More on Linux Crontab Curl Tutorial
Sometimes for a website you only want to do a web server task once or a few times a day, and you, ideally, wouldn’t want to run into the remote possibility that two people could perform the task at precisely the same time. This scenario happens often when using server-side languages like ASP.Net and PHP. If it didn’t matter that two people might do the same thing at once then you could do the job within you web-based PHP, and you may consider using a date and time test in your web-based PHP to do your “once or a few times” task in your PHP. This latter idea has the advantage that file ownership issues will probably not crop up because the web user will be manipulating any media or HTML files you may be dealing with, rather than the administrator user that will probably be overseeing your “once or a few times” cron job that we show you here today.
So the task today is to keep the rjmprogramming.com.au’s landing page “Tutorials” dropdown up to date with the latest blog postings, rather than having to manually attend to this with interactive (HTML code) edits and (s)ftp uploads. You may be wondering why not just write the landing page in PHP, and this is a fair point, but the strategy here at this domain is that the landing page and its “fellow” webpages should be independent of MySql, which, under big loads, can go down, and here we use a watchdog to fix that, should it happen. Previously we had a very similar task you can read about with the Linux Crontab Curl Primer Tutorial below. It is so similar, let’s use curl, PHP and crontab to do this task like with that previous task. So this is command line, and this is batch work, but that worry in other types of batch work where you need to worry about where you are as you do any file management, is handled quite well by the wrapping of the job in curl, which takes as its argument a URL, so, implicitly, the place to work with file management, is handled by curl and the PHP we write (specifically where the code uses dirname(__FILE__)).
Here is the downloadable PHP programming source code you could call tutorial_options.php and you will see, if you examine the code that it is very specific to conditions here at this rjmprogramming.com.au domain, so please be aware of that, but maybe its concepts can help you out with something else you are tackling. A list of the specifics you would want to address (should you adapt it for your own purposes) include:
- the PHP explode methodology (in the code above) assumes there are two select dropdowns (spelt in lowercase) in the underlying HTML code (for rjmprogramming.com.au’s landing page) we are rewriting, and that it is the second one we update with information gleaned (via SQL query) from the WordPress MySql database the PHP reads (where we piece together a $retval string of the HTML option tags belonging to the tutorials HTML select tag)
- the MySql database access hostname, username, password, database name, database table name
- the Linux web server file owner used during the (commented out) chown command you may need to consider to allow the crontab’s user be able to leave the files owned by the website user
PHP commands of interest in today’s tutorial include:
Maybe it is just me, but liking “Onion of the 4th dimension” the kind of thinking I gravitate towards is:
- supervisory command line arrangements
- all “replace” type functionality … to get the problem into a shape suitable for the bits down deeper into the “core” of your “onion” … Linux-speak would say “kernel” of your “onion”
- parameters and arguments … continuing on that theme of slotting something (the user is specifically interested in, data-wise) in dynamically for something that “represents” it in the code … sort of, like an “interpretive” approach (often people like to say “scripting”)
Programmers can think in entirely different ways, and if other methods work then that is great (woh!) but if they do not work for any platform (sometimes you don’t intend to cater for every platform known to humans and mangoes … damn … was doing so well … persongoes), and these ideas may really irk many programmers, but you need to consider styles of work that suit your way of thinking best.
So, to do this task we wrote new PHP code and cloned the crontab code used in the tutorial below but change the minute of the hour the task is run, and what is run, in crontab on the Linux web server for the domain at rjmprogramming.com.au … happy studies!
Previous relevant Linux Crontab Curl Primer Tutorial is shown below.
Linux Crontab Curl Primer Tutorial
Linux (or Unix) is a command line environment that supports interactive and non-interactive task management. Often non-interactive tasks are known as batch processes, often intended not to rely on any user intervention, unless there is a problem.
Today, in this tutorial (which builds on Linux Background Primer Tutorial), we see the power of combining Curl under the auspices of crontab to run a webpage piece of functionality once a day at a specified time.
Some CMS systems like Drupal use crontab to perform actions on the database at regular intervals.
For Windows systems you might want to look at Windows Task Schedular for similar functionality.
Linux has several approaches to activating a task, some (not mentioning all the ways one process can fork or launch another process) of them being:
- interactively (from the command line)
- kick off a process from command line, and place in in the background, without supervision, necessarily, via the suffix &
- kick off a process without supervision, necessarily, via crontab
- kick off a process in the background (without supervision, necessarily) via nohup
- kick off a process from command line, and place it in the background, without supervision, necessarily (and optionally change its state via fg and bg)
The previous Linux Background Primer Tutorial is well worth reading below.
Linux Background Primer Tutorial
Linux (or Unix) is a command line environment that supports interactive and non-interactive task management. Often non-interactive tasks are known as batch processes, often intended not to rely on any user intervention, unless there is a problem.
In the pre-GUI days batch processes were very prominent, but they still are today, as far as actually getting things done. It is just that the more glamorous activities pass our notice more, but there are servers out there churning away with batch processes, dreaming of the day when they’ll work out how they can make that cup of coffee for you in the morning. And do we ever offer them even a cup’o’tea … it’s a disgrace, so it is.
Linux has several approaches to activating a task, some (not mentioning all the ways one process can fork or launch another process) of them being:
- interactively (from the command line)
- kick off a process from command line, and place in in the background, without supervision, necessarily, via the suffix &
- kick off a process without supervision, necessarily, via crontab
- kick off a process in the background (without supervision, necessarily) via nohup
- kick off a process from command line, and place it in the background, without supervision, necessarily (and optionally change its state via fg and bg)
The last four above are often best performed on a shell script, whether that be a Bash, Korn, Bourne or Cshell script. There are some good general tips for batch process scripts:
- never assume where you are, rather specify the directory to be in, via “cd”
- never assume that the environment variables will be exactly the same as for an interactive session
- be more thorough to log errors and log activity to files to see, later, what happened
- it is not good to proceed on errors (unless you have set up independent means of checking) because there will be no human to interactively decide that the problem should not stop the job proceeding
- pretty obviously, don’t expect anybody to answer an interactive entry requirement (but you can simulate lots of interactive input via redirected standard input eg. < [filename])
Take a look at some example Linux background processing at this tutorial. In it you will see some crontab work where the parameters are (as explained here … thanks):
MIN = Minute 0-60
HOUR = Hour [24-hour clock] 0-23
MDAY = Day of Month 1-31
MON = Month 1-12 OR jan,feb,mar,apr …
DOW = Day of Week 0-6 OR sun,mon,tue,wed,thu,fri,sat
COMMAND = Command to be run Any valid command-line
The */6 * * * * /Applications/MAMP/htdocs/pdftoimage.sh of our usage runs /Applications/MAMP/htdocs/pdftoimage.sh every six minutes.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
