Troubleshooting MySql Query Error Tutorial
Please see the parallels with the previous Troubleshooting Landing Page Crontab Curl Tutorial as we review a “same fix” “different symptom” regarding diskspace on our RJM Programming domain Apache/PHP/MySql web server …
Let’s go over the symptoms found with our web server on 20/4/2020. Lots of the workings of “This Blog” were fully functional including …
- navigating from Landing Page to “This Blog”
- navigation from links within “This Blog”
- search textbox searches within “This Blog”
- tag link navigation within “This Blog” … but …
- ☞ emoji button “Is Mentioned By” (involving MySql query completion, behind the scenes) hung … as did …
- isolating (into a URL and up at the web address bar) the PHP calling the MySql query into an execution completed with no result … and …
- plucking out the MySql query from PHP execution above (we could get to that point with the execution) and feeding that into a (brilliant, marvellous, curiously refreshing) phpMyAdmin SQL window then clicking “Go” button resulted in …
#1030 - got error 28 from storage engine
- Google search for “#1030 – got error 28 from storage engine” got us to …
- MySQL 1030 got error 28 from storage engine – Quick Fix! …
We’ve seen many of our customers experience the error ‘1030 got error 28 from storage engine’. The error occurs when there is no enough disk space left for MySQL to run properly. When the server has a lack of disk space or inodes which prevents MySQL from writing to the disk.
… as the end of the “away from web server” Analysis stage to the Troubleshooting Cycle, leading to the “at the web server” Analysis and Fix Deployment stage to the Troubleshooting Cycle …
- logged into web server via …
ssh -p 22 [priviledgedUsername]@[webServerIPaddress]
- we did “$ df -k /” and “$ df -i /” and these were okay … but …
- and subsequent “$ ls -l /tmp | more” … amongst others … showed …
-rw------- 1 51840000 Apr 20 03:22 magick-292610qF3dwPahJP5
-rw------- 1 51840000 Apr 20 03:22 magick-29261HlkxLJfysIlp
-rw------- 1 51840000 Apr 20 03:24 magick-29261QTbh8WfGJWFP
-rw------- 1 51840000 Apr 20 03:44 magick-338887dWYhlpIY8my
-rw------- 1 51840000 Apr 20 03:44 magick-33888K6VdS4hp0CSV
-rw------- 1 51840000 Apr 20 03:45 magick-33888lYrilhZ2fiOb
-rw------- 1 42226688 Apr 20 03:49 magick-35006u4MoeBAUQRaC
-rw------- 1 51840000 Apr 20 03:50 magick-35007wI1WBpZhu9GI
-rw------- 1 46220288 Apr 20 03:50 magick-35017acMtBUkIA4Cp
-rw------- 1 51840000 Apr 20 03:50 magick-35017l-VQQLoigCCL
-rw------- 1 0 Apr 20 03:50 magick-350870ZorZ6dH6tDu
- that “$ rm -f /tmp/magick-*” released a lot of /tmp diskspace … resulting in …
- got us back to …
$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/usr/tmpDSK 544256 22696 493912 5% /tmp
$ exit
- which caused all the issues mentioned in section way above to start working again … yay!
$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/usr/tmpDSK 544256 516556 52 100% /tmp
This was the “on the job” fix with somebody seeing the symptom. On the RJM Programming web server “crontab” scheduled jobs is a …
13 4 * * * ksh -c 'for i in `find /tmp -name "magick-*" -atime +2`; do rm -f $i; done'
… that we “reappraised to a day’s worth of ImageMagick temporary files still there (that only happens with a diskspace issue problem) alternative and backup approach” …
13 4 * * * ksh -c 'for i in `find /tmp -name "magick-*" -atime +2`; do rm -f $i; done'
17 6 * * * ksh -c 'for i in `find /tmp -name "magick-*" -mmin -1440`; do rm -f $i; done'
… entries that would have fixed this issue in a scheduled way, that was set up after we were faced with Troubleshooting Landing Page Crontab Curl Tutorial. You may wonder “why just a once a day approach” … well, we try not to clobber a legitimate use of the wonderful ImageMagick command line functionality we talk about a bit at this blog.
Previous relevant Troubleshooting Landing Page Crontab Curl Tutorial is shown below.
Troubleshooting Landing Page Crontab Curl Tutorial
Back earlier in December (2019) the crontab/curl work of Landing Page Mobile Phone Crontab Curl Tutorial must have been disrupted.
Why think that?
The RJM Programming Landing Page dropdown of blog tutorial links was missing something … anyone, anyone? … yes, Sunil Gavaskar … its blog posting links. And how are they created each day … anyone, anyone? … yes, Laika, look down to Landing Page Mobile Phone Crontab Curl Tutorial to read all about it.
Do you think it would be good to try, via ssh access what crontab/curl automates on the RJM Programming web server, but on the command line?
Excellent suggestion!
$ curl HTTP://www.rjmprogramming.com.au/PHP/tutorial_options.php
<b>Error 3: Error writing file '/tmp/MYpxajPE' (Errcode: 28)</b><option selected='selected' value=' '>Tutorials (show blog, toggle sort order) ...</option>
Ah! Do you think a diskspace examination of the /tmp disk on this Linux web server would reveal anything useful?
You’re on a roll!
$ df -k /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
/usr/tmpDSK 544256 513316 3292 100% /tmp
Wow, that’s bad! Do you think a file listing examination of the /tmp disk on this Linux web server would reveal anything useful?
Have you been reading the script to all this?
$ ls -l /tmp
total 490508
drwxrwxrwt 114 owner group 294912 Dec 2 03:54 ./
dr-xr-xr-x. 25 owner group 4096 Dec 1 04:49 ../
-rw-r--r-- 1 owner group 32 Aug 9 2014 adminer.key
-rw-r--r-- 1 owner group 27 Aug 26 2014 aexception.is
drwx------ 2 owner group 4096 Jun 16 2018 ansible_Hinjci/
-rw-r--r-- 1 owner group 225887 Dec 2 02:04 basegetmelist.is
-rw-r--r-- 1 owner group 225887 Dec 2 02:01 basegetmelist.wis
-rw-r--r-- 1 owner group 43615 Jan 18 2015 basegetmelist.wisold
-rw-r--r-- 1 owner group 17099 Apr 8 2014 BatsAndBall.tcl
-rw-r--r-- 1 owner group 130989 Apr 23 2018 CDB-simple_after_LibreOffice.htm
-rw-r--r-- 1 owner group 20624 Jun 4 2015 check-mk-agent.rpm
drwx------ 2 owner group 4096 May 11 2019 clamav-030407a30e41469e50714e18d06cfef6/
drwx------ 2 owner group 4096 Nov 10 2018 clamav-05c7969f7c92127c0a67d1e1e030ff7a/
drwx------ 2 owner group 4096 Jun 16 2018 clamav-0c9c5d08dd88487d2db8ab0d89844a98/
# more follows ...
-rw------- 1 owner group 20041728 Dec 1 02:32 magick-299501ChMis2EYamN
-rw------- 1 owner group 31457280 Dec 1 02:32 magick-29950E5FvDZxbZUTb
-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30009gmKHkoN2pnsz
-rw------- 1 owner group 31457280 Dec 1 02:32 magick-30011B1tagrDAr7pR
-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30011VUw-Xwaqo2lT
-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30043kgDfzEiS4YhO
-rw------- 1 owner group 31457280 Dec 1 02:34 magick-30089dJLmu7BUm4Cw
-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30133nUvTOGPaB5XG
-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30133qaJFxqr0Kbgr
-rw------- 1 owner group 7234560 Dec 1 02:45 magick-33938vgJrHboBpWeI
-rw------- 1 owner group 31457280 Dec 1 02:59 magick-36847TC-kxmkRqR4V
-rw------- 1 owner group 31457280 Dec 1 03:00 magick-36847YUdME4k4jgwh
-rw------- 1 owner group 31457280 Dec 1 02:59 magick-368918jQfT0gouz5z
-rw------- 1 owner group 31457280 Dec 1 03:00 magick-36891NSRHciA7uHb8
-rw------- 1 owner group 31457280 Dec 1 03:00 magick-3707285W0V0oGoH97
-rw------- 1 owner group 31457280 Dec 1 02:59 magick-37072lAYUvoSex9wk
-rw------- 1 owner group 31457280 Dec 1 03:00 magick-37073XLevidVmQCPl
# more follows ...
-rw-rw-rw- 1 owner group 87155 Mar 26 2019 minify_0dca26f4ae4988d406871896dc562350
-rw-rw-rw- 1 owner group 15260 Mar 26 2019 minify_0dca26f4ae4988d406871896dc562350.gz
-rw-rw-rw- 1 owner group 85916 Mar 26 2019 minify_2972bf58405bbbc339884b94935eb6bf
-rw-rw-rw- 1 owner group 17938 Mar 26 2019 minify_2972bf58405bbbc339884b94935eb6bf.gz
-rw-rw-rw- 1 owner group 12447 Mar 26 2019 minify_67325ffb44e64f839c31db871aa8f6de
-rw-rw-rw- 1 owner group 2824 Mar 26 2019 minify_67325ffb44e64f839c31db871aa8f6de.gz
-rw-rw-rw- 1 owner group 185518 Mar 26 2019 minify_f8189fe934d7fecd140658eef38fcf74
-rw-rw-rw- 1 owner group 34816 Mar 26 2019 minify_f8189fe934d7fecd140658eef38fcf74.gz
-rw------- 1 nobody root 2227 Apr 25 2014 mNQtmxm4OS
-rw-r--r-- 1 owner group 0 Apr 28 2015 msans.ans
-rwxr-xr-x 1 owner group 741 Apr 28 2015 msans.ksh*
-rw------- 1 mysql mysql 0 Jun 19 2018 MYchYpJx
-rw-r--r-- 1 owner group 0 Sep 12 2017 mysocket1337.tmp
-rw-r--r-- 1 owner group 386 Oct 21 2014 no.worries
drwxr-xr-x 2 owner group 4096 Oct 21 2014 okold/
-rw------- 1 owner group 0 Jan 11 2017 pdfFafOZA
Those magick-* files seem problematic! Do you think a look around the net would reveal anything useful?
Astounding! Look at what are ‘/tmp/magick-*’ files? – ImageMagick …
When IM can’t do its work in main memory, it uses disk. If the process completes normally, the temporary disk files are deleted. If the process fails, these files are often not deleted.
If you have no IM processes running, you can safely delete these files.
Knowing that, and seeing the dates of those problematic /tmp/magick-* files, am I right in thinking you’ll tabulate a short term and long term strategy remedy to the problem?
And do fish swim?!
| Short term strategy | Long term strategy Within ksh shell script that crontab runs each minute, add (allowing 40 minutes before saying the ImageMagick must have failed) … |
|---|---|
|
|
Feel free to see a PDF stream of consciousness presentation of us dealing with this issue.
Previous relevant Landing Page Mobile Phone Crontab Curl Tutorial is shown below.
Landing Page Mobile Phone Crontab Curl Tutorial
That work with the RJM Programming Landing Page we were last looking at a couple of days ago (with Landing Page Mobile Phone Tutorial) was all fine and good, but the introduction of a new HTML indexmobile.html for mobile phone usage, specifically, needs melding into existant web server arrangements.
For the most part “web server arrangements” has little to do with HTML and Javascript, which is usually client side work, unless, perhaps you involve Ajax.
But even ignoring Ajax, and ignoring server side languages like PHP (or ASP.Net) … remember …
As we continue in this series of tutorials we don’t pretend to be experts on aesthetics but we will show you some HTML and Javascript (no PHP, deliberately (read Linux sudo nohup Watchdog Primer Tutorial to get this in context, for us)) functionality ideas that could supplement some great graphical design ideas you have to start something yourself, perhaps.
… from Landing Page Primer Tutorial … how can HTML (with its associated Javascript) make use of “web server” anything much other than being a place it gets stored?
Well, that web server is a server … a computer … with its own processes … ours is a Linux web server “computer” … with batch processing capabilities … easiest to access via crontab, and we first talked about how the RJM Programming Landing Page could be supplied with an HTML select element filled with RJM Programming blog tutorials with the most recent ones up the top, via the “crontab” running of a PHP web application that reads the WordPress MySql database to derive the contents of this HTML select element plonked into the Landing Page when we presented More on Linux Crontab Curl Tutorial as shown below.
So here we have a before and after scenario … that doesn’t involve hair nor weight loss … so here is the crontab before (which we’ll keep but add another similar process onto) combining its talents with PHP and curl … the “CC dynamic duo” …
59 23 * * * curl http://www.rjmprogramming.com.au/PHP/tutorial_options.php 2> /home/rjmprogr/public_html/PHP/tutorial_options.bad
… but we need to tell you a great thing about curl … you can access, easily, a single $_GET[] PHP parameter, because Linux doesn’t think “?” is very special (but it does have a “background process meaning” for “&” making it hard to use curl with more than one $_GET[] PHP parameter involved) … leading us to our new bold crontab record below to cater for “indexmobile.html” updating …
59 23 * * * curl http://www.rjmprogramming.com.au/PHP/tutorial_options.php 2> /home/rjmprogr/public_html/PHP/tutorial_options.bad
57 23 * * * curl http://www.rjmprogramming.com.au/PHP/tutorial_options.php?mobile=mobile 2> /home/rjmprogr/public_html/PHP/tutorial_options.bad
… and what needed doing with the PHP?
Here is the downloadable PHP programming source code you could call tutorial_options.php changed for the purposes of being capable of updating “indexmobile.html” as per this link.
The effect on mobile phone usage of the RJM Programming Landing Page tutorial dropdown (HTML select element) list of RJM Programming blog tutorials (updated daily) is seen with today’s tutorial picture.
Previous relevant More on Linux Crontab Curl Tutorial is shown below.
More on Linux Crontab Curl Tutorial
Sometimes for a website you only want to do a web server task once or a few times a day, and you, ideally, wouldn’t want to run into the remote possibility that two people could perform the task at precisely the same time. This scenario happens often when using server-side languages like ASP.Net and PHP. If it didn’t matter that two people might do the same thing at once then you could do the job within your web-based PHP, and you may consider using a date and time test in your web-based PHP to do your “once or a few times” task in your PHP. This latter idea has the advantage that file ownership issues will probably not crop up because the web user will be manipulating any media or HTML files you may be dealing with, rather than the administrator user that will probably be overseeing your “once or a few times” cron job that we show you here today.
So the task today is to keep the rjmprogramming.com.au’s landing page “Tutorials” dropdown up to date with the latest blog postings, rather than having to manually attend to this with interactive (HTML code) edits and (s)ftp uploads. You may be wondering why not just write the landing page in PHP, and this is a fair point, but the strategy here at this domain is that the landing page and its “fellow” webpages should be independent of MySql, which, under big loads, can go down, and here we use a watchdog to fix that, should it happen. Previously we had a very similar task you can read about with the Linux Crontab Curl Primer Tutorial below. It is so similar, let’s use curl, PHP and crontab to do this task like with that previous task. So this is command line, and this is batch work, but that worry in other types of batch work where you need to worry about where you are as you do any file management, is handled quite well by the wrapping of the job in curl, which takes as its argument a URL, so, implicitly, the place to work with file management, is handled by curl and the PHP we write (specifically where the code uses dirname(__FILE__)).
Here is the downloadable PHP programming source code you could call tutorial_options.php and you will see, if you examine the code that it is very specific to conditions here at this rjmprogramming.com.au domain, so please be aware of that, but maybe its concepts can help you out with something else you are tackling. A list of the specifics you would want to address (should you adapt it for your own purposes) include:
- the PHP explode methodology (in the code above) assumes there are two select dropdowns (spelt in lowercase) in the underlying HTML code (for rjmprogramming.com.au’s landing page) we are rewriting, and that it is the second one we update with information gleaned (via SQL query) from the WordPress MySql database the PHP reads (where we piece together a $retval string of the HTML option tags belonging to the tutorials HTML select tag)
- the MySql database access hostname, username, password, database name, database table name
- the Linux web server file owner used during the (commented out) chown command you may need to consider to allow the crontab’s user be able to leave the files owned by the website user
PHP commands of interest in today’s tutorial include:
Maybe it is just me, but liking “Onion of the 4th dimension” the kind of thinking I gravitate towards is:
- supervisory command line arrangements
- all “replace” type functionality … to get the problem into a shape suitable for the bits down deeper into the “core” of your “onion” … Linux-speak would say “kernel” of your “onion”
- parameters and arguments … continuing on that theme of slotting something (the user is specifically interested in, data-wise) in dynamically for something that “represents” it in the code … sort of, like an “interpretive” approach (often people like to say “scripting”)
Programmers can think in entirely different ways, and if other methods work then that is great (woh!) but if they do not work for any platform (sometimes you don’t intend to cater for every platform known to humans and mangoes … damn … was doing so well … persongoes), and these ideas may really irk many programmers, but you need to consider styles of work that suit your way of thinking best.
So, to do this task we wrote new PHP code and cloned the crontab code used in the tutorial below but change the minute of the hour the task is run, and what is run, in crontab on the Linux web server for the domain at rjmprogramming.com.au … happy studies!
Previous relevant Linux Crontab Curl Primer Tutorial is shown below.
Linux Crontab Curl Primer Tutorial
Linux (or Unix) is a command line environment that supports interactive and non-interactive task management. Often non-interactive tasks are known as batch processes, often intended not to rely on any user intervention, unless there is a problem.
Today, in this tutorial (which builds on Linux Background Primer Tutorial), we see the power of combining Curl under the auspices of crontab to run a webpage piece of functionality once a day at a specified time.
Some CMS systems like Drupal use crontab to perform actions on the database at regular intervals.
For Windows systems you might want to look at Windows Task Schedular for similar functionality.
Linux has several approaches to activating a task, some (not mentioning all the ways one process can fork or launch another process) of them being:
- interactively (from the command line)
- kick off a process from command line, and place in in the background, without supervision, necessarily, via the suffix &
- kick off a process without supervision, necessarily, via crontab
- kick off a process in the background (without supervision, necessarily) via nohup
- kick off a process from command line, and place it in the background, without supervision, necessarily (and optionally change its state via fg and bg)
The previous Linux Background Primer Tutorial is well worth reading below.
Linux Background Primer Tutorial
Linux (or Unix) is a command line environment that supports interactive and non-interactive task management. Often non-interactive tasks are known as batch processes, often intended not to rely on any user intervention, unless there is a problem.
In the pre-GUI days batch processes were very prominent, but they still are today, as far as actually getting things done. It is just that the more glamorous activities pass our notice more, but there are servers out there churning away with batch processes, dreaming of the day when they’ll work out how they can make that cup of coffee for you in the morning. And do we ever offer them even a cup’o’tea … it’s a disgrace, so it is.
Linux has several approaches to activating a task, some (not mentioning all the ways one process can fork or launch another process) of them being:
- interactively (from the command line)
- kick off a process from command line, and place in in the background, without supervision, necessarily, via the suffix &
- kick off a process without supervision, necessarily, via crontab
- kick off a process in the background (without supervision, necessarily) via nohup
- kick off a process from command line, and place it in the background, without supervision, necessarily (and optionally change its state via fg and bg)
The last four above are often best performed on a shell script, whether that be a Bash, Korn, Bourne or Cshell script. There are some good general tips for batch process scripts:
- never assume where you are, rather specify the directory to be in, via “cd”
- never assume that the environment variables will be exactly the same as for an interactive session
- be more thorough to log errors and log activity to files to see, later, what happened
- it is not good to proceed on errors (unless you have set up independent means of checking) because there will be no human to interactively decide that the problem should not stop the job proceeding
- pretty obviously, don’t expect anybody to answer an interactive entry requirement (but you can simulate lots of interactive input via redirected standard input eg. < [filename])
Take a look at some example Linux background processing at this tutorial. In it you will see some crontab work where the parameters are (as explained here … thanks):
MIN = Minute 0-60
HOUR = Hour [24-hour clock] 0-23
MDAY = Day of Month 1-31
MON = Month 1-12 OR jan,feb,mar,apr …
DOW = Day of Week 0-6 OR sun,mon,tue,wed,thu,fri,sat
COMMAND = Command to be run Any valid command-line
The */6 * * * * /Applications/MAMP/htdocs/pdftoimage.sh of our usage runs /Applications/MAMP/htdocs/pdftoimage.sh every six minutes.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
If this was interesting you may be interested in this too.
