MySql Watchdog Migration Transition Solution Tutorial

MySql Watchdog Migration Transition Solution Tutorial

Onto yesterday’s MySql Watchdog Migration Transition Preparations Tutorial, yes, with Korn Shell scripting, thanks to the advice from How to use nslookup in bash to verify is DNS is configured properly? and bash – How to get the primary IP address of the local machine on Linux and OS X? – Stack Overflow …


#!/bin/ksh
# will_the_real_rjmprogramming_com_au_holler.ksh

nsl="`nslookup rjmprogramming.com.au`";
myip="`ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1'`";
if [ "`echo ${nsl} | grep -c ${myip}`" != "0" ]; then
if [ -d '/home/rjmprogr/public_html/wordpress/' -a "$1" = "doit" ]; then
echo y > /home/rjmprogr/public_html/wordpress/mysql_needs.attention
chmod 777 /home/rjmprogr/public_html/wordpress/mysql_needs.attention
else
echo "yes";
fi
exit;
fi
if [ -d '/home/rjmprogr/public_html/wordpress/' -a "$1" = "doit" ]; then
echo "";
else
echo "no";
fi
exit;


… we can now “play dead” (by taking those blue codelines, which also appear in the MySql Watchdog, and now, instead, have …


if [ "2" != "4" ] ; then
ksh /etc/init.d/will_the_real_rjmprogramming_com_au_holler.ksh doit
else
echo y > /home/rjmprogr/public_html/wordpress/mysql_needs.attention
chmod 777 /home/rjmprogr/public_html/wordpress/mysql_needs.attention
fi


…) regarding web servers whose IP address do not match the DNS looked up for rjmprogramming.com.au as a domain, with that “attention seeking” /home/rjmprogr/public_html/wordpress/mysql_needs.attention file creation that our MySql Watchdog works off as the basis to swing into action … more reliable than …

Previous relevant MySql Watchdog Migration Transition Preparations Tutorial is shown below.

MySql Watchdog Migration Transition Preparations Tutorial

When we commented, in AlmaLinux Apache/PHP/MySql WordPress Migration Tutorial …

Yes, we’re migrating again (as we were talking about with Apache/PHP/MySql Web Hosting Website Migration ssh Tutorial), though as you read this blog posting, if hot off the press here on 19/7/2024, we’ll not have transferred the DNS over yet, which we’ll outline the reasons for into the future. But, today, we’re excited to outline some steps after …

… it was, mainly, regarding our MySql Watchdog ( korn shell scripting in the Linux Apache/PHP/MySql web server /etc/init.d scripts called via crontab ) that concerned us most. Why (Nala getting distracted, ever since MySql Down Thinking Followup Tutorial … or …)?

Well, understandably our Crazy Domains web hosts were kind enough to …

• take on the migration … thanks … and not, just blithely …
• make any “old scripting” assumptions ( ie. we came across the new web server’s /etc/init.d folder as empty (and /etc/rc.d/init.d folder, for your AlmaLinux Apache2 daemon process folder arrangements not empty, but missing our own tailored scripting (and that scripting and our crontabbing used /etc/init.d mentions) … so …) … especially any to do with anything even resembling …
• crontab … nor, heaven forbid …
• curl … (the use of which, up to now, we’ve only ever combined with argument URLs that mentioned rjmprogramming.com.au rather than any numerical IP address … hence, our concerns) … scripting being helped out by crontab … that had any personal tailoring ( ie. a crontab -l
  # showed default AlamaLinux arrangements only

This can be very powerful stuff, and you could, during a period before the DNS changeover, if you had a matching crontab situation happening, having two watchdogs on two different IP addresses (only one of which is pointed to proper DNS rjmprogramming.com.au arrangements trying to pick up a MySql service, and though only one would get through picking up the true rjmprogramming.com.au MySql service, the other would, at the very least, be doing superfluous activity because of the way we scripted it. We scripted for …

• one DNS arrangement for the one IP address related to rjmprogramming.com.au … rather than …
• one DNS arrangement for the one IP address related to rjmprogramming.com.au and another IP address being readied for a future DNS rearrangement that it will map to … and after that …
• one DNS arrangement where that new IP address can newly become the DNS for rjmprogramming.com.au and another obsolete (but still having a working crontab happening) IP address awaits a decommission

That is the power, and also the worry, regarding how curl and crontab can be such useful Linux web server tools!

For number 3 scenario, perhaps we can be quick decommissioning, but we imagine we’ll want to have another test period after the DNS changeover. We’ll see. Anyway, on the road towards solutions we’re pretty sure a first step is …

• before thinking of starting up any “inhouse scripting componentry” crontab schedulings on the new IP address web server …
• populate our empty /etc/init.d … with …
• matching script content and permissions and ownership as is (and then, was) happening on the current (but soon to be decommissioned) IP address web server

This transfer of scripts is not suitable for our FileZilla sftp transferring arrangements, and we do not feel like tackling any big shake up of arrangements here.

But, we ended up with a zipping and unzipping methodology that went like …

• pick a folder that is hidden to the public (for both IP addresses) but that your Linux administration user (in our case “root”) can get to and our FileZilla web server user (in our case “rjmprogr”) can plonk to and from, for us, that being (notice, regarding Apache, no “public_html” mention) … /home/rjmprogr
• for current, but soon to be decommissioned IP address … ssh login … cd /etc/init.d … zip /home/rjmprogr/etcinitd.zip *
• use relevant FileZilla on current, but soon to be decommissioned IP address setup to copy this back to your local operating system, ours being macOS …
• use relevant FileZilla on new, but soon to be the DNS IP address setup to copy this local operating system etcinitd.zip over to /home/rjmprogr
• for new, but soon to be the DNS IP address … ssh login … cd /etc/init.d … unzip /home/rjmprogr/etcinitd.zip *
• if happy ( via … ls -l /etc/init.d/* … checking and comparing between IP addresses ), for current, but soon to be decommissioned IP address … ssh login … rm -f /home/rjmprogr/etcinitd.zip # or do this via appropriate FileZilla session
• if happy ( via … ls -l /etc/init.d/* … checking and comparing between IP addresses ), for new, but soon to be the DNS IP address … ssh login … rm -f /home/rjmprogr/etcinitd.zip # or do this via appropriate FileZilla session

… that we were really happy with.

For a small number of script files, though, an animated GIF presentation we’ve provided (and don’t you just hate it when you only get the knack of it near the end of the job … sheeesh!) you might want to use …

• open two macOS Terminal windows side by side (and not overlapping) …
• left one has ssh login to current, but soon to be decommissioned IP address … cd /etc/init.d
• right one has ssh login to new, but soon to be the DNS IP address … cd /etc/init.d
• for each ( mainly *.*sh for us ) (korn shell usually for us) script of interest (we’ll call thesc.ksh below) …
  1. # no return key press below …
  2. left % cat thesc.ksh
  3. # you might want to use Terminal app’s Edit -> Copy the highlighted “thesc.ksh” to help (via Terminal app’s Edit -> Paste) with …
  4. right % vi thesc.ksh
  5. at left, as necessary, press Return key
  6. at left highlight from the bottom ( probably “exit” ) up to the top ( probably “#!/bin/ksh” ) and use Terminal app’s Edit -> Copy
  7. at right, as necessary, press Return key
  8. at right, if content already there, use appropriate <esc_ape>999dd command to clear to nothing then …
  9. at right, press “i” key (for Insert) …
  10. at right, use Terminal app’s Edit -> Paste
  11. at right, maybe press <esc_ape>w (to have an interim look) ahead of …
  12. at right, maybe press <esc_ape>wq!
• attend to any ownership (research chown and chgrp) or file permission (research chmod) mismatches you would tweak to via ls -l /etc/init.d/* … checking and comparing between IP addresses
• and consider only leaving “inhouse scripting” in /etc/init.d on new, but soon to be the DNS IP address here, for now (and into the future rewrite scripting for /etc/rc.d/init.d reuse)

Previous relevant MySql Down Thinking Followup Tutorial is shown below.

MySql Down Thinking Followup Tutorial

When we discussed Linux Watchdog Primer Tutorial in that very generic fashion below, we were being that … generic. Today we turn to specifics in that regard. On the rjmprogramming.com.au domain we use a …

• (software) watchdog which checks for the health of the MySql Service, and if not healthy, restore it to health … which is all fine and good from the point of view of the domain rjmprogramming.com.au … but depending on what the user was doing we could …
• code for intervention within the MySql using piece of software, and writing out the database error to the webpage, and present alternative navigation

… on the understanding with that latter thought, that we put into play today for a WordPress 4.1.1 blog, we realized we needed to intervene in …


[DocumentRoot]/ITblog/wp-includes/functions.php


… in the emboldened code intervention as below …



// Otherwise, be terse.
status_header( 500 );
nocache_headers();
$ubitsare=explode("/", str_replace("/?p=","",$_SERVER['REQUEST_URI']));
if (sizeof($ubitsare) > 2) {
if (strpos(str_replace("-","%20",$ubitsare[2]), "slideshow.htm") === false) {
header("Location: /slideshow.html?title=" . explode("&", str_replace("-","%20",str_replace("/?p=","",$ubitsare[2]))[0]));
exit;
} else {
header( 'Content-Type: text/html; charset=utf-8' );
}
} else {
header( 'Content-Type: text/html; charset=utf-8' );
}
?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"<?php if ( is_rtl() ) echo ' dir="rtl"'; ?>>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title><?php _e( 'Database Error' ); ?></title>

</head>
<body>
<h1><?php _e( 'Error establishing a database connection'); ?></h1>
</body>
</html>
<?php
die();
}



… where the redirection to /slideshow.html effectively “Cuts to the Chase” for the gist of that blog posting you never got to, as MySql is currently down.

Notice how specific the actions can be when you write code “dependent” on another software component, as distinct from the “watchdog” approach we often want to be “independent” in its thinking.

Guess this begs the question? How did we work out where to intervene? We just got to folders on the rjmprogramming.com.au web server with the WordPress (Codex) software and went (showing you the one that yielded the nicest result for us) …


pwd // and we are at [DocumentRoot]/ITblog
cd wp-includes
fgrep 'Error establishing a database connection' *.php


Previous relevant Linux Watchdog Primer Tutorial is shown below.

Linux Watchdog Primer Tutorial

“

Do you have a computer operational problem? What the … what is that humanoid on about? Well, is there a computer operation you have to do routinely, to fix an ongoing problem, that relies on your personal intervention that, one day, you’d like to not have to worry about, or better still … you usually intervene, but would like a backup approach should you get sick and can’t do it … here’s where you need Fido a watchdog

… am not recommending a Golden Retriever, unless you want your process licked rather than attended to? (had you forgotten this was a question, Cedric Linux Nala)? …

oh, moi?)

“

Why is bellybutton fluff blue? But we digress, or at Buckingham Palace one digresses.

Right, back at watchdogs, there are schools of thought …

1. The process is down for too long and we need to do something about it, because customers are leaving
2. If you need a watchdog to save your bacon, then (clearly you’re not with it, because the dog’s eaten the bacon and) there is something else fundamentally wrong or something you do not understand, which is what you really should resolve, either way

My view is that, if the underlying process would take years to understand or if it is written in legacy code, I’d go more with the former idea, especially as there is delight in creating a really good watchdog (training one? not so easy) … it can be really hard to do … for this reason cannot give code here really, because there is no “out of the box” that is a responsible approach to advise … you have to study the issue and cut it into its components, unit test the solutions to the components of your watchdog solution, and retest with the interactions of those components. However, this is a coding enthusiast’s view, and is a bit short-sighted, perhaps. In any case, what will save the day is that this decision will probably be made for you by an operations expert, if you work in a large organization.

Some other watchdog considerations should be …

1. Is the attempt to automate the solution that the watchdog will provide technically possible … may not be?!
2. We can resolve it with personal intervention … can the watchdog simulate each step of the human intervention? If so, go for that approach if possible.
3. Be very careful of approaches that involve mouse clicks, as they are quite often relative to too many other environmental issues … try to restrict the watchdog solution to command line/scripting/keyboard ideas … on Windows, AutoHotKey is an excellent recording program of interest (would recommend just using it for keyboard recordings, if using it for a watchdog … by the way, tomorrow (tomorrow arrived today) we do a tutorial showing you how to create an AutoHotKey terminate-and-stay-resident program on Windows).
4. Have we identified the real intervention points? If not, you might succeed some of the time, but not all the time, and you may cause damage on those times when you have made some assumptions, with your incomplete understanding.

Here is an example. You have an overnight batch process run, and it falls over at a certain point, and you get paged at 3 or 4 (it’s bound to be AM). It has been tentatively decided you might want to create a watchdog … what are some considerations …

1. What do log files tell me? Find out.
2. Is it a single thing that is missing that would resolve the problem once and for all? If yes, well, you know not to deviate from this one thing … ignore ideas below.
3. Of the few problems, is it worth proceeding with the watchdog idea, because the number of separate issues can often cause a factor of ten more complication issue points, and maybe you should stick with human intervention.
4. Break the watchdog problem into these problem issues as a separate unit-testable piece of scripting code (or whatever your watchdog solution entails) … test each for success … retest for their interaction with each other (ie. that they don’t interfere with each other).

The title of this tutorial mentions Linux but generic thinking like above covers other operating system thought patterns, but there are some Linux (or Unix) tools that are great Linux commands that we should point out … thanks to Hscripts.com and CyberCiti.biz for this …

1. crontab ( eg. */5 * * * * Username /path/to/command # where /path/to/command gets run by Username every five minutes … arranged via crobtab -e ) … for Windows, equivalent would be Task Schedular in Windows Primer Tutorial
2. nohup
3. bg
4. nice

And here are some of the practicalities of a watchdog …

1. Where does it run? If at more than one place, consider each place separately. If it runs on more than one computer, then clearly this is important. Does the directory and file permissions allow the watchdog to run, but do not allow other users to misuse it? (Please say yes here.) A generic thing about crontab or nohup (or Windows start) scripting arrangements are that you should not assume the environments of these processes is the same as your current running command line process … you should write as if you have just logged on and have done nothing … so just about the first decision of the script is to “cd” itself to the proper place where it was designed to run.
2. Which user(s) (on whichever computer(s)) can run it?
3. When does it run? If the solution is uncomplicated enough, maybe you can use a pre-emptive approach. For example, you know the problem is to do with a file missing when a non-critical process fails but later on that file is looked for, then why not pre-emptively get the watchdog to create that file (with default data) ahead of the process run crash point time.
4. As part of the question above, does one subprocess need to end before another starts? If yes, you need to intervene in such a way that that process architecture remains, and you need to work out an independent way for your independent watchdog to step in, at the correct time, and take over the same task, as required. But if it gets to this, don’t you understand the underlying process well enough to have a crack at doing the “real” solution (for all time)? Have a think, now, and keep checking in on the issue?
5. You need to log the workings of the watchdog both for information and for further research which might help in achieving a “real” solution (without the watchdog) further down the track.

So why was this posting called a tutorial? Well, there’s some homework. You see there are these Daleks, and we sort of need to know when they’re going to invade Earth again, and Dr Who is not always available, so, was wondering, if it wouldn’t be too inconvenient … if you wouldn’t mind writing that watchdog to detect a Dalek invasion and shoo them off … 6 hours … okay?

“

Woofsky!

“

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.