Linux CentOS Web Server Domlogs Diskspace Automating Solution Tutorial

It’s one thing to find a neat “operating” one off command line command solution to a web server issue, but it could be that you do not want to live with having to worry about this part of the web server maintenance, constantly and vigilantly. On a Linux web server like ours, most of this …

Automation of a Procedure

… we leave to crontab. We mainly team …

• crontab -l # lists crontab jobs
• crontab -e # sometimes, but no need this time, edits the crontab jobs in place … which we like to encase in a …
• ksh -c “[command]” # Korn Shell framework, often … but today we add to our “every minute Korn Shell based set of checks” via
  * * * * * /etc/init.d/cron_is_working.ksh
  # …
• vi /etc/init.d/cron_is_working.ksh # an every minute set of checks suits, today, because our solution is one that applies whenever you might want to execute it

Those additional ksh -c “[command]” x 3 commands …


ksh -c "find /usr/local/apache/domlogs -name 'rjmprogramming.com.au' -size +784297745c -exec cat /dev/null > /usr/local/apache/domlogs/rjmprogramming.com.au \;"
ksh -c "find /usr/local/apache/domlogs -name 'rjmprogramming.com.au-bytes_log' -size +25332007c -exec cat /dev/null > /usr/local/apache/domlogs/rjmprogramming.com.au-bytes_log \;"
ksh -c "find /usr/local/apache/domlogs -name 'rjmprogramming.com.au-ssl_log' -size +709422957c -exec cat /dev/null > /usr/local/apache/domlogs/rjmprogramming.com.au-ssl_log \;"


… we had great help, regarding their construction, from https://www.unix.com/shell-programming-and-scripting/139001-deleting-files-based-their-size.html towards adding into crontab via “the every minute” /etc/init.d/cron_is_working.ksh we were careful to check out back with individual command checking. In other words …

Make no assumptions when it comes to operating procedures on a web server, especially regarding where they are run and whether you have permission to perform them, and during testing use benign replacement Linux commands like “ls” during this unit testing, until absolutely sure of the procedural integrity.

And so, onto yesterday’s Linux CentOS Web Server Domlogs Diskspace Issue Tutorial useful research and development we can augment that with “sleep at night” easing of some of the diskspace issues on this RJM Programming web server.

But logs are often there for a reason and you should check out that you can debug issues, living with this “occasionally clobbering of a log” procedure idea for a diskspace solution.

---

Previous relevant Linux CentOS Web Server Domlogs Diskspace Issue Tutorial is shown below.

Running a Linux CentOS Apache/PHP/MySql web server has its ups and downs, the most visited issues for us involving diskspace in two ways …

• diskspace itself ( detectable via df -k / )
• diskspace related inode count limit ( detectable via df -i / )

And so, as another chapter in our “diskspace management” saga, in a similar vein to MySql Repair Table Diskspace Error Log Consequence Tutorial was the discovery yesterday, that dk -k / remained at 97% (anything around 95% and above is not good) well past an Apache Web Server service and MySql Web Server service restart (the time before this getting us back 3% of diskspace regarding logs) got us looking further afield. We discovered huge logs in what we call the …

Domlogs area

These logs were gigabytes big, and got us wondering …

“can i delete logs in /usr/local/apache/domlogs”

… and we were glad to get the tip that we could do clearing, as long as we did not want information from out of these extensive logs (which you should look into yourself) and that the mechanism should be a …

• (what we like to call a) “clobber” … rather than …
• “deletion”

… in that a “clobber” is like “clear all the records in a file but keep its name and permissions always intact” ( like, in SQL delete * from mytable ) whereas a “deletion” on Linux often involves the “rm” command where the file itself disappears ( like, in SQL drop table mytable ). An example of a command (which got us going “aaaaahhh”) was …


cat /dev/null > /usr/local/apache/domlogs/rjmprogramming.com.au


… to get diskspace back and not affect the usual web server and RJM Programming domain website operations.

---

Previous relevant MySql Repair Table Diskspace Error Log Consequence Tutorial is shown below.

If you are managing a web server, you can run into times when …

• a total solution to a problem needs more research that will take time … but, perhaps …
• there is a solution that “eases the pain” of web server management and operations, in our case, “diskspace” issues, awaiting that total solution

So it was here at this RJM Programming CentOS WHM Linux dedicated Apache/PHP/MySql web server, recently, as it ran out of diskspace, and we researched the issue, again, via the “old chestnuts” (at least for us) …

• ssh session at command line …
• 
df -k /

  … for overall disk diskspace reading … and, sidling up to a “base folder” of interest, via “cd” …
• 
find . -type f -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}


… landed us on another file that appeared, but was not frontmost in our thinking, on that other day when we presented MySql Repair Table Failure Diskspace Consequence Tutorial to outline a MySql Table Failure Consequence to diskspace.

Well, this “MySql Table Failure Consequence” has a second part to its story regarding diskspace. Yes, in order to try for a “total solution” we attempt this phpMyAdmin …


REPAIR TABLE wps_comments;


… SQL but it fails. And though the WordPress Blog is functional, still, just because you go from Defcon 5 to Defcon 1 ³/₄ in your “Systems Operator” thinking, then, doesn’t mean there aren’t other issues here! Oh, no. no, no, no … yes!

That MySql table repair requirement is logged, but where? Have a read of the excellent The MySQL Log Files – MySQL Reference Manual [Book], especially its section entitled “The Error Log”, as well as those …


FLUSH LOGS;


… phpMyAdmin SQL that might come in handy!

Yes, this “vs-rmetcalfe.au.syrahost.com.err”, for us here at RJM Programming web server, looked familiar?! Yes, it appeared in the list of big files during the research and development mentioned in MySql Repair Table Failure Diskspace Consequence Tutorial “Part 1” below, but was not our focus that day.

So, are we allowed to clear that log periodically, because we examined it, and it contained the same old records throughout, just with a different timestamp? We did an experiment. First, at the ssh command line, we went …


mv vs-rmetcalfe.au.syrahost.com.err vs-rmetcalfe.au.syrahost.com.err-bad


… on the understanding that we could rename, back, should we cause MySql problems. And, before the punchline, “mv” retains the “chown” and “chgrp” and “chmod” file permissions, happily, here. Well, nothing bad happened. Oddly, no new “vs-rmetcalfe.au.syrahost.com.err” was created, and that gave us an uneasy feeling, so we put things back to the way they were (renaming backwards) and saw the proper logfile grow again (sort of sadly and happily at the same time), but along the way we saw this “way backdated” and tiny “vs-rmetcalfe.per.syra.net.au.err” with appropriate file permissions from the old days sitting there, and so, tried …


cp vs-rmetcalfe.au.syrahost.com.err vs-rmetcalfe.au.syrahost.com.err-bad
cp vs-rmetcalfe.per.syra.net.au.err vs-rmetcalfe.au.syrahost.com.err


… which resulted in …

1. a “hived off” okay record of a set period of time in that “vs-rmetcalfe.au.syrahost.com.err-bad” file, left, existing, for future perusal, as required … as well as …
2. a “non disrupted” scenario for MySql error logging, starting with that tiny 1 byte “vs-rmetcalfe.per.syra.net.au.err” start
3. all file permissions and ownership as they should be … as well as, if you can imagine …
4. do this at a particular time, once a day … cccrrrooonnntttaaabbb! … you get …
   • a day’s worth of MySql logging available for perusal (during office hours)
   • maximum two day’s worth of logging diskspace used (at its worst just before the commands above, with added “-f” force switches are applied …
     
54 4 * * * ksh -c "cp -f /var/lib/mysql/vs-rmetcalfe.au.syrahost.com.err /var/lib/mysql/vs-rmetcalfe.au.syrahost.com.err-bad ; cp -f /var/lib/mysql/vs-rmetcalfe.per.syra.net.au.err /var/lib/mysql/vs-rmetcalfe.au.syrahost.com.err"

     ) … with the new “crontab” scheduling as above … as distinct from “? day’s worth, depending on web server uptime, which has been known to be as high as 90 days” diskspace being eaten up and eventually causing us web server failures all over the place

Ahhhhhhh! What a relief!

---

Previous relevant MySql Repair Table Failure Diskspace Consequence Tutorial is shown below.

The previous MySql Repair Table Primer Tutorial outlined some phpMyAdmin administering of MySql (database) maintenance via SQL like …


REPAIR TABLE wps_comments;


… but what about if the SQL above fails or times out? Well, there can be implications, as we discovered via this useful link, thanks, that told us …

The . TMD file is an intermediate data file for a table that needs to recreate its data file. So you can remove it because it’s normally used as a temporarly file but you could rename the file and check what happens just in case.

This advice prompted us to lookup a previous “diskspace concern” blog posting we’d written called Troubleshooting CentOS Web Server Logfile Diskspace Issue Tutorial, its “take home”, always, for us, being that Linux statement …


find . -type f -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}


… gleaned via disk space – Linux utility for finding the largest files/directories – Super User, that has been so helpful for us over the years.

You’ll see this command starring again with today’s animated GIF presentation, but you might wonder about where “diskspace” became our “theme of concern”? Well, while the “REPAIR TABLE wps_comments” SQL was running we happened to have an RJM Programming ssh session going, monitoring …


df -k /


… (for our Linux CentOS Apache/PHP/MySql dedicated web server) at regular intervals, and it started at 97% (used), but ballooned out to 99% (used), alarmingly, in that short, maybe, two minutes of monitoring! So, again, thanks all you helpers and advisors out there on the net, helping us arriving to the position where we got 2% of diskspace back by removing that MySql TMD file caused by a failure with a “REPAIR TABLE” MySql SQL command!

---

Previous relevant MySql Repair Table Primer Tutorial is shown below.

Today’s tutorial continues the MySql (command line) discussion last heard about with MySql Stored Procedures Primer Tutorial as shown below, to illustrate that MySql has functionality to repair database tables, if they run into trouble.

When you troubleshoot a problem regarding an Apache/PHP/MySql website you can tend to think, firstly, when a bit of functionality is not working (when it has been working fine for a while now) that the problem is with the software code being unable to cater with some unusual situation.

The Apache log may get you closer to discovering the problem, but you may get to know some troubleshooting shortcuts, to rule out the database, or not, by checking the relevant database table using MySql from the command line, or the brilliant phpMyAdmin, as a GUI interface to the database.

Not always, but sometimes, you can be surprised to find that MySql database tables do have problems from time to time, and that MySql marks these tables as “crashed”, as you can see in today’s MySql Repair Table Primer Tutorial.

The functionality shown in the tutorial regarding phpMyAdmin MySql statement REPAIR TABLE [tableName] and the two MySql command line mysqlcheck or, if MyISAM format, myisamchk choices are not guaranteed to fix the problem all the time, and sometimes it can be to do with the size of the table involved, or with the size of the temp area set aside on your server. When a REPAIR TABLE operation runs too long, in phpMyAdmin the table will be still marked as crashed and you may see the string “(in use)” shown in the Collation column of the table screen. In such cases you can:

• continue waiting for repair process to finish … or …
• use a restore operation to restore the table (that you have backed up recently) … or … (the next two options lose data, and so should only be done when you know the table is independent of others) …
• truncate the table with the phpMyAdmin “empty” icon (this may not work for a while, but becomes an option as time goes on) … or …
• drop the table (then recreate it) with the phpMyAdmin “drop” icon (this may not work for a while, but becomes an option as time goes on)

---

Previous relevant MySql on Command Line Primer Tutorial is shown below.

Today’s tutorial follows up on MySql Stored Procedures Primer Tutorial as shown below, to illustrate that MySql database (and PHP server-side language) are friends of the command line, and you can use all these tools in batch processing, even though you most often associate PHP and MySql with interactive web use (or maybe you don’t?!).

This blog talks a fair bit about batch processing and command line work, not so much from the point of view of nostalgia, but more from the point of view that lots of computer software functionality can still be modelled on a procedural set of tasks which can be planned so that interactivity stops at a certain point, and then you should let loose the power of the computer to actually do things (the joy of batch processing). Do enough batch processing, and you feel what you feel like when you have trodden softly on this earth.

Anyway, enough of the rave, but with the tutorial we interactively, on our local webserver’s MAMP environment, create a new database called testdb, use the previous tutorial’s code to create some MySql tables and stored procedures in that database, so that we have one record in each of our two tables that we show you via phpMyAdmin (in an interactive way) and via a Mac OS X laptop’s Terminal application’s Linux bash environment command line (via the mysql program in a Linux command line environment).

This example is simple, but you build off the simple, and imagine the big, and imagine the procedural, and imagine the compartmentalization of tasks … if you want to have fun that is … the bungee jumpers are all just faking it!

So today, as well as with the previously relevant tutorial we used (MySql) stored procedures, and we’ll let Wikipedia explain this important database tool below.

A stored procedure is a subroutine available to applications that access a relational database system. A stored procedure (sometimes called a proc, sproc, StoPro, StoredProc, sp or SP) is actually stored in the database data dictionary.

Typical use for stored procedures include data validation (integrated into the database) or access control mechanisms. Furthermore, stored procedures can consolidate and centralize logic that was originally implemented in applications. Extensive or complex processing that requires execution of several SQL statements is moved into stored procedures, and all applications call the procedures. One can use nested stored procedures by executing one stored procedure from within another.

Stored procedures are similar to user-defined functions (UDFs). The major difference is that UDFs can be used like any other expression within SQL statements, whereas stored procedures must be invoked using the CALL statement.[1]

Thanks to Wikipedia for the quote above, and would recommend that if you want to get the most out of what is possible with databases, like MySql you also study triggers, and for that will leave you with a Wikipedia entry about this topic. Happy databasing!

---

Previous relevant MySql Stored Procedures Primer Tutorial is shown below.

Today’s tutorial follows up on phpMyAdmin interface to MySql and PHP Primer Tutorial in that we again use the brilliant phpMyAdmin to oversee the results of some PHP code which uses MySql calls to create tables called POINT and CIRCLE used to store information defining a circle, and then it creates three MySql stored procedures in the database to help add circle data with the single MySql statement that goes CALL AddCircle(x, y, radius); via the use of those stored procedures. Let’s see below what Wikipedia says about Stored Procedures generally.

A stored procedure is a subroutine available to applications that access a relational database system. A stored procedure (sometimes called a proc, sproc, StoPro, StoredProc, sp or SP) is actually stored in the database data dictionary.

Typical use for stored procedures include data validation (integrated into the database) or access control mechanisms. Furthermore, stored procedures can consolidate and centralize logic that was originally implemented in applications. Extensive or complex processing that requires execution of several SQL statements is moved into stored procedures, and all applications call the procedures. One can use nested stored procedures by executing one stored procedure from within another.

Stored procedures are similar to user-defined functions (UDFs). The major difference is that UDFs can be used like any other expression within SQL statements, whereas stored procedures must be invoked using the CALL statement.[1]

Here is some downloadable PHP programming source code which shows the results of the MySql SQL requests made and can be renamed to ourmysqlstoredprocedure.php as required.

Here is some downloadable supervisory PHP programming source code which gathers the MySql SQL requests made and can be renamed to ourmysql_storedprocedure.php as required.

If you want to develop your own live usage (have provided a live usage link here which will not work but you can use if you are a beginner, to get used to mysql errors, which will occur after you hit either button of the link (these errors indicate bad MySql connection details which are the same reason the database dropdown is not full of options), or you can use this link to see the raw MySql SQL involved in piecing this tutorial together) of these two PHP source codes then you could fix up the hard codings for MySql host/username/password up the top of ourmysql_storeprocedure.php (where you may notice that the default host in the code is localhost:8889 which is the default host string for MySql (ie. port 8889 is used) when using a MAMP (Mac laptop) local web server (which uses localhost:8888 as its local “domain name” for http usage)) or you can keep the same code and use a URL like:

[your-domain-name-plus-a-bit-maybe]/ourmysql_storeprocedure.php?host=[your MySql host address]&username=[your MySql username]&password=[your MySql password]&database=[your optional MySql default database name within the looked up list presented]

---

Previous phpMyAdmin interface to MySql and PHP Primer Tutorial is relevant and shown below.

Transcript:

You never hear much about the data when you hear about great PHP products, but we
all know it is the data that differentiates the quality of the end result.

That is probably because database products like MySql, SqlServer, Oracle SQL, Advantage and Access
are pretty good at what they do, and emphasise reliability rather than flashiness.

MySql and PHP have a great open source interface with phpMyAdmin, which is so good,
you forget that it is not the default MySql administrator’s interface product.

Let’s have a look at this WordPress database and a bit of how it looks, looking
through the prism of phpMyAdmin …

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.

---

If this was interesting you may be interested in this too.