WordPress Upgrade Crontab Curl Tutorial

WordPress Upgrade Crontab Curl Tutorial

Yesterday‘s work with the WordPress upgrade from 3.0.3 to 4.1.1 … and “letting go” (of 3.0.3) thoughts mentioned …

… but not yet moving to any generic mappings via the WordPress 3.0.3’s index.php nor any .htaccess / httpd.config arrangements for this, just yet, as we have an issue with the blog posts during this interim period where the “?p=[postId]” doesn’t always match, and we have to plan what to do about this, and not be too kludgy … for the future.

… we’ve, today, been working on a “not too” kludgy approach, and there seem to be a lot of good aspects to the solution, especially being able to again deploy good ol’ Crontab and Curl … and they drove the fastest milk cart, in the West … so this is a small time scan after daily publish period of perusal to check for and fix the mismatched IDs method. The method, though, may stuff up some category and tag arrangements with the WordPress 3.0.3 blog postings during this interim period.

The solution today involved the concept of doing a MySql database SQL query that joins two tables from different databases …


SELECT t1.ID, t2.ID, t1.post_title, t2.post_title, substr(t1.guid, instr(t1.guid, '/ITblog/')) as t1g, substr(t2.guid, instr(t2.guid, '/wordpress/')) as t2g
FROM itblog.wps_posts t1
JOIN rjmprogr_wordpress.wp_posts t2
ON t2.post_title = t1.post_title and t2.post_status in ('publish','future') and t1.post_status in ('publish','future')
WHERE t2.ID != t1.ID


… and want to thank this useful link for advice here.

The way to approach this solution takes a well trodden path … nothing much unusual here … the wonderful, the marvellous, the refreshingly curious and the curiously refreshing phpMyAdmin is used to “unit test” the heart of the solution … ie. “finding the candidates” … thinking on it the idea could be summarized with a lot of database work …

1. query the database with “SELECT” to “find the candidates”
2. for each candidate, “UPDATE” and/or “DELETE” and/or “INSERT” to database to act on the problem and fix

… and phpMyAdmin is greatly useful to “unit test” step 1 above, in particular.

Time for the PHP code (that we Curl) now, that you could call old_new_wp_mismatch_id.php, which uses that unit tested SQL query, then does “UPDATE” to change existing records of, in order of preference …

1. change the WordPress 3.0.3 ID and guid to suit the WordPress 4.1.1 ID
2. in case of ID duplicate clashes change both WordPress 3.0.3 ID (+ guid) and WordPress 4.1.1 ID (+ guid) to a new “biggest in both” value

So now that the code is developed, it goes through a “unit testing” phase with Curl but no “Crontab”.

Then finally “Crontab” batch processing is introduced, and this needs less testing, unless you want to simulate a set time run of it. The bold bit of the SQL …


SELECT t1.ID, t2.ID, t1.post_title, t2.post_title, substr(t1.guid, instr(t1.guid, '/ITblog/')) as t1g, substr(t2.guid, instr(t2.guid, '/wordpress/')) as t2g
FROM itblog.wps_posts t1
JOIN rjmprogr_wordpress.wp_posts t2
ON t2.post_title = t1.post_title and t2.post_status in ('publish','future') and t1.post_status in ('publish','future')
WHERE t2.ID != t1.ID


… allows for the idea that an interactive “non-Crontab” call of old_new_wp_mismatch_id.php that would be meaningful just after creating the linking blog posts in both WordPress 3.0.3 and WordPress 4.1.1, but we will proceed with the Crontab arrangement for surety of the process. After all, a ham sandwich can reduce concentration levels … case in point … we rest our cases.

This also meant that the time to “do the Crontab” can be tied down better than “a small time scan after daily publish period of perusal to check for and fix the mismatched IDs” as we discussed above, to result in a Crontab record like …


17 1 * * * curl http://www.rjmprogramming.com.au/PHP/old_new_wp_mismatch_id.php


… so that it happens once a day before the one two “Scheduled” blog post posts (which is how we normally arrange things) turns into a “Published” one ones.

Anyway, hope you get an idea or more from today’s tutorial.

Previous relevant WordPress Upgrade Links Tutorial is shown below.

WordPress Upgrade Links Tutorial

Here is today’s “moving forward” moment regarding our WordPress upgrade from version 3.0.3 to 4.1.1. You may be wondering how relevant this is to you, and it may not be immediately applicably relevant, but you may find the approaches of generic interest when you upgrade any website and are in that in between period of settling the new website in, but not ready to let the old one go.

Another consideration regarding “let the old one go” is, if you do move on, will the dead links on the search engines pointing at your old version mean your domain is penalized because of these dead links. Perhaps an .htaccess / httpd.config arrangement left behind will fix this, but am not up to that yet. Experts are welcome to leave comments. This, I am not sure about, but within your own blog broken links can be checked by a WordPress plugin you can read about at Broken Link Checker.

Okay, so we’re not “throwing out the baby with the bath water” and so how can we chip away at getting users to use the new blog website? How about …

1. the Recent Posts widget links
2. the blog Entry Title (class=’entry-title’) light blue links on the blog

… but not yet moving to any generic mappings via the WordPress 3.0.3’s index.php nor any .htaccess / httpd.config arrangements for this, just yet, as we have an issue with the blog posts during this interim period where the “?p=[postId]” doesn’t always match, and we have to plan what to do about this, and not be too kludgy … for the future.

So what are the code changes for steps 1 and 2 above … both, further below, need new wajax.js (and same for nothing.js) function …


function newblogmap(tworpinnerHTML, kj) { // ?p=[postId] wordpress old to /%postname%/ permalink ITblog new ... April, 2015 ... RJM
var iqwa, newidea, nideatwo, aih, aihtwo, aword, awords, niword, retval = tworpinnerHTML;
if (tworpinnerHTML.indexOf("?p=") != -1) {
newidea = tworpinnerHTML.split("");
for (iqwa=0; iqwa<(newidea.length - 1); iqwa++) {
if (iqwa == kj) {
aih = newidea[iqwa].split(">");
awords = aih[(aih.length - 1)].split(" ...");
//if (awords[0].indexOf("Walking Trip") != -1 && document.URL.indexOf("?x=x") != -1) alert("*" + awords[0] + "*");
niword = awords[0].toLowerCase().replace(/\//g, "").replace(/,/g, "");
//if (awords[0].indexOf("Walking Trip") != -1 && document.URL.indexOf("?x=x") != -1) alert("!" + niword + "*");
niword = niword.trim();
//if (awords[0].indexOf("Walking Trip") != -1 && document.URL.indexOf("?x=x") != -1) alert("$" + niword + "*");
niword = "/ITblog/" + niword.replace(/ /g, "-") + "/";
if (awords[0].indexOf("Walking Trip") != -1) niword = "/ITblog/walking-trip/";
nideatwo = newidea[iqwa].split("?p=");
if (nideatwo.length > 1) {
aih = nideatwo[1].split('"');
aihtwo = nideatwo[1].split("'");
if (aih[0].length < aihtwo[0].length) { aihtwo = aih[0].split("&"); aword = "/wordpress/?p=" + aihtwo[0]; } else { aih = aihtwo[0].split("&"); aword = "/wordpress/?p=" + aih[0]; } retval = retval.replace(aword + "&", niword + "?").replace(aword + "&", niword + "?").replace(aword, niword).replace(aword, niword); } } } } return retval; }

1. the Recent Posts widget links ... and we talked about this with WordPress Recent Post Image Follow Up Tutorial and/or WordPress Recent Post Image Follow Up Tutorial ... bold bits below ... affects wajax.js (and same for nothing.js) ...
   
...
function winit() {
...
for (var j=0; j < allPs.length; j++) { if (allPs[j].id == "text-4") { ... } else if (allPs[j].id == "text-5") { ... } else if (allPs[j].id == "recent-posts-2") { // Recent blog post Ajax previews 27/11/2014 var ihs = allPs[j].innerHTML.split("?p="); var myposts; var aeight=new Array("aone", "atwo", "athree", "afour", "afive", "asix", "aseven", "aeight"); var ieight = aeight.length - 1; for (var jhs=(ihs.length - 1); jhs>=1; jhs--) {
myposts = ihs[jhs].split('"');
if ((wisiPad || wisTouch)) {
if (allPs[j].innerHTML.indexOf('a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" t') != -1) {
allPs[j].innerHTML = allPs[j].innerHTML.replace('a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" title="', 'a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" style="margin-bottom:20px;" ontouchend=" yehbut(); " id="' + aeight[ieight] + '" ontouchstart=" bpost=' + myposts[0] + '; if (mtimer) { clearInterval(mtimer); } tickcnt = 0; mtimer = setInterval(mchecker, 1000); " title="You can wait for the long hover functionality for a Blog Post preview ... ');
} else {
allPs[j].innerHTML = allPs[j].innerHTML.replace('" href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '"', ' ... you can wait for the long hover functionality for a Blog Post preview" href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" style="margin-bottom:20px;" ontouchend=" yehbut(); " id="' + aeight[ieight] + '" ontouchstart=" bpost=' + myposts[0] + '; if (mtimer) { clearInterval(mtimer); } tickcnt = 0; mtimer = setInterval(mchecker, 1000); " ');
}
if (1 == 2) allPs[j].innerHTML = newblogmap(allPs[j].innerHTML, eval(jhs - 1)); // not yet for mobile - map to new blog URL with permalink /%postname%/
} else {
if (allPs[j].innerHTML.indexOf('a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" t') != -1) {
allPs[j].innerHTML = allPs[j].innerHTML.replace('a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" title="', 'a href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" onmouseout=" yehbut(); " id="' + aeight[ieight] + '" style="margin-bottom:20px;" onmouseover=" bpost=' + myposts[0] + '; setTimeout(xget, 4000); " title="You can wait for the long hover functionality for a Blog Post preview ... ');
} else {
allPs[j].innerHTML = allPs[j].innerHTML.replace('" href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '"', ' ... you can wait for the long hover functionality for a Blog Post preview" href="http://www.rjmprogramming.com.au/wordpress/?p=' + myposts[0] + '" onmouseout=" yehbut(); " id="' + aeight[ieight] + '" style="margin-bottom:20px;" onmouseover=" bpost=' + myposts[0] + '; setTimeout(xget, 4000); " ');
}
allPs[j].innerHTML = newblogmap(allPs[j].innerHTML, eval(jhs - 1)); // map to new blog URL with permalink /%postname%/
}
ieight = ieight - 1;
}
}
...
}
...

2. the blog Entry Title (class='entry-title') light blue links on the blog ... bold bits below ... affects wajax.js (and same for nothing.js) ...
   
...
function winit() {
...
for (var j=0; j < allPs.length; j++) { hdivid = allPs[j].id; if (typeof hdivid == "undefined") { hdivid = ""; } else { if (hdivid.indexOf("post-") != -1) { hdivids = (hdivid.substring(hdivid.indexOf("post-")) + " ").split(" "); hdivid = hdivids[0]; hdivids = allPs[j].innerHTML.split(' class="entry-title">');
if (hdivids.length > 1) {
clname = ("~" + hdivids[1]).replace('~<',' ').replace('~',''); hdivids = clname.split("<"); if (hdivid != "" && hdivids[0] != "") { if (hdivids[0].indexOf(">") != -1) {
clname = hdivids[0];
hdivids = clname.split(">");
hdivids[0] = hdivids[hdivids.length - 1];
if ((wisiPad || wisTouch)) {
wisiPad = wisiPad;
} else {
allPs[j].innerHTML = newblogmap(allPs[j].innerHTML, 0); // map to new blog with permalink /%postname%/
}
}
if (hrsel.length == 0) {
hrsel = "<select onchange='location.href=this.value;' id='hrsel' style='display:NONE;'><OPTION value='#" + hdivid + "'>" + hdivids[0] + "</option></select><br>";
} else if (hrsel.indexOf(hdivids[0]) == -1) {
hrsel = hrsel.replace("</option></select>", "");
if (hdivids.length > 1) {
clname = ("~" + hdivids[1]).replace('~<',' ').replace('~',''); hdivids = clname.split("<"); if (hdivids[0].indexOf(">") != -1) {
clname = hdivids[0];
hdivids = clname.split(">");
hdivids[0] = hdivids[hdivids.length - 1];
if ((wisiPad || wisTouch)) {
wisiPad = wisiPad;
} else {
allPs[j].innerHTML = newblogmap(allPs[j].innerHTML, 0); // map to new blog with permalink /%postname%/
}
}
if ((hrsel + "*").indexOf("<option ") == -1 || hrsel.length == 0) {
if (hdivid != "" && hdivids[0] != "") {
if (hrsel.length == 0) {
hrsel = "<select onchange='location.href=this.value;' id='hrsel' style='display:NONE;'><option value='#" + hdivid + "'>" + hdivids[0] + "</option></select><br>";
} else if (hdivids[0].substring(0,1) >= "A" && hdivids[0].substring(0,1) <= "Z" && hrsel.indexOf(hdivids[0]) == -1) {
hrsel = hrsel.replace("<OPTION ", "<option value='#" + hdivid + "'>" + hdivids[0] + "</option><OPTION ");
} else if (hdivids[0].substring(0,1) >= "A" && hdivids[0].substring(0,1) <= "Z") {
hrsel = hrsel.replace("<OPTION ", "<option ");
}
}
} else {
if (hdivid != "" && hdivids[0] != "") {
if (hrsel.length == 0) {
hrsel = "<select onchange='location.href=this.value;' id='hrsel' style='display:NONE;'><option value='#" + hdivid + "'>" + hdivids[0] + "</option></select><br>";
} else if (hdivids[0].substring(0,1) >= "A" && hdivids[0].substring(0,1) <= "Z" && hrsel.indexOf(hdivids[0]) == -1) {
hrsel = hrsel.replace("</option></select>", "</option><option value='#" + hdivid + "'>" + hdivids[0] + "</option></select>");
}
}
}
}
}
}
...
}
...


Hope this also can help in some way for your work.

Previous relevant WordPress Upgrade Implications Tutorial is shown below.

WordPress Upgrade Implications Tutorial

Yesterday we saw with WordPress Upgrade Follow Up Tutorial as shown below that WordPress has a great Export and Import plugin to help migrate data after you upgrade your WordPress version. By the way, it is worth noting with this, that two WordPress default Widgets are useful as sanity checks for how the data migration is going ...

1. Archives and/or
2. Calendar

... particularly if you are like us, and publish once a day.

Anyway, it is now time, once you are happy with data migration to turn to the implications of a happy data migration which is one where the default permalink "?p=[postId]" is equivalent for old and new websites, but the new website also accepts "/%postname%/" permalinks (in our case). This means that, with care, instead of "/wordpress/" being in users' URLs you can ease them into "/ITblog/" replacing that in their URLs. So why the trepidation, why the "with care" bit? Well, I'm not sure, to be honest, whether all this contravenes a fairly big search engine (SEO) sin of having duplicate data. I'm, of course, hoping that with the search engine view of the Blog Post titles having that extra (middle name) "James" in them, plus a different "/%postname%/" permalink URL associated with them (but unfortunately with the same content) whether this will be enough to potter along for a while with both WordPress versions, while all the teething issues are sorted out, or whether the data migration will cause grief at the search engines ... am an optimist here, but will keep you posted on that. You may wonder, why don't we just cut over ... well, maybe, one day, but am sure it is a thing to ease into, rather than a snap decision, especially as the snap decision does not improve anything about the "duplicate content" doubt up at the search engines.

So, back at the matter at hand ... considerations of what can improve once a good data migration is achieved. Here's one we can think of. If you are an attentive reader at this blog you may know about the PHP code of recent-posts-2.php which was first talked about with WordPress Recent Post Image Primer Tutorial and last changed (and talked about first regarding that) with FFmpeg Image Optimization Primer Tutorial. Well, we changed the code as per this link because now wherever a URL has "/wordpress/" it can now be safely mapped to "/ITblog/" and be hooking into the new WordPress blog website that way, as reflected by zero.html

So over coming days recognising these functionality improvements could be the go (like, do we do some .htaccess work or rather what we lean towards now, some mapping amendments at the WordPress 3.0.3 index.php end of things?), and it occurs to us to leave several days (to try things out (perhaps on a MAMP local web server, first)) for WordPress Blog upgrade work, to facilitate the changeover methodically. In the meantime here is today's functionality change.

Did you know?

With using permalink /%postname%/ it sets you, on a Linux web server, in particular, to think about URL uppercase/lowercase confusion. After all, it is natural that somebody will wonder, with a URL such as ...

https://www.rjmprogramming.com.au/ITblog/swift-dictionaries-and-arrays-primer-tutorial/

... whether ...

http://www.rjmprogramming.com.au/itblog/swift-dictionaries-and-arrays-primer-tutorial/

... or even ...

http://www.rjmprogramming.com.au/ITBLOG/swift-dictionaries-and-arrays-primer-tutorial/

... would also work, and with Apache / mod_rewrite / .htaccess / httpd.conf considerations ... yes, it is possible (as long as mod_rewrite is installed). Let's look at the web server alternative to ITblog folder called itblog (gobsmacked Windows (perhaps ASP.Net web server) afficianados cannot have this differentiation but also don't have the issue to worry about in the first place ... here's the HTML a element information from w3schools in lowercase and (bits of) uppercase (with its URL) ... anyone for a bit of I.T. history?) and what that new web server folder needs to make this happen to have this extra "convenience" ...

1. httpd.conf change (for us, here, because we only allow .htaccess functionality, on a folder by folder decision basis ... changes to httpd.conf need an Apache restart to happen, but a change to .htaccess should not need such a restart for the change to take effect)
   
<Directory [path to new itblog directory]>
# ... other directives...
AllowOverride All
</Directory>

2. .htaccess (in new itblog directory ... chmod 644 ... in large part, the same as WordPress installation's .htaccess file created in the ITblog directory)
   
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /itblog/
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /ITblog/index.php [L]
</IfModule>
# END WordPress
3. index.html (in new itblog directory)
   
<html> <body onload=" location.href = document.URL.replace('/itblog', '/ITblog'); "> </body> </html>

4. index.php (in new itblog directory)
   
<?php header("Location: ../ITblog/index.php?" . $_SERVER['QUERY_STRING']); ?>


... meaning that a misspelled URL with "/itblog" (which is quite understandable) should function the same as the "/ITblog" version.

Previous relevant WordPress Upgrade Follow Up Tutorial is shown below.

WordPress Upgrade Follow Up Tutorial

WordPress has a great Export and Import plugin to help migrate data after you upgrade your WordPress version, perhaps.

This came to the fore yesterday when we started (there is a lot more to do) on that migration path going from WordPress version 3.0.3 to version 4.1.1 as you can read a bit about with WordPress Upgrade Primer Tutorial as shown below.

So the two parts to the Export and Import equation are ...

1. Export from WordPress 3.0.3
2. Import into WordPress 4.1.1

... now you may naively think that you will be able to export once and import once, but various things may get in your way achieving that ...

1. the small settings for max_execution_time (eg. 30 should be temporarily 360) and max_input_time (eg. 60 should be temporarily 120) and memory_limit (eg. 70M should be temporarily 128M) in php.ini need to suit the memory requirements needed to create the Export file you upload to the live server
2. even fixing requirement for step 1 you may (or the symptom of needing to do something as advised in step 1), as you Export, get the PHP error message "Fatal error: Maximum execution time of ? seconds exceeded in ..." ... where ? was 30 for us here ... this error message will appear in the last line of the Export file so
   tail -2 wordpress.2014-04-01.2014-07-01.xml
   would have been a Linux programmatical way of checking for the scenario shown a bit below
3. the small settings for upload_max_filesize (eg. 2M should be temporarily 32M) and post_max_size (eg. 8M should be temporarily 32M) in php.ini need to suit the size of the Import file you upload to the live server
4. even fixing requirement for step 3 you may (or the symptom of needing to do something as advised in step 3), as you Import, get the PHP error message "Fatal Error: Allowed Memory Size of ? Bytes Exhausted" ... where ? is a fairly big number ... and this error message may show on your web page window

So it is probably better to do this job in chunks. We think if there is a blog posting each day a three month span (or perhaps one month span) of Export could be good if comments are included as well. And how to exclude the comments? Supposing your Start Date is April 2014 and End Date is June 2014 then, at Linux ...


$ cat wordpress.2014-04-01.2014-07-01.xml | sed '/^[ \t]*<wp:comment>/s//<wp:comment>/g' | sed '/^[ \t]*<\/wp:comment>/s//<\/wp:comment>/g' > xwordpress.2014-04-01.2014-07-01.xml
$ cat xwordpress.2014-04-01.2014-07-01.xml | awk '/<wp:comment>/,/<\/wp:comment>/' > comment_2014_4.xml
$ comm -23 xwordpress.2014-04-01.2014-07-01.xml comment_2014_4.xml > no_comment_2014_4.xml


... would result in a suitable no_comment_2014_4.xml to Import into the WordPress MySql database. We thank this link and this link and this link.

More WordPress upgrade information tomorrow. Thanks for reading.

WordPress Upgrade Primer Tutorial

It's early days, and there is more to be done, but here at this WordPress blog we're transitioning to version 4.1.1 from 3.0.3

Three major decisions involved ...

1. Do I keep the same theme?
   • This one depends a bit, and we feel the answer stems from how you go piecing together on a local MAMP web server, for the MacBook Pro example, and how long you think it would take to transition in a change of theme. With this upgrade we tinkered with themes Twenty Fourteen, Twenty Thirteen and Twenty Fifteen with degrading success, and plumped instead to reinstall our previous theme Twenty Ten. Think it was a good decision, as if you are sticking with WordPress default themes, Twenty Ten is still a good theme in my opinion. Of course, you may have your own theme you like in mind and then you should invest the hours it will take to transition all the styling changes inherent in a shift of theme.
2. Do I change the permalink arrangement?
• In our previous WordPress blog we were using the default "?p=[postId]" arrangement which the search engines do not like ... and you can understand why, in that to glean the best categorization of a webpage's content gist, the search engines have to parse the whole page, whereas a permalink arrangement such as ...
  
/%postname%/

  teamed up with a meaningful title, can tell a search engine so much about the webpage content just from its URL. So we went ahead and tried for the best option we'd read about, as above, and it all worked well at MAMP but getting to the live server we needed the great advice of this link from a WordPress forum, and it panned out for us that the WordPress install was doing everything right as far as its ".htaccess" file was concerned, but in our "httpd.conf" Apache configuration file we needed to specify ...
  
<Directory [path to new WordPress document root directory]>
# ... other directives...
AllowOverride All
</Directory>

  to make things better
3. Do I filter comments more?
   • Definitely that would be good, and the great choice is between the Akismet plugin and Captcha option, or both, and as we suspected, we think more people like Akismet here, and I've always liked it more too, because Captcha excludes some users from ever accessing your blog. Akismet was already installed with WordPress, and activation for a personal blog like this one just involves getting an API_KEY from Akismet ... so thanks a lot.

As you may imagine, this is an ongoing story, with more twists and turns to come. Hope you'll join us for more information into the future, and hope to see you back at the new blog soon.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.