Restoring cPanel backup to system without cPanel

cPanel is a web front end for “reseller” hosting accounts, and it’s very popular with web designers reselling hosting services. It’s very simple to use, and allows the web designers to set up virtual hosting accounts without giving them any real control over the server – self-service and fool proof. It’s also an expensive thing to license. It makes sense for a self-service low-cost hosting provider, where the customers do all the work, but for small-scale or “community” hosting providers you’re talking big money.

I’ve just had to rescue a number of web sites from a developer using one of these hosting services, and they’ve got a lot of sites. And the only access to the virtual server is through cPanel (and FTP to a home directory). I logged in to cPanel and there’s an option to create a backup of everything in one big tarball, and this looked like just what I wanted to get them all at once. However, it was designed to upload and unpack in another cPanel environment.

Getting out the home directories is pretty straightforward. They end up in a directory called “homedir”, and you just move it to where you want them – i.e. ~username/www/. But how about restoring the dump of the MySQL databases. Actually, that’s pretty simple too. They’re in a directory called “mysql”, but instead of it being one big dump, each is in it’s own file – and without the create commands, which are in another with the extension “.create” instead of “.sql”. Loading them all manually is going to be a time-wasting PITA, but I’ve worked out the the following shell script will do it for you if you run in while in the backup’s mysql directory:

for name in `find . -name “*.create”`; do
cat $name `echo $name | sed s/.create/.sql/` | mysql
done

You obviously have to be in the directory with the files (or edit find’s specification) and logged in as root (or add the root login as a parameter to the mysql utility).

You’ll also want to set the user/password combination on these. The tarball will have a file called mysql.sql in its root directory. Just feed it in thus:

mysql < mysql.sql

Please be aware that I figured this out looking at the files in the dump and NOT by reading any magic documentation. It works on the version of cPanel I encountered, and I was restoring to FreeBSD. By all means add a comment if you have a different experience when you try it, and don’t go this way if you’re not sure how to operate a MySQL database or you could do a lot of damage!

The final hurdle is configuring Apache for all these new sites. cPanel creates a directory in the dump called “userdata”, and this seems to contain a file with information about each web site. I decided to automate and wrote the following script:


#!/bin/sh

# Convert cPanel dump of "userdata" in to a series of Apache .conf files
# (c) F J Leonhardt 17 April 2014 - www.fjl.co.uk
# You may use this script for your own purposes, but must not distribute it without the copyright message above left intact

# Directory to write config files
# Normally /usr/local/etc/apache22/Include but you might want to write
# them somewhere else to check them first!

confdir=/usr/local/etc/apache22/Include

# oldhome and newhome are the old and new home directories (where the web sites are stored
# oldtestname and newtestname are used (together with a sub-domain) to implement test web sites before
# they have a real domain name pointed at them. They will be substituted in server names and aliases

oldhome=/data03/exampleuser/public_html
newhome=/home/exampleuser/www
oldtestname=exampleuser.oldisp.co.uk
newtestname=newuser.fjl.org.uk

# Now some static information to add to all virtual hosts
# vhost is the IP address or hostname you're using for virtual hosting (i.e. the actual name of the server)
# serveradmin is the email address of the server admin
# logfiles is the directory you want to put the log files in (assuming you're doing separate ones). If
# you do this you must uncomment the lines that write the .conf file

vhost=web.exampleuser.com
serveradmin=yourname@example.com
logdir=/var/log

getvalue()
{
grep ^$1: $name | sed s!$1:\ !! | sed s!$oldtestname!$newtestname!
}

# Start of main loop We DO NOT want to process a special file in the directory called "main" so
# a check is made.

for name in `ls`; do
if [ "$name" != "main" ]
then
echo -n "Processing $name "

if grep ^servername: $name >>/dev/null
then

# First we get some info from the file

sitename=`getvalue servername`
serveralias=`getvalue serveralias`
documentroot=`getvalue documentroot`

# Below we're setting the .conf pathname based on the first part of the file name (up to the first '.')
# This assumes that the file names are in the form websitename.isp.test.domain.com
#
# If the sitename in the source file is actually the name of the site (rather than a test alias) use
# this instead with something like:
#
# Basically, you want to end up with $givensitename as something meaningful when you see it
#
#givensitename=$sitename

givensitename=`echo $name | cut -d \. -f1`

confname=$confdir/$givensitename.conf

echo to $confname

echo "" >$confname
echo -e \\tServerAdmin $serveradmin >>$confname
echo -e \\tServerName $sitename >>$confname
for aname in $serveralias; do
echo -e \\tServerAlias $aname >>$confname
done
echo -e \\tDocumentRoot `echo $documentroot | sed s!$oldhome!$newhome!` >>$confname
echo -e \\tErrorLog $logdir/$givensitename-error.log >>$confname
echo -e \\tCustomLog $logdir/$givensitename-access.log combined >>$confname
echo "
" >>$confname

#from check that servername present
else
echo "- ignoring file - no servername therefore wrong format?"
fi

#fi from check it wasn't called "main"
fi
done

All of the above assumes you’re familiar with setting up virtual hosting on an Apache 2.2 http server in an UNIX-like environment. It’s just too complicated to explain that in a single blog post. Drop me a line if you need assistance.

Heartbleed bug not as widespread as thought

Having tested a few servers I’m involved with, many of which are using old or very old versions of OpenSSL, I can’t say I’ve found many with the problem. You can test a server here: http://filippo.io/Heartbleed/ on a site recommended by Bruce Schneier.

So what’s going on? Does this affect very specific nearly-new releases. This story could turn out to be a serious but solvable problem, and a media panic. I recall spending most of 1999 doing interviews on how the “year 2000 bug” was going to be a damp squib, but it’s early days yet.

Heartbleed bug

Someone’s finally found a serious bug in OpenSSL. It allows a remote attacker to snoop around in the processes memory, and this is seriously bad news because this is where you will find the private keys its using. They’re called “private keys” because, unlike public keys, they need to remain private.

This is going to affect most web sites using https, and secure email (if you’re using it – most aren’t). But before user’s rush off to change their passwords (which are different for each site, aren’t they?) – there’s no point in doing this if an attacker is watching. The popular press reckons your passwords are compromised; I don’t. If I understand it correctly, this exploit theoretically allows an attacker to intercept encrypted traffic by pretending to be someone else, and in doing so can read everything you send – including your password. So don’t log in until the server is fixed. They can’t read your password until you use it.

To cure this bug you need a new version of OpenSSL, which is going to be a complete PITA for server operators who aren’t on-site. Hell, it’ll be a PITA even if you are on-site with the servers. Once this is done you’ll also need new certificates, and the certificate authorities aren’t geared up for everyone in the world changing at once.

But the big fun one is when you can’t update OpenSSL. It’s used everywhere, including in embedded systems for which there was never any upgrade route. I’m talking routers, smart TVs – everythign.

I believe that SSH isn’t affected by this, which is one good thing, but I’m waiting for confirmation. Watch this space.

But, if you’re using a secure web site to log in over SSL, consider the password compromised if you’ve used it in the last few days and be prepared to change it soon.

Criminals using self-assessment tax filing deadline to drop Trojans

I’ve intercepted rather a lot of these:

From: <gateway.confirmation@gateway.gov.uk>
To: <**************>
Date: Mon, 3 Feb 2014 20:33:49 +0100
Subject: Your Online Submission for Reference 485/GB6977453 Could not process

The submission for reference 485/GB6977453 was successfully received and was not processed.

Check attached copy for more information.

This is an automatically generated email. Please do not reply as the email address is not monitored for received mail.

Someone (via France, and the sender certainly does not speak proper English) is taking advantage of people’s panic about getting self-assessment tax forms in before the 31st January deadline to avoid a fine The attached ZIP file contains an executable with a .scr extension. It doesn’t show as being anything recognisable as nasty, so someone’s planned this well. Be careful; this is slipping through ISP malware scanners (and all the Windoze desktop scanners I’ve checked it against).

 

FreeBSD 10.0 and ZFS

It’s finally here: FreeBSD 10.0 with ZFS. I’ve been pretty happy for many years with twin-drive systems protected using gmirror and UFS. It does what I want. If a disk fails it drops it out and sends me an email, but otherwise carries on. When I put a replacement blank disk it can re-build the mirror. If I take one disk out, put it into another machine and boot it, it’ll wake up happy. It’s robust!

So why mess around with ZFS, the system that puts your drives in to a pool and decides where things are stored, so you don’t have to worry your pretty little head about it? The snag is that the old ways are dying out, and sooner or later you’ll have no choice.

Unfortunately, the transition hasn’t been that smooth. First off you have to consider 2Tb+ drives and how you partition them. MBR partition tables have difficulties with the number of sectors, although AF drives with larger sectors can bodge around this. It can get messy though, as many systems expect 512b sectors, not 4k, so everything has to be AF-aware. In my experience, it’s not worth the hassle.

The snag with the new and limitless “GPT” scheme is that it keeps safe copies of the partition at the end of the disk, as well as the start. This tends to be where gmirror stores its meta-data too. You can’t mix gmirror and GPT. Although the code is hackable, I’ve got better things to do.

So the good new is that it does actually work as a replacement for gmirror. To test it I stuck two new 3Tb AF drives into a server and installed 10.0 using the new procedure, selecting the menu option zfs on root option and GPT partitioning. This is shown in the menu as “Experimental”, but seems to work. What you end up with, if you select two drives and say you want a zfs mirror, is just that.

Being the suspicious type, I pulled each of the drives in turn to see what had happened, and the system continues without a beat just like gmirror did. There were also a nice surprises when I stuck the drives back in and “onlined” them:

First-off the re-build was almost instant. Secondly, HP’s “non-hot-swap” drive bays work just fine for hot-swap under FreeBSD/ZFS. I’d always suspected this was a Windoze nonsense. All good news.

So why is the re-build so fast? It’s obvious when you consider what’s going on. The GEOM system works a block level. If the mirror is broken it has no way of telling which blocks are valid, so the only option is to copy them all. A major feature of ZFS, however, is that the directories and files have validation codes in the blocks above, going all the way to the root. Therefore, by starting at the root and chaining down, it’s easy to find the blocks containing changed data, and copy them. Nice! Getting rid of separate volume managers and file systems has its advantages.

So am I comfortable with ZFS? Not yet, but I’m a lot happier with it when its a complete, integrated solution. Previously I’d only been using on data drives in multi-drive configurations, as although it was possible to install root on ZFS, it was a real PITA.

Advertorial in Process Engineering Control & Maintenance

The relationship between journals and advertisers has always been tricky, with many of them forced to say nice things, or at least avoid saying anything bad concerning major advertisers. In my day as an editor I was free to say what I liked, as no advertiser could afford to stop advertising because it was the best route to reaching potential customers before the Internet.

Times have certainly changed, and today marks a new low. We’ve intercepted several spammed messages offering to sell editorial in Process Engineering Control and Maintenance. Normally I wouldn’t draw attention to this, but they were sent to a spamming list and picked up by no less than six honeypots – addresses than no legitimate sender of bulk mail should be using. Therefore they’re fair game.

Dear Public Relations Manager

I deal with the editorial content for the Process Engineering Control & Maintenance publication, and are just putting together our editorial feature pages within our February edition, this is a very special edition as this will not only be distributed to our exclusive 100,000 named circulation but an extra 5,000 copies will also be distributed at MAINTEC, Sustainability Live & National Electronics Week to the wide range of purchasing professionals that attend.

I wanted to contact you to see if you would be able to provide some editorial content for this special edition.

The only cost to include a press release within this special edition would be a small editorial set up fee of just £85…

…As I am only able to offer this editorial opportunity to the first few companies to respond to this offer, please email me the editorial content that you would like to include, and please confirm that you would be happy to pay the £85 set up fee.

Kind Regards

******* ******** CIE

[name and telephone number deleted]

If you’re one of the 105,000 people “lucky” enough to get a copy of the magazine, you have been warned.

 

 

 

Direct Response monitored alarms fail to show

Not to an alarm call out, but they had an appointment at 9am today to talk about their monitoring service. At 9:30 they called to say they weren’t coming with the excuse that they’d tried to call to confirm the appointment but couldn’t get through. Except they confirmed it yesterday afternoon and there’s someone on the hot-line number they claim to have used since 6am today.

Okay, they double booked slots and got caught with their pants down and this is the best they could come up with, but a company trying to sell an ARC service, not showing for an appointment has to be the biggest no-no going. LOL!

They’re actually possibly worth talking to, because they use the rather interesting Risco panels. Risco is an Israeli company, and they’re upping the game by integrating CCTV and IDS in one system with PIR detectors that will take a snapshot of what triggered them and sending to the ARC. The lady on the phone said they just wanted to demonstrate this, and I couldn’t resist even though we’re happy with the British-made Texecom kit (although we use Risco beam sensors already).

However, this is the same Direct Response that got hauled before the OFT and clobbered in 2009 for telling porky pies about their monitored alarms getting a priority response from the police. The caller also claimed the alarms were made in Iran (“or somewhere like that”). And they’re still using the same old sales tactics (“We are calling as part of an awareness campaign, and four people in your area will be selected at random for a free alarm worth £999”, without mentioning the £400 installation fee up front and claiming a £5/week monitoring fee – I’ll be pleasantly surprised if this bit is true).

The appointment’s been re-made for 9am on Monday. Let’s see. In fairness, I did warn the first and second callers that they hadn’t called a normal householder. All they gotta do is Google me.

Botnet shows itself with New Year spam :)

The crims have been at it again this Christmas season (more elsewhere). The latest interesting activity has been a flood of emails with :) as the subject and “Happy new year !” as the text-only payload. Don’t feel left out if you didn’t get one, as they’re only being sent to email addresses made of random numbers at various domains I monitor.

What are the crims up to? Probably testing out mail servers to see if they’ll accept things to random addresses. Every domain should, and deliver them to a human postmaster (not that many net newbies are even aware of this rule). However, there’s nothing to say they can’t also go to analysis tools.

What makes this latest caper interesting is that the botnet they’re coming from doesn’t show up on the usual lists of such things – it’s either new or extended rapidly from an old one. New botnets popping up after Christmas aren’t uncommon as the seasonal fake greeting cards and amazon purchase confirmation trojans are relentless in the days before, together with the lack of staff available over the holiday to deal with them. However, I find this one unusual as most of the IP addresses used to send out the probes are from Europe (Germany and Spain in particular).

 

Faulty screen on Lenovo S10-3 10″ laptop

My trusty and very portable S10-3 – one of the best laptops ever made in my opinion – died a couple of months ago. Well its screen went black. Or it went all-white, to be precise. And I mean black OR white; every pixel was either full-on or full off.

The rest of the machine appeared to be fine – it could be heard booting and it appeared on the network – you just couldn’t use the screen.

Today I fixed it. There was a loose connection where the LCD panel cable joins the motherboard. Unfortunately, it’s been lying in a pile waiting to go to Lenovo’s service centre in Germany all this time because I couldn’t figure out how to check for loose connections. Like most laptops, dissassembly isn’t obvious. Fortunately, like most Lenovo (nee IBM) laptops, it’s actually built with servicing in mind. So here’s the trick:

Remove the battery and undo all obvious screws on the back cover. There four captive screws on the cover plate, behind which lurk the winchester, DIMM and cellular modem (if fitted). Don’t be fooled; they don’t come out! You can remove the winchester if you wish, but watch out – it has two of its own retaining screws and two more screws that are part of the cover you’ve just removed. You could go mad looking for them if you miss this point.

There are then six black M2 screws to remove to the case, and three very small screws under where the battery fits. Remove them all, and it will look like nothing’s changed.

Next you have to lever the keyboard off. It’s actually very easy if you lever in the right place, which is along the top edge. It ‘snaps in’ at the corners; gentle levering with a small flat screwdriver and finger nails pops it off easily.

To disconnect the ribbon cable connector, pull the black bit of the PCB socket clip forward and up. (Good luck getting the cable back in, from underneath, and closing the clip again with adult sized-hands!) You can then put the keyboard aside, and undo three further black M2 screws, which are found at either edge and the centre of the silver plate thing you’re looking at. Then you need to prise the top of the case off – the sliver bit comes with it. Again, this is much easier than it sounds if you lever with a small screwdriver and get your fingernails underneath.

The planar (motherboard if you’re younger than a certain age), is now laid before you. The LCD cable is obvious at the top left; they’ve even labelled it. Although it looks like it’s taped down, it just pulls in and out; reseating it did the trick for me.

If you need to dismantle the screen/lid assembly (or if you’re curious, like me), you can detach the power cables that come in on the right hinge and undo a couple of screws at each side to remove it completely. To open it you need to remove the screws hidden under the self-adhesive rubber pads in the corners. Then you need to flex the screen frame quite dramatically, working around the edge, until it un-snaps (if you see what I mean). Let’s just say it’s easier to replace the lid as one unit if you’re breaking for spares.

Anyway, my little friend is back and I’m happy. It’s just a shame the manufactures are pandering to the craze for fondleslabs and had dropped the 10″ form factor for truly portable “proper” computers, able to run software other than games, Facebook and surfing the web. Now that ASUS has dropped the Eee book you’re looking at something like the ThinkPad E145, which I was about to buy in spite of its extra bulk, weight and cost.

Unfortunately, the S10-3 and closely related models in the field  are currently not replaceable until fashion swings back.