Your Smart TV is watching YOU

There were a couple of  interesting presentations at Black Hat yesterday Aaron Grattafiori and Josh Yavor from iSEC Partners and Seungjin Lee from Korea University were both talking about hijacking Smart TVs. These devices are Internet connected and basically do a lot of their stuff using web browser technology, including JavaScript and other well known attack vectors. iSEC Partners were testing Samsung TVs in particular, but they all work pretty much the same way and apparently the manufacturers’ programmers haven’t done much to consider the security aspects.

Grattafiori was particularly keen to point out that the cameras on such devices were as susceptible to hijacking as anything else.

He went on “Because the TV only has a single user, any type of compromise into an application or into Smart Hub, which is the operating system — the smarts of the TV — has the same permission as every user, which is, you can do everything and anything.”

He suggested you might want to  make sure the TV in your bedroom has it’s lens covered with a sticky label.

Earlier this year Samsung has issued a software update for the TVs affected by the security flaws described in Las Vegas, but the fact they’re all using flaky browser technology means we should all be wary of them.

 

Mark Shuttleworth’s Ubuntu Edge Dream

Mark Shuttleworth’s software company, Canonical Ltd, trying to raise $32M to build the first 40,000 units of a smart-phone type device that can run Ubuntu Linux. I predict he’s raise the money, and make the handsets. But the idea will tank anyway. Here’s why.

The concept of a ‘phone capable of running a desktop OS is easy to understand. When you want to use the desktop Ubuntu side you plug it in to a real monitor and keyboard – say one at home and one in the office. When you’re on the move it will run Android Linux (for Android is simply Linux with an Android graphical shell). You carry your environment with you, and carry on working wherever you are, assuming you have a monitor and keyboard available. If you run the Ubuntu graphical environment on the move, using the handset’s touch-screen it’s going to be pretty painful.

People investing about £600 will get a ‘phone, if they’re ever made. Is this an investment, or a pre-ordering deal? I think it’s up to you whether you invest enough to get a ‘phone, or buy even more equity as an investment in the future of the device, but I suspect a lot of people will simply be after the latest gadget. Whether £600 is too much for the Penguinistas, remains to be seen.

I think they stand a good chance of raising the money because they’re selling a dream that’s been around in various forms since the dawn of personal computing. One of the early incarnations would be the Apple IIc, which looked a bit like a portable typewriter when cut free from its monitor. With it you could carry your computer back from the office, but it didn’t catch on. Then, came the Tandon Data Pac, a hard disk cartridge. With a cartridge slot in PCs at the locations you needed to work, you could carry the important part of your environment with you. In those days, Microsoft didn’t do anything prevent hard disk transplants, so this was a realistic idea. But it didn’t catch on. Whether there are 40,000 people in the world who still have this dream is a good question.

Now we have laptop/notebook/netbook PCs, which are easy enough to carry in a briefcase if you get the right kind. I have always had the right kind, starting with the Cambridge Z88, moving on to the Sony Vaio and currently the Lenovo S10-3. At around 1Kg, they’re truly portable but although the Lenovo is modern, it was only on the market for a year or two as the 10″ screens format isn’t for well received by the masses. They demands big and fast, and they aren’t really worried about the battery life as long as they look cool. People often ask me “where can I get one of those”, and I tell them. (Currently only Asus and Acer producing a highly portable laptop/netbook). The snag is that when they get one they then “must” run Office 365, or some similar bloatware that a small CPU can’t handle fast.

If you don’t need battery life and the ability to work on the move, but simply want to carry your PC to and from the office, there are small form factor machines also from ASUS and Acer. If you want really small there’s the Fit-PC2 which can actually fit in a pocket. I must admit, I bought one because I thought it was a neat design. These are all Intel based and can run unmodified Windows, and yet they haven’t really caught on either. The Ubuntu Edge will not run Windows; it runs Linux. This means it won’t run Microsoft Office, ever. My experience has shown this is a big problem for a lot of people. There’s nothing wrong with OpenOffice; it’ll work with Microsoft Office files and vice versa. It’s free, whereas Microsoft Office costs and small fortune. Yet in nearly every case, people who I’ve set up with OpenOffice for cost reasons have hankered after the Microsoft version, and most have gone out and bought it (or otherwise acquired it) within a year.

The CPU for the Ubuntu Edge has yet to be announced, but based on size, battery life and heat dissipation it’s very unlikely to be Intel, or even Intel compatible. The only thing that will fit will be RISC, and given the binary nature of Linux distributions it’ll be the second-best choice of ARM. Or will its users be expected to compile everything from source? No. It’ll be an ARM and the models that are capable of running Linux with a GUI at nearly the right speed will still rip through the battery at an alarming rate.

The final nail in its coffin will be the way people currently commute with their computing environment. This comes down to cheap and cheerful thumb drive, if you can find a ubiquitous Windose PC at both ends, or on-line applications such as Google Docs if you’re really serious about it; all your data and applications on every web browser, and impossible to lose at that. If you can find a keyboard and monitor at both ends, you’re probably going to find a web browser anyway so why bother to carrying your stuff on a mobile ‘phone instead? It’s a solution to a problem that has been a “difficult sell” for 30 years, and which has now been solved by the Internet. Okay, this allows you to use an Android ‘phone between PCs, but you could just get an Android ‘phone to plug that gap in your life.

Email addresses used by comment spammers on WordPress

On studying the behaviour of comment spammers I became interested in the email addresses they used. Were they genuine and where were they from? Well of course they’re not likely to be genuine, but it is possible to force them to register with an address if they want their comments to appear – even if they don’t. Here’s what I found:

When the spammers were required to register, these are the domain names they registered with:

Domain Percent
hotmail.com 25%
mailnesia.com 19%
Others (unique) 16%
gmail.com 7%
o2.pl 7%
outlook.com 5%
emailgratis.info 4%
gmx.com 2%
poczta.pl 2%
yahoo.com 2%
more-infos-about.com 1%
aol.com 1%
go2.pl 1%
katomcoupon.com 1%
tlen.pl 1%
acity.pl 1%
dispostable.com 1%
live.com 1%
mail.ru 1%
se.vot.pl 1%
acoustirack.com <1%
butala.htsail.pl <1%
cibags.com <1%
eiss.xoxi.pl <1%
justmailservice.info <1%
laposte.net <1%
pimpmystic.com <1%
twojewlasnem.pl <1%
wp.pl <1%

Where the authenticity of the address is more questionable, although the sample a lot larger, the figures are as follows:

Domain Percent
gmail.com 40%
yahoo.com 11%
Other (unique) 6%
hotmail.com 6%
aol.com 4%
ymail.com 2%
googlemail.com 2%
gawab.com 2%
bigstring.com 1%
zoho.com 1%
t-online.de 1%
inbox.com 1%
web.de 1%
yahoo.de 1%
arcor.de 1%
live.com 1%
freenet.de 1%
yahoo.co.uk 1%
comcast.net 1%
mail.com 1%
gmx.net 1%
gmx.de 1%
outlook.com <1%
live.cn <1%
hotmail.de <1%
msn.com <1%
livecam.edu <1%
google.com <1%
live.de <1%
rocketmail.com <1%
gmail.ocm <1%
wildmail.com <1%
moose-mail.com <1%
hotmail.co.uk <1%
care2.com <1%
certify4sure.com <1%
snail-mail.net <1%
1701host.com <1%
cwcom.net <1%
maill1.com <1%
wtchorn.com <1%
chinaadv.com <1%
noramedya.com <1%
o2.pl <1%
vegemail.com <1%
vp.pl <1%
24hrsofsales.com <1%
kitapsec.com <1%
peacemail.com <1%
whale-mail.com <1%
wp.pl <1%
aim.com <1%
animail.net <1%
bellsouth.net <1%
blogs.com <1%
email.it <1%
mailcatch.com <1%
rady24.waw.pl <1%
titmail.com <1%
fastemail.us <1%
btinternet.com <1%
harvard.edu <1%
onet.pl <1%
yahoo (various international) <1%
akogoto.org <1%
concorde.edu <1%
freenet.com <1%
leczycanie.pl <1%
mail15.com <1%
speakeasy.net <1%
yale.edu <1%
123inholland.co.nl <1%
SolicitorsWorld.com <1%
apemail.com <1%
buysellonline.in <1%
email.com <1%
help.com <1%
ipad2me.com <1%
ismailaga.org.tr <1%
live.fr <1%
myfastmail.com <1%
mymail.com <1%
ngn.si <1%
redpaintclub.co.uk <1%
stonewall42.plus.com <1%
traffic.seo <1%
xt.net.pl <1%
a0h.net <1%
accountant.com <1%
alphanewsroom.com <1%
att.net <1%
auctioneer.com <1%
brandupl.com <1%
canplay.info <1%
charter.net <1%
cluemail.com <1%
darkcloudpromotion.com <1%
earthlink.com <1%
earthlink.net <1%
eeemail.pl <1%
emailuser.net <1%
excite.com <1%
fastmail.net <1%
gmai.com <1%
gouv.fr <1%
h-mail.us <1%
hotmail.ca <1%
hotmailse.com <1%
hotmalez.com <1%
imajl.pl <1%
jmail.com <1%
juno.com <1%
live.co.uk <1%
mac.com <1%
mailandftp.com <1%
mailas.com <1%
mailbolt.com <1%
mailnew.com <1%
mailservice.ms <1%
modeperfect3.fr <1%
mymacmail.com <1%
nyc.gov <1%
op.pl <1%
peoplepc.com <1%
petml.com <1%
pornsex.com <1%
qwest.net <1%
rosefroze.com <1%
sbcglobal.net <1%
ssl-mail.com <1%
t-online.com <1%
thetrueonestop.com <1%
turk.net <1%
virgilio.it <1%
virginmedia.com <1%
windstream.net <1%
yaahoo.co.uk <1%
yahoo.com.my <1%
yazobo.com <1%
yopmail.com <1%
zol.com <1%

A few words of warning here. First, these figures are taken from comments that made it through the basic spam filter. Currently 90% of comments are rejected using a heuristic, and even more blocked by their IP address, so these are probably from real people who persisted rather than bots. They’re also sorted in order of hits and then alphabetically. In other words, they are ranked from worst to best, and therefore zol.com has least, or equal-least, multiple uses.

It’s interesting to note that gmail was by far the most popular choice (40%) when asked to provide a valid email address but when this was used to register this dropped to 7%, with Hotmail being the favourite followed by other freemail services popular in East Europe and Russia (many single-use and counted under “Other”). Does this mean that Gmail users get more hassle from Google when they misbehave? The use of outlook.com had an even bigger reduction in percentage terms – again suggesting it’s a favourite with abusers.

Another one worth noting is that mailnesia.com was clearly popular as a real address for registering spammers, but was not used even once as a fake address. This is another of those disposable email address web sites, Panamanian registered – probably worth blacklisting. emailgratis.info is also Panamania registered but heads to anonymous servers that appear to be in North Carolina.

Where you see <1% it means literally that, but it’s not insignificant. It could still mean hundreds of hits, as this is a sample of well over 20K attempts.

If you have WordPress blog and wish to extract the data, here’s how. This assumes that the MySQL database your using is called myblog, which of course it isn’t. The first file we’ll create is that belonging to registered users. It will consist of lines in the form email address <tab> hit count:

echo 'select user_email from wp_users ;' | mysql myblog | sed 1d | tr @ ' ' | awk '{ print $2 }' | sed '/^$/d' | sort | uniq -c | sort -n | awk '{ print $2 "\t" $1}' > registered-emails.txt

I have about a dozen registered users, and thousands of spammers, so there’s no real need to exclude the genuine ones for the statistics, but if it worries you, this will get a list of registered users who have posted valid comments:

select distinct user_email from wp_users join wp_comments where not comment_approved='spam' and ID=user_id;

To get a file of the email addresses of all those people who’ve posted a comment you’ve marked as spam, the following command is what you need:

echo "select comment_author_email from wp_comments where comment_approved='spam';" | mysql myblog | sed 1d | tr @ ' ' | awk '{ print $2 }' | sed '/^$/d' | sort | uniq -c | sort -n | awk '{ print $2 "\t " $1}' > spammer-emails.txt

If you want a list of IP addresses instead, try:

echo "select comment_author_IP from wp_comments where comment_approved='spam';" | mysql myblog | sed 1d | sort | uniq -c | sort -n | awk '{ print $2 "\t " $1}' > spammer-ip-addresses.txt

As I firewall out the worse offenders there’s no point in me publishing the results.

If you find out any interesting stats, do leave a comment.

David Cameron on Google Porn

I’ve been watching with dismay David Cameron’s statements on the Andrew Marr show at the weekend; he’s attacked Google and other big companies for not blocking illegal pornography. Let’s be clear: Google et al, already do, as far as is possible. The Prime Minister is simply playing politics, and in doing so is exposing his complete lack of understanding about matters technological and social.

It’s not just the coalition government; Edward Miliband trumped him in stupidity by saying that the proposed plans “didn’t go far enough”, which is his usual unthinking response to anything announced by the government that’s might be popular.

Cameron’s latest announcement is to force ISPs to turn on “no porn” filters for all households (optionally removed, so it’s not State censorship). I’d be fascinated to hear him explain how such a filter could possibly work, but as my understanding of quantum mathematics isn’t that good it I may yet be convinced. Don’t hold your breath waiting.

The majority of the population won’t be able to understand why this is technical nonsense, so let’s look at it from the social point-of-view. People using the Internet to distribute child-abuse images do not put them on web sites indexed by Google. If Google finds any, they will remove them from search results and tell the police, as would everyone else. Paedophiles simply don’t operate in the open – why would they? They’re engaged in a criminal activity and don’t want to be caught, and therefore use hidden parts of the Internet to communicate, and not web sites found by Google!

Examining the illegal drugs trade is a useful model. It’s against the law, harmful and regarded as “a bad thing” by the overwhelming majority. The police and border security spend a lot of time and money tackling it, but the demand remains and criminal gangs are happy to supply that demand. So how successful has 100 years of prohibition been? Totally ineffective, by any metric. With 80% of the prison population on drugs IN PRISON it should be obvious that criminals will continue to supply drugs under any circumstances, if there’s a demand. If anything, proscribing drugs has made it more difficult to deal with the collateral effects by making the trade and users much more difficult to track.

So, if we can’t stop drugs (a physical item) getting in to prisons (presumably amongst most secure buildings in the country) , does anyone seriously think it’s possible to beat the criminals and prevent illegal porn being transmitted electronically to millions of homes across  the country? David Cameron’s advisors don’t appear to have been able get him to understand this point.

Another interesting question is whether I should opt to have the porn filter removed from my connection. The only way such a filter could possibly be effective is if it banned everything on its creation, and then only allowed what was proven safe through. There are generally considered to be over 500 million web sites out there, with 20,000 being added every month. That’s sites; not individual pages. The subset that can realistically be examined and monitored to make sure they are safe is going to be quite small, and as a security researcher, I need to retrieve everything. So am I going to have to ‘phone my ISP and say “yes please, want to look at porn”? Actually, that won’t be a problem for me because I am my own ISP. The government doesn’t even know I exist; there is no register of ISPs (or even a definition of the term). There are probably tens of thousands in the country. So I shall await a call from Mr Cameron’s office with a full technical explanation of this filtering  scheme with interest.

Fortunately for the Prime Minister, his live speech on the subject scheduled for 11am has been displaced by a load of royal reporters standing outside a hospital and Buckingham Palace saying “no news yet” on the supposed imminent arrival of the Duke and Duchess of Cambridge’s first child.

 

New kind of distraction email bomb attack

AppRiver

I got an interesting note from AppRiver, in which Fred Touchette, one of their analysts explains a technique used by criminals, which they first noticed in January. I haven’t seen it, nor any evidence of specific cases, but it’s food for thought.

The idea is to mail-bomb a user with thousands of spam emails containing random content over a period of several hours. Mr Touchette’s theory is that this is done to cause the user to delete the whole lot unread, and in doing so to miss an important email from their bank or similar, and therefore fail to notice a fraud attempt.

I’m not so convinced about this MO to cover bank fraud, but it would certainly be useful to someone stealing a domain name. A registrar will contact the administrative contact with a chance to block the transfer of a domain when any attempt to move it is made. This is a weak system; banks would normally require positive confirmation and not rely on the receipt and reading of an email before doing anything drastic.

If the criminals have your email login, necessary to manage something like a bank account, they will have no need to prevent you from reading emails with a mail-bomb. They just have make sure they read and delete your mail before you do, which isn’t hard if they’re keen. AppRiver’s advice, nonetheless, is to call all your banks to warn them someone might be attempting to compromise your account. I’m sure they’ll thank you politely if you do.

You can read Appriver Threatscape Report for yourself. Most of it’s unsurprising if you follow threats yourself, but this detraction technique as an attack vector is worth taking seriously, regardless of its prevalence in the wild. AppRiver is based in Florida and provides web and email security and filtering services. I met them at a London trade show and they seemed like a decent bunch.

Pipe stdout to more than one process on FreeBSD

There are odd times when you may wish to use the stdout of a program as the stdin to more than one follow-on. bash lets you do this using a > to a command instead of just a file. I think the idea is that you just have to make sure the command is in ( ) and it works. One up to bash, but what about somthing that will work on standard shells?

The answer is to use a named pipe or fifo, and it’s a bit more hassle – but not too much more.

As an example, lets stick to the “hello world” theme. The command sed s/greeting/hello/ will replace the word “greeting” on stdin with “world” on stdout – everything else will pass through unchanged. Are you okay with that? Try it if you’re not comfortable with sed

Now I’m going to send a stdout to two different sed instances at once:

sed s/greeting/hello/
sed s/greeting/world/

To get my stdout for test purposes I’ll just use “echo greeting”. To pipe it to a single process we would use:

echo greeting | sed s/greeting/hi/

Our friend for the next part is the tee command (as in a T in plumbing). It copies stdin to two different places, stdout and (unfortunately for us) a file. Actually it can copy it to as many files as you specify, so it should probably have been called “manifold”, but this is too much to ask for an OS design that spells create without the training ‘e’.

Tee won’t send output to another processes stdin directly, but the files can be a fifos (named pipes). In older versions of UNIX you created a pipe with the mknod command, but since FreeBSD moved to character only files this is deprecated and you should use mkfifo instead. Solaris also uses mkfifo, and it came in as far back as 4.4BSD, but if you’re using something old or weird check the documentation. It’ll probably be something like mknod <pipename> .

Here’s an example of it in action, solving our problem:

mkfifo mypipe
sed s/greeting/hello/ < mypipe &
echo greeting | tee mypipe | sed s/greeting/world/  
rm mypipe

It works like this: First off we create a named pipe called mypipe. Next (and this is the trick), we run the first version of sed, specifying its input to come from “mypipe”. The trailing ‘&’ is very important. In case it had passed you by until now, it means run this command asynchronously – or in background mode. If we omitted it, sed would sit there waiting for input it would never receive, and we wouldn’t get the command prompt back to enter the further commands.

The third line has the tee command added to send a copy of stdout to the pipe (where the first sedis still waiting). The first copy is piped in the normal way to the second sed.

Finally we remove the pipe. It’s good to be tidy but it doesn’t matter if you want to leave it there use it again.

As a refinement, pipes with names like “mypipe” in the working directory could lead to trouble if you forgot to delete it or if another job picks the same name. Therefore it’s better to create them in the /tmp directory and add the current process ID to the name in order to avoid a clash. e.g.:

mkfifo /tmp/mypipe.1.$$

$$ expands to the process-ID, and I added a .1. in the example so I can expand the scheme to have multiple processes receiving the output – not just one. You can use tee to send to as many pipes and files as you wish.

If you run the example, you’ll probably get “hello world” output on two lines, but you might get “world hello”. The jobs have equal status, so there’s no way to of knowing which one will complete first, unless you decided to put sleep at the start of one for force the issue.

When I get around to it a more elaborate example might appear here.

When is a Psychic not a Fraud?

The Daily Mail has had to pay £125,000 in compensation and costs to self-styled psychic Sally Morgan following an article that claimed she was perpetrating a fraud on her audience. Specifically, it said she got her messages from an earpiece rather than from beyond the grave. Going on reports of the proceedings, it appears that she didn’t use an ear-peice (or at least the Daily Mail couldn’t prove it), and therefore the article was libellous. On the face of it, some poorly researched journalism, but whoever would have thought it mattered in the case of someone pretending to hear things from the spirit world?

What the Daily Mail got wrong was that they decided to publish something as fact that couldn’t be backed up – specifically that she used an earpiece. If I say Sally Morgan, and all psychics, are not receiving any super-natural help for their performances or readings it’s up to them to prove otherwise. They’d have a hell of a job doing that, so I’m not worried in the slightest.

Ofcom, the broadcasting regulator, has guidelines about such things. Basically, you’re not allowed to pretend that psychics are real – every so often you have to say that it’s all a load of rubbish, and broadcast for entertainment purposes only. Majestic TV, the company behind Psychic Today, have today been fined £12.5K for not doing this, and inciting credulous punters to call their premium rate telephone lines for “accurate readings”.

I’m at a bit of a loss as to why Mr Justice Tugendhat (the judge in the Sally Morgan vs. Associated Newspapers) came to his decision. Sally Morgan is no more a psychic than a plank of wood, simply because there is no such thing. Exposing her methods incorrectly was wrong, but what’s the compensation all about? People who want to believe this rubbish will do so anyway, so her livelihood is safe. Whether a judge should be awarding damages to anyone who’s livelihood depends on pretending something is true when it isn’t is another matter. So he could have found in her favour and awarded her £1.

As it is Sally Morgan’s web site waffles on about her being “vindicated”, without going in to any detail about what the ruling was about. People reading it might well infer that the judge found her to be a genuine psychic, which isn’t the case at all.

Ofcom’s guidelines on these things seem eminently sensible (see ruling), but unfortunately their reach is limited to broadcasting and publishing. The growth of phone-in TV shows, known as audience participation, where the viewers are incited to dial premium rate numbers to hear a load of stuff made up about their future, is worrying but at least it’s regulated. The Internet isn’t, and I fear this kind of activity will soon spread to there.

If any psychic out there would care to prove they are genuine I will, of course, modify my view. Be warned, I’m a qualified magician and not as easy as the usual marks to fool.

 

Using MX records to create backup mail server

There’s a widely held misunderstanding about “main” and “backup” MX records in the web developer world. The fact is that there no such thing! Anyone who tells you different is plain wrong, but there are a lot of web developers who believe there is the case, and some ISPs give in and provide them as it’s simpler than arguing. It’s possible to use two MX records in some crazy scheme that looks like a backup server, in practice it does very little to help and quite possibly rather a lot to hinder. It won’t make your email more robust in practical terms.

If you are using an email server at a data centre, with reasonable expectation of an always-on connection, you need a single MX record. If your processing requirements are great you can have multiple records at the same level to spread the load between peered servers, but none would be a backup any more than any other. Senders simply get one server at random. I have a single MX record.

“But you must have a backup!”, is the usual response. I do, of course, but it has nothing to do with having multiple MX records. Let me explain:

A domain’s MX record gives the address of the server to which its email should be sent. In practice, this means the company’s mail sever; or if they have multiple servers, the incoming one. Most companies have one mail server address, and this is fine. If that mail server dies it needs to be repaired or replaced, and the replacement gets the same address.

But what of having a second MX record with an alternative, lower-priority server? It may sound good, but it’s nuts. Think about it – the company’s mail server is where the mail ends up. It’s where the users expect to log in and read it. If you have an alternative server, the mail will go there instead, but the user’s won’t be able to read it. This assumes that the backup is on a different site, available if only the first site goes down. If it’s on the same site it’s even more pointless, as it will be affected by the same connectivity issues that took the first one offline. Users will have their existing mail on the broken server, new mail will be on a different server, and you’ll be in a real bugger’s muddle trying to reconcile the two later. It makes much more sense to just fix the broken one, or switch in a backup at the same location and on the existing IP address. In extremis, you can change the MX record to point to a replacement server elsewhere.

There’s a vague idea that if you don’t have a second MX, mail will be lost. Nothing can be further from the truth. If your one and only mail server is off-line, the sender’s server will queue up the message and keep trying until it comes back – it will normally do this for a week. It won’t lose it. Most mail servers will report back to the sender if it hasn’t been able to get through for four hours, so they’ll know there’s a problem and won’t worry that you haven’t replied.

If you have two mail servers, one on a different site, the secondary server will start receiving emails when the first one goes off-line. It’ll just queue them up, waiting to forward them to the primary one, but in this case the sender won’t get notification of the delay. Okay, if the primary server is off-line for more than a week it will prevent mail loss – but why would the primary server possibly be off-line for a week – the company won’t function unless it’s repaired quickly.

In the old days of dial-up, before POP3 came in to being, some people did use SMTP in a way where a server in a data centre forwarded to the remote site when it connected. I remember Cliff Stanford had a PC mail client called Turnpike that did just this in the early days of Demon. But SMTP was designed for always-on connections and POP3 was designed for dial-up, so POP3 won out.

Let’s get real: There are two likely scenarios for having a mail server off-line. Firstly, the hardware could be dead. If so, get it repaired, and in less than a week. Secondly, the line to the server could be down, and this could be medium-term if someone with a JCB has done a particularly “good job” on it. This gives you a week to make alternative arrangements and direct the mail down another line, which is plenty of time.

So, the idea of having a “backup” MX is pointless. It can only send mail to an off-site server; it doesn’t prevent any realistic mail loss and your email ends up where your users can’t get it until the primary server is repaired. But is there any harm in having one if it makes you feel better?

Actually, in practice, yes. It does make matters worse. In theory mail will just pile up on a spare server and get forwarded later. However, this spare server probably isn’t going to be up to the same specification as the primary one. They never are – they sit there idling, with nothing to do nearly all the time. They won’t necessary have the fastest line; their spam and virus filtering will be out-of-date or non-existent and they have a finite amount of disk space to absorb mail. This can really matter if you end up storing and forwarding a large amount of spam, as is often the case these days. The primary server can be configured to discard it quickly, but this isn’t a job appropriate for the secondary one. So it builds up until it’s ancient and meagre disk space is exhausted, and then it tells the sender to give up trying due to a “disk full” error – and the email is bounced off in to the ether. It’d have been much better to leave it on the sender’s server is the first place.

There are other security issues to having a secondary server. One problem comes with spam filtering. This is best done at the end of the line; it’s not the place of a secondary server to determine what gets delivered and what doesn’t. For starters, it doesn’t see the corpus of legitimate emails, so won’t be so adept at comparing and sorting. It’s probably going to be some old spare kit that’s under-powered for modern spam processing anyway. However, when it stores and forwards it, the primary server will see it comes from a “friend” rather than a dubious source in a lawless part Internet. Spammers do use secondary MX records as a back door to get around virus and spam filters for this very reason.

You could, of course, specify and configure a secondary mail server to be up to the job, with loads of disk space to prevent a DoS attack and fully functional spam filters, regularly maintained and sharing Bayesian analysis data and local rules with the actual server. And then have this expensive resource sitting there doing nothing all day but converting electricity in to heat. Realistically, it’s not going to happen.

By now you may be wondering, if multiple MX records are so pointless, why they exist? It’s one of these Internet myths; a paradigm that users feel comfortable with, without questioning the technology behind it. There is a purpose, but it’s not for “backup”.

When universal Internet email was new, messages would be sent to a user “@” computer, and computers were normally shared, so each would have multiple possible users. The computer would receive the email and put it in the mailbox corresponding to the user part of the address.

When the idea of sending email to a domain rather than a specific server came in to being, MD and MF records also came in to being. A MD record gave the IP address of the server where mail should end up (the Mail Destination). An MF record, if it existed, allowed the mail to be forwarded through another machine first (Mail Forward). This was sometimes necessary, for example if the MD was on a dial-up connection or behind a firewall and unable to accept direct connections over the Internet. The mail would go to the MF instead, and the MF would send it to the MD – presumably by batching it up and opening a line, transiting a firewall or using some other non-public mechanism.

In the mid 1980’s it was felt that having both MD and MF records placed double the load on DNS servers, so unified MX records, which could be read with a single lookup, were born. To allow for multiple levels of mail forwarding through firewalls, they were prioritised to 99 levels, although if you need more than three for any scheme you’re just being silly.

Unfortunately, the operation of MX records, rather than the explicitly named MF and MD, is a bit subtle. So subtle it’s often very misunderstood.

The first thing you need to understand is that email delivery should be controlled from the DNS for the domain, NOT from the individual mail servers that exist on that domain. This may not be obvious, but this is how it’s designed to work, and when you think of it, a central point of control is a good thing.

Secondly, DNS records should be universal. Every computer on the Internet, making the same DNS query, should get the same result. With the later addition of asymmetric NAT, there is now an excuse for varying this, but you can come unstuck if you get it wrong and that’s not what it was designed for.

If you want to reconfigure the route that mail takes to a domain, you do it by editing the single master DNS record (zone file) for that domain – you leave the multiple mail servers alone,

Now consider this problem: an organisation (called “theorganisation”) has a mail server called A. It’s inside the theorganisation’s firewall, for its own protection. Servers on the Internet can’t talk to A directly, because the firewall won’t let them through, but local users send and receive mail through it all day long. To receive external mail there’s another server called B, this time outside the firewall. A rule on the firewall allows specific traffic from B to get to A. The relevant part of the zone file may look something like this (at least logically):

MX 1 A.theorganisation
MX 2 B.theorganisation

So how do these simple lines tell the world, and servers A and B, how to operate? You need to understand the rules…

When another server, which I shall call C, sends a message to alice@theorganisation it will look up the MX records for theorganisation, and see the records above. C will then attempt to contact alice at the lowest numbered MX it finds, which points to server A. If C is located within the same department, it will be within the firewall and mail can be delivered directly; otherwise the firewall will block it. If C can’t contact A because of the firewall it will try the next highest on the list, in this case B. B is on the Internet, and will accept connections from C (and anyone else). The message arrives at B for Alice, but alice is not a user of B. However, B knows that it’s not the final destination for mail to theorganisation because the MX record says there’s a lower numbered server called A, so it attempts to forward it there. As B is allowed through the firewall, it can deliver the message to A, where it’s finally put in alice’s mailbox.

This may sound a bit complicated, but the rules for MX records can be made to route mail through complex paths simply by editing the DNS zone file, and this is how it’s supposed to work. The DNS zone file MX records control the path the mail will take. If you try to use the system for some contrary purpose (like a poor-man’s backup), you’re going to come unstuck.

There is another situation where you might want multiple MX records: If your mail server has multiple network interfaces on different (redundant) lines. If the MX priority value is the same for both, each IP address will (or should) be used at random, but if you have high-cost and low-cost lines you can change the priority to favour one route over another. With modern routing this artifice may not be necessary – let the router choose the line and mangle the IP addresses in to one for you. But sometimes a simple solution works just as well.

In summary, MX record forwarding is not designed for implementing backup mail servers and any attempt to use them for the purpose is going to do more harm than good. The ideas that this is what they’re all about is a myth.

 

CIA and GCHQ implicated in spying shocker

I was somewhat surprised to hear that the news that CIA (or NSA) has been snooping on Internet interconnections has surprised anyone (read that twice). In the computing world, this has been the assumption since the Internet became commercial, and probably before. There’s a widely held belief that Facebook was built on CIA money, and although I’ve not seen any evidence to prove it, strategically it all makes sense. Social networking promises a rich goldmine for the intelligence services. If they weren’t digging in it I’d want to know why I was funding them with my taxes. Amazon and IBM are currently in a spat over who gets a contract for a CIA “cloud” data centre. Of course they’re all connected!

GCHQ in Cheltenham
GCHQ in Cheltenham

Here in the UK there’s now a kerfuffle about whether GCHQ is involved in using this data. Snooping without a warrant is considered “not cricket”, and I’ve just watched Sir Malcolm Rifkind (chairman of the Intelligence and Security Committee) backing up the Prime Minister in saying that the UK agency was acting within the law, and wasn’t listening in on conversations without a ministerial warrant. This has never been the issue. Tracking those conversations that take place is not the same as listening in on them, and as I understand it, is perfectly legal. In fact ISPs are required to record the details of such conversations, but not the content unless they so wish. They know who calls who, but not what was said.

The public has no right to get all precious about this invasion of privacy. They signed it away when they signed up to Twitter, Facebook or whatever other freebie social networking service they joined. These services exist to mine personal data on their users to sell advertising, or just to sell. If you’re happy about telling a multi-national corporation with dubious morals what you think and who you associate with, why should you be unhappy about your elected government knowing the same things? If you don’t trust them, vote them out.

Personally, I don’t use Yahoo, Facebook or any other service for “social networking” – and not just because I have a life. If you choose to, don’t be naïve about it.

FreeBSD 8.4 released today

FreeBSD 8.4 has just been released. But I thought we were up to 9.1? Actually version 8 is still being maintained for those who don’t want to change too much in one go, as is the way for FreeBSD.

FreeBSD Logo
FreeBSD 8.4 released

Given this conservatism, why bother upgrading from 8.3 to 8.4? If you want the latest, why not go straight to  9.1; otherwise be conservative and leave well alone? This time I might upgrade, because 8.4 contains fixed versions of BIND and OpenSSL. Certain high-profile DDoS attacks amplified by a feature of BIND are a good enough reason to suggest everyone keeps up with the latest BIND release.

You could, of course, update BIND and OpenSSL by just pulling them from the repository but there are a number of other good bug fixes in there anyway, especially in several on the Ethernet drivers. And ZFS has been improved, if you want crazy disks.

I’m not expecting 9.2 to show up until early next year, if convention holds, which is a pity because some of the BIND and OpenSSL problems were found after 9.1 was released. Don’t wait until January, apply the patches now! (Follow the links above).