July 2013 – Frank Leonhardt's Blog

Mark Shuttleworth’s Ubuntu Edge Dream

Mark Shuttleworth’s software company, Canonical Ltd, trying to raise $32M to build the first 40,000 units of a smart-phone type device that can run Ubuntu Linux. I predict he’s raise the money, and make the handsets. But the idea will tank anyway. Here’s why.

The concept of a ‘phone capable of running a desktop OS is easy to understand. When you want to use the desktop Ubuntu side you plug it in to a real monitor and keyboard – say one at home and one in the office. When you’re on the move it will run Android Linux (for Android is simply Linux with an Android graphical shell). You carry your environment with you, and carry on working wherever you are, assuming you have a monitor and keyboard available. If you run the Ubuntu graphical environment on the move, using the handset’s touch-screen it’s going to be pretty painful.

People investing about £600 will get a ‘phone, if they’re ever made. Is this an investment, or a pre-ordering deal? I think it’s up to you whether you invest enough to get a ‘phone, or buy even more equity as an investment in the future of the device, but I suspect a lot of people will simply be after the latest gadget. Whether £600 is too much for the Penguinistas, remains to be seen.

I think they stand a good chance of raising the money because they’re selling a dream that’s been around in various forms since the dawn of personal computing. One of the early incarnations would be the Apple IIc, which looked a bit like a portable typewriter when cut free from its monitor. With it you could carry your computer back from the office, but it didn’t catch on. Then, came the Tandon Data Pac, a hard disk cartridge. With a cartridge slot in PCs at the locations you needed to work, you could carry the important part of your environment with you. In those days, Microsoft didn’t do anything prevent hard disk transplants, so this was a realistic idea. But it didn’t catch on. Whether there are 40,000 people in the world who still have this dream is a good question.

Now we have laptop/notebook/netbook PCs, which are easy enough to carry in a briefcase if you get the right kind. I have always had the right kind, starting with the Cambridge Z88, moving on to the Sony Vaio and currently the Lenovo S10-3. At around 1Kg, they’re truly portable but although the Lenovo is modern, it was only on the market for a year or two as the 10″ screens format isn’t for well received by the masses. They demands big and fast, and they aren’t really worried about the battery life as long as they look cool. People often ask me “where can I get one of those”, and I tell them. (Currently only Asus and Acer producing a highly portable laptop/netbook). The snag is that when they get one they then “must” run Office 365, or some similar bloatware that a small CPU can’t handle fast.

If you don’t need battery life and the ability to work on the move, but simply want to carry your PC to and from the office, there are small form factor machines also from ASUS and Acer. If you want really small there’s the Fit-PC2 which can actually fit in a pocket. I must admit, I bought one because I thought it was a neat design. These are all Intel based and can run unmodified Windows, and yet they haven’t really caught on either. The Ubuntu Edge will not run Windows; it runs Linux. This means it won’t run Microsoft Office, ever. My experience has shown this is a big problem for a lot of people. There’s nothing wrong with OpenOffice; it’ll work with Microsoft Office files and vice versa. It’s free, whereas Microsoft Office costs and small fortune. Yet in nearly every case, people who I’ve set up with OpenOffice for cost reasons have hankered after the Microsoft version, and most have gone out and bought it (or otherwise acquired it) within a year.

The CPU for the Ubuntu Edge has yet to be announced, but based on size, battery life and heat dissipation it’s very unlikely to be Intel, or even Intel compatible. The only thing that will fit will be RISC, and given the binary nature of Linux distributions it’ll be the second-best choice of ARM. Or will its users be expected to compile everything from source? No. It’ll be an ARM and the models that are capable of running Linux with a GUI at nearly the right speed will still rip through the battery at an alarming rate.

The final nail in its coffin will be the way people currently commute with their computing environment. This comes down to cheap and cheerful thumb drive, if you can find a ubiquitous Windose PC at both ends, or on-line applications such as Google Docs if you’re really serious about it; all your data and applications on every web browser, and impossible to lose at that. If you can find a keyboard and monitor at both ends, you’re probably going to find a web browser anyway so why bother to carrying your stuff on a mobile ‘phone instead? It’s a solution to a problem that has been a “difficult sell” for 30 years, and which has now been solved by the Internet. Okay, this allows you to use an Android ‘phone between PCs, but you could just get an Android ‘phone to plug that gap in your life.

28-July-1328-July-13

Email addresses used by comment spammers on WordPress

On studying the behaviour of comment spammers I became interested in the email addresses they used. Were they genuine and where were they from? Well of course they’re not likely to be genuine, but it is possible to force them to register with an address if they want their comments to appear – even if they don’t. Here’s what I found:

When the spammers were required to register, these are the domain names they registered with:

Domain	Percent
hotmail.com	25%
mailnesia.com	19%
Others (unique)	16%
gmail.com	7%
o2.pl	7%
outlook.com	5%
emailgratis.info	4%
gmx.com	2%
poczta.pl	2%
yahoo.com	2%
more-infos-about.com	1%
aol.com	1%
go2.pl	1%
katomcoupon.com	1%
tlen.pl	1%
acity.pl	1%
dispostable.com	1%
live.com	1%
mail.ru	1%
se.vot.pl	1%
acoustirack.com	<1%
butala.htsail.pl	<1%
cibags.com	<1%
eiss.xoxi.pl	<1%
justmailservice.info	<1%
laposte.net	<1%
pimpmystic.com	<1%
twojewlasnem.pl	<1%
wp.pl	<1%

Where the authenticity of the address is more questionable, although the sample a lot larger, the figures are as follows:

Domain	Percent
gmail.com	40%
yahoo.com	11%
Other (unique)	6%
hotmail.com	6%
aol.com	4%
ymail.com	2%
googlemail.com	2%
gawab.com	2%
bigstring.com	1%
zoho.com	1%
t-online.de	1%
inbox.com	1%
web.de	1%
yahoo.de	1%
arcor.de	1%
live.com	1%
freenet.de	1%
yahoo.co.uk	1%
comcast.net	1%
mail.com	1%
gmx.net	1%
gmx.de	1%
outlook.com	<1%
live.cn	<1%
hotmail.de	<1%
msn.com	<1%
livecam.edu	<1%
google.com	<1%
live.de	<1%
rocketmail.com	<1%
gmail.ocm	<1%
wildmail.com	<1%
moose-mail.com	<1%
hotmail.co.uk	<1%
care2.com	<1%
certify4sure.com	<1%
snail-mail.net	<1%
1701host.com	<1%
cwcom.net	<1%
maill1.com	<1%
wtchorn.com	<1%
chinaadv.com	<1%
noramedya.com	<1%
o2.pl	<1%
vegemail.com	<1%
vp.pl	<1%
24hrsofsales.com	<1%
kitapsec.com	<1%
peacemail.com	<1%
whale-mail.com	<1%
wp.pl	<1%
aim.com	<1%
animail.net	<1%
bellsouth.net	<1%
blogs.com	<1%
email.it	<1%
mailcatch.com	<1%
rady24.waw.pl	<1%
titmail.com	<1%
fastemail.us	<1%
btinternet.com	<1%
harvard.edu	<1%
onet.pl	<1%
yahoo (various international)	<1%
akogoto.org	<1%
concorde.edu	<1%
freenet.com	<1%
leczycanie.pl	<1%
mail15.com	<1%
speakeasy.net	<1%
yale.edu	<1%
123inholland.co.nl	<1%
SolicitorsWorld.com	<1%
apemail.com	<1%
buysellonline.in	<1%
email.com	<1%
help.com	<1%
ipad2me.com	<1%
ismailaga.org.tr	<1%
live.fr	<1%
myfastmail.com	<1%
mymail.com	<1%
ngn.si	<1%
redpaintclub.co.uk	<1%
stonewall42.plus.com	<1%
traffic.seo	<1%
xt.net.pl	<1%
a0h.net	<1%
accountant.com	<1%
alphanewsroom.com	<1%
att.net	<1%
auctioneer.com	<1%
brandupl.com	<1%
canplay.info	<1%
charter.net	<1%
cluemail.com	<1%
darkcloudpromotion.com	<1%
earthlink.com	<1%
earthlink.net	<1%
eeemail.pl	<1%
emailuser.net	<1%
excite.com	<1%
fastmail.net	<1%
gmai.com	<1%
gouv.fr	<1%
h-mail.us	<1%
hotmail.ca	<1%
hotmailse.com	<1%
hotmalez.com	<1%
imajl.pl	<1%
jmail.com	<1%
juno.com	<1%
live.co.uk	<1%
mac.com	<1%
mailandftp.com	<1%
mailas.com	<1%
mailbolt.com	<1%
mailnew.com	<1%
mailservice.ms	<1%
modeperfect3.fr	<1%
mymacmail.com	<1%
nyc.gov	<1%
op.pl	<1%
peoplepc.com	<1%
petml.com	<1%
pornsex.com	<1%
qwest.net	<1%
rosefroze.com	<1%
sbcglobal.net	<1%
ssl-mail.com	<1%
t-online.com	<1%
thetrueonestop.com	<1%
turk.net	<1%
virgilio.it	<1%
virginmedia.com	<1%
windstream.net	<1%
yaahoo.co.uk	<1%
yahoo.com.my	<1%
yazobo.com	<1%
yopmail.com	<1%
zol.com	<1%

A few words of warning here. First, these figures are taken from comments that made it through the basic spam filter. Currently 90% of comments are rejected using a heuristic, and even more blocked by their IP address, so these are probably from real people who persisted rather than bots. They’re also sorted in order of hits and then alphabetically. In other words, they are ranked from worst to best, and therefore zol.com has least, or equal-least, multiple uses.

It’s interesting to note that gmail was by far the most popular choice (40%) when asked to provide a valid email address but when this was used to register this dropped to 7%, with Hotmail being the favourite followed by other freemail services popular in East Europe and Russia (many single-use and counted under “Other”). Does this mean that Gmail users get more hassle from Google when they misbehave? The use of outlook.com had an even bigger reduction in percentage terms – again suggesting it’s a favourite with abusers.

Another one worth noting is that mailnesia.com was clearly popular as a real address for registering spammers, but was not used even once as a fake address. This is another of those disposable email address web sites, Panamanian registered – probably worth blacklisting. emailgratis.info is also Panamania registered but heads to anonymous servers that appear to be in North Carolina.

Where you see <1% it means literally that, but it’s not insignificant. It could still mean hundreds of hits, as this is a sample of well over 20K attempts.

If you have WordPress blog and wish to extract the data, here’s how. This assumes that the MySQL database your using is called myblog, which of course it isn’t. The first file we’ll create is that belonging to registered users. It will consist of lines in the form email address <tab> hit count:

echo 'select user_email from wp_users ;' | mysql myblog | sed 1d | tr @ ' ' | awk '{ print $2 }' | sed '/^$/d' | sort | uniq -c | sort -n | awk '{ print $2 "\t" $1}' > registered-emails.txt

I have about a dozen registered users, and thousands of spammers, so there’s no real need to exclude the genuine ones for the statistics, but if it worries you, this will get a list of registered users who have posted valid comments:

select distinct user_email from wp_users join wp_comments where not comment_approved='spam' and ID=user_id;

To get a file of the email addresses of all those people who’ve posted a comment you’ve marked as spam, the following command is what you need:

echo "select comment_author_email from wp_comments where comment_approved='spam';" | mysql myblog | sed 1d | tr @ ' ' | awk '{ print $2 }' | sed '/^$/d' | sort | uniq -c | sort -n | awk '{ print $2 "\t " $1}' > spammer-emails.txt

If you want a list of IP addresses instead, try:

echo "select comment_author_IP from wp_comments where comment_approved='spam';" | mysql myblog | sed 1d | sort | uniq -c | sort -n | awk '{ print $2 "\t " $1}' > spammer-ip-addresses.txt

As I firewall out the worse offenders there’s no point in me publishing the results.

If you find out any interesting stats, do leave a comment.

22-July-13

David Cameron on Google Porn

I’ve been watching with dismay David Cameron’s statements on the Andrew Marr show at the weekend; he’s attacked Google and other big companies for not blocking illegal pornography. Let’s be clear: Google et al, already do, as far as is possible. The Prime Minister is simply playing politics, and in doing so is exposing his complete lack of understanding about matters technological and social.

It’s not just the coalition government; Edward Miliband trumped him in stupidity by saying that the proposed plans “didn’t go far enough”, which is his usual unthinking response to anything announced by the government that’s might be popular.

Cameron’s latest announcement is to force ISPs to turn on “no porn” filters for all households (optionally removed, so it’s not State censorship). I’d be fascinated to hear him explain how such a filter could possibly work, but as my understanding of quantum mathematics isn’t that good it I may yet be convinced. Don’t hold your breath waiting.

The majority of the population won’t be able to understand why this is technical nonsense, so let’s look at it from the social point-of-view. People using the Internet to distribute child-abuse images do not put them on web sites indexed by Google. If Google finds any, they will remove them from search results and tell the police, as would everyone else. Paedophiles simply don’t operate in the open – why would they? They’re engaged in a criminal activity and don’t want to be caught, and therefore use hidden parts of the Internet to communicate, and not web sites found by Google!

Examining the illegal drugs trade is a useful model. It’s against the law, harmful and regarded as “a bad thing” by the overwhelming majority. The police and border security spend a lot of time and money tackling it, but the demand remains and criminal gangs are happy to supply that demand. So how successful has 100 years of prohibition been? Totally ineffective, by any metric. With 80% of the prison population on drugs IN PRISON it should be obvious that criminals will continue to supply drugs under any circumstances, if there’s a demand. If anything, proscribing drugs has made it more difficult to deal with the collateral effects by making the trade and users much more difficult to track.

So, if we can’t stop drugs (a physical item) getting in to prisons (presumably amongst most secure buildings in the country) , does anyone seriously think it’s possible to beat the criminals and prevent illegal porn being transmitted electronically to millions of homes across the country? David Cameron’s advisors don’t appear to have been able get him to understand this point.

Another interesting question is whether I should opt to have the porn filter removed from my connection. The only way such a filter could possibly be effective is if it banned everything on its creation, and then only allowed what was proven safe through. There are generally considered to be over 500 million web sites out there, with 20,000 being added every month. That’s sites; not individual pages. The subset that can realistically be examined and monitored to make sure they are safe is going to be quite small, and as a security researcher, I need to retrieve everything. So am I going to have to ‘phone my ISP and say “yes please, want to look at porn”? Actually, that won’t be a problem for me because I am my own ISP. The government doesn’t even know I exist; there is no register of ISPs (or even a definition of the term). There are probably tens of thousands in the country. So I shall await a call from Mr Cameron’s office with a full technical explanation of this filtering scheme with interest.

Fortunately for the Prime Minister, his live speech on the subject scheduled for 11am has been displaced by a load of royal reporters standing outside a hospital and Buckingham Palace saying “no news yet” on the supposed imminent arrival of the Duke and Duchess of Cambridge’s first child.

3-July-133-July-13

New kind of distraction email bomb attack

I got an interesting note from AppRiver, in which Fred Touchette, one of their analysts explains a technique used by criminals, which they first noticed in January. I haven’t seen it, nor any evidence of specific cases, but it’s food for thought.

The idea is to mail-bomb a user with thousands of spam emails containing random content over a period of several hours. Mr Touchette’s theory is that this is done to cause the user to delete the whole lot unread, and in doing so to miss an important email from their bank or similar, and therefore fail to notice a fraud attempt.

I’m not so convinced about this MO to cover bank fraud, but it would certainly be useful to someone stealing a domain name. A registrar will contact the administrative contact with a chance to block the transfer of a domain when any attempt to move it is made. This is a weak system; banks would normally require positive confirmation and not rely on the receipt and reading of an email before doing anything drastic.

If the criminals have your email login, necessary to manage something like a bank account, they will have no need to prevent you from reading emails with a mail-bomb. They just have make sure they read and delete your mail before you do, which isn’t hard if they’re keen. AppRiver’s advice, nonetheless, is to call all your banks to warn them someone might be attempting to compromise your account. I’m sure they’ll thank you politely if you do.

You can read Appriver Threatscape Report for yourself. Most of it’s unsurprising if you follow threats yourself, but this detraction technique as an attack vector is worth taking seriously, regardless of its prevalence in the wild. AppRiver is based in Florida and provides web and email security and filtering services. I met them at a London trade show and they seemed like a decent bunch.

1-July-132-July-13

Pipe stdout to more than one process on FreeBSD

There are odd times when you may wish to use the stdout of a program as the stdin to more than one follow-on. bash lets you do this using a > to a command instead of just a file. I think the idea is that you just have to make sure the command is in ( ) and it works. One up to bash, but what about somthing that will work on standard shells?

The answer is to use a named pipe or fifo, and it’s a bit more hassle – but not too much more.

As an example, lets stick to the “hello world” theme. The command sed s/greeting/hello/ will replace the word “greeting” on stdin with “world” on stdout – everything else will pass through unchanged. Are you okay with that? Try it if you’re not comfortable with sed

Now I’m going to send a stdout to two different sed instances at once:

sed s/greeting/hello/
sed s/greeting/world/

To get my stdout for test purposes I’ll just use “echo greeting”. To pipe it to a single process we would use:

echo greeting | sed s/greeting/hi/

Our friend for the next part is the tee command (as in a T in plumbing). It copies stdin to two different places, stdout and (unfortunately for us) a file. Actually it can copy it to as many files as you specify, so it should probably have been called “manifold”, but this is too much to ask for an OS design that spells create without the training ‘e’.

Tee won’t send output to another processes stdin directly, but the files can be a fifos (named pipes). In older versions of UNIX you created a pipe with the mknod command, but since FreeBSD moved to character only files this is deprecated and you should use mkfifo instead. Solaris also uses mkfifo, and it came in as far back as 4.4BSD, but if you’re using something old or weird check the documentation. It’ll probably be something like mknod <pipename> .

Here’s an example of it in action, solving our problem:

mkfifo mypipe
sed s/greeting/hello/ < mypipe &
echo greeting | tee mypipe | sed s/greeting/world/  
rm mypipe

It works like this: First off we create a named pipe called mypipe. Next (and this is the trick), we run the first version of sed, specifying its input to come from “mypipe”. The trailing ‘&’ is very important. In case it had passed you by until now, it means run this command asynchronously – or in background mode. If we omitted it, sed would sit there waiting for input it would never receive, and we wouldn’t get the command prompt back to enter the further commands.

The third line has the tee command added to send a copy of stdout to the pipe (where the first sedis still waiting). The first copy is piped in the normal way to the second sed.

Finally we remove the pipe. It’s good to be tidy but it doesn’t matter if you want to leave it there use it again.

As a refinement, pipes with names like “mypipe” in the working directory could lead to trouble if you forgot to delete it or if another job picks the same name. Therefore it’s better to create them in the /tmp directory and add the current process ID to the name in order to avoid a clash. e.g.:

mkfifo /tmp/mypipe.1.$$

$$ expands to the process-ID, and I added a .1. in the example so I can expand the scheme to have multiple processes receiving the output – not just one. You can use tee to send to as many pipes and files as you wish.

If you run the example, you’ll probably get “hello world” output on two lines, but you might get “world hello”. The jobs have equal status, so there’s no way to of knowing which one will complete first, unless you decided to put sleep at the start of one for force the issue.

When I get around to it a more elaborate example might appear here.