SMTP – Frank Leonhardt's Blog

When Yahoo ran BT Internet’s customer email for them, it wasn’t great. We all know they had problems coping with spammers hammering away trying to deliver scams and marketing messages to BT’s punters, putting the whole system in to paranoid anti-spam mode on occasions. But it could have been worse, and now it is.

Since Critical Path (now owned by Openwave Messaging) took over running the shambles in May 2013, they appear to have hit on the bright idea of not accepting more than 49 emails a day from any one server. What? Yes, you read that correctly. If the server tries to send message fifty it gets a delayed email response:

Deferred: 421 Too many messages (1.5.6.1) from xxx.xxx.xxx.xxx

Sendmail (or other normal MTA) will simply continue trying to send it for a week, but if you have more than fifty messages a day on average to BT punters the queue is never going to empty. And fifty messages isn’t a lot. Suppose you’re a company and someone wants their work emailed forward to BT Internet? That could easily be fifty for one luser. And if you’re an web host, one of your customers is probably going to want all the email for a domain to go to a @btinternet.com address, and they’ll likely set it up without you even knowing about it.

This has being going on for over a year now, with a possible reduction in the limit last autumn. There was a theory going around that it would reject domains if the SPF record was inconclusive. Although SPF sounded like a good idea for the first five minutes, it’s rubbish when used as a naive check on mail that’s been forwarded.

I’ve been able to get some unsatisfactory information out of BT on this issue. Basically their policy is to “throttle” mail from an IP address if they think more than a certain proportion of it is spam, based on SPF records and suchlike. In the case of a user having all their mail forwarded to a BT Internet box, a high proportion of it is going to be spam; it’s inevitable. And a check of the SPF record is obviously going to fail (doah!)

BT luser forums are full of complaints about this, although the cause is misunderstood. Users get bounce messages, but it’s the server log that tells the whole picture, and as it’s often delayed they believe that a hokey “fix” has actually worked and others follow.

So what can be done about it? The obvious answer is to stop using BT Internet mail. They’ve shown a complete unwillingness to address this issue, and will doubtless make some excuse that most users are unaffected – that’s to say the other large ISPs and freemail services; direct business-to-BT Internet Luser is a small fraction. If that doesn’t work for you, the minimum you should do is ban anyone forwarding mail to @btinternet.com through your servers. Then make sure that domains you host have the correct SPF records. If you don’t, and one exceeds the limit, the IP address will be blocked and prevent your other customers from using it too.

No one who knows anything about spam control will rely on SPF, of course. But if there is someone who knows what they’re doing at Openwave, their voices are clearly being ignored.

If you’re a BT customer and you use email, based on the fact this problem has gone unresolved for a year now, the only advice I can give is to move away. Which is inrtersting, because this April BT announced plans to charge their ex-punters 60 to keep their (broken) @btinternet.com domain names – the same price as BT Internet’s broadband offering anyway. Done, you will be.

Gmail Fail

Note: You may wish to read this follow-up article, which contains a solution.

A couple of weeks ago, users started complaining that people using GMAIL (and possibly iCloud) were having their emails bounced back to them from my servers. This is odd – most complaints on the Internet are from users of dodgy hosting companies having their mail rejected by GMail as likely spam. But I haven’t blacklisted Google, and all other mail is working, so they must have been mistaken.

But as soon as I could, I tried it for myself. And sure enough, a bounce came back. The relevent bit is:

Technical details of temporary failure:
TLS Negotiation failed: generic::failed_precondition:
               starttls error (0): protocol error

On fishing around in Sendmail logs, I found evidence that this has been going on all over the place:

sm-mta[84848]: STARTTLS=server, error: accept failed=-1, SSL_error=1, 
               errno=0, retry=-1, relay=mail-qg0-f50.google.com [209.85.192.50]
sm-mta[84848]: STARTTLS=server: 84848:error:1408A0C1:SSL
               routines:SSL3_GET_CLIENT_HELLO:no shared cipher:/usr/src/secure
               /lib/libssl/../../../crypto/openssl/ssl/s3_srvr.c:1073:
sm-mta[84848]: t7QJXCPI084848: mail-qg0-f50.google.com [209.85.192.50] did
               not issue MAIL/EXPN/VRFY/ETRN during connection to MTA

Oh my! The STARTTLS stuff isn’t working because there’s no shared cypher! Hang on a minute, there isn’t supposed to be. Who told Google they could use STARTTLS on port 25. It’d be neat if it worked, but it’s not configured – at least not with a certificate from a public CA. It actually works just fine if you are cool with self-signed (private) certificates. So what is Google playing at?

In the wake of Edward Snowden, people have started worrying about this kind of thing, so companies like Google are trying to be seen doing something about it, and encrypting mail might seem like a good idea. Unfortunately STARTTLS is a bad idea. The rationale behind STARTTLS was to add encryption to a previously unencrypted port’s traffic. If the sender issued a STARTTLS as part of the protocol it could switch in to TLS mode if it knew how; otherwise it would just work as normal. The IETF was very keen on this in the late 1990’s as an easy fix, citing all sorts of iffy reasons, generally to do with having two ports; one standard and one encrypted. They thought it would be confusing, requiring different URLs and not allow for opportunistic automatic encryption of the kind Google seems to be attempting.

As far as I’m concerned, this is rubbish. Having clearly defined encrypted and unencrypted ports means you know where you are. It either is or it isn’t. If you say something must be encrypted, turn off the unencrypted port. STARTTLS allows a fall-back to plain text if you specify the clear text port; and if you have a man-in-the-middle you’ll never know that the STARTTLS was stripped from the negotiations. It opens up a vulnerability that need not be there, all for the sake of saving a port. And time is on my side in this argument. Since 1999 the implementation of encrypted ports has really taken off, with https, smtps (in spite of 465 being rescinded), imaps – you name it – all servers and clients support it and you know where you are.

So what’s this sudden clamouring for the insecure STARTTLS? Naivety on the part of the large internet companies, or a plot to make people think their email traffic is now safe from snoopers when its not?

I’ve reported this problem and I await an answer from Google, but my best guess is that they’re speculatively using STARTTLS, and then barfing and throwing their toys from the pram when it doesn’t work because the verify can’t be done. Having thought about it, I’m okay with the idea of trying STARTTLS as long as you don’t mind about the CA used for the certificate; and if you can’t negotiate a TLS link, go back to plain text. In many ways it’d be better to use the well known port 465 for TLS, and if it can’t be opened, go to plain text on 25. Except there’s no guarantee that port 465 is on the same server as port 25, and it’s normally configured to require SASL authentication. As everyone knows, apart from Google it seems, assumption is the mother of all foul-ups.

Encryption is a good idea, but making assumptions about Port 25 being anything other that straight SMTP is asking for trouble.

Tag: SMTP

BT Internet mail is broken – Deferred: 421 Too many messages (1.5.6.1) from xxx.xxx.xxx.xxx

Problems receiving mail from GMail – STARTTLS is a bad idea