Quicken doesn’t like Amex QIF format

If you’re trying to import data into Quicken 98 (or compatible) having downloaded it from the American Express web site you’ll get some odd and undesirable results. There are two incompatibilities.

Firstly the date field (D) needs to have a two-digit year, whereas Amex adds the century.

Secondly, for reasons I haven’t quite figured out yet, the extra information M field sometimes results is a blank entry (other than the date).

You can fix this by editing the QIF file using an editor of your choice, but this gets tedious so here’s a simple program that will do it for you. It reads from stdin and writes to stdout so you’ll probably use it in a script or redirect it from and to a file. I compile it to “amexqif”. If there’s any interest I’ll tweak it to make it more friendly.

It’s hardly a complex program, the trick was figuring out what’s wrong in the first place.

#include <stdio.h>
#include <string.h>

char buf [200];

int main()
{
int lenread;
char *str;
        while ((str = fgets(buf, 200,stdin))) // (()) to placate idiot mode on CLANG
        {
                if ((lenread = strlen(str)) > 1)        // Something other than just the newline
                {
                        if (str[0] == 'M')
                                continue;       // Random stuff in M fields breaks Q98


                        if (str[0] == 'D' && lenread == 12)     // Years in dates must be two digit
                                strcpy (str+7, str+9);
                }
                printf("%s",str);
        }
}

Digital Postage Stamps

The Royal Mail hasn’t just lost your item, it’s lost the plot completely.

While the news media has been obsessed with what civil servants might have been doing after work in Downing Street they have overlooked the latest bonkers development from the Post Office – “digital stamps”.

The gimmick is that every new stamp will have a 2D barcode on one side. According to the Royal Mail’s Nick Landon, “Introducing unique barcodes on our postage stamps allows us to connect the physical letter with the digital world and opens up the possibilities for a range of new innovative services in future.” This was followed by promises that it would be possible to link the codes to videos, and by scanning them with an App you could send “birthday messages” and other videos.

Just because something’s possible, it doesn’t mean its a good idea Nick! But what’s the harm, eh?

Well look a bit further – from the start of 2023 you won’t be able to use any of your existing stamps. That’s right – they’re being withdrawn. In a statement Royal Mail has said:

“Mail posted with non-barcoded Definitive stamps after 31 January 2023, will be treated in the same way as if there is insufficient postage on an item….Any item that has insufficient postage is subject to a surcharge. Surcharge fees can be found on our website.”

Please generate and paste your ad code here. If left empty, the ad location will be highlighted on your blog pages with a reminder to enter your code. Mid-Post

What you’re supposed to do now is find all your “old fashioned” stamps and post them off to the Royal Mail, who will send you the new digital ones in return. What a waste of time and money – theirs and ours. Why not just accept the old stamps people have paid for until they run out? I’ve asked but received no further comment.

So let’s just assume the Royal Mail hasn’t completely lost its senses and there’s a better reason for this than using an App to “send” Shaun the Sheep videos, or to make money by cancelling stamps already paid for that people won’t get around to replacing in time.

One answer would be to make the stamps machine readable. Possibly, but that’d also make them much easier to forge. You could machine-read an existing stamp anyway; barcode technology is quicker and more forgiving, which is also its weakness.

Perhaps they’re worried about counterfeit stamps? Printing a barcode isn’t difficult. Unless…

I’ve looked at the stamps and they’ve got what’s probably a 47×16 matrix. Allowing for ECC and alignment marks that’s still going to be something like a 480-bit number – enough to give every stamp printed its own serial number from now until the end of time. This would also explain how scanning one could be used to deliver a unique video message to the recipient. If this is the plan – every stamp is unique – they could spot when the same stamp passed through their scanners twice, thus spotting when a forgery has been used.

The flaw in this brilliant plan is that the Royal Mail will have no way of telling if the stamp its currently scanning is the original or the forgery. If a forger has used your stamp number before you did, I predict an almighty row.

It’s the LAW (GDPR as an excuse)

GDPR

In the 2000s it was “It’s necessary for our QA procedure”. Now it’s GDPR. Basically, the technical sounding response to shut people up when they complain. As a qualified ISO-9000 auditor I used to had a lot of fun calling their bluff in the first case.

With data protection it might seem more clear cut than having an encyclopaedic knowledge of ISO9000:2000. After all DPA 2018 (that which implemented GDPR) isn’t that dissimilar to its predecessors, and has a much tighter scope. However, it’s more open for interpretation and we’re waiting for some test cases.

However, what it doesn’t cover are situations like this:

Dear Mr Leonhardt, 
Hope you're well; It is law to speak to the account holder.
Kind regards, Salvin Tingh
Morrisons Online Customer Service Team

I won’t bore you with the full details of what led to this attempted put-down, but briefly I emailed Morrisons about a mistake they’d made on an order. On receiving no response I called (and they sorted it out efficiently, over the phone). A week later I got an email response, and I said it was too late but it was sorted out, thanks very much. A week later, another reply that suggested they hadn’t read the first one. I said “Sorted, thanks, and I’ll just use the ‘phone in future”.

Next week’s reply was along the lines that they couldn’t verify I was the customer. I replied that perhaps they should have tried (they know my email address and telephone number), but don’t worry it’s sorted. A week later the above arrived (name changed to protect the guilty).

Leaving aside the principles of good customer service – if you need to check someone’s identity before solving a problem then do so – one might wonder what law he might be talking about. You see, data protection laws are not as wide-ranging as people think.

Basically, the law relates to sensitive information about an identifiable individual. Stronger protections exist depending on the sensitivity of the information (e.g. race, religion, biometrics and the usual stuff). But if it’s not sensitive information about an identifiable information it’s definitely out-of-scope.

In this case, Mr Tingh was dealing with a customer’s problem. He wasn’t being asked to divulge sensitive information to a possible third party. It’s possible (and desirable) that company procedures required that he make sure it really was the customer complaining, but that’s hardly “the law”. And had I been an imposter claiming I hadn’t received my sausage, the worst that would happen was someone else got a couple of quid refunded unexpectedly. Does Morrisons get that kind of thing often, one wonders?

And it also begs the question, if they were so concerned about whether a customer complaint about an order, emailed in with the full paperwork, really was from the household in question they need only pick up the phone; or check the email address? Neither of these is fool proof, but in the circumstances one might have thought this good enough. Did he want me to visit the shop show the manager my passport?

But to reiterate, The Data Protection Act (colloquially referred to as GDPR) is there to protect information pertaining to an individual. A company would have a duty to ensure it was talking to the right person if giving out sensitive information, but when someone is reporting the non-delivery of a vegan sausage to the suppler there is no sensitive information involved. They only need to check your identity if its really necessary.

Other protections in the DPA include transparent use of an individual’s data, not storing more than is necessary or for longer than necessary, and ensuring it’s accessible to the individual concerned, not leaked and is accurate (corrected if needs be). The European GDRP added provisions for portability, forcing companies to make your data available to competing services at your request.

So when someone tries to fob you off with “data protection”, stop and think if the above actually apply. And if you’re trying to fob someone off, don’t try to bluff a data security expert.

Chip crisis? What chip crisis?

We’ve all seen the mass media going on about a chip shortage – or “crisis” as everything seems to be called these days. Silicon chips are unobtainable, apparently. And industry leaders are blaming their inability to meet demand for products on the “chip shortage”. But does this mean we should believe them?

Industry leaders are brilliant at blaming their mistakes on outside factors. Chips, and IT in general, is an obvious scapegoat.

It’s important to differentiate between a “chip shortage” and demand outstripping supply for particular ICs. Cryptocurrency mining is soaking up GPUs like there’s no tomorrow, so you could say there’s a GPU supply crisis. The the boyz will have to make do with plain old HD murder simulators for a while.

The automotive sector always had an interesting supply chain. They beat the price down to the last penny and order “just enough” semiconductors ahead of time meet their anticipated demand – if they guess wrong then it’s on them, and in a pandemic they’re going to lose their nerve and order less.

And then there are the usual “flood/fire/zombie invasion” stories on silicon foundries that accompany every supply crisis. I’m not having it.

The facts (remember “facts” from the old days?) tell a different story when you look at the units shipped. Okay, this lumps in NVidia GPUs with 741 op-amps but it still paints a picture.

The fact is that the supply of semiconductors continues to go up year on year. The latest predictions for 2021 are suggesting there’s been a 27% increase over 2020, and that was a significant increase over 2019. Business is booming.

So, if there’s a semiconductor supply crisis, please tell me which semiconductors are actually out of stock? Automotive manufacturers who failed to pre-order enough to meet demand of their particular custom chips are going to have to wait. And they might find that having beaten the price down, they’re not top of the list when it comes to rushing through a special order fast unless they pay a bit more.

Wicked Parents let Kid Survive on Walkers Crisp Diet!

If the parents have done anything wrong it’s going public, putting themselves in the firing line for abuse from the ignorant. This may seem a simple case of bad parenting to people who know very little about mental illness or children.

The fact is that when a child (or adult) has it in their head that they’ll only accept certain things as food you have no choice but to give it to them. No choice apart from force feeding, that is. And force feeding someone in that condition isn’t going to cure them off it; make them worse more likely.

And while we’re at it, don’t think of blaming the parents for allowing their child to get into this mental state. It happens in the “best” of families and in otherwise perfectly normal adults.

Sometimes we call a firmly held belief in something that isn’t true, based on available evidence, psychosis. But we’re not consistent. In fact, certain groups see such beliefs as virtuous. A believer in the existence of a supreme being is described as pious. Fringe political group work on conspiracy theory too, sometimes breaking their nonsense into mainstream thought. We know communism doesn’t work, but some will keep the red flag flying based on all sorts of excuses that it just hasn’t been done right before.

Meanwhile another group have convinced themselves that Covid-19 is some kind of hoax. It’s a widespread problem, and not just in uneducated communities; a quick look at the evidence is all you need to prove otherwise, but using selection they can make a case that’s good enough for them.

If your misplaced belief has an appropriate number of adherents it becomes a religion or political movement, and gets equal air time on the BBC. People use the idea that if others agree with them, they must be right and no contrary evidence is considered. It was probably made up by those seeking to undermine the “truth” anyway.

If, however, you alone believe there are aliens living in the house next door, you have a psychotic disorder.

So, before we jump to conclusions about mental illness, and in particular psychotic beliefs, perhaps we should first evaluate what we believe against the evidence.

Let’s have a serious talk about lorry drivers.

Every news outlet and fool politician is banging on about the idea that Brexit has led to a shortage of 100,000 lorry drivers in the UK. This story is too good to check for those still smarting over the lost referendum, or have some other axe to grind. Unfortunately for them, I have checked the story, and it’s a pretty shabby state of affairs.

Let’s start with this figure of 100,000. It comes from the Road Haulage Association, a lobby group. And they claim to have calculated it.

When pushed, it all gets a bit vague, and it might surprise you to know that they were claiming a shortage of 50,000 in 2015 – before anyone had heard of Brexit. They always claim a shortage of about this number. They say it was calculated by surveying their members, and other means – such as looking at vacancies. They also subtract the number of drivers registered with them from the number of lorries registered with the DVLA and add that. Seriously.

Tesco has recently stated it needs another 800 drivers. A quick look at their staff vacancies adverts shows they’re actually looking for just three.

Richard Walker from Iceland, another arch-Europhile, has taken the opportunity to get his mug into newspapers by talking about “Cancelling Christmas”. This is the same Richard Walker who gets publicity for environmental initiatives yet flies around in a private helicopter.

The Road Haulage Association will also tell you there are about 600,000 lorry drivers in the UK. The Office for National Statistics, which knows what people do for a living, reckons there are half that number. Again, the RHA is counting the wrong thing – HGV licenses. Just because someone has an HGV license it doesn’t mean they’re actually a lorry driver. Many people, myself included, have one so I can hire a lorry when I need one – such as for transporting stuff to Scout camps. At the time I got it, the cost was £70. It’s not unreasonable to want to drive something large privately.

Another group with HGV licenses are firemen. Those big red things they drive around in are too big to be classified as cars, so they do the HGV test. I believe Princess Anne had one once, so she could drive large horse boxes.

So I’m not going to take anything the Road Haulage Association takes seriously until they use better methods for obtaining their statistics. It’s almost as if they had an agenda. Actually… it’s a lobby group and its head – Richard Burnett – is a long-time campaigner against Brexit (and by extension the present government).

So what is the truth of the matter if you go to a sober source such as the ONS for figures? There is indeed a shortage of HGV drivers – they say the number has dropped by 55,000 in the last 18 months, of which 47,000 were the last year. However, this isn’t caused by Brexit. In Q2 2020 there were 25,000 EU drivers working in Britain; a year later there are 24,350. This is about the same as the 2015 figure. There was, however, a blip in numbers, peaking at 42,460 – and that happened after Brexit. This fell as drivers returned home during the pandemic; boring but true. And it’s only a minor factor in the current shortage. About 12,500 lost drivers out of 55,000 (18%) were EU nationals. Every country across Europe is reporting similar shortages, apart from Romania as far as I can tell. They’re also complaining in the USA; as far away from Brexit as you can get.

In reality, far the largest drop in working driver numbers comes from retirement – or more accurately leaving to find other jobs. It’s as simple and boring as that. But the story doesn’t end there, as it’s also been claimed that more people are retiring than passing tests. Unfortunately the figures don’t bear this out either.

In 2010-2014 there were 15,500 new licenses issued, with 7500 retiring. In the second half of the decade there were 25,500 new licenses and 8600 retiring. The fact is that there are 230,000 licensed drivers under the age of 45 alone in the UK who are choosing not to drive commercially. They’ve got fed up and taken other jobs, or are using the license privately. The average age of British lorry drivers is now about 55, clearly pointing to trouble ahead.

If you want to figure out what’s going on behind the headlines, and the Twitter experts who have never even driven a lorry in their lives, you eventually end up following the money. In this case the RHA (a lobby group, remember) is making the case for the government to favour their sector. Of course they’re going to highlight any problem, and demand the government does something about it – and more specifically, throw money at it. The thing is that the logistics industry hires their own strategic planners; experts in the field of logistics. They should have seen this coming and done something about it, instead of bleating for the government to bail them out now.

The truth may be simple; if the pay and conditions for lorry drivers were better, more people would do it. And that’s entirely up to the logistics companies to solve. Some have undoubtedly been using cheap foreign labour in the last few years, which has gone home during the pandemic – and they’re the ones that have been hit the worst. And now they want the taxpayers to bail them out for having treated their drivers badly.

Update

Now we’re being told that 5000 visas are being made available to hire in foreign drivers. That’s great. But why would foreign drivers from the EU even want to work here? There’s a shortage across Europe, and they have better conditions working there. France, Germany and Belgium have laws that mean drivers don’t have to work on Sundays. And if you’ve ever compared a French and British transport cafes, the continentals win hands-down.

Update 2

So now Richard Burnett (RHA) has started panic buying of fuel by claiming there was going to be a shortage due to the lack of tanker drivers, and the hysterical media has picked it up. I’m sure the timing has nothing to do with the Labour Party conference.

Sources:

All figures in this article come from the Office for National Statistics, the Road Haulage Association or European/American government sources. I’ll make the ONS spreadsheets (the reliable stuff) available when I can figure out how.

FreeBSD in Godden Green

What is going on with FreeBSD in Godden Green in Kent, UK? Jobsite has been spamming me with junior/mid-level programmer roles mentioning FreeBSD for months now, and I’m getting curious!

I have an alert set up so whenever FreeBSD is mentioned I get a ping, as I like to know what’s going on. This isn’t one of the usual suspect AFAIK – they might even be interesting!

Amazon Echo vulnerable in Smart Speaker battle

When Google launched its smart speaker it was playing catch-up with Amazon. The Echo had an established ecosystem, and unless Amazon blew it, this lead looked unassailable. The field was Amazon’s to lose.

Since then, Amazon’s arrogance seems to have taken it towards such a losing strategy. Glitzy launches of new gadgets are not enough to maintain a lead. I have a sample of pretty much every Echo device ever sold, and the newer ones aren’t that much better than the old ones. The build quality was always good, and they work.

What could damage the Echo is the slide in functionality.

Most people assumed that the rough edges – things you should be able to do but couldn’t – would be addressed in time. Google stole a march by recognising the person speaking, but Amazon has caught up. Sort-of. Meanwhile Google has been catching up with Amazon on other functionality and ecosystem.

What Amazon is failing to realise is that they’re selling smart speakers. This is the core functionality. They came up with the technology to link speakers in groups, so you could ask for something to be played “Upstairs”.

This is still there, but it’s been made almost useless. In the beginning you could play anything you wanted on an Echo. All music purchased direct from Amazon was added to your on-line library. There was also Amazon’s Prime music service. The latter has gone down hill recently, with the good stuff moved to a separate “full” streamin service. The ability to play your own music by uploading your MP3 files to your library. This facility has just “gone”, as of the start of the year.

Loyal Amazon customer assumed that it would go the other way, and that you’d be able to stream from your local source to your smart speaker groups. Amazon has blocked this, although some third party skills can play media to a single Amazon speaker. Not so smart.

Now Echo users are about to be hit again. From next month feed of BBC Radio, and other things, is changing. You’ll still be able to get them, but only on a BBC skill. The effect of this is that you can’t use an Echo as a radio alarm clock and more, the alarms will be confined to built in sounds. No longer will I be able to wake up to Radio 4’s Today program at 6am. Unfortunately I will still have to wake up at that time.

Echo Dot with Time Display – but now no use as a radio alarm

Ironically, one of Amazon’s enhancements is an Echo Dot with a time display. Just in time for it to be made useless by the software.

Looking at the change, I also strongly suspect you won’t be able to play a radio station on a group of speakers either. The speaker group technology is limited to Amazon’s own streaming service.

The Echo/Alexa system used to just work. Unless Amazon reverses these catastrophic decisions, it just doesn’t work. And now the public has a taste for this functionally, someone else can walk in and provide it.

Why Python is a terrible language for education

The interpreted language Python is a lot of fun. It’s great for quick and dirty lash-ups, and has list comprehensions whilst being easier to use that Haskell. There are many great reasons why you would never deploy it in a production environment, but that’s not what this article is about.

In the UK, the government decided that schoolchildren needed to learn to code; and Python was picked as the language of choice.

Superficially it looks okay; a block structured BASIC and relatively easy to learn. However, the closer I look, the worse it gets. We would be far better off with Dartmouth BASIC.

To fundamentally understand programming, you need to fundamental understand how computers work. The von Neumann architecture at the very least. Sure, you can teach CPU operation separately, but if it’s detached from your understanding of software it won’t make much sense.

I could argue that students should learn machine code (or assembler), but these days it’s only necessary to understand the principle, and a high level language like BASIC isn’t that dissimilar.

If you’re unfamiliar with BASIC, programs are made up of numbered lines, executed in order unless a GOTO is encountered. It also incorporates GOSUB/RETURN (equivalent to JSR/RTS), numeric and string variables, arrays, I/O and very little else. Just the basic building blocks (no pun intended).

Because of this it’s very quick to learn – about a dozen keywords, and familiar infix expression evaluation, and straightforward IF..THEN comparisons. There are also a few mathematical and functions, but everything else must be implemented by hand.

And these limitations are important. How is a student going to learn how to sort an array if a language has a built-in list processing library that does it all for you?

But that’s the case for using BASIC. Python appears at first glance to be a modernised BASIC, although its block structured instead of having numbered lines. That’s a disadvantage for understanding how a program is stored in sequential memory locations, but then structured languages are easier to read.

But from there on, it gets worse.

Types

Data types are fundamental to computing. Everything is digitised and represented as an appropriate series of bits. You really need to understand this. However, for simplicity, everything in python is treated as an object, and as a result the underlying representation is completely hidden. Even the concept of a type is lost, variables are self-declaring and morph to whatever type is needed to store what’s assigned to them.

Okay, you can do some cool stuff with objects. But you won’t learn about data representation if that’s all you’ve got, and this is about teaching, right? And worse, when you move on to a language for grown-ups, you’ll be in for a culture shock.

A teaching language must have data types, preferably hard.

Arrays

The next fundamental concept is data arrays; adding an index to a base to select an element. Python doesn’t have arrays. It does have some great built in container classes (aka Collections): Lists, Tuples, Sets and Dictionaries. They’re very flexible, with a rich syntax, and can be used to solve most problems. Python even implements list comprehensions. But there’s no simple array.

Having no arrays means you have to learn about the specific characteristics of all the collections, rather than simple indexing. It also means you won’t really learn simple indexing. Are we learning Python, or fundamental programming principles?

Structuring

Unlike BASIC, Python is block structured. Highly structured. This isn’t a bad thing; structuring makes programs a lot easier to read even if it’s less representative of the underlying architecture. That said, I’ve found that teaching an unstructured language is the best way to get students to appreciate structuring when it’s added later.

Unfortunately, Python’s structuring syntax is horrible. It dispenses with BEGIN and END, relying on the level of indent. Python aficionados will tell you this forces programmers to indent blocks. As a teacher, I can force pupils to indent blocks many other ways. The down-side is that a space becomes significant, which is ridiculous when you can’t see whether it’s there or not. If you insert a blank line for readability, you’d better make sure it actually contains the right number of spaces to keep it in the right block.

WHILE loops are support, as are FOR iteration, with BREAK and CONTINUE. But that’s about it. There’s no DO…WHILE, SWITCH or GOTO.

You can always work around these omissions:

do
<something>
until <condition>

Becomes:

while True: 
<something>
if <condition>:
break

You can also fake up a switch statement using IF…ELSEIF…ELSEIF…ELSE. Really? Apart from this being ugly and hard to read, students are going to find a full range of control statements in any other structured language they move on to.

In case you’re still simmering about citing GOTO; yes it is important. That’s what CPUs do. Occasionally you’ll need it, or at least see it. And therefore a teaching language must support it if you’re going to teach it.

Object Orientation

And finally, we come on to the big one: Object Orientation. Students will need to learn about this, eventually. And Python supports it, so you can follow on without changing language, right? Wrong!

Initially I assumed Python supported classes similar to C++, but obviously didn’t go the whole way. Having very little need to teach advanced Python, I only recently discovered what a mistake this was. Yes, there is a Python “class”, with inheritance. Multiple inheritance, in fact. Unfortunately Python’s idea of a class is very superficial.

The first complete confusion you’ll encounter involves class attributes. As variables are auto-creating, there is no way of listing attributes at the start of the class. You can in the constructor, but it’s messy. If you do declare any variables outside a method it silently turns them into global variables in the class’s namespace. If you want a data structure, using a class without methods can be done, but is messy.

Secondly, it turns out that every member of a class is public. You can’t teach the very important concepts of data hiding; how to can change the way a class works but keep the interface the same by using accessors. There’s a convention, enforced in later versions, that means prefixing a class member with one or more underscores makes it protected or private, but it’s confusing. And sooner or later you discover it’s not even true, as many language limitations are overcome by using type introspection and this rides a coach and horses through any idea of private data.

And talking of interfaces, what about pure virtual functions? Nope. Well there is a way of doing it using an external module. Several, in fact. They’re messy, involving an abstract base class. And, in my opinion, they’re pointless; which is leading to the root cause why Python is a bad teaching language.

All Round Disaster

Object oriented languages really need to be compiled, or at least parsed and checked. Python is interpreted, and in such a way as it can’t possibly be compiled or sanity checked before running. Take a look at the eval() function and you’ll see why.

Everything is resolved at run-time, and if there’s a problem the program crashes out at that point. Run-time resolution is a lot of fun, but it contradicts object orientation principles. Things like pure virtual functions need to be checked at compile time, and generate an error if they’re not implemented in a derived class. That’s their whole point.

Underneath, Python is using objects that are designed for dynamic use and abuse. Anything goes. Self-modifying code. Anything. Order and discipline are not required.

So we’re teaching the next generation to program using a language with a wide and redundant syntax and grammar, incomplete in terms of structure, inadequate in terms of object orientation, has opaque data representation and typing; and is ultimately designed for anarchic development.

Unfortunately most Computer Science teachers are not software engineers, and Python is relatively simple for them to get started with when teaching. The problem is that they never graduate, and when you end up dealing with important concepts such as object orientation, the kludges needed to approximate such constructs in Python are a major distraction from the underlying concepts you’re trying to teach.

Talkmobile APN data settings for Android

If you’re trying to get Talkmobile working with the current version of Android and have tried various settings on the Web with no luck. The Talkmobile web site itself is also incorrect. Here are the real ones as of right now…

Go to “Access Point Names” under setting somewhere. You’ll see Vodafone ones already there, probably. Ignore them.

Create a new one. Call it “Talkmobile” or whatever you fancy. The only three settings you need to change are:

APN Name: talkmobile.co.uk

User name: wap

Password: wap

Proxy: 212.183.137.12

Port: 8799

APN Type: * (if this doesn’t work try “Default”)

I haven’t given the MMS settings because I leave them blank and avoid rip-off charges!