If you’re trying to import data into Quicken 98 (or compatible) having downloaded it from the American Express web site you’ll get some odd and undesirable results. There are two incompatibilities.
Firstly the date field (D) needs to have a two-digit year, whereas Amex adds the century.
Secondly, for reasons I haven’t quite figured out yet, the extra information M field sometimes results is a blank entry (other than the date).
You can fix this by editing the QIF file using an editor of your choice, but this gets tedious so here’s a simple program that will do it for you. It reads from stdin and writes to stdout so you’ll probably use it in a script or redirect it from and to a file. I compile it to “amexqif”. If there’s any interest I’ll tweak it to make it more friendly.
It’s hardly a complex program, the trick was figuring out what’s wrong in the first place.
#include <stdio.h>
#include <string.h>
char buf [200];
int main()
{
int lenread;
char *str;
while ((str = fgets(buf, 200,stdin))) // (()) to placate idiot mode on CLANG
{
if ((lenread = strlen(str)) > 1) // Something other than just the newline
{
if (str[0] == 'M')
continue; // Random stuff in M fields breaks Q98
if (str[0] == 'D' && lenread == 12) // Years in dates must be two digit
strcpy (str+7, str+9);
}
printf("%s",str);
}
}
I’ve seen all sorts of stuff on forums about how to process command line argument in C or C++. What a load of fuss and bother. There’s a standard getopt() function in the ‘C’ library, similar to the shell programming command, but it’s not great.
The main problem with getopt() is that it produces its own error message. Although this saves you the trouble, it can be a bit user unfriendly for the user, especially when things get complex. For example, you might have mutually exclusive arguments and want to print out a suitable message. That said, it’ll work most of the time.
Here’s a page showing a good explanation for the GCC version, whcih is pretty standard.
But rolling your own is not hard. Here’s a skeleton I use. It’s pretty self-explanatory. My rules allow single options (e.g. -a -b) or combined options (e.g. -ab), or any mixture. “–” ends options, meaning subsequent arguments are going to be a actual arguments.
Please generate and paste your ad code here. If left empty, the ad location will be highlighted on your blog pages with a reminder to enter your code.
Mid-Post
If you want to pass something as an option, such as a filename, you can. -fmyfile or -f myfile are both handled in the example.
You can add code to detect a long option by adding “if (!strcmp(p,”longname”)) … just after the char c. But I don’t like long options.
#include <stdio.h>
void process(char *s)
{
printf("Processing %s\n",s);
}
int main (int cnt, char **a)
{
int i;
for (i=1; i<cnt && a[i][0] == '-'; i++)
{
char *p = &a[i][1];
char c;
if (*p == '-')
{
i++;
break;
}
while (c = *p)
switch (*p++)
{
case 'a':
printf("Option a\n");
break;
case 'b':
printf("Option b\n");
break;
case 'f':
if (!*p && i+1 < cnt)
printf("Value for f=%s\n", a[++i]);
else
{
printf("Value for f=%s\n", p);
while (*p)
p++;
}
break;
default:
printf("Bad switch %c\n",c);
}
}
for (;i<cnt;i++)
process(a[i]);
}
The above code assumes that options precede arguments. If you want to mix them the following code allows for complete anarchy – but you can end it using the “–” option, which will take any following flags as arguments. As a bonus it shows how to add a long argument.
#include <stdio.h>
#include <string.h>
void process(char *s)
{
printf("Processing %s\n",s);
}
int main (int cnt, char **a)
{
int i;
int moreargs=1;
for (i=1; i<cnt; i++)
{
if (moreargs && a[i][0] == '-')
{
char *p = &a[i][1];
char c;
if (*p == '-')
{
moreargs = 0;
continue;
}
if (!strcmp(p,"long"))
{
printf("Long argument\n");
i++;
continue;
}
while (c = *p)
switch (*p++)
{
case 'a':
printf("Option a\n");
break;
case 'b':
printf("Option b\n");
break;
case 'f':
if (!*p && i+1 < cnt)
printf("Value for f=%s\n", a[++i]);
else
{
printf("Value for f=%s\n", p);
while (*p)
p++;
}
break;
default:
printf("Bad switch %c\n",c);
}
}
else
process(a[i]);
}
}
How do you force a string into proper case in a Unix shell script? (That is to say, capitalise the first letter and make the rest lower case). Bash4 has a special feature for doing it, but I’d avoid using it because, well, I want to be Unix/POSIX compatible.
It’s actually very easy once you’ve realised tr won’t do it all for you. The tr utility has no concept on where in the input stream it is, but combining tr with cut works a treat.
I came across this problem when I was writing a few lines to automatically create directory layouts for interpreted languages (in this case the Laminas framework). Languages of this type like capitalisation of class names, but other names have be lower case.
Before I get started, I note about expressing character ranges in tr. Unfortunately different systems have done it in different ways. The following examples assume BSD Unix (and POSIX). Unix System V required ranges to be in square brackets – e.g. A-Z becomes “[A-Z]”. And the quotes are absolutely necessary to stop the shell globing once you’ve introduced the square brackets!
Also, if you’re using a strange character set, consider using \[:lower:\] and \[:upper:\] instead of A-Z if your version of tr supports it (most do). It’s more compatible with foreign character sets although I’d argue it’s not so easy on the eye!
Anyway, these examples use A-Z to specify ASCII characters 0x41 to 0x5A – adjust to suit your tr if your Unix is really old.
To convert a string ($1) into lower case, use this:
lower=$(echo $1 | tr A-Z a-z)
To convert it into upper case, use the reverse:
upper=$(echo $1 | tr a-z A-Z)
To capitalise the first letter and force the rest to lower case, split using cut and force the first character to be upper and the rest lower:
This is tested on FreeBSD in /bin/sh, but should work on all BSD and bash-based Linux systems using international character sets.
You could, if you wanted to, use sed to split up a multi-word string and change each word to proper case, but I’ll leave that as an exercise to the reader.
It’s been ten years since the original EU ePrivacy Directive (Regulation of the European Parliament and of the Council concerning the respect for private life and the protection of personal data in electronic communications and repealing Directive 2002/58/EC) came into effect in the UK. It’s implement as part of the equally wordy Privacy and Electronic Communications Regulations 2012 (PECR) One thing it did was require companies with websites to allow users to opt in to cookies. I’ve written about this before, but since then the amount of tracking cookies has become insane, and most people would choose to opt out of that much survailance.
The problem is that with so many cookies, some websites have effectively circumvent the law by making it impractical to opt out of all of them. At first glance they offer an easy way to turn everything off and “save settings”, but what’s not so clear is that they hide an individual option for every tracking cookie company they have a deal with. And there can be hundreds, each of which needs to be individually switched off using a mouse.
These extra cookies – and they’re the ones you don’t want – are usually hidden behind a “vendors” or “partners” tab. With the site shown below this was only found by scrolling all the way down and clicking on a link.
This kind of thing is not in the spirit of the act, and web sites that do this do not “care about your privacy” in any way, shape or form. And if you think these opt-in/out forms look similar, it’s because they are. Consent Management Platforms like Didomi, OneTrust and Quantcast are widely used to either set, or obfuscate what you’re agreeing to.
An update to the ePrivicy directive is now being talked about that says it must be “easy” to reject these tracking cookies, which isn’t the case now.
Meanwhile some governments are cracking down. In January, Google and Facebook both got slapped with huge fines in France from the Commission Nationale de l’Informatique et des Libertés, which reckoned that because it took more clicks to reject cookies than accept them, Google and Facebook were not playing fair.
“Several clicks are required to refuse all cookies, against a single one to accept them. [The committee] considered that this process affects the freedom of consent: since, on the internet, the user expects to be able to quickly consult a website, the fact that they cannot refuse the cookies as easily as they can accept them influences their choice in favor of consent. This constitutes an infringement of Article 82 of the French Data Protection Act.”
I’m inclined to agree. And on top of a fines of €150 and €60 respectively, they’re being hit with €100K for each extra day they allow the situation to remain.
Unfortunately we’re not likely to see this in the UK. The EU can’t actually agree on the final form of the ePrivacy regulations. The UK, of course, is no longer in the EU and may be able to pass its own laws while the EU argues.
Last year the Information Commissioner’s office did start on this, taking proposals to the G7 in September 2021. Elizabeth Denham told the meeting that a popup with a button saying “I Agree” wasn’t exactly informed consent.
However, this year the government is going in the other direction. It’s just published plans to do away with the “cookie popup” and allow websites to slurp your data. Instead sites will be required to give users clear information on how to opt out. This doesn’t fill me with confidence.
Also in the proposals is to scrap the need for a Data Protection Impact Assessment (DPIA) before slurping data, replacing it with a “risk-based privacy management programme to mitigate the potential risk of protected characteristics not being identified”.
I don’t like the idea of any of this. However, there’s a better solution – simply use a web browser that rejects these tracking cookies in the first place.
The Royal Mail hasn’t just lost your item, it’s lost the plot completely.
While the news media
has been obsessed with what civil servants might have been doing
after work in Downing Street they have overlooked the latest bonkers
development from the Post Office – “digital stamps”.
The gimmick is that every new stamp will have a 2D barcode on one side. According to the Royal Mail’s Nick Landon, “Introducing unique barcodes on our postage stamps allows us to connect the physical letter with the digital world and opens up the possibilities for a range of new innovative services in future.” This was followed by promises that it would be possible to link the codes to videos, and by scanning them with an App you could send “birthday messages” and other videos.
Just because
something’s possible, it doesn’t mean its a good idea Nick! But
what’s the harm, eh?
Well look a bit
further – from the start of 2023 you won’t be able to use any of
your existing stamps. That’s right – they’re being withdrawn.
In a statement Royal Mail has said:
“Mail posted with
non-barcoded Definitive stamps after 31 January 2023, will be treated
in the same way as if there is insufficient postage on an item….Any
item that has insufficient postage is subject to a surcharge.
Surcharge fees can be found on our website.”
So let’s just
assume the Royal Mail hasn’t completely lost its senses and there’s
a better reason for this than using an App to “send” Shaun the
Sheep videos, or to make money by cancelling stamps already paid for
that people won’t get around to replacing in time.
One answer would be
to make the stamps machine readable. Possibly, but that’d also make
them much easier to forge. You could machine-read an existing stamp
anyway; barcode technology is quicker and more forgiving, which is
also its weakness.
Perhaps they’re worried about counterfeit stamps? Printing a barcode isn’t difficult. Unless…
I’ve looked at the stamps and they’ve got what’s probably a 47×16 matrix. Allowing for ECC and alignment marks that’s still going to be something like a 480-bit number – enough to give every stamp printed its own serial number from now until the end of time. This would also explain how scanning one could be used to deliver a unique video message to the recipient. If this is the plan – every stamp is unique – they could spot when the same stamp passed through their scanners twice, thus spotting when a forgery has been used.
The flaw in this
brilliant plan is that the Royal Mail will have no way of telling if
the stamp its currently scanning is the original or the forgery. If a
forger has used your stamp number before you did, I predict an
almighty row.
Over the last few months I’ve noticed huge increase is spam with a “Reply To:” field set to a gmail address. What the miscreants are doing is hijacking a legitimate mail server (usually a Microsoft one) and pumping out spam advertising a service of some kind. These missives only work if the mark is able to reply, and as even a Microsoft server will be locked down sooner or later, so they’ll never get the reply.
The reason for sending this way is, of course, spam from a legitimate mail server isn’t going to be blacklisted or blocked. SPF and other flags will be good. So these spams are likely to land in inboxes, and a few marks will reply based on the law of numbers.
To get the reply they’re using the email “Reply-To:” field, which will direct the reply to an alternative address – one which Google is happy to supply them for nothing.
The obvious way of detecting this would be to examine the Reply-To: field, and if it’s gmail whereas the original sender isn’t, flag it as highly suspect.
I was about to write a Spamassassin rule to do just this, when I discovered there is one already – and it’s always been there. The original idea came from Henrik Krohns in 2009, but it’s time has now definitely arrived. However, in a default install, it’s not enabled – and for a good reason (see later). The rule you want is FREEMAIL_FORGED_REPLYTO, and it’s found in 20_freemail.cf
Enabling FREEMAIL_FORGED_REPLYTO in Spamassassin
If you check 20_freemail.cf you’ll see the rules require Mail::SpamAssassin::Plugin::FreeMail, The FreeMail.pm plugin is part of the standard install, but it’s very likely disabled. To enable this (or any other plugin) edit the init.pre file in /usr/local/etc/mail/spamassassin/ Just add the following to the end of the file:
You’ll then need to add a list of what you consider to be freemail accounts in your local.cf (/usr/local/etc/mail/spamassassin/local.cf). As an example:
Note the use of ‘*’ as a wildcard. ‘?’ matches a single character, but neither match a ‘.’. It’s not a regex! There’s also a local.cf setting “freemail_whitelist”, and other things documented in FreeMail.pm.
Then restart spamd (FreeBSD: service spamd restart) and you’re away. Except…
The problem with this Rule
If you look at 20_freemail.cf you’ll see the weighting is very low (currently 0.1). If this is such a good rule, why so little? The fact is that there’s a lot of spam appearing in this form, and it’s the best heuristic for detecting it, but it’s also going to lead to false positives in some cases.
Consider those silly “contact forms” beloved by PHP Web Developers. They send an email from a web server but with a “faked” reply address to the person filling in the form. This becomes indistinguishable from the heuristic used to spot the spammers.
If you know this is going to happen you can, of course add an exception. You can even have the web site use a local submission port and send it to a local mailbox without filtering. But in a commercial hosting environment this gets a bit complicated – you don’t know what Web Developers are doing. (How could you? They often don’t).
If you have control over your users, it’s probably safe to up the weighting. I’d say 3.0 is a good starting point. But it may be safer to leave it at 0.1 and examine the results for what would have been false positives.
In the 2000s it was “It’s necessary for our QA procedure”. Now it’s GDPR. Basically, the technical sounding response to shut people up when they complain. As a qualified ISO-9000 auditor I used to had a lot of fun calling their bluff in the first case.
With data protection it might seem more clear cut than having an encyclopaedic knowledge of ISO9000:2000. After all DPA 2018 (that which implemented GDPR) isn’t that dissimilar to its predecessors, and has a much tighter scope. However, it’s more open for interpretation and we’re waiting for some test cases.
However, what it doesn’t cover are situations like this:
Dear Mr Leonhardt, Hope you're well; It is law to speak to the account holder. Kind regards, Salvin Tingh Morrisons Online Customer Service Team
I won’t bore you with the full details of what led to this attempted put-down, but briefly I emailed Morrisons about a mistake they’d made on an order. On receiving no response I called (and they sorted it out efficiently, over the phone). A week later I got an email response, and I said it was too late but it was sorted out, thanks very much. A week later, another reply that suggested they hadn’t read the first one. I said “Sorted, thanks, and I’ll just use the ‘phone in future”.
Next week’s reply was along the lines that they couldn’t verify I was the customer. I replied that perhaps they should have tried (they know my email address and telephone number), but don’t worry it’s sorted. A week later the above arrived (name changed to protect the guilty).
Leaving aside the principles of good customer service – if you need to check someone’s identity before solving a problem then do so – one might wonder what law he might be talking about. You see, data protection laws are not as wide-ranging as people think.
Basically, the law relates to sensitive information about an identifiable individual. Stronger protections exist depending on the sensitivity of the information (e.g. race, religion, biometrics and the usual stuff). But if it’s not sensitive information about an identifiable information it’s definitely out-of-scope.
In this case, Mr Tingh was dealing with a customer’s problem. He wasn’t being asked to divulge sensitive information to a possible third party. It’s possible (and desirable) that company procedures required that he make sure it really was the customer complaining, but that’s hardly “the law”. And had I been an imposter claiming I hadn’t received my sausage, the worst that would happen was someone else got a couple of quid refunded unexpectedly. Does Morrisons get that kind of thing often, one wonders?
And it also begs the question, if they were so concerned about whether a customer complaint about an order, emailed in with the full paperwork, really was from the household in question they need only pick up the phone; or check the email address? Neither of these is fool proof, but in the circumstances one might have thought this good enough. Did he want me to visit the shop show the manager my passport?
But to reiterate, The Data Protection Act (colloquially referred to as GDPR) is there to protect information pertaining to an individual. A company would have a duty to ensure it was talking to the right person if giving out sensitive information, but when someone is reporting the non-delivery of a vegan sausage to the suppler there is no sensitive information involved. They only need to check your identity if its really necessary.
Other protections in the DPA include transparent use of an individual’s data, not storing more than is necessary or for longer than necessary, and ensuring it’s accessible to the individual concerned, not leaked and is accurate (corrected if needs be). The European GDRP added provisions for portability, forcing companies to make your data available to competing services at your request.
So when someone tries to fob you off with “data protection”, stop and think if the above actually apply. And if you’re trying to fob someone off, don’t try to bluff a data security expert.
I’ve just heard Ursula von der Leyen (President of the European Commission) announce to the European Parliament that Kremlin propaganda network Russia Today (now restyled RT) is to have its license to operates revoked (i.e. it’s being banned). This is a terrible idea.
There are many media organisations committed to undermining the western “establishment” using a heavily slanted narrative. However, allowing them to exist is what makes us different from Putin’s Russia. By banning one of them we’d be playing the same game as Putin, the CPC and every other repressive regime around the world.
Putin’s Russia isn’t about to invade Ukraine, and neither will China be marching in to Taiwan. They’ll be doing nothing until the Winter Olympics are over.
It’s not complicated. Putin wants to assert Russian influence over the former Soviet Union and has no intention of allowing one of the Ukrainian factions to take the country in the direction of Poland and other Warsaw Pact allies. This has nothing to do with Russia’s security – to believe otherwise you’d have to think the West had designs for an invasion of Russia. This is nonsense. Putin probably isn’t paranoid.
Western media paints the Ukrainian situation as a noble government in Kiev and a bunch of evil separatists. It’s not. It’s an ongoing nasty civil war, with no end in sight. Because of the geopolitical situation, no one is doing anything about it.
Some factions in Ukraine, which is a big place, have neo Nazi roots. They’re not the separatists. The Russians have a history with the Ukrainian Nazis, and seeing them on the streets of Kiev is going to have an effect of their thinking. The President, government and most Ukrainians are certainly not Nazi sympathisers, of course – far from it.
My guess would be that Putin has had enough of the instability on his boarders and plans to do something about it. This would be logical, but it will be the people of the region that suffer if this takes the form of military action. I fear that it will, but after the sport is over.
Note to history
When this was written, the BBC was reporting that Russia was pulling its troops and tanks back from the border and Putin’s insistence that he had no plans to invade. The headline is mocking the BBC and other media outlets. As predicted, Russia declared the pro Russian areas of Ukraine independent of Kiev the day after the Olympics ended, and at time of writing, is believed to have sent (more) troops in to “defend” them – i.e invaded Ukraine.
Intel’s latest wheeze for its CPUs is Software Defined Silicone (SDSi). The deal is that you buy the CPU at one price and then pay extra for a license to enable more stuff.
Basically, the CPU has an interface that you can access if you have an Authentication Key Certificate (AKC) and have purchased a Capability Activation Payload (CAP) code. This will then enable extra stuff that was previously disabled. Quite what the extra stuff is remains to be seen – it could be extra instructions or enabling extra cores on a multi-core chip, or enabling more of the cache. In other words, you buy extra hardware that’s disabled, and pay extra to use it. What’s even more chilling is that you could be continuously paying licenses for the hardware you’ve bought or it’ll stop working.
It’s not actually defining the silicone in software like a FPGA, as you’d expect from euphemistic name. Software Defined Uncrippling would be more honest, but a harder sell.
But this is nothing
new. I remember IBM doing this with disk drives in the 1970’s. If
you upgraded your drive to double the capacity an IBM tech turned up
and removed a jumper, enabling the remaining cylinders. Their
justification was that double the capacity meant double the support
risk – and this stuff was leased.
Fast forward 20
years to Intel CPUS. Before the Intel 80486 chips you could provide
whatever input clock you wanted to your 80386, just choosing how fast
it went. Intel would guarantee the chip to run at a certain speed,
but that was the only limiting factor. Exceed this speed at your own
risk.
The thing was that
the fast and slow CPUs were theoretically identical. It’s often the
case with electronic components. However, manufacturing tolerances
mean that not all components end up being the same, so they’re
batch tested when the come off the line. Those that pass the toughest
test get stamped with a higher speed and go in the fast bucket, where
they’re sold for more. Those that work just fine at a lower speed
go into the slower bucket and sell for less. Fair enough. Except…
It’s also the
nature of chip manufacture that the process improves over time, so
more of the output meets the higher test – eventually every chip is
a winner. You don’t get any of the early-run slow chips, but you’re
contracted to sell them anyway. The answer is to throw some of the
fast chips into the slow bucket and sell them cheap, whilst selling
others at premium price to maintain your margins.
In the early 1990’s
I wrote several articles about how to take advantage of this in PCW,
after real-world testing of many CPUs. It later became known as
overclocking. I also took the matter up with Intel at the time, and
they explained that their pricing had nothing to do with
manufacturing costs, and everything to do with supply and demand.
Fair enough – they were honest about it. This is why AMD gives you
more bang-per-buck – they choose to make things slightly better and
cheaper because that maximises their profits too.
With the
introduction of the 80486, the CPU clock speed was set in the package
so the chip would only run at the speed you paid for. SDSi is
similar, except you can adjust the setting by paying more at a later
date. It also makes technical sense – producing large quantities of
just one chip has huge economies of scale. The yield improves, and
you just keep the fab working. In order to have a product range you
simply knobble some chips to make them less desirable. And using
software to knobble them is the ultimate, as you can decide at the
very last minute how much you want to sell the chip for, long after
it’s packaged and has left the factory.
All good? Well not
by me. This only works if you’re in a near monopoly position in the
first place. Microsoft scalps its customers with licenses and
residual income, and Intel wants in on that game. It’s nothing
about being best, it’s about holding your customers to ransom for
buying into your tech in the first place. This hasn’t hurt
Microsoft’s bottom line, and I doubt it’ll hurt Intel’s either.
