[HN Gopher] UK court clears post office staff convicted due to '...
___________________________________________________________________
UK court clears post office staff convicted due to 'corrupt data'
Author : ashergill
Score : 248 points
Date : 2021-04-23 10:14 UTC (12 hours ago)
(HTM) web link (www.theguardian.com)
(TXT) w3m dump (www.theguardian.com)
| PopGreene wrote:
| "It is hard to imagine a more stupid or more dangerous way of
| making decisions than by putting those decisions in the hands of
| people who pay no price for being wrong." - Thomas Sowell
| robalfonso wrote:
| Did no one ever ask where was the money? These people all had
| these huge short falls, why did no one go to find the cash?
|
| That's financial crimes 101
| DanBC wrote:
| Post Office told the post masters that they were short, so many
| post masters made up the shortfall from their own pocket,
| expecting the books to eventually balance and to get repaid.
|
| When people were unable to continue making up that shortfall
| this was seen as further evidence of their criminality:
| "they've spent the money", "they've hidden the money", and not
| "they never had the money".
| robalfonso wrote:
| It's insane they stopped pulling the thread and the defence
| didn't push that, even if you spent it there would be
| evidence. I would have been highly skeptical that all of it
| just disappeared Into thin air across the entire
| group....nuts
| Vuska wrote:
| The company I work for ships hundreds of packages through RM. The
| RM tech I've seen is a mess. Makes me wonder what it's like
| behind the scenes. Just one lowlight I've come across, this
| comment can be found in the HTML for one of their portals:
| <!-- $Revision: #6 $ $Change: 54072 $ $DateTime: 2004/02/16
| 15:56:30 $" -->
| emdowling wrote:
| One nitpick: Royal Mail and Post Office are two separate
| companies with independent boards. Royal Mail is the network
| and carrier, while the Post Office is the primary entry point
| into that network (they also offer access to a bunch of other
| services not related to the Royal Mail). Doesn't make your
| point any less valid, but wanted to call out the distinction.
| Silhouette wrote:
| _One nitpick: Royal Mail and Post Office are two separate
| companies with independent boards._
|
| Though as a nitpick of your nitpick, they weren't truly
| independent until the relevant provisions of the Postal
| Services Act 2011 came into effect on 1 April 2012. What we
| know today as the "Post Office" and "Royal Mail" had a long
| history before that.
| emdowling wrote:
| TIL. Thanks! I've only lived in the UK for 6 years so don't
| know much about the history before that. Appreciate it!
| BillinghamJ wrote:
| As a sub-nitpick, I would definitely say Royal Mail itself is
| certainly the primary entry point into the network too. But
| Post Office is super helpful in providing supporting services
| for many government-related things like passport photos,
| certification, applying for things etc
| lloydatkinson wrote:
| Sounds like the kind of bullshit BT and OpenReach pull too.
| Claim to be two unrelated companies and yet one owns parts of
| the other and the same boards run both - all so they can pass
| customer problems between the two I definitely.
| DanBC wrote:
| They have different boards.
|
| https://www.openreach.com/about-us/our-leadership-and-
| govern...
|
| https://www.bt.com/about/bt/our-company/group-
| governance/boa... Openreach Mike
| McTighe Chairman Clive Selley CEO Matt Davies
| Chief Finance Officer Edward Astle Non-executive
| Board member Liz Benison Non-executive Board member
| Andrew Barron Non-executive Board member Jon Furmston
| Secretary to the board Simon Lowth BT Group nominee
| BT Jan du Plessis Chairman Philip Jansen Chief
| Executive Simon Lowth Group Chief Financial Officer
| Adel Al-Saleh Non-independent, non-executive director
| Sir Ian Cheshire Independent non-executive director
| Iain Conn Senior independent director and independent non-
| executive director Isabel Hudson Independent non-
| executive director Mike Inglis Independent non-
| executive director Matthew Key Independent non-
| executive director Allison Kirkby Independent non-
| executive director Leena Nair Independent non-
| executive director Sara Weller Independent non-
| executive director Rachel Canham Company Secretary &
| General Counsel, Governance
| gambiting wrote:
| I'm honestly surprised anyone complains about that split -
| it was introduced specifically so that BT wouldn't control
| the entire telecom infrastructure in this country, and
| OpenReach was formed to provide equal access to all
| operators - BT being only one of them. This is an
| _extremely_ good solution to what used to be a massive
| inequality problem previously. So no, BT and Openreach aren
| 't split for some bullshit reason, they were ordered by the
| court to split in order to protect consumer rights and
| increase competition, goals which were overwhelmingly
| achieved due to that split.
|
| And yes, the negative side is that every time something
| goes wrong, BT _really_ can 't fix it any faster, it's all
| down to OpenReach to maintain the network. But on the other
| hand, it _always_ goes through OpenReach, whether you are
| with TalkTalk, BT or Sky, so the entity responsible for
| maintaing the network isn 't the entity selling you
| broadband for home.
| zinok wrote:
| It would be an extremely good solution if it worked as
| intended. In fact Openreach were not fully independent
| from BT for most of their existence, and they operated
| the network in a way which was extremely favorable to BT
| for a long time.
|
| Thus, the two companies extracted an exorbitant rent for
| the formerly public goods they controlled. The fact that
| some of this rent went to inefficiencies of running two
| separate companies on an illusionary arm's length basis
| does not really improve matters.
| emdowling wrote:
| This sounds like a similar situation to Telstra in
| Australia, which was forced to split into two entities -
| one a wholesale network provider that was open to all
| operators, the other a consumer operator that
| (supposedly) operates under the same rules as everyone
| else.
| chrisseaton wrote:
| I think you're possibly confusing the Royal Mail with the Post
| Office, there. You're talking about the Royal Mail. This
| article is about the Post Office.
| afandian wrote:
| This started in Royal Mail and was inherited by Post Office.
| https://corporate.postoffice.co.uk/our-media-
| centre#/pressre...
| gadiyar wrote:
| Nick Wallis has been documenting the entire thing here for a long
| time: https://www.postofficetrial.com
|
| Today's update isn't there yet but should be shortly.
| bennysomething wrote:
| BBC radio 4 did a thing about this, even when the post office
| knew they kept going throwing people in prison. It's so
| depressing. What's also depressing is that people trusted this
| software. How did the defence teams never question it properly
| the first time. I mean if it's a ledger, prove it works.
| vmception wrote:
| > How did the defence teams never question it properly the
| first time. I mean if it's a ledger, prove it works.
|
| Post office employee can't afford a lawyer that would do any
| extra work
|
| Earn enough so you can afford your rights... and appeals court
| where that actually matters :)
| kristjankalm wrote:
| This is unreal. Shitty software sending people to prison without
| anyone in the process considering what exactly is the likelihood
| of hundreds of postmasters simultaneously becoming thieves
| overnight.
| LorenPechtel wrote:
| I suspect they didn't think they became thieves overnight, but
| that the new system caught existing thieves.
| meowface wrote:
| Yeah, this is the part I'm having trouble understanding. A few
| people, sure. But all these postal workers committing fraud,
| with many insisting there must be something wrong with the
| software? How did this not get discovered before they were all
| convicted and sentenced?
|
| And according to the article, the full number may actually be
| something like _900 people_.
|
| >Campaigners believe that as many as 900 operators, often known
| as subpostmasters, may have been prosecuted and convicted
| between 2000 and 2014.
|
| How do you make this mistake almost 1000 times over 14 years
| before someone suspects the system data may not be quite right?
| Also, even if you do completely believe the data, how can you
| convict them all without additional supporting evidence, like
| new purchases that don't seem to fit their salary, suspicious
| bank transactions or balances, records of unusual system access
| or them actually manipulating data, etc.
| karatinversion wrote:
| The judgement from TFA is available here:
| https://www.judiciary.uk/judgments/hamilton-others-v-post-
| of...
|
| It pains a very bad picture of the Post Office, including:
|
| - an expert witness from Fujitsu, who developed the system,
| "had been aware of at least two bugs which had affected
| Horizon Online[...], but had failed to say anything about
| them or about any Horizon issues in his statements";
|
| - POL arranged a number of conference calls to discuss
| problems with the system; "instruction was then given that
| those emails and minutes should be, and have been,
| destroyed";
|
| - "there was a culture, amongst at least some in positions of
| responsibility within POL, of seeking to avoid legal
| obligations when fulfilment of those obligations would be
| inconvenient and/or costly"
|
| Further, once a number of convictions had been secured, the
| Post Office then used those convictions in later trials as
| evidence that the Horizon system was robust and reliable.
|
| All in all, a prima facie criminal conspiracy by the Post
| Office.
| baybal2 wrote:
| > How do you make this mistake almost 1000 times over 14
| years before someone suspects the system data may not be
| quite right?
|
| It's very much a case of assumed infallibility of "scientific
| evidence," which in this case were computer records.
|
| It's also very much a case of UK judges greatly, greatly
| disregarding the process, _which fully reneges on their
| oath._
|
| Country's legal system can't function if you have judges who
| can lightheadedly throw out the process out of the window
| 1000 times over 14 year.
| JetSetWilly wrote:
| > It's very much a case of assumed infallibility of
| "scientific evidence," which in this case were computer
| records.
|
| I wonder if any of the prosecuted were in Scotland?
|
| In Scots Law there's a fundamental rule of Corroboration:
| https://en.wikipedia.org/wiki/Corroboration_in_Scots_law
|
| There must be two source of independent evidence for
| someone to be convicted of a crime. I'll be interested to
| see (if there's genuinely no corroborating evidence beyond
| the computer records) how many prosecutions went ahead
| north of the border.
| g_p wrote:
| Given this appeal took place in England (and not in the
| Supreme Court), it was all English verdicts which were
| overturned as I understand.
|
| The requirement for corroboration in such a situation
| would probably be met by having someone "speak to" the
| digital evidence and audit trail.
|
| For example, if you have CCTV evidence, the CCTV is one
| piece of evidence, and it would be corroborated by a
| witness statement of the victim identifying them from the
| CCTV.
|
| Corroboration is an important and useful safeguard, but I
| don't think it would necessarily have outright prevented
| this. Perhaps it would - maybe it would have raised the
| bar on scrutiny of the evidence, by there being a general
| higher expectation?
| JetSetWilly wrote:
| Hmn possibly. I suppose I am interested to see if there
| is a practical difference because there's some debate
| about whether corroboration is a good thing to have or
| not, when you can have one piece of evidence (like DNA
| evidence) which is very high certainty.
|
| I'd expect there was prosecutions north of the border
| seeing as the post office is UK-wide so be good to see
| how they went.
| himinlomax wrote:
| This reminds me of what happened after 9/11, the fear of dirty
| bomb was all the rage so the US government deployed a network
| of Geiger counters. They arrested a number of dangerous dirty
| bombers, all of whom were cancer patients spotted by the
| detector at the subway station nearest Johns Hopkins radiation
| treatment facility.
|
| It took weeks to fix the problem.
| LorenPechtel wrote:
| At least when my wife hit that in the Shanghai/Pudong airport
| (residue from a heart scan, not cancer) they resolved it in a
| few minutes of talking.
|
| On the other hand, I think Shanghai didn't check well enough
| --there was one simple test they could have done but didn't:
| Hand held geiger counter, see what's hot. Body equally hot,
| baggage not hot, it's medical.
|
| Why couldn't the US cops do the same thing?
| himinlomax wrote:
| They implemented the system without even thinking of the
| false positives. Eventually they added that to the
| procedures, but they harassed quite a few people before
| that happened. Cancer patients on top of that, many of whom
| were probably half dead already.
| gambiting wrote:
| >>considering what exactly is the likelihood of hundreds of
| postmasters simultaneously becoming thieves overnight
|
| I mean, I don't think anyone assumed they suddenly and
| inexplicably became thieves, just that the fancy new software
| finally caught people who have been scamming the post office
| for years. Obviously the software was completely wrong and it's
| criminal what happened to those people.
| jsight wrote:
| I agree. My first thought on hearing this was that they'd
| look at the priors and realize there had to be a mistake.
|
| My second thought was that most accounting departments I've
| worked with actually wouldn't do that, would blame fraud, and
| then would congratulate themselves at how much better they've
| gotten at detecting it!
| kristjankalm wrote:
| yes, this reasoning does make sense. but given the human cost
| it should only make sense if there's a significant prior: in
| most of these cases there was no previous evidence
| whatsoever, just a new system, and boom, thieves.
|
| I think the core point here is how imbalanced this process
| was: postal system builds a new accounting program that shows
| money is missing. these people were convicted solely on the
| evidence that software said so, there was no burden on them
| to show that the money was _actually_ missing. I mean, hard
| for me to grasp how is that possible. anyone can write a
| program that shows something. how is this sufficient proof to
| send people to prison? does it not need to touch some
| objective reality at some point?
| gambiting wrote:
| Yeah I mean if your brand new software discovered that a
| retail shop was suddenly missing PS50k/month in income,
| surely you'd do full inventory to confirm PS50k worth of
| goods is actually missing. No idea how you would do that in
| a post office, but I guess take an inventory of stamps and
| any other services sold?
| g_p wrote:
| This would normally be the role of a forensic accountant.
|
| My suspicion is that the Post Office wanted to do this
| "at scale" and "automate", and just assumed blindly their
| own records were accurate, because well... They must be!
|
| Had they actually tried to investigate these as one by
| one offences, you'd gather evidence of individuals
| concerned making huge cash transactions to buy expensive
| cars and holidays. And when you didn't find any evidence
| of this unexplained enrichment (as there wasn't any),
| your investigator would point this out, and you'd realise
| you didn't have a case.
|
| Similarly a photograph of the subpostmaster getting into
| their outright-owned Lamborghini would have been useful
| evidence there. The absence of any of the evidence of
| this enrichment seems absent throughout. Let alone the
| detailed forensic accounting to determine what was
| actually taken. I suspect the issue was they simply
| didn't have any way to tell what should have been there,
| other than what the defective horizon system said... They
| were trying to run at national scale, without enough
| ground truth information to validate their assumptions
| and detect the issue.
| jsight wrote:
| Wow, we should get raises for the fine job we are doing at
| keeping people from stealing from our agency!
| [deleted]
| gertrunde wrote:
| Also - more technical background:
| https://www.computerweekly.com/news/252496560/Fujitsu-bosses...
| davidhyde wrote:
| Those postmasters defending charges of theft against them in the
| 2000s should not have had to prove that the computer system they
| were forced to use had bugs. In order to prosecute them in the
| first place, the Post Office should have had to prove, beyond a
| shadow of a doubt and without risk of bias, that the computer
| system was correct. So, independent review at the very least, not
| testimony from parties with a vested interest in the outcome.
| This mess was as much a failure of the UK legal system as it was
| of the active efforts of the Post Office and Fujitsu to
| deceitfully protect their own interests above the postmasters
| affected.
|
| It's like a murderer giving evidence against a random stranger
| and being believed at face value because they provided all the
| evidence first hand.
| [deleted]
| robk wrote:
| Computer said so and they blindly followed. Offshoring victims
| :(
| dd82 wrote:
| same with facial recognition.
|
| https://www.nytimes.com/2020/06/24/technology/facial-
| recogni...
| lupire wrote:
| An arrest based on (mechanical) eye witness evidence is not
| the same thing as a conviction, at all.
|
| The security photo is directly viewable by the police and
| the accused.
| dignick wrote:
| As the post office is such an old organisation (350 years), it
| used to have its own armed guard, has its own investigations
| branch and conducts its own prosecutions. The police wouldn't
| get involved because the post office was considered to have
| jurisdiction. Taken from this, which is a great listen:
| https://soundcloud.com/privateeyenews/page-94-the-private-ey...
| Mauricebranagh wrote:
| I used to work for BT post split from the Posties and when I
| commented that the procedure for IB/SD investigations was
| very rigorous i.e. all interviews taped two copies of the
| tape kept.
|
| I said this is just like if the police where investigating
| your for murder and I was told ah well in the bad old days
| people used to fall down stairs on occasion
| anonymousDan wrote:
| They should be stripped of those powers immediately in my
| opinion. It's an anachronism and as been shown by this
| incident they are not fit for purpose.
| Ichthypresbyter wrote:
| Private prosecutions are not unusual in England, although
| the tide may be turning against them. For instance, the
| RSPCA recently announced that it would stop bringing
| private prosecutions for animal cruelty [0], which it has
| done since before there were police. There was apparently
| pressure from MPs for them to do so, after some fairly
| high-profile cases where they were seen as being too eager
| to prosecute.
|
| Of course, the CPS (Crown Prosecution Service) has always
| had the right to take over and discontinue a private
| prosecution.
|
| [0]https://www.theguardian.com/world/2021/jan/28/rspca-
| plans-to...
| Roonerelli wrote:
| Really great work by Private Eye. They did all the
| investigation and broke the story. None of the big news
| outlets were on it at all
| toyg wrote:
| On internal UK news, there is nothing as good as Private
| Eye. Every two weeks they publish more "hard" material than
| newspapers do in a month. I'm a subscriber, the value for
| money is simply ridiculously good.
| khc wrote:
| Is there something equivalent for the US that people
| recommend?
| dignick wrote:
| Same. I struggle to find time to get through each
| edition, but I don't mind paying the subscription to
| support quality journalism.
| hermitcrab wrote:
| I have also recently subscribed to Private Eye - mostly
| because there is now so little other investigative
| journalism going on in the UK that I think they deserve
| some support (the main papers are nearly all owned by
| billionaire mates of the Conservative Party). It is also
| quite funny.
| temporama1 wrote:
| Any independent review would inevitably be done by the same
| type of "expert" that wrote the software in the first place.
|
| No doubt it's some sprawling, insane Java monstrosity Manhattan
| project or suchlike.
|
| "Yeah - 10,000 classes - completely fine and not crazy at all."
| ClumsyPilot wrote:
| Surely that contradicts the meaning of independant
| jonplackett wrote:
| These people need to be paid massive compensation for having
| their lives ruined, paid for by whoever did such a bad job on
| this system.
| rgblambda wrote:
| >>paid for by whoever did such a bad job on this system.
|
| Blame whoever signed off on the system. Can't fix bugs that
| aren't reported.
| _vertigo wrote:
| It's really not about the bugs. The bugs were unfortunate,
| but bugs happen. The problem is taking the word of the system
| at face value and not investigating further even when dozens
| of people's livelihood and freedom are hanging in the
| balance.
|
| Oh, and also the bit about spending 2 decades covering
| everything up and trying to clamp down on the investigation
| rather than admitting you got it wrong, once again at the
| expense the subpostmasters..
| mcguire wrote:
| " _It 's really not about the bugs. The bugs were
| unfortunate, but bugs happen._"
|
| Once upon a time, computer programming attempted to be a
| profession. Fortunately for all of us who write code for a
| living, we no longer have to live under the threat of that
| responsibility.
| giantg2 wrote:
| "The problem is taking the word of the system at face value
| and not investigating further even when dozens of people's
| livelihood and freedom are hanging in the balance."
|
| We do the same thing with breathalyzers in most of the US.
| No independent people allowed to inspect the system for
| bugs.
| deepspace wrote:
| The breathalyser issue is immensely frustrating. In the
| few cases where the software has been allowed to be
| examined, they found egregious bugs. Not to mention that
| the one-size-fits-all measurement model is inaccurate for
| people outside a very narrow metabolic range.
|
| The trouble is that if you speak up about it, people ask:
| "why are you defending drunk drivers?" It's like innocent
| until _proven_ guilty flies out the window.
| giantg2 wrote:
| "It's like innocent until proven guilty flies out the
| window."
|
| Honestly, this is now the public perception (and the
| system) operates these days. I had a trooper recently
| hold a charge that he _knew_ was incorrect and it carried
| with it pretrial restrictions that no other charge would.
| The state police say there 's nothing wrong with
| subjecting people to pretrial restrictions under charges
| that they _know_ to be incorrect. The attitude is "screw
| you, criminal" (just a summary offense).
|
| Some states actually get it right and use blood tests.
| That means that some blood is saved if the defense wants
| to have it tested (evidence preservation).
| matheusmoreira wrote:
| All sorts of police field tests have significant rates of
| false positives. People actually get arrested on the
| basis of such "evidence" all the time. The justice and
| law enforcement systems essentially operate on the notion
| that these things are "good enough".
| giantg2 wrote:
| I've just recently had a trooper make 4 "mistakes" in
| court and in official reports. One of which I believe was
| an outright lie. The system doesn't care. If this were
| any other witness they would be discredited, but because
| they are law enforcement, they get a free pass. The
| agencies won't even handle the complaints correctly or
| file this information as Guglio data for when future
| cases request it (if found to be unreliable they can lose
| thier job). The system actors (law enforcement and
| judicial) in most states have special privileges in
| keeping information private - so special in many cases
| that if a complaint against a judge turns up exculpatory
| evidence that you have no right to it. The reason they
| state is to uphold the integrity of, and the public trust
| in, the system. I would think transparency would do that
| better. The only way that transparency would hurt those
| objectives is when wrongdoing is ignored or the
| punishments are so lenient to offend the public sense of
| justice.
|
| But hey, I'm just a stupid peon, so what do I know.
| arethuza wrote:
| It's far worse than just having bugs - they _knew_ there were
| bugs and covered it up even when they knew what impact it was
| having. That 's the bit I find genuinely shocking.
| jonplackett wrote:
| Well, whoever was in charge of that is candidate #1 to open
| their chequebook!
| andi999 wrote:
| I do not think this is how this works.
| wizzwizz4 wrote:
| It would be preferable to the way it works, though.
| Mauricebranagh wrote:
| Unfortunately (As a sort of insider X BT ) the Postal side
| always had much worse employee relations.
|
| Part of which was caused the very "antagonistic" IB or SD
| like the US postal Inspectors.
|
| There was a bit guilty by suspicion tendency that went on
| and I suspect some of this culture was embedded in the
| organisation.
|
| Certainly having yourself or your staff investigated by SD
| was considered very stressful even after the "bad old days"
| TedDoesntTalk wrote:
| no idea what those acronyms mean. X BT? IB? SD? WTF?
| skissane wrote:
| X = ex-, as in formerly
|
| BT = British Telecom. British telco, which used to be
| part of the government-owned Post Office, but was
| separated from it in 1981 and then privatised in 1984.
| The delivery services part of the Post Office (Royal
| Mail) was separately privatised in 2013; but the retail
| post office business (Post Office Ltd) remains under full
| government ownership, albeit most of the individual post
| offices are privately run by franchisees - and it was
| these franchisees who were being prosecuted
|
| IB = Investigation Branch -
| https://www.postalmuseum.org/blog/the-post-office-
| investigat...
|
| SD = Security Division
| arethuza wrote:
| Scots law is supposed to require two separate items of evidence
| to bring a prosecution - doesn't seem to have protected them as
| there are a number of cases being reviewed in Scotland as well.
|
| https://www.bbc.co.uk/news/uk-scotland-25639645
| hermitcrab wrote:
| Some of these postmasters died before being able to clear their
| names. It is a huge miscarriage of justice and a national
| disgrace. I don't suppose any of the guilty parties will be
| punished though.
| MaxBarraclough wrote:
| You make a solid point. A similar argument applies to closed-
| source voting machines built by the lowest bidder, something
| Schneier has written about.
|
| https://www.schneier.com/blog/archives/2018/04/securing_elec...
| lbriner wrote:
| Unfortunately, that sounds straight-forward but isn't:
|
| Q) Did you or any of the people you got to examine the software
| found any way that what the defendents said was true?
|
| A) No
|
| Q) Then you are guilty beyond reasonable doubt.
|
| I think the bigger issue here is around the power that a large
| organisation wields to duck and dive and use corporate tricks
| to manipulate how it played out. For example, the fact that so
| many people had been accused could have been analyzed if it was
| known e.g. Last year 5 convictions, this year, 700!
| jjk166 wrote:
| No, that's still presumption of guilt.
|
| If you're going to convict someone of stealing PS59,000, the
| very first thing you should have to show is that PS59,000
| actually got stolen. If there is reasonable doubt that the
| crime took place, no one can be guilty beyond that reasonable
| doubt. If the defendant claims the computer system got it
| wrong, it's not enough to say you are unaware of bugs, the
| prosecution should have to show that the computer's output
| was consistent with the results of doing the calculation by
| another method.
| arethuza wrote:
| Not only that PS59,000 was actually stolen but that the
| accused received the money. There was no evidence that the
| post masters charged ever had the money in their
| possession!
| lbriner wrote:
| What you are both arguing doesn't match up with the
| facts. It was proved beyond reasonable doubt multiple
| times. The "proof", (which we now know was flawed) was
| that the system had shown that what they had sold didn't
| tally with what was sent to the Post Office, to refute
| that proof, the defence have to show another plausible
| explanation.
|
| Yes, if they could have proved they also received what
| was stolen, that would have been a slam dunk but there
| are enough plausible reasons why they can't find the
| money. Maybe it was given to friends and family as cash,
| maybe it was used to gamble or to pay off some criminal.
|
| It isn't much different than somebody saying, "you did it
| because we found your DNA". The Courts or Jury are
| inclined to believe it because "science" and if the
| defence are not on their game enough to show how "because
| DNA" is not always watertight, the defendent is seen as
| guilty beyond reasonbale doubt.
| wizzard wrote:
| > Yes, if they could have proved they also received what
| was stolen, that would have been a slam dunk but there
| are enough plausible reasons why they can't find the
| money. Maybe it was given to friends and family as cash,
| maybe it was used to gamble or to pay off some criminal.
|
| If this was one or two cases, then sure, maybe they were
| really smart about hiding the money. However, there were
| hundreds of convictions. What is the more likely
| explanation?
| c3534l wrote:
| > but there are enough plausible reasons why they can't
| find the money
|
| There is a strange presumption in here. It is true that
| lack of evidence doesn't always means there's evidence
| that there was no crime. But that shouldn't matter. A
| crime should only be prosecutable if it is demonstrable.
| We shouldn't say "oh, well the prosecution had a really
| hard case, we should just convict this person anyway
| because it wasn't fair to those lawyers." That's such a
| perverse way of reasoning about it.
| ClumsyPilot wrote:
| "The "proof", ..what they had sold didn't tally with what
| was sent to the Post Office"
|
| Post office looses packages all the time, should someone
| go to jail for that too?
|
| If their stuff doesn't tally, they are disorganised, they
| loose stuff or have idiots. Thats their problem. Maybe
| it's post office employees stealing shit.
|
| Why do we immediately assume postmasters have abything to
| do with it without a shred of evidence?
| jjk166 wrote:
| > The "proof", (which we now know was flawed) was that
| the system had shown that what they had sold didn't tally
| with what was sent to the Post Office
|
| No, it claimed that what they had sold didn't tally, a
| claim they never proved. The defense put forward another
| plausible explanation - that the software was incorrect,
| and the prosecution obviously didn't prove the software
| was accurate.
|
| Even if the computer was right and there was a genuine
| discrepancy in the tally, you then need to prove that
| this person was the one responsible for it. Certainly in
| this case, there couldn't possibly have been sufficient
| evidence to prove they were the ones that did it if it
| was never done to begin with. Absence of evidence isn't
| evidence of absence, but it sure as hell isn't proof of
| presence.
|
| In the "we found your DNA" analogy, you're finding my DNA
| in my workplace where nothing has actually gone missing -
| how does that prove I am guilty of theft?
| matthewheath wrote:
| > Not only that PS59,000 was actually stolen but that the
| accused received the money.
|
| Can only speak for English and Welsh law, but this isn't
| accurate. Theft is prosecuted under the Theft Act 1968
| and does not require the accused to actually receive the
| goods or money stolen. All the accused need do to
| "appropriate" property is assume the rights of the owner
| e.g, if the accused had access to someone's bank account
| and they sent money to a third-party, that's still theft
| because they assumed the rights of the owner (to transfer
| the money) even though the money didn't go to the accused
| themselves.
| avs733 wrote:
| Or breathalyzer convictions in the us...where charges are often
| dismissed if a subpoena for the code is granted
| quacked wrote:
| Do you have a source for that? I'd believe it immediately,
| but I'd like to be able to spread it around.
| fitblipper wrote:
| Me to! Please share. :)
| avs733 wrote:
| I remember reading a fairly in depth article that I cannot
| find...but some sources I'm able to ID at the moment that
| discuss the issue:
|
| https://freedom-to-tinker.com/2009/05/11/breathalyzer-
| source...
|
| https://lawreader.com/?p=12801
|
| https://www.tradesecretslaw.com/2008/02/articles/practice-
| pr...
|
| https://arstechnica.com/tech-policy/2009/05/buggy-
| breathalyz...
| deepspace wrote:
| The most comprehensive article I have been able to find
| is this one:
| https://www.nytimes.com/2019/11/03/business/drunk-
| driving-br... (needs login).
| LatteLazy wrote:
| While I agree more broadly, you cannot expect the Post Office
| to prove a negative. How would they prove conclusively that the
| software had no bugs at all under any circumstances? That's a
| pretty steep QA bill imho.
| ClumsyPilot wrote:
| If their code is a mess and has errors, and there are people
| at post office that know this, they should be fined on the
| spot for false representation of contemp of the court.
|
| If their system were up to date, written in a safe language,
| has unit tests and an independant review said it was solid,
| then it is just one acceptable piece of evidence.
|
| What i dont get is - where was the money? Supposedly hundreds
| of people stoke huge amount of money, and none of them had it
| in a bank, bought a new car, or showed any signs of suddenly
| becoming wealthier. Where did the judge think the money go
| to, they ate it? How was this not suspicious?
| LatteLazy wrote:
| The amazing thing about these cases are exactly how many
| shitty things had to happen (and did) for this to occur.
| Like you say, where is the money? And why didn't anyone
| spot that more sub-post-masters were getting charged than
| almost all other employee types? Why didn't anyone manage
| to reproduce the error? Why did managers hide the reports
| (who does that?)?
|
| I personally think this is partly down to the fact people
| don't get state defence lawyers anymore in the uk. You
| could accuse me of fraud with zero evidence and I likely
| would have to plead guilty as I don't have 20k for the down
| payment for a lawyer...
|
| What a shit storm. Now watch as nothing changes...
| pmichaud wrote:
| Sure, I can agree with that -- it's way too hard to prove a
| big system isn't buggy. So then you also can't use its output
| as evidence in court, right? You have to have other evidence
| that you can prove isn't faulty. Can't have it both ways.
| bennysomething wrote:
| If you are taking someone to court, maybe be cautious, could
| it be a bug, are there similarities across the all cases
| here? Etc.
| throwaway823882 wrote:
| This is the proof that how we write software is inherently
| wrong, if we allow innocent lives to be destroyed because we
| don't want to write it differently.
|
| Imagine not doing inspections of new building construction
| because it would be costly.
| guitarbill wrote:
| That is a tempting conclusion, but consider if the software
| was 100% correct per some specification, and the spec was
| wrong?
|
| No, the problem is greater than that. Decisions that affect
| people should not be made solely by computers or
| algorithms, and those decisions should be made transparent
| and auditable. If that leads to different/better ways of
| writing software, good. It's a larger societal issue
| though.
| etothepii wrote:
| The Grenfell inquiry would like to talk to you ...
| wutbrodo wrote:
| > This is the proof that how we write software is
| inherently wrong, if we allow innocent lives to be
| destroyed because we don't want to write it differently.
|
| I think there's a lot of room for writing software better,
| including expanded source access for public systems and
| formal verification when critical.
|
| But the failure in this case isn't technical, it's legal.
| It's rational to decide that occasional bugs in a mail
| software system are acceptable, and not worth the cost of
| designing a system's development around formal-
| verification. What's obviously insane is treating such a
| system as if it's bug-free beyond a reasonable doubt, and
| ruining innocent people's lives over it.
|
| There are a lot of forms of gross incompetence and
| negligence that we're all fine with because they're so
| common. Failing to reason about software systems and their
| pitfalls, or consult with those who are capable of doing
| so, is an extremely-common and often-dangerous example (cf
| dumbass Senators grilling Zuckerberg with their 1970s
| understanding of how technology functions).
|
| The blame here lies squarely on the prosecutors, judges,
| etc who are responsible for these verdicts. They should be
| ashamed of themselves.
| mcguire wrote:
| As long as none of the blame goes to the programmers,
| we're all good.
| davidhyde wrote:
| Agreed that you cannot expect the Post Office to prove a
| negative but, if they cannot prove a negative, they should
| not be able to use their computer system as evidence of theft
| and fraud. Especially if this is the only evidence they have.
| ganzuul wrote:
| So government software, technically cybernetic software,
| should be proven to be correct. Will automatic theorem
| proovers be able to accommodate this?
| _vertigo wrote:
| Not necessarily proven to be correct, proven to be
| correct _beyond a reasonable doubt_. That's the standard.
| jschwartzi wrote:
| They can actually use formal methods to prove that their
| software is bug free. This technique is often used in safety-
| critical systems to ensure that they function as-specified.
| As long as the specification is correct, the software system
| should perform to specification under all input conditions.
|
| https://en.wikipedia.org/wiki/Formal_methods
|
| We should consider the cost of QA and of engineering process
| against the cost to these 39 people of their freedom and a
| large part of their lives due to an accounting error in the
| software.
| justincormack wrote:
| That wasn't the issue. Fujitsu, who operated the system,
| had access to post office branch systems with full access,
| but this was denied. The postmasters were prosecuted
| individually without good enough representation. The 2019
| judgement is a good read about how the prosecutions
| happened and how the evidence was presented [1]. Really,
| formal methods and bugs were not issues, this was a system
| with humans in and someone decided that some of the humans
| should be blamed for issues, because the balance of power
| let them.
|
| [1] https://www.judiciary.uk/wp-
| content/uploads/2019/12/bates-v-...
| Mauricebranagh wrote:
| More than that there was at least one Suicide
| DaiPlusPlus wrote:
| Formal proof of program correctness tends to require that
| the software's purpose lies in a very narrow, and extremely
| well-defined problem space. The _Horizon_ software in-
| question is a general-purpose line-of-business system,
| which presumably has to react to ever-changing business
| requirements - that's probably the hardest space to
| implement formal-methods in - with little benefit for
| doing-so precisely because requirements change so often.
|
| The places where you do see formal-methods would be in, for
| example, FADEC for aircraft engines, or an operating system
| process scheduler.
| da_chicken wrote:
| You don't need a formal proof for the entire system as a
| whole. You could simply have a formal proof of specific
| functions of the software. Those functions which will
| have their data audited, for example.
| himinlomax wrote:
| Formal methods require a comprehensive specification.
| Usually, if you have a specification comprehensive enough
| for formal verification, you already have 90% of the
| benefits, which is why it's (so far) only really useful in
| safety critical applications with a very small scope. I'm
| not going to take a huge risk in betting that the postal
| service didn't have anything resembling a serious spec in
| this case.
| ClumsyPilot wrote:
| Thats fine, but if they choose to handle money with a
| joke app, they should eat the losses
| jjk166 wrote:
| They don't need to prove that their software has no bugs, but
| they do need to prove that what their software claims is
| true.
| cletus wrote:
| The first bizarre part to me about this fiasco is that
| accounting, as a discipline, is one that is designed to catch
| errors. Put it another way: it assumes errors will occur. This is
| why in shops, for example, you'll have manual stocktaking (ie
| let's verify what's in the store is what the computer thinks is
| in the store) and in any business you'll have reconciliation
| processes to find and remedy errors.
|
| This highlights a key part of systems design. A key question you
| should be asking is: what happens when this fails? Note that's
| "when" not "if".
|
| So something like Horizon should be used to flag cases for
| reviews. If a branch is found ot have a cash shortfall suggesting
| possible theft then there has to be a reconciliation possible to
| identify if the computer system was wrong.
|
| Bugs happen too. How do they ever have confidence in the system
| and fix bugs if they can't determine if a given flag is a false
| or true positive?
|
| But instead the system's output was taken as gospel with no
| possibility of verification. I'm of the belief that if you can't
| verify anything the system outputs, particularly for something in
| a discipline so used to verification as a concept, then that
| signal is worthless. The fact that convictions happened as a
| result of this is a crime. This is the UK and not the US so sadly
| that compensation will probably be limited to nonexistent.
|
| As an aside, this is exactly why electronic voting should be
| outlawed. You need paper ballots (that can be counted
| electronically) as a verification measure. And the fact that we
| even have to debate that makes me sad.
| throwaway823882 wrote:
| > electronic voting should be outlawed
|
| Nationally regulated, sure. Verified with a physical copy (or a
| different system), sure. But banned altogether? You might as
| well ban _everything in the world that is digital_ , as none of
| them are fool-proof.
|
| Voting isn't even that important. The wrong guy gets picked,
| what happens? Same bullshit as if the right guy got picked. If
| your choices are "Hitler" or "Jesus", then your system is just
| fucked up, and making voting fool-proof isn't the way to fix
| it.
|
| In addition, electronic voting would be a boon to democracy. It
| would provide another avenue for maligned minorities in remote
| areas be able to vote, when things like paper ballot voting in
| the middle of a pandemic might fail or be error-prone (esp.
| when a fascist fucks with the postal system), or local
| authorities enforce racist requirements like a physical ID
| card.
| cletus wrote:
| If you vote on a touch screen and it prints out a paper
| ballot, that's fine as long as that's a legible ballot, like
| not just a QR code or something. The voter should have
| confidence in the output.
|
| Likewise, if you use a pen or pencil to fill out a ballot
| that then is counted electronically, that too is fine.
|
| In both cases there's a paper ballot as a source of truth and
| that's what's key.
| heraclius wrote:
| > Voting isn't even that important. The wrong guy gets
| picked, what happens? Same bullshit as if the right guy got
| picked.
|
| If voting is unimportant, why do you care about racist
| requirements for physical ID cards? Perhaps there might be
| some sort of connexion between the two!
| throwaway823882 wrote:
| It's more important that you are able to participate than
| what the result is. Better to have an insecure system where
| 10 million people get to vote, than a secure system where
| only 10 people get to vote.
| mcguire wrote:
| Anyone expect that Post Office Ltd. and Fujitsu will face any
| significant repercussions?
|
| Yeah, me neither.
| bennysomething wrote:
| I hope the people who served time get millions. I hope the people
| who covered it up go to prison.
| _0o6v wrote:
| A shocking injustice. Innocent people went to prison for years.
| There was clearly a cover up at Fujitsu and the Post Office, and
| those accountable should now be prosecuted.
| WarOnPrivacy wrote:
| I'd wager a stuck pig that Fujitsu was a major campaign
| contributor. If they still are, convictions are a lot less
| likely.
| DanBC wrote:
| https://www.judiciary.uk/judgments/hamilton-others-v-post-of...
|
| The judgment is blistering.
| robertlagrant wrote:
| The previous judgements sent lots of people to jail, so let's
| not congratulate the criminal justice system very much.
| gpvos wrote:
| _> In the latest chapter of one of the biggest miscarriages of
| justice in English legal history, 39 people who were prosecuted_
|
| Meanwhile in the Netherlands, ~26000 people have been branded as
| fraudsters by the tax office due to a way too strict child
| benefits law. More than 100 probably entirely innocent people
| fled the country. Even the compensation that is now promised is
| only slowly trickling towards them, and likely to be snatched up
| by debt collectors - including even the tax office itself, which
| is still partly unrepentant. Okay, they haven't been sent to jail
| directly, but the scale of this is huge.
| toomanybeersies wrote:
| We had a similar thing happen with unemployment benefits in
| Australia [1], which arguably led to several suicides.
|
| [1] https://en.wikipedia.org/wiki/Robodebt_scheme
| lovetocode wrote:
| This is insanity. We need legislators, lawyers and judges who are
| tech competent.
| robertlagrant wrote:
| An undermentioned problem: how could something this bad have held
| up in court?
| lupire wrote:
| The idea that 700 people in the same job were all committing the
| same crime and constantly getting caught is insane. This is a
| perfect example of Orwell's description of fascist Britain, where
| the people are made slaves of the state.
| simonh wrote:
| Over several years and 20,000 post office branches it's not a
| huge percentage. I suppose they assumed the new system was
| revealing corruption that had gone unnoticed under the previous
| system. That's in no way an excuse or justification for the
| knowing, deliberate suppression of evidence that went on here.
| lupire wrote:
| What job has 3.5% rate of criminal prosecutions, with 0
| eyewitness evidence?
| drcongo wrote:
| It's well worth listening to the radio show linked at the bottom
| of the article to understand just how heartbreaking this story
| is.
| LatteLazy wrote:
| The Real travesty here is that people can't afford to pay for
| their lawyers (let alone a software expert or QA to actually look
| at the code or test it) , they aren't entitled to representation,
| so they have no option but to plead guilty.
| PaulKeeble wrote:
| Software is in the walls. At some point legislators are going to
| come and ask the question how we stop things like this happening
| and if the Fujitsu's of the world don't have an answer then we
| can expect regulation that will likely embed practices that don't
| help.
|
| I don't think we take software reliability seriously enough, most
| of our focus is on speed of release, ever quicker cycles and it
| being OK to break things. This culture ruined these peoples
| lives. Things must change. This isn't a unique issue to Fujitsu
| it is something most of the software industry is doing, this
| story could be about just about any piece of software.
| jjk166 wrote:
| > I don't think we take software reliability seriously enough,
| most of our focus is on speed of release, ever quicker cycles
| and it being OK to break things. This culture ruined these
| peoples lives.
|
| I think people see a false dichotomy between making things
| quickly and making them safe. The fact is in the development of
| any complex thing, you're going to have bugs, and generally
| that's okay. But things should be designed to fail safe. Making
| something that throws errors when something unexpected happens
| is actually faster and easier than trying (and possibly
| failing) to handle edge cases; had Fujistu taken that simpler,
| easier approach then all this pain and suffering would have
| been avoided.
| DoubleGlazing wrote:
| > I don't think we take software reliability seriously enough,
| most of our focus is on speed of release, ever quicker cycles
| and it being OK to break things.
|
| This drives me up the walls. At my last job (food ordering
| startup the CEO had the attitude that releasing code that was
| 95% functional was Okay, remaining issues could be fixed as we
| went along.
|
| As a result, one developer overlooked a bug that cost the
| company EUR300,000, loyalty discounts weren't being deducted
| from payments to take-aways. They then had the cheek to demand
| take-aways pay them back.
|
| Then they launched a major upgrade to the system at 5pm on a
| Friday - two hours before their busiest time of the week. It
| collapsed a few hours later and it was impossible to roll back
| because they didn't include a roll-back SQL script for the DB.
| It took till the following Tuesday to fix it.
|
| The DB schema was all over the place and as a result it was
| slow. Entity Framework couldn't handle it and the SQL it was
| generating was terrible. Me being the only one with decent SQL
| knowledge had to replace all the bad EF queries with raw inline
| SQL.
|
| Despite this, they still carried on deploying without a care in
| the world. I was told to stop moaning about QA. We didn't have
| QA or testing staff, the CEOs attitude being why pay for QA
| staff when our clients will do it for free?
| coldcode wrote:
| I worked at a place that internal customers complained QA
| took too long, so IT said fine, we won't do any. Then they
| complained the software didn't work... people sure can be
| stupid.
| detaro wrote:
| Indeed, getting rid of QA instead of improving it just
| because people think it's slow is indeed kind of stupid.
| lbriner wrote:
| How do you know this culture existed 20 years ago when this
| system was developed? It is almost certain that a corporate
| developer in the 1990s would be 100% waterfall.
|
| The issue is very often related to massively complex corporate
| requirements (the Post Office makes me cringe, even today, with
| the complexity of their postal system) and then coupled with
| the ever-present need to keep costs low, especially when
| designing something so complex.
|
| I doubt anyone building this thought it would be OK to break
| things!
| rajin444 wrote:
| > I don't think we take software reliability seriously enough,
| most of our focus is on speed of release, ever quicker cycles
| and it being OK to break things.
|
| This is extremely domain dependent, and should be handled as
| such. And in some cases it already is - look at the testing /
| verification space shuttle code goes through vs your friends
| cat video side project website.
| lupire wrote:
| Throwing innocent people on jail based on lies (with bonus,
| corrupt government officials colluding with foreign entities)
| is the problem here, not software bugs.
| WarOnPrivacy wrote:
| Software-based convictions are all about leveraging black box
| propriety, to hide the flaws that boost conviction numbers.
|
| and
|
| 'Justice' is only used ironically now.
| ccsnags wrote:
| Software bugs happen. The trick is to have proper management
| of the release that takes into account the inevitability of
| bugs while incentivizing bugs to be identified and fixed
| without the stakeholders of the project being in a position
| to have to defend a project as if it is perfect.
|
| I cannot imagine how it must have felt being under the boot
| of an entire government and it's corporate partners due to a
| bug. This is why we are important. A poorly managed IT system
| with bad incentives puts lives in danger. It is a literal
| threat to the safety of society. This cannot be stressed
| enough.
| srswtf123 wrote:
| > At some point legislators are going to come and ask the
| question how we stop things like this happening
|
| If the problem is the software, then _use less software_.
| Perhaps we shouldn 't simply take it as a given that moving
| processes into software isn't always the right move?
| simplerman wrote:
| > I don't think we take software reliability seriously enough,
| most of our focus is on speed of release, ever quicker cycles
| and it being OK to break things. This culture ruined these
| peoples lives. Things must change. This isn't a unique issue to
| Fujitsu it is something most of the software industry is doing,
| this story could be about just about any piece of software.
|
| This won't change until executives go to jail.
|
| A few years ago, we were fighting against tight deadline and
| skipping unit tests, QA, processes, etc. Someone brought up one
| of the recent major breach (Equifax?). Developers started to
| say that people will go to jail. Basically, devs were using
| this breach to imply that they will personal responsibility for
| releasing a product that might have security flaws. Our
| director laughed and said no one will go to jail and if our
| product ever got in trouble, they will personally take
| responsibility.
| jedimastert wrote:
| > I don't think we take software reliability seriously enough,
| most of our focus is on speed of release, ever quicker cycles
| and it being OK to break things.
|
| The phrase "move fast and break things" should be seen as
| cautionary, not aspirational.
| ChrisMarshallNY wrote:
| _> I don 't think we take software reliability seriously
| enough, most of our focus is on speed of release, ever quicker
| cycles and it being OK to break things. This culture ruined
| these peoples lives. Things must change. This isn't a unique
| issue to Fujitsu it is something most of the software industry
| is doing, this story could be about just about any piece of
| software._
|
| Damn straight. I'm _really_ big on software Quality. It 's kind
| of my driving passion.
|
| It has been my experience, that an attitude of Quality is
| actively discouraged in today's "rush to a crappy, lashed-
| together-with-baling-wire-and-bandaids MVP" SV culture.
|
| We glorify and make heroes of those that deliberately publish
| garbage, but make money doing so.
|
| When we look to an industry to police itself; it never does.
| But the rules and regulations applied from non-domain-expert
| politicians are often ineffective, burdensome, and really only
| apply to a bygone era (See ISO 9001/CMMI).
| robertlagrant wrote:
| Actually there's a huge amount of self policing. Engineers
| are the ones at the forefront of inventing and tooling more
| ways to test.
| ChrisMarshallNY wrote:
| Yes, and no.
|
| We have some marvelous CI/D tools at hand, but the execs
| are the ones that push to release before ripe, and they
| won't let things like auto-test failures get in the way of
| MVP.
|
| There was a comment here, some time ago, that was made by
| someone that proclaimed themselves to have started and
| successfully exited a number of companies. It went
| something like _" If you do not get physically sick,
| looking at the code in your MVP, you are spending too much
| time, worried about code quality."_
|
| I think that's a pretty good summary of today's startup
| zeitgeist.
| 8note wrote:
| I think the bugs left in by the SV culture are less important
| than the one they do fix.
|
| The most important bug is that the software doesn't solve the
| problem that you have. It doesn't matter how reliably it
| doesn't solve your problem
| ChrisMarshallNY wrote:
| _> The most important bug is that the software doesn 't
| solve the problem that you have._
|
| And we should add:
|
| _Unless we can 't do so without introducing any additional
| problems, while solving that problem in a manner that
| truly_ solves _it; as opposed to making it_ appear
| _solved._
|
| We really are often best off, with the problem, if the cure
| is worse than the disease.
|
| When I was younger, we had a saying:
|
| _To err is human, but it takes a computer to really fuck
| things up._
| Uberphallus wrote:
| It's been 15 years since I've seen CMMI mentioned, and I was
| glad I hadn't.
| quercusa wrote:
| Do you object to CMM as a model or just the CMMI/9001
| industry?
| Uberphallus wrote:
| Both. It's very well geared towards maintaining a certain
| standard of quality and predictable project throughput in
| rather well defined projects, and it certainly makes the
| job easier in procurement, but it's totally detached from
| what the SWE world is outside of that.
|
| I can see the point of such models in certain areas, like
| military, aerospace, naval, or, to stay on topic,
| Horizon, where dev is outsourced, somewhat critical,
| specs rather set in stone, and non experts need to
| measure how capable an organization is to deliver, but
| for anything else it just feels like unnecessary meta-
| management that brings significant organisational and
| development overhead.
| ChrisMarshallNY wrote:
| They had a good idea, but they applied "old world"
| thinking to it.
|
| The single biggest issue with software development, is
| that it is _incredibly_ dynamic.
|
| Static solutions don't work, and CMMI is a _very_ static
| solution. Sadly, a lot of quality practices are static.
|
| Dynamic solutions are _really_ difficult to get right,
| and tend to depend on a lot of hard-to-quantify
| variables, like the experience and talents of individuals
| on a team.
|
| For example, I am quite good at designing fairly complex
| systems, as long as I am doing it alone. I can hold some
| fairly ambitious designs in my head; which allows me a
| great deal of flexibility. I can start with a fairly
| "fuzzy" architectural model (I call it my "napkin
| sketch"), and begin a project fairly quickly. As the
| project progresses, I can apply some massive structural
| changes, and pivot fairly easily.
|
| However, the minute I need to communicate this plan, the
| whole shooting match comes to a screeching halt.
|
| Team overhead is a really big deal, and I believe it is
| seldom factored into our plans, in any kind of realistic
| manner.
| lupire wrote:
| Maybe don't use an "incredibly dynamic system" as
| evidence in criminal cases, then.
| ChrisMarshallNY wrote:
| Yup.
|
| AI is gonna pour rocket fuel on this stuff. There's
| already a great deal of talk about replacing lawyers with
| AI.
| onlyfortoday2 wrote:
| as a QA Tester THIS IS VERY TRUE
|
| agile is a terrible way of working
| mnw21cam wrote:
| I still think https://xkcd.com/2030/ has to be taken seriously.
| You can put a whole load of verification effort into your
| software, which will undoubtedly make it more reliable. But you
| are still likely to have some kind of corner case where it
| breaks down. Software is complex enough for this to be
| universally true.
|
| The key is how we respond when the software fails. The
| https://en.wikipedia.org/wiki/Therac-25 case shows an example
| of what not to do - when hospitals started reporting their
| machines giving lethal radiation doses to people, the
| manufacturer doubled down on the computers-are-infallible
| rhetoric, where they should have put every last effort into
| investigating. Likewise, the post office should have noticed
| that a rather excessive number of postmasters were apparently
| fiddling the books, and investigated. Instead, after it was
| fairly obvious that the computer was wrong, they pushed the
| computers-are-infallible line right through the courts, and
| that is what earned them the "affront to justice" judgment.
| zentiggr wrote:
| Something about "pride goeth before a fall" and "the one
| thing you can expect a manager to do is whatever shields them
| from liability".
| Rexxar wrote:
| I agree with the sentiment but the example taken for software
| in this xkcd is wrong. There is a fundamental problem of
| trust when using software for voting systems that is not
| linked to the reliability of software but to the nature of
| voting systems and the properties we want.
| michaelt wrote:
| Well, there are two problems and which is the fundamental
| one depends on your prior assumptions.
|
| Some would say it's impossible to build a secure electronic
| voting system, _even if your supplier and their employees
| were completely trustworthy_ because between physical
| tampering, state-level adversaries, the state of the art in
| software development and the impossibility of proving a
| negative, such security has never been seen before.
|
| In other words, that it's an unsolvable technical problem.
|
| Others would say it's impossible to build a secure
| electronic voting system _even if we were capable of
| creating flawless bug-free and tamper-proof software and
| hardware_ because the supplier will always be able to
| introduce undetectable bugs if they want to, and no
| supplier can ever be perfectly trustworthy.
|
| In other words, that it's an unsolvable social problem.
| kosievdmerwe wrote:
| Yeah, electronic voting is essentially like having a
| person in the voting booth that you have to tell your
| vote and trust that they will tally it correctly. [1]
|
| It doesn't matter whether voting machines are actually
| secure, they probably mostly are right now, but whether a
| layperson can have faith in the system.
|
| Paper voting is very secure if you involve people from
| opposing parties in the process and attacks are not very
| scalable. Most people can think of and understand
| mitigations for certain kinds of attacks. And if paper
| voting is too expensive for your country, you have bigger
| issues. [2]
|
| [1] https://www.youtube.com/watch?v=LkH2r-sNjQs
|
| [2] That said, I don't see how secure electronic voting
| can possibly be cheaper than paper voting. For voting
| machines to be secure, you have to manufacture them in a
| very audited manner, with little to no foreign sourcing
| of parts, you can't leave the machines unattended for
| long periods of time (aka, reusing them between elections
| is probably a no-go) and you have to build them in manner
| that is secure against voters tampering with them in
| their private booth.
| Silhouette wrote:
| _The key is how we respond when the software fails._
|
| I agree, but if the first step to solving a problem is
| understanding that it exists then the first principle here
| must be to acknowledge that software systems are fallible and
| therefore any surprising or reasonably contested result they
| produce should be treated with proper caution until further
| information can be gathered.
|
| So many of the problems we see when modern technology goes
| wrong start with assuming it didn't. At that point, it's not
| even about how you respond to the failure, because you're
| denying that the failure ever happened. Big software
| companies with considerable lobbying power seem to be
| particularly good at convincing people who aren't technical
| experts, including most politicians, judges, juries and
| reporters, that this is the case.
|
| A corollary to this is that we desperately need more
| technological awareness among our politicians, lawyers,
| journalists and other relevant professions. Tech has become
| too big to be a minor issue you delegate to some random
| advisor in a basement office. It affects almost everything we
| do today, sometimes profoundly, and failing to understand
| that will inevitably lead to some horrible outcomes as we've
| seen all too vividly today.
| marcinzm wrote:
| This seems more an issue of bureaucratic incentives than
| software. Fujitsu wanted to hide bugs to look better for future
| contracts. The Post Office wanted to hide bugs to deflect blame
| from central leadership and be able to scapegoat people at
| will. The judicial systems seems to have either not cared or
| had incentives for some quick prosecutions.
|
| Software doesn't exist in a vacuum and software will never be
| perfect. Trying to solve systematic problems by holding one
| part to impossible standards will just make things worse rather
| than better.
| neolefty wrote:
| Yes, it seems clear that people knew about these problems --
| they were _obvious_ at one level of management -- and they
| worked together to cover them up.
| ClumsyPilot wrote:
| Heads have to start rolling for this, or we will end up in
| a dystopian nightmare where any corporate organisation can
| ruin your life for no reason.
|
| It will be like USSR except more unpredictable because it
| can come from any direction
| zentiggr wrote:
| I think we're already over the edge of that, it's more
| urgent than you think.
|
| A couple of insensitive Facebook posts gets you dropped
| from consideration for a job... no matter how long ago
| and how much you may have matured in the meantime.
|
| Google implements FLoC and cohorts start identifying
| political leanings, medical conditions, mental health
| issues, anything that's legally potentially
| discrimination territory... how do you know that someone
| deduced a cohort topic and denied you <something> based
| on that...
|
| Tip of the iceberg. Data aggregators already have opaque
| records on probably everybody alive, just find the one
| with data about your person of interest.
|
| This needs to be a complete change of awareness and
| ethics and global law... otherwise we're going to have
| the movie "The Circle" come completely true as opposed to
| being just around the corner.
| serial_dev wrote:
| > At some point legislators are going to come and ask the
| question how we stop things like this happening
|
| Bob Martin talks about it a lot, how the software developers of
| the world need to have an "oath", like the hippocratic oath.
| Two posts that summarize things well (but there might be more
| where he talks about these things)
| https://blog.cleancoder.com/uncle-bob/2011/01/17/software-cr...
| https://blog.cleancoder.com/uncle-bob/2015/11/18/TheProgramm...
| PaulKeeble wrote:
| Doctors don't follow the hippocratic oath in practice, it
| just isn't a real consideration. If they did none of these
| covid long haulers or ME patients would have been tortured
| into worse conditions, nor would all those mentally ill
| patients have been locked up. Medicine treats the oath like
| software developers treat most best practices that reduce bug
| counts, as a nice to have but no one has time for.
| throwaway210222 wrote:
| "software developers... need to have an "oath", like the
| hippocratic oath."
|
| More importantly, the employers of software engineers must
| have ZERO option to emply a software engineer (anywhere on
| earth) that doesn't have the same oath.
|
| Doctors have a monospony on their services that makes their
| oath work: the hospital manager cannot just go hire un-oathed
| doctors.
|
| Never going to happen in software. Ever.
| vageli wrote:
| The practice of medicine was not a licensed endeavor at its
| inception, and that changed over time. With that in mind,
| what makes you say "Never going to happen in software.
| Ever."?
| vlovich123 wrote:
| An oath isn't going to do anything without any way to enforce
| it legally. The Hippocratic oath is neat but the real teeth
| are in enforcement against malpractice like civil and
| criminal lawsuits and a licensing body. You see similar
| things for lawyers and certain engineers (in commonwealth
| countries "professional engineer" is a restricted title like
| Lawyer or MD). Note that just doing that won't solve all
| problems either. These licensing bodies regularly publish
| enforcement actions, so malfeasance continues. Nominally they
| can help whistleblowers but, as with all regulatory bodies,
| there's always a risk of regulatory capture making such
| actions still peril-filled.
|
| Moreover it's not even clear this particular work even fall
| under traditional definitions that would required a licensed
| engineer as those deal with public safety (bridge
| construction, buildings, etc) and something like this doesn't
| really. We'd need an updated definition that takes into
| account the software needs of the world (privacy and
| security, etc).
| mavhc wrote:
| They made a computer that can't add
| quickthrower2 wrote:
| A reminder to people who think they are safe from their
| government because they've "done nothing wrong" or "have nothing
| to hide"
| whyleyc wrote:
| There's a great 10 episode Podcast on this debacle on BBC Sounds:
|
| https://www.bbc.co.uk/sounds/series/m000jf7j
|
| It's really well paced and includes contributions from many of
| the sub-postmasters affected by this scandal.
| gerjomarty wrote:
| I hadn't heard about the story until the BBC started re-running
| this series this week. Absolutely shocking that flaws in the
| system were dismissed and suspicion thrown on the sub-
| postmasters instead.
| blfr wrote:
| Was there actual wrongdoing that the buggy system allowed and
| made difficult/impossible to trace or was it bugs all the way
| down?
| fitblipper wrote:
| Crappy software sends people to prison. Crappy software keeps
| people in prison
| (https://www.techdirt.com/articles/20210222/12462746295/arizo...)
| mariuolo wrote:
| > The Post Office settled the civil claim brought by 555
| claimants for PS57.75m - amounting to PS12m after legal costs -
| without admitting liability
|
| That's some PS20'000 each. A pittance for years of suffering and
| inability to work.
| noja wrote:
| > software engineer Richard Roll
|
| Risky click.
| gm3dmo wrote:
| Never gonna give you up.
| cabernal wrote:
| This and the John Deere bug posted earlier make me a bit
| concerned over the accumulating evidence of unreliable software
| ruining people's lives...
|
| What can be done? Mandatory audits, pen testing?
|
| If this is an organizational problem, more vacation? limiting
| overtime? rethinking employee incentives?
| danpalmer wrote:
| Pentesting and auditing aren't great solutions here. They can
| be useful on small scopes but a big system like this, it's
| unlikely to be hugely impactful - it will find things, but who
| knows if it finds enough.
|
| In the UK in the wake of the 2008 banking crisis, a number of
| positions in banks became criminally liable for issues under
| them. If you're director-level or above (I think?) then you may
| be ultimately put in prison for negligence or issues like that
| which occur in your department. This is rare, not sure if it's
| been used yet, but it effected a cultural change in consumer
| banking as a bunch of execs suddenly had their necks on the
| line if someone under them did something wrong. I don't believe
| this is too hard-line in practice, I think a defence is "look
| at all these reasonable steps we take, we couldn't have
| foreseen this", but it had the impact (source, a good friend of
| mine is bordering on this level in a UK bank).
|
| I wonder if a similar thing could work in a wider way across
| more industries - not with the intention of criminally
| punishing lots of people, but with the aim to change the
| culture around responsibility to the public and other
| stakeholders in the work that we do.
| Chris2048 wrote:
| Standards. Just say certain things, payment systems, need to
| meet certain levels of auditability (does it record all
| relevant data, and can I see them after the fact), verification
| (is the data correct and can I prove that) and privacy.
| icegreentea2 wrote:
| It's not about positive incentives, it's about the lack of
| negative incentives. More true negative incentives need to be
| shifted onto the production side, back onto the corporations,
| its officers, its middle management, and if required down to
| the individual contributor.
|
| Corporate structure helps diffuse and deflect responsibility.
| Each group (executive leadership, middle management, and ICs)
| gets to diffuse and deflect responsibility and liability onto
| each other.
|
| We already have all the positive incentives in the world - cash
| money. It's not enough.
| viraptor wrote:
| > What can be done?
|
| Not taking software results as a fact. Software report stating
| X in court should be equivalent to "the person who wrote this
| in a hurry would say X, but it's not a sworn testimony".
|
| We should have the person presenting any report like that be
| personally responsible for the contents. If they aren't
| willing, it shouldn't be presented.
| Silhouette wrote:
| _We should have the person presenting any report like that be
| personally responsible for the contents. If they aren 't
| willing, it shouldn't be presented._
|
| I don't think making it personal works at scale. You can't
| reasonably expect everyone giving evidence in court, say
| every individual police officer who is a witness to a
| speeding offence, to be a technical expert on the
| technological tools they are given to do their job.
|
| Instead, as you implied in the previous paragraph, the weight
| given to any evidence derived from technology should be
| proportionate to the credibility of that technology. If it's
| a device that has to be vetted and approved according to
| strict regulatory standards and in court there are two other
| concurring sources of evidence, that's clearly a much
| stronger case than a single reading from a single device
| whose calibration has reasonably been called into question at
| trial that is being presented as the only evidence in that
| trial.
| viraptor wrote:
| > say every individual police officer who is a witness to a
| speeding offence, to be a technical expert on the
| technological tools they are given to do their job.
|
| That's what I was going for. If the officer doesn't
| understand the limitations of their tool, they shouldn't
| testify in court beyond "I pointed it that way and read the
| number, as trained".
|
| There are existing cases where the speed reading is
| contested because the handheld speed cameras can move
| slightly and bounce first off the side mirror then off the
| reg plate giving you "extra speed".
|
| My point was that if you say "that person was speeding" you
| should be responsible for that statement afterwards, but
| you can say "I used the provided tool and got reading X",
| at least the doubt is there.
| Silhouette wrote:
| FWIW, I'm reasonably sure that's exactly what does
| normally happen in that particular case. Police officers
| sometimes speak in a slightly stilted way in court here
| in the UK, partly because they use words carefully chosen
| to be statements of fact as they know them and not to
| draw conclusions that are a matter for the court to
| decide.
| reedf1 wrote:
| Anyone have more technical detail on the software or the bugs
| therein?
| [deleted]
| londons_explore wrote:
| Presumably to put someone in prison for being a money thief, one
| would need to prove where that money went...
|
| Were all these people accused of theft with not a single record
| of the yachts they bought with all the money they supposedly
| stole?
|
| I would assume most of these people would be able to turn over a
| complete financial record of their lives (ie. I was paid PSx, I
| paid taxes of PSy, and here is a bank statement showing how I
| spent it, and here is whats leftover). How exactly can you
| imprison someone for theft of money if they can present that?
| zinok wrote:
| Post Offices handle a large amount of cash, much more than any
| other business of their size. Many of the sub-post offices in
| question would be paying out pensions and welfare benefits in
| cash to a large proportion of local customers. If someone was
| stealing from the post office, they could easily do so in cash.
| ClumsyPilot wrote:
| So 500 peiple stole millions and the prosecution cannot show
| where a single penny went, noone even got a new car or TV?
| Did they eat the money?
|
| And all the evodence the prosecution has are electronic
| records, entirely in their control, which they could fake and
| which were never checked by a third party for basic errors?
| This is a colossal miscarriage of justice
| lupire wrote:
| It's hard to spend $70K in cash, though.
| zinok wrote:
| Perhaps it is, but if there had been credible evidence of a
| theft 'I would not have been able to spend all that money
| in cash' is not the basis of a solid defense.
|
| There have been genuine cases where accountants, bank
| managers, and so on have embezzled large sums of money,
| including in cash, and spent it all untraceably on things
| like feeding a gambling addiction.
| switch007 wrote:
| For background and more information, Private Eye Special Report:
| "JUSTICE LOST IN THE POST: How the Post Office wrecked the lives
| of its own workers" (PDF) https://www.private-
| eye.co.uk/pictures/special_reports/justi...
| switch007 wrote:
| Typical BBC, not mentioning the man who committed suicide.
| unpopularopp wrote:
| >So far, nobody at the Post Office or Fujitsu has been held
| accountable
|
| And this is the most important part.
| WarOnPrivacy wrote:
| See, now I'm looking at US & UK Gov corruption and can't tell
| who is mimicking who.
| jibbit wrote:
| Such a terrible story. I'm surprised it hasn't been more
| prominent within the tech community. Many dozens of lives were
| ruined.
| _joel wrote:
| Unfortunately people died during this time too and did so with
| this hanging over their head. An absolute scandal, but there's
| no inquiry into the directors involved. Not yet, at least.
| spacemanmatt wrote:
| I just came from another thread (here) where the subject was
| Google arbitrarily ruining businesses and lives based on
| algorithmic fraud detection gone wrong. I'd estimate the issue
| is alive with U.S. techies at the very least.
| handelaar wrote:
| Of _course_ it 's Fujitsu, purveyor of nearly every nonfunctional
| hit-and-run government IT contract in the UK and Ireland.
|
| As far as I can gather this malignancy escapes permanent legal
| destruction primarily by shedding all of its staff every 20
| minutes
| cmsefton wrote:
| Private Eye magazine (a satirical investigative news magazine)
| has covered this for many years, and have an excellent report for
| anyone interested: https://www.private-
| eye.co.uk/pictures/special_reports/justi... [PDF]
|
| Glad to see them finally have their names cleared, and can only
| hope prosecutions will follow as a result, utterly shameful how
| the Post Office, Fujitsu and others behaved. For example:
|
| > A Fujitsu programmer from the time, Richard Roll, who would
| become a key witness in the sub-postmasters' high court case
| against the Post Office in 2019, told the Eye that Horizon was
| one the company's few profitable contracts. Among other private
| sector deals, it was also lining up a key role in the mother of
| all government IT splurges, New Labour's PS12bn NHS IT project
| (Eyes passim ad nauseam). Fujitsu could ill-afford either bad
| publicity or the penalties that came with software faults. "We
| would have been fined," said Roll, who worked at the company
| between 2001 and 2004. "So the incentive was to pretend it
| [software error] didn't happen", while running "a constant
| rolling programme of patches to fix the bugs". Fujitsu "would
| basically tell the Post Office what they wanted to hear". So
| prolific did Roll's bug-fixing team become it won the company's
| President's Award for outstanding corporate contribution in 2002.
| And the quick-fix, ask-no-questions approach that suited Fujitsu
| financially enabled the Post Office to hold the line that blame
| for all branch shortfalls must lie with the sub-postmaster.The
| Fujitsu insider concluded that errors leaving sub-postmasters out
| of pocket were inevitable. Could that mean hundreds of them?
| "Given there were [about] 20,000 post offices when I was at
| Fujitsu and the sort of problems we were dealing with all the
| time, yeah," he told the Eye. "Sounds reasonable."
| justincormack wrote:
| The judgement is a good (long) read
| https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-...
| milonshil wrote:
| https://ustreama.com/2021/04/23/ufc-261-live-stream-how-to-w...
| dang wrote:
| Related articles that submitters and commenters have pointed out:
|
| https://www.bbc.co.uk/news/business-56859357 (also from today)
|
| https://www.computerweekly.com/news/252496560/Fujitsu-bosses...
|
| https://www.private-eye.co.uk/pictures/special_reports/justi...
| [pdf]
|
| https://www.bbc.co.uk/sounds/series/m000jf7j [podcast series]
|
| Some past related threads - pretty sure there have been others:
|
| _UK Post Office: Error-laden software ruined staff lives_ -
| https://news.ycombinator.com/item?id=26905528 - April 2021 (3
| comments)
|
| _UK legal system assumes that computers don 't have bugs_ -
| https://news.ycombinator.com/item?id=25518936 - Dec 2020 (24
| comments)
|
| _Post Office scandal: Postmasters celebrate victory against
| convictions_ - https://news.ycombinator.com/item?id=24661321 -
| Oct 2020 (2 comments)
|
| _Faults in Post Office accounting system led to workers being
| convicted of theft_ -
| https://news.ycombinator.com/item?id=21795219 - Dec 2019 (103
| comments)
|
| _Post Office hires accountants to review sub-postmasters '
| computer claims_ - https://news.ycombinator.com/item?id=4143107 -
| June 2012 (1 comment)
| lambda_dn wrote:
| What's more likely, hundreds of Postmasters where thieves or the
| system had a few bugs. How did this even happen?
| vanilla-almond wrote:
| _Repeating this comment that I posted yesterday...it is unfair?_
|
| Will any developers involved in this horrible scandal ever will
| be held accountable for their work?
|
| I wonder if the developers who were responsible for such a bug-
| infested piece of software realise their work has destroyed
| people's lives? (They presumably never met the users of their
| software or were so distant from end-users that they never
| considered the consequences of their actions.)
|
| Do those developers even realise it was their incompetence that
| caused untold misery? Or are they completely detached from the
| events in this scandal and see themselves as simply cogs in the
| 'system' and thus blameless?
|
| Blame must be apportioned to management. But also I feel it's too
| easy as a developer to see yourself as part of a team and thus
| absolved of any individual blame. You're subsumed in the "team" -
| and ultimately no-one takes responsibly.
|
| Even with management at fault, one cannot deny that it was the
| developers who produced absolute garbage.
|
| I hope the developers who worked on this system, no matter how
| much they feel they are not responsible for the failure of this
| project, will reflect on how the impact of software they built
| had devastating consequences on people's lives.
| segmondy wrote:
| I'm saving this article to show developers that your software can
| ruin lives. It doesn't have to be used in aerospace or health
| care to matter.
| martingoodson wrote:
| I'm sure it's a coincidence that Fujitsu was also heavily
| involved in the NHS IT fiasco which cost the NHS PS10B. 'the
| biggest IT failure ever seen'. The Fujitsu UK chairman is also a
| large Conservative party donor of course - also a complete
| coincidence. https://www.vice.com/en/article/59x7wz/fujitsu-uk-
| sues-depar...
| temporama1 wrote:
| As: a postmaster
|
| When: I use this software
|
| Then: I should not be falsely imprisoned for 3 years.
| temporama1 wrote:
| Tough crowd
| haunter wrote:
| https://www.computerweekly.com/news/252496560/Fujitsu-bosses...
|
| >For the first 10 years of Horizon's existence, transaction and
| account data was stored on terminals in each branch before being
| uploaded to a central database via ISDN. Our source says this
| part of the system simply did not work.
|
| >"The cash account was a piece of software that sat on the
| counter NT box, asleep all day," he said. "At the end of the day,
| or a particular point in the day, it came to life, and it ran
| through the message store from the point it last finished. It
| started at a watermark from yesterday and combed through every
| transaction in the message store, up until the next watermark.
|
| >"A lot of the messages in there were nonsense, because there was
| no data dictionary, there was no API that enforced message
| integrity. The contents of the message were freehand, you could
| write whatever you wanted in the code, and everybody did it
| differently. And then, when you came back three weeks later, you
| could write it differently again."
|
| And down further
|
| >Speaking to Computer Weekly in 2015, the anonymous source told
| us: "The asynchronous system did not communicate in real time,
| but does so using a series of messages that are stored and
| forwarded, when the network connection is available. This means
| that messages to and from the centre may trip over each other. It
| is perfectly possible that, if not treated properly, messages
| from the centre may overwrite data held locally."
|
| >Four years later, former Fujitsu engineer Richard Roll wrote in
| a witness statement to the High Court: "The issues with coding in
| the Horizon system were extensive. Furthermore, the coding issues
| impacted on transaction data and caused financial discrepancies
| on the Horizon system at branch level."
|
| BUT the most important part
|
| >So far, nobody at the Post Office or Fujitsu has been held
| accountable
| TheOtherHobbes wrote:
| That's not (even) the most important part.
|
| The most important part is that the PO used these actions to
| claw back "stolen" money from its postmasters. This money
| appears to have ended up in its profit and loss account.
|
| If true this means that instead of the postmasters stealing
| from the PO, _the PO was stealing from its postmasters._
|
| There's been at least one claim - in the Daily Telegraph, so
| questionably credible, but never mind - that a document exists
| proving that senior management were aware that the accusations
| against postmasters were untrue, but carried on regardless.
|
| If that document exists it changes the narrative from
| accidental tech failure and management incomprehension to
| something less wholesome.
| DaedPsyker wrote:
| BBC (broadcast so don't have link) said that under the
| previous CEO that an investigation was shelved into the
| accusations. Given the number of accusations I have to wonder
| if there was a cover-up.
|
| Jail sentences, bankruptcy and suicide has been caused,
| management that oversaw this need to face prosecution.
| WarOnPrivacy wrote:
| The crappy state of rural UK broadband (circa 2010) is proudly
| on display here.
|
| ref:
| https://www.ingenia.org.uk/Ingenia/Articles/c05470e5-337f-4b...
|
| Maybe the FCC went all NYPD-World-Police on the UK - popped
| over there to run things for a while.
| gm3dmo wrote:
| I don't think the state of broadband can be the cause here.
| Banks, supermarkets and even GP surgeries were able to
| support complex accounting systems or patient records for
| decades.
|
| Seems like the Futisu team running Horizon decided to
| reinvent everything badly.
|
| Much of government IT was being given to consultancies like
| Fujitsu/EDS in the 15 years since 1994. These contracts ended
| badly: https://www.computerweekly.com/news/1280096810/Why-
| did-EDS-c... especially for the public paying the bills.
|
| Martha Lane Fox and the GDS pointed out the folly of this
| approach in 2010 https://gds.blog.gov.uk/story-2010/
|
| They've done an amazing job overall, but hubris overcame them
| with things like Verify https://www.google.com/amp/s/www.comp
| uterweekly.com/news/252...
| Mauricebranagh wrote:
| Didn't have BB back then - this is the sort of application D
| Chanel was designed for.
| Mauricebranagh wrote:
| Written from the POV of a former BT billing systems developer -
| The system was designed (fucking badly) before the widespread
| existence of ADSL.
|
| This is what happens when you outsource core financial systems
| to low cost bidders with dubious tech chops building a message
| queue system is not fraking rocket science at this point.
|
| Back when I worked on the ground up billing system for Telecom
| Gold (aka Dialcom) we did this as the existing mish mash of
| dodgy code that Dialcom offered (Sorry Eric) was not up to
| standard.
|
| We had large amounts of internal auditing built in and we
| tracked discrepancies to the Penny.
| lbriner wrote:
| I think a lot of "normal" people like the idea of holding
| corporates accountable but how would that actually work?
|
| The CEO blames one of their directors; the Director blames the
| supplier; the supplier blames the requirements documentation;
| the Business Analysts blame the culture for creating confusing
| and conflicting requirements.
|
| Yes, you can hold the organisation accountable but then the
| people who worked there back then are long gone, they don't
| care if the Post Office gets fined PS500M.
|
| You only have to look at the enquiry into the flammable
| cladding scandal which was entirely down to fraud, yet, there
| are people who have not been arrested over their
| misrepresentation of their products.
| jjk166 wrote:
| At some (or perhaps more than one) point there was someone
| who was responsible for ensuring that the system put in place
| complied with requirements and that it was functioning as
| intended. They didn't do their job. They can point their
| finger any which way, but that won't absolve them of
| dereliction of duty.
| xbar wrote:
| Apply the same criminal liability that applies to boards,
| CFOs and CEOs for financial statements for all other
| statements?
| NovemberWhiskey wrote:
| That's fine, but Sarbanes-Oxley only applies criminal
| penalties for knowing or willful mis-statements.
| nitwit005 wrote:
| The problem was people lying about the quality of the
| evidence. There's nothing exotic about prosecuting people for
| lying to investigators or courts.
| rlpb wrote:
| > I think a lot of "normal" people like the idea of holding
| corporates accountable but how would that actually work?
|
| Exactly which specific problem is "holding corporates
| accountable" trying to fix?
|
| If it's that postmasters were being falsely convicted, then
| the way to fix that is to raise the burden of proof
| significantly. I hope this case has done that, and next time
| a court will not accept "computer says so".
|
| With that fixed, the corporates would have to take the
| (falsely reported) losses; they wouldn't be able to pass it
| on to the postmasters like they did. Then the consequences of
| the problem will remain with the people responsible.
|
| Is that sufficient?
| rectang wrote:
| No, it is not sufficient.
|
| The problem is that it is possible to design malicious
| systems which through incentives, ensure that illegal acts
| will take place, yet only low-level actors are ever
| punished. The people who architected the systems and made
| the decisions _statistically guaranteeing_ illegal activity
| escape punishment through plausible deniability and abscond
| with their ill-gotten gains.
|
| Besides this scandal, see the failure to punish any
| executives after the 2007 crash, or Carrie Tolstedt and
| John Stumpf of Wells Fargo who even after clawbacks retired
| tens of millions of dollars ahead, etc.
| pas wrote:
| The quick and dirty way is to somehow tie their
| power/privileges/financial-situation to that of those who
| they have power over. (And make it stick for many years.)
|
| There's a big missing culture of fixing problems in
| corporations. Which of course must start with acknowledging
| the problem. Which of course means that people reporting
| problems shouldn't face negative consequences. Which means
| that the current cultural gap is not just a nice empty void,
| it's an actively hostile roiling psychological chasm of
| corporate warfare.
|
| So if random CEO knew about some problems that actively
| harmed the employees and did nothing, and later a court says
| that the company did wrong, the CEO automatically has to pay
| some fines too.
|
| And it should be possible to share (but not completely
| delegate) this responsibility down the corporate hierarchy,
| to incentivize executives/VPs/managers/team-leads to do the
| right thing.
|
| Of course this would need a political culture that is
| motivated to develop, fine-tune and enforce such a framework.
| -\\_(tsu)_/-
| ww520 wrote:
| CEO and senior executives are paid to take responsibility of
| the actions of their subordinates, otherwise why would they
| get the big bucks?
| DanBC wrote:
| There's a short but good podcast about the trial and how it
| affected people here:
| https://www.bbc.co.uk/sounds/series/m000jf7j
| redis_mlc wrote:
| This is a similar story.
|
| When ATMs were introduced in Canada in the 70s/80s, it was common
| to believe they were infallible. When customers claimed they were
| short-changed by machines, often they were prosecuted for fraud
| or attempted theft.
|
| I'm sure HNers can think of dozens of ways a machine could be
| wrong ...
|
| https://en.wikipedia.org/wiki/Automated_teller_machine
|
| Also, regarding the Postmaster article, note that somebody
| working on that project would likely face great difficulty in
| convincing anybody there was a systems problem.
| haunter wrote:
| https://www.computerweekly.com/news/252496560/Fujitsu-bosses...
|
| >For the first 10 years of Horizon's existence, transaction and
| account data was stored on terminals in each branch before being
| uploaded to a central database via ISDN. Our source says this
| part of the system simply did not work.
|
| >"The cash account was a piece of software that sat on the
| counter NT box, asleep all day," he said. "At the end of the day,
| or a particular point in the day, it came to life, and it ran
| through the message store from the point it last finished. It
| started at a watermark from yesterday and combed through every
| transaction in the message store, up until the next watermark.
|
| >"A lot of the messages in there were nonsense, because there was
| no data dictionary, there was no API that enforced message
| integrity. The contents of the message were freehand, you could
| write whatever you wanted in the code, and everybody did it
| differently. And then, when you came back three weeks later, you
| could write it differently again."
|
| And down further
|
| >Speaking to Computer Weekly in 2015, the anonymous source told
| us: "The asynchronous system did not communicate in real time,
| but does so using a series of messages that are stored and
| forwarded, when the network connection is available. This means
| that messages to and from the centre may trip over each other. It
| is perfectly possible that, if not treated properly, messages
| from the centre may overwrite data held locally."
|
| >Four years later, former Fujitsu engineer Richard Roll wrote in
| a witness statement to the High Court: "The issues with coding in
| the Horizon system were extensive. Furthermore, the coding issues
| impacted on transaction data and caused financial discrepancies
| on the Horizon system at branch level."
|
| BUT the most important part
|
| >So far, nobody at the Post Office or Fujitsu has been held
| accountable
| coldcode wrote:
| Is there no legal support for challenging the source code of
| the product in the UK?
| moomin wrote:
| There's barely any legal support at all these days. That's
| what all that "tough on crime" and "stop waste" nonsense in
| newspapers gets you: large chunks of the criminal justice
| system barely work anymore.
| tyingq wrote:
| The link out to another story[1] has some interesting details...
|
| _" In December 2019, at the end of a long-running series of
| civil cases, the Post Office agreed to settle with 555
| claimants._"
|
| So settlements in 555 of the original 700+ prosecutions.
|
| _" It accepted it had previously "got things wrong in [its]
| dealings with a number of postmasters", and agreed to pay PS58m
| in damages. The claimants received a share of PS12m, after legal
| fees were paid."_
|
| But 80% of the settlement money went to lawyers. Ugh.
|
| [1] https://www.bbc.com/news/business-56718036
| FpUser wrote:
| Since the government was in a wrong I do not understand at all
| why they are not ordered to compensate all legal expenses as
| well.
| WarOnPrivacy wrote:
| Government gets to write the laws. That is, when lobbyists
| let them.
| FpUser wrote:
| My understanding is that the court still has the power (at
| least in theory) to order legal expense compensation.
| lupire wrote:
| The percentage isn't the problem. The problem is of the
| settlement amount doesn't include damages and also legal fees,
| both of which should be the responsibility of the perpetrators.
| tyingq wrote:
| _" The percentage isn't the problem"_
|
| I disagree. Even the ambulance chasers here in the U.S. take
| around 40% as their contingency fee. 80% is just...wow.
|
| Edit: "ambulance chasers" in this context means very
| opportunistic lawyers that are primarily motivated by money,
| and not helping their clients. I don't see how that term is
| disparaging any victims/clients. The comparison is that even
| outright greedy lawyers aren't taking half+ of the
| settlement. In this case, using PS250/hr, the lawyers spent
| 88 lawyer years worth of time (184k hours).
| lupire wrote:
| You're comparing apples to oranges. "Ambulance chasers" (a
| terrible slur that looks down on weak victims pursuing
| justice), offer their services in a competitive market. If
| they charge too much, again, that should be determined by
| having a separate pool for fees separate from damages, and
| be a dispute between the perpetrator and the lawyer, not
| the victim and the lawyer.
|
| The cost of the legal work is uncorrelated to the size of
| the damages.
|
| Limiting legal fees just makes it not cost effective to
| pursue justice for smaller damages with more complex cases.
|
| It's absurd bordering on evil to say the problem here is
| that people got paid too much for their excellent work
| (fighting against the resources of a corrupt major
| corporation and a corrupt major world government!) not that
| the perpetrators was under punished for their horrific
| crime.
|
| The heroes who saved 700 people's lives deserve the money
| more than super-wealthy psychopathic perpetrators.
| hobs wrote:
| Then why not give them 99.9% of the take if they are such
| big heroes?
|
| Because for the lawyers to get all the money each time
| harm happens means they more from harm to people than the
| people themselves benefit, this is a perverse incentive
| to keep the system exactly as it is for people who often
| become our lawmakers.
|
| This also applies to 80/20 splits.
| lupire wrote:
| This response bears no relation to the topic at hand. As
| said earlier, the damages and the legal fees are two
| separate things that shout be kept separate.
| robertlagrant wrote:
| > a terrible slur that looks down on weak victims
| pursuing justice
|
| No, you've misunderstood entirely.
| mschuster91 wrote:
| > "Ambulance chasers" (a terrible slur that looks down on
| weak victims pursuing justice), offer their services in a
| competitive market.
|
| I wonder how other countries get by without "ambulance
| chasers". The only country I know that has them is the
| US, and their existence is the sign that something is
| fundamentally wrong.
| zinok wrote:
| It was an extremely complex case which was very hard to
| prove, against companies which belong to the establishment
| and had been shown the benefit of the doubt by the legal
| system on multiple occasions.
| tyingq wrote:
| I found the actual settlement here:
| https://www.onepostoffice.co.uk/media/47518/20191210-glo-
| con...
|
| There's obviously a lot of detail there, but it does still
| feel to me like more than PS12M should have gone to the
| actual post workers. That's ~22k each.
| hourislate wrote:
| What is the restitution in these cases? Will the victims be
| compensated for their losses and will the UK Gov and Fujitsu be
| held responsible?
| gandalfian wrote:
| Unfair but as a spectator so frustratingly lacking any proper
| answers. It seems nobody could ever even work out if any money
| was missing or not. Let alone why. No closure. Just official
| judgement that no one knows...
| raesene9 wrote:
| An (IMO) Interesting question is how to reduce the risks of
| things like this happening.
|
| Where evidence from IT systems is being used as a large part of a
| prosecution, it seems that it should have some kind of scrutiny
| as to how those systems operate.
|
| One option would be allowing the defence to see details of how
| the system works, testing that was done and known bugs, but that
| would require a lot of expensive work by legal defence teams,
| especially where the system is complex.
|
| Another option would be some kind of certification of IT system
| operation, but again it would be hard/expensive to do and very
| incompatible with rapid development techniques.
| mikehollinger wrote:
| > An (IMO) Interesting question is how to reduce the risks of
| things like this happening.
|
| I look forward to finding out if this was a "fraud system gone
| wrong" or a more basic ledger system failing to do sums
| correctly.
|
| Partially addressing your question though, if you were to
| insert the words "AI" and "bias" into the sentence we as an
| industry are starting to figure this out. The certification and
| testing processes you mentioned are there in cases where a
| team's mature enough to have both a data and model lifecycle
| worked out. You see words like MLOps trying to describe how to
| do that effectively in production.
|
| For example, my work has both a design approach (in both the
| product design touchy/feely sense and software architecture
| sense) that includes questions and practices that will help to
| reason through data needed to address a problem, what can go
| wrong with that, and how things look when it goes wrong. The
| last bit is the most interesting one to me. In terms of
| practical engineering, inference results generally should have
| some sense of lineage - of data, model, and training services
| which explain how you got to a given answer, including what
| inputs were considered or ignored.
|
| An interesting side topic with this is that poor
| implementations can result in inexcusable differences that
| affect downstream systems. For example, if a particular model
| has predicted something like "this transaction is suspected to
| be fraud" it better be consistent from run to run, and the
| input data better be consistent over time. If either of those
| changed - explaining that to the consumers of the data is
| essential to them understanding that either the model changed,
| the data changed, or both.
| spideymans wrote:
| >An (IMO) Interesting question is how to reduce the risks of
| things like this happening.
|
| Corroborating evidence. In this case, _where was the evidence
| that this money was ever in their possession_? Was it ever
| sitting in their bank account? Was it buried in the back yard?
| Did they buy fancy sports cars or houses? The prospect of
| thousands of people stealing money without a trace of the cash
| is fantastical.
|
| In general, I'd say electronic evidence should need to be
| corroborated with physical or other types of evidence to
| achieve a conviction. It's too easy for electronic records to
| be falsified, either through software bugs or outright
| malicious intent.
| BillinghamJ wrote:
| I'm very sure this system was certified in a multitude of ways.
| No certification process would prevent this.
|
| The real issue here was that Post Office refused to recognise
| that, although computers themselves are mostly infallible,
| computer programs are never infallible. They conducted their
| activities and took actions based on assuming the reporting was
| flawless.
|
| Then the really serious problem is that in cases where the
| fallibility became more visible, they consistently and
| systematically covered it up and pressed forward with their
| incredibly aggressive enforcement work anyway, knowing how much
| damage it was doing.
|
| This is unquestionably an issue of abuse of power and position.
| citrin_ru wrote:
| > although computers themselves are mostly infallible
|
| What do you mean? Hardware is fallible too, just less often
| than software. This may cause problem on its own e. g. bit
| flips in non-ECC memory, HDD which lie (reply to flush cache
| before data is actually written) or HW can trigger software
| errors, e. g. HW can crash at random moment and SW can be not
| designed to handle this properly.
| simonswords82 wrote:
| Outrageous that so many people's lives were blown up by this.
| Relieved to hear the court ordered in their favour.
|
| I wonder if the post masters can now go after the Post Office for
| damages?
| mnw21cam wrote:
| The phrase "Affront to justice" is key here. To be honest, I am
| completely shocked that this wasn't sorted out several years
| ago when it was all over the papers and it was completely
| obvious what had happened. But that key phrase allows the
| wholesale claiming of damages.
| Silhouette wrote:
| It's also noteworthy that these injustices originated from
| private prosecutions brought by the Post Office. That is a
| relatively unusual legal action in this country, where almost
| all criminal prosecutions are brought by the state. Given the
| damage that a wrongful criminal prosecution can cause,
| including imprisonment and having a criminal record, the
| compensation awarded could be considerable and there is
| already talk of the Post Office needing extra government
| funding to cover the cost.
|
| Another small point of interest that doesn't seem to be
| making the mainstream reporting yet is that under our legal
| system the state prosecutor (the Crown Prosecution Service)
| has the power to take over and, if appropriate, shut down any
| private prosecution. When the inevitable inquiries publish
| their conclusions, the fact that so many bad prosecutions
| were successfully brought over such a long period might
| reflect poorly not only on the Post Office and on the courts
| and lawyers involved in the convictions but also on the CPS
| for not intervening. This could become politically
| significant, because the current Leader of the Opposition was
| in charge of the CPS around 2009-2013, the last five years
| when most such prosecutions were being brought. That could
| leave him in an awkward position if he's attacked over his
| record during the next general election campaign, given that
| his party is exactly the one that's supposed to stand up for
| working class "little guys" like the victims in these cases.
| lupire wrote:
| > the convictions of 39 former postmasters ... the UK's most
| widespread miscarriage of justice.
|
| There's no way this is true.
|
| > There were more than 700 prosecutions based on Horizon
| evidence. The commission and the Post Office are asking anyone
| else who believes their conviction to be unsafe to come forward.
|
| On second thought, I guess it may be, since even after the abuse
| was proven they are still holding innocent people on false
| charges.
| FpUser wrote:
| >"since even after the abuse was proven they are still holding
| innocent people on false charges."
|
| Well, same government first destroyed immigration papers and
| then deported and otherwise ruined the lives of their own
| citizens ( Windrush scandal ). I'd love to see the perpetrators
| in jail but fat chance.
| notimetorelax wrote:
| A lesson to test your code and take action based on costumer
| feedback. I'm curious to learn what was Fujitsu's position during
| those investigations.
| noir_lord wrote:
| Not sure I'd take action based on what someone who makes fancy
| dress/theatrical clothes suggests tbh.
| jedimastert wrote:
| What is this in reference to?
|
| Edit: I get it.
| noir_lord wrote:
| > costumer
|
| I think he/she meant customer, I found the idea of someone
| who makes fancy dress giving technical feedback amusing.
| frameset wrote:
| The typo of "Costumer" where they probably meant
| "Customer".
| [deleted]
| meowster wrote:
| duplicate -ish
|
| 175 and 53 comments also posted 3 hours ago:
| https://news.ycombinator.com/item?id=26913183
___________________________________________________________________
(page generated 2021-04-23 23:00 UTC)