The Worked Example

On the first night of the parallel run, the new engine disagreed with the old one 4,116,802 times.

I had built the comparison harness myself, so my first assumption was that the harness was wrong. It wasn’t. Every live award in the caseload had been calculated twice — once by ENTITLE, the Department’s engine of record since 1987, and once by the replacement — and wherever the two figures differed, the harness wrote a line to a file. The file was four million lines long. I got about forty lines in before I saw the shape of it.

CASE 00047112   OLD  8901   NEW  8902   +1
CASE 00047115   OLD 12207   NEW 12208   +1
CASE 00047118   OLD  6034   NEW  6035   +1

Weekly amounts, in pence. Every disagreement was exactly one penny, and the old engine was always the lower.

A defect with a signature that clean is a gift; it took one morning to find. The regulation says the weekly amount is the annual amount divided by fifty-two, rounded to the nearest penny. The replacement does exactly that. ENTITLE does this:

DIVIDE WS-ANNUAL-PENCE BY 52
    GIVING WS-WEEKLY-PENCE.

COBOL truncates unless you tell it otherwise; rounding costs a keyword, ROUNDED, and the keyword isn’t there. So ENTITLE keeps the quotient and discards the remainder, and for the half of the caseload whose annual amount leaves a remainder of twenty-six pence or more, discarding the remainder and rounding it disagree — by a penny, always downward. Four million claimants, a penny a week short, since the ninth of March 1987.

Above the division there was a comment.

* AMENDED 12/02/87 M.H. --
* 'ROUNDED' REMOVED PER MEMO 85/223 (MR PEMBERTON)

The programme director read that twice and said: if we’re going to stand in front of the Permanent Secretary and tell him the Department has been doing its sums wrong for fifty-two years, I want to know why first. Find M.H.

M.H. was Maureen Hale, who joined the DHSS at Longbenton in 1984 and retired from the Department in 2009. We offered her a call. She insisted on coming in; she said she wanted to see the code again, the way you’d want to see a house you’d lived in. In the meeting room she put on her glasses, read the printout for a while, and spoke before anyone had asked a question.

“I wrote ROUNDED. I’ll tell you that first. I wrote it the way the regulation said, and it failed acceptance.”

Acceptance, in 1987, was Memo 85/223: two typed pages from J. S. Pemberton, Assistant Secretary, setting out the calculation for the new computerised system. The memo said what the regulation said. Then it did what specifications have always done, which is to add an example, so that the people downstream would have something to check against.

The weekly amount is found by dividing the annual amount by 52 and rounding to the nearest penny.

Example: annual amount £2,341.00; weekly amount £45.01.

“Twenty-three forty-one over fifty-two is forty-five point nought one nine two,” Maureen said. “Round that and you get forty-five and two pence. The example says forty-five and a penny. My output said tuppence. The memo was the acceptance test. I was failing it.”

“You couldn’t have queried it?” the director asked.

She gave him a look I hope someone gives me when I’m seventy-eight. “He was an Assistant Secretary. I was an AP-grade programmer in Newcastle. You did not ring an Assistant Secretary to tell him his own memo disagreed with itself. And anyway, the words could have meant a few things — nearest penny, nearest whole penny, banker’s rounding, who knows. The example could only mean one thing. When the text and the example disagree, you trust the example, because the example is the thing somebody actually worked.”

She took her glasses off. “Nobody had worked it. That’s the joke. But I didn’t know that for thirty years.”

I know it now, because the file survives. Registry papers from 1985 went to Kew decades ago, and Memo 85/223 is in a folder with a treasury tag through the corner, and on the file copy, in the margin, in pencil, is the long division itself. Pemberton did it by hand: 234100 over 52, quotient 4501, remainder 48, double-ruled. The remainder is right there. Forty-eight fifty-seconds — comfortably past the half — everything needed to round correctly, on the page, in his own hand. He wrote down the quotient and ruled off.

A slip, possibly. Possibly not: men of his generation and grade had it in the wrist that you never rounded upward in the claimant’s favour without written authority, and truncation may have felt less like an error than like prudence. He died in 1996. The file records what he did. It doesn’t record which of him did it.

Here is the part I find hardest to explain to people outside the trade: the penny had already survived two rewrites before we ever arrived. ENTITLE was replatformed in 1999, for the millennium, and again in 2014, and both times the acceptance criterion was the same, because it is cheap to test and easy to defend: the new system shall reproduce the outputs of the old. Golden-master testing, we call it. Both programmes ran millions of cases through both engines, achieved a perfect match, and were signed off as correct — correctness meaning, by then, agreement with ENTITLE, the way Maureen’s correctness had meant agreement with the example. Ours was the first programme in fifty years to go back to the words. That was policy, not virtue: the calculation code was drafted by models working from the statute book, because the legacy estate was the thing being retired, and nobody thought of the consequence, which is that for the first time since 1985 somebody — something — did the division the way the regulation described. That is the only reason there was anything to find.

We found a great deal more once we knew to look. The impact assessment took three weeks and read like a survey of everything a number touches on its way through a country. The welfare-rights charities’ calculator matches ENTITLE to the penny, deliberately; they reverse-engineered the truncation in 2011, and the ticket that closed the work says matches DWP production; the regulation is aspirational. Two commercial software houses certify their products against archived ENTITLE outputs. Local authorities compute passported support from the awarded figure. Our own debt-recovery module runs the calculation in reverse to reconstruct overpayments. And in the tribunal record there are decided cases — decided, closed, precedent-bearing — in which arrears were computed on ENTITLE’s figures. Priya, my test lead, put it in one line at the end of the assessment: after fifty years there is no such thing as an internal detail; everything the engine does, someone is standing on.

Then the lawyers did their arithmetic, which is the arithmetic that decides things. Entitlement is statutory. The engine has therefore been underpaying roughly half the caseload by a penny a week for fifty-two years. Median sum owed per person: £11.37. Interest per the remediation rules. Net arrears across all affected histories, perhaps £188 million — held by millions of people, a striking share of them dead, whose estates would need tracing. Estimated administrative cost of finding, verifying, contacting and paying them: £1.9 billion. Fixing the engine going forward while paying nothing was examined and abandoned in an afternoon; entitlement being statutory cuts both ways, and the Department would lose the first tribunal case brought by anyone with a calculator.

The third option was one paragraph long, and it is the option that happened. Conform the legislation to the established method of calculation.

I sat with parliamentary counsel while she drafted it. Her problem was that a statute cannot cite COBOL; the algorithm had to be restated in statutory language, and she is good at her work, so it came out clean:

The weekly amount is to be found by— (a) expressing the annual amount in pence; (b) dividing that amount by 52; and (c) disregarding any remainder; and an amount is to be treated as always having been correctly found if found by that method.

Clause (c) is three words of primary legislation doing the work of a missing keyword. The deeming provision closes the past. It went through committee inside the uprating bill in under forty minutes, four members present, no division. Maureen asked to see the Schedule before it was introduced, and read clause (c) aloud, once. “Disregarding any remainder,” she said. “Fifty-two years. It’s finally documented.”

Royal Assent came in March. ENTITLE is switched off in the autumn, and the replacement now implements Schedule 1 and passes every test — which it would do, since I wrote the tests from the Schedule and the Schedule was written from the code. There is a straight line from Pemberton’s pencil through Maureen’s keyboard to the statute book, and nothing along it now that anyone could call a mistake.

The explanatory notes include a worked example, for the avoidance of doubt. Annual amount £2,341.00; weekly amount £45.01. Counsel took it from the evidence bundle. I don’t believe she knew where it was first worked.