Stress Testing a Bank: What happens when you run out of account numbers?
Have you ever heard of a bank running out of account numbers?
We have.
In fact, we've done it. Three times.
The numbers game
In the UK, bank account numbers have eight digits, and they are paired with a six-digit sort code that identifies the bank they belong to.
Each digit can be any number from 0 to 9, which means ten possibilities in every position. That's one million sort codes, and 100 million account numbers for each.
With so many combinations, it's hard to imagine a situation where any bank might run out. So what happened?
Building a shadow bank
At Starling, we like to test our systems to their absolute limits.
One of the ways we test our bank is with a simulator: a piece of software that mimics the behaviour of real customers and staff. It runs in a replica of our production environment that we call 'demo'.
We use our simulator to test new features and changes before we release them to our customers, and to simulate the bank of the future. It's always at least three times busier than our production environment, which helps us to be confident that we are ready to scale.
It opens accounts for fake people who live at made-up addresses. It buys pretend coffee with imaginary cards. It puts pocket money into Spaces for children who don't exist. It even commits fictional financial crime.
For almost a decade, our simulator has been building a shadow bank.
In its enthusiasm to always stay ahead of Starling itself, it has been burning through account numbers at an alarming rate. And on three occasions in the past ten years, it has assigned the very last available account number of a sort code.
The invisible maths of banking
With 100 million account number combinations for each sort code, you might be wondering how we've been able to run out of account numbers at all. Just how big is our shadow bank?
But the basic maths of digit combinations only tells part of the story.
In theory, you can have 100 million accounts per sort code. But no bank actually does it. It's far too risky.
Imagine your account number is
12345678and mine is12345687. You've just won the lottery, and they ask for your bank details to send over your prize.You're excited. You type quickly. You accidentally swap the last two numbers around.
Suddenly, I'm the one celebrating.
The modulus check
To prevent catastrophic typos, the UK banking system uses a mathematical trick called a modulus check, which relies on a hidden number called a weight.
Whenever we generate a new account number, we run a test. We take each digit, multiply it by its weight, and add all the results together. Finally, we take the total and divide it by 11.
If there is nothing left over - the number divides perfectly - it's a valid number and we open the account.
If there's a remainder, we discard the number and generate a new one.
This invisible check means two things:
- Account numbers are unlikely to differ by just one digit. This stops you accidentally sending your lottery winnings to me.
- The set of account numbers for each sort code is different. This means your account number is less likely to also belong to another customer at another bank.
It's much safer, but it does have a downside. If only one in eleven randomly generated account numbers is valid, then we only have nine million valid account numbers per sort code, not 100 million.
And that's why our shadow bank has run out so many times.
Grinding to a halt
The first time we ran out of account numbers, it took us by surprise. Onboarding in our demo environment stopped and no new accounts could be opened.
This was a problem.
We rely on our demo environment to make sure our changes are safe to release. And, at Starling, we release code dozens of times every day.
So we treat even a small disruption as an incident.
Luckily, we have an excellent incident management process and, if our demo environment stalls, engineers drop what they're doing and focus on getting us back up and running.
When we ran out of account numbers, we had two priorities:
- Unblock the simulator. Get a new sort code up and running so we can continue opening accounts.
- Prevent it happening for real. Make sure we never run out of account numbers in production.
The solution didn't have to be perfect.
We take pride in our bespoke tools at Starling, but we try to remain pragmatic. Just because something breaks doesn't mean we need to gold-plate a fix. We are always asking whether a fully-featured solution will actually give us a competitive edge, or whether a simple fix will allow us to scale quickly and safely without unnecessary complexity.
In this case, we knew we had plenty of account numbers to play with. So once we had a new sort code in place, we focussed on building automated alerts to let us know when they started to run low again. Then, we added fail-safes in case they ever run very low. And, we wrote instructions to our future selves to explain the technicalities of pressing a new sort code into service.
Now, we're confident we'll get a warning long before we run out of account numbers in production. And we've already run through the process several times in demo, so we know we'll be ready when it happens.
Meanwhile, our simulator is free to keep opening accounts with wild abandon.
Breaking things on purpose
This simulated exhaustion is a perfect example of life as an engineer at Starling. We push our test environment to its absolute limit so that our real-world systems never reach theirs.
Breaking our demo environment isn't a failure. It's the exact reason the shadow bank exists.
We take what we've broken, refine our logic, update our approach, and make sure it won't happen again.
To find out more about what it's like to work here, take a look at our blog post How Starling Builds a Bank.