Two months ago, I changed teams at Microsoft, and moved from the Office security test team to the Outlook test team. I’m excited to focus on more than just security, but I still love security testing. I’m fortunate to be at the BlackHat Vegas conference this week, learning more about how other people approach security testing.
One of the more entertaining presentations was Barnaby Jack’s ATM jackpot bugs. Seeing an ATM spit out money from a quick local or remote attack makes for good theater.
But I was more impressed by the process taken to find the bugs, and the testability steps he needed, and I think they have broader applicability outside of the narrow space of ATM penetration testing.
Testability is about the control and observability of software. An ATM is an extreme example of a black box with (usually) limited interfaces exposed to the customer – just the stripe reader, pin pad, screen, and money dispenser.
- Obtaining physical ATM’s – Barnaby needed to obtain several ATM’s, and placed them in his house. A similar situation when testing cloud or server software is whether you have full access to the running bits, for at least some of your testing.
- Access to the motherboard and debugger interface – In order to understand the software on the ATM, he needed to attach a debugger, and to do so get access to the motherboard and JTAG interface.
- Access to the USB port – One of his bugs relied on a local firmware upgrade, using the USB port of the motherboard. Although the money in the ATM is in a safe, the motherboard was much easier to access with a standard key.
- Access to network traffic – His remote attack relied on understanding the proprietary network protocol, and the remote monitoring interface. Instead of relying on the network via the phone, he was able to use the TCP/IP interface in the ATM.
- Injecting explorer.exe – First step for Barnaby was getting the Windows CE running in the machine to run an explorer.exe shell. This allowed much better understanding of the files and processes.
- Copying files to a PC and using Visual Studio – Barnaby copied the executables locally, and used Visual Studio for some of his analysis and debugging.
- API’s to inject hooks for observability and control – in order to both understand and control the running software, he injected code, both with the windows hook api and detours style hooks.
- Understanding the proprietary checksum for firmware upgrades – Barnaby needed to take the time to understand the proprietary encrypted and checksummed firmware format to make his firmware upgrade work.
All of these show the dedication needed to understand software well enough to find cool bugs, and apply to non-security testing too.
Having attended StarEast and BlackHat, there certainly differences in the testers attending each conference, with more elitism in the security test community, more secrecy, and more value placed on the bugs found. But the basic testing skills still apply, including getting testability in order to find bugs.