At one point do you simply just cut power to the entire company?
in reading a Mother Jones piece on high-frequency trading problems I came across the following:
in the Jersey City offices of a midsize financial firm called Knight Capital, panic was setting in. A program that was supposed to have been deactivated had instead gone rogue, blasting out trade orders that were costing Knight nearly $10 million per minute. And no one knew how to shut it down. At this rate, the firm would be insolvent within an hour. Knight's horrified employees spent an agonizing 45 minutes digging through eight sets of trading and routing software before they found the runaway code and neutralized it.
At $10 million / minute you'd think the thing to do is kill electrical power to everything right off the bat and worry about the resulting cleanup later rather than combing the source code. Companies in the cloud computing space might loose track of stuff that's running. That's why Netflix created Janitor Monkey - software who's purpose is to go around and try to clean up abandoned but still running servers. You'd think that the financial services world would be somewhat better contained though.
And of course there's still the crazy speed with which financial markets are obsessed:
Stock exchanges can now execute trades in less than a half a millionth of a second
There's really a lot of thinking that needs to be done about the sort of market-oversight involved there. e.g. there appears to possibly have been some insider trading with some seeming to get the results of a US government report about 400ms prior to it's official release. It might seem ridiculous to prosecute for releasing info 400ms ahead of time but if in that short timeframe you can execute hundreds of thousands of trades can you really decline to chase this down as the SEC appears to have done?
As the article notes most of the problems so far seem to have been accidental but there seems room for a maliciously-created algorithm to reap quite a bit of havoc. How do you prevent problems? There's talk of various types of "kill switches" to shutdown trading, but humans may be adverse to pulling such triggers manually as the article notes. Another interesting idea are things such as quote lifetimes - crazy to think about how short a lifetime might have stopped some previous problems:
Even a minimum quote life of just 50 milliseconds "would have eliminated the flash crash," says Eric Hunsader, the CEO of Nanex, a company that makes software for high-speed traders.
The per-trade tax also seems to make a fair bit of sense.