It is hard to agree with his conclusion that the market is unstable. The flash crash was a huge exogenous shock of unknown (at the time) origin and the market self corrected in a few minutes. The market did such a good job that the author did not even notice the crash until a specialist pointed it out.
If only all multithreaded programs behaved that well.
[edit: added parenthetical "(at the time)", thanks chr15.]
This really isn't what caused the flash crash. The SEC either doesn't have a complete understanding of what happened, or is omitting swaths of data from their report. Nanex did a much better job writing this all up, and weeks before the SEC published their report.
75,000 e-mini contracts is not an astronomical amount. Orders of this size are not uncommon.
To really understand what happened one must answer the question: what are the market conditions we have created such that a normal sized order can cause such a dislocation?
A lot of this has to do with the practices of the HFT 'liquidity providers.' This is a complex topic, but the flash crash demonstrated what a lot of people have been saying for years: volume != liquidity, the HFT shops generate volume, not liquidity and engage in some practices (quote-stuffing, sub-pennying etc) which serve to systemically destabilize the markets. (and overload the market infrastructure)
No body just dumps all 75,000 in one transaction. The order is filled over a period of time. W/R didn't dump either, actually further analysis shows that they sold the majority of their contracts after the crash
One thing that I think isn't getting enough attention is it wasn't even a Wall Street "master of the universe" like GS, just a relatively staid firm out on Main Street in the Midwest.
From the report..and where much of the ludicrousness surround HFT exists:
"Between 2:45:13 and 2:45:27, HFTs traded over 27,000 contracts, which accounted for about 49 percent of the total trading volume, while buying only about 200 additional contracts net... "
While buying only 200 additional contracts...HFT algo's pinging the market with orders never placed :D
A) You can't ping the market with "orders never placed." You can place orders which may or may not be filled. In this case, they were filled.
B) They are talking about trade volume, not order volume. Further, "200 additional contracts net". Thus means the HFTs immediately purchased 27,000 contracts and later resold 26,800 of them to someone else. Lastly, your laptop can probably process 27,000 trades in less than one second.
Very true. If you made the problem worse by placing "sell at any price" orders, you lost money. On the other hand, if you (as many HFT's did) tried to buy at the bottom and push the price back up to normal, you made money.
This is why the market tends to correct errors quickly. Fixing things == making money, breaking things == losing money.
How can you blame the retail investor for having stop-loss orders to lock in profits and praise some HFT's for "fixing" the flash crash by buying at the bottom when it was other HFT's that precipitated the flash crash to begin with?! Maybe that's why the retail investor is getting out of the stock market and leaving it to the machines to fight amongst themselves.
Read the SEC report. The flash crash was precipitated by a fat fingered mutual fund. It was exacerbated, among other things, by the stop loss orders this price decrease triggered.
...stop-loss orders to lock in profits...
Go learn what a stop loss order is. I blame the retail investor for placing an order he doesn't understand.
Nobody said people placed simple STOP orders. I think it's pretty obvious to even n00biest of the traders and investors that in such volatile markets as we had in last couple of years you should use STOP LIMIT.
So all those people that wanted to protect their hard earned investments (incl. 401k's) at a reasonable level "broke" things and HFT "corrected" them!? In my world that's actually called stealing.
From the comment I responded to: "...unless you lost a bundle w/ stop-loss orders..."
If you placed stop limit orders, you didn't lose a bundle and you also didn't harm the market very much. If you placed a stop limit on Accenture at 40, you lost $1/share. That's a hit, but not a ridiculous one. That's the penalty for telling the market that Accenture should be priced at $40 when it should really be priced at $41.
A lot of retail investors actually did place stop loss orders. That's why Accenture traded at $0.01 (note: those trades were broken). Bargain seeking HFT's and daytraders bought at $0.01, sold at $0.05, and quickly pushed the price of Accenture back up to $40.0 or so.
The market will hurt you if you give it bad information. The worse the information, the more you lose. If you don't understand this you should not trade.
We are not slaves of the Market, we shouldn't give it any information, it should give us information. That's the utility of the market you know.
When the game becomes "who has the fastest program to rip off this stupid risk-averse 401k guy trying to make few basis points above CD" Mr. Market totally defied it's purpose. That's when we should call the butcher and rip some tentacles off.
> We are not slaves of the Market, we shouldn't give it any information, it should give us information.
When we're buying or selling, we have to give the market information as to our buy or sell price. We have to give information to the market in order to participate in it. The information coming out of the market, after all, is merely the aggregation of the information we put into it.
If the information you give to the market is "sell at any price" when what you actually meant was "sell at $40 or more", you messed up. Don't blame the market when you give it the wrong information and it takes advantage of your mistake; if you offer a ridiculous bargain, the job of the market is to allow others a chance to jump at that bargain, which is exactly what they did.
Again, you're thinking of retail investors being like herd of stupid sheep.
First of all, very elitist from you (even though I suppose you're one of them too).
And second, actually not true. As I said, no (or almost no if you like) retail investor is stupid. All they wanted to do is to protect their positions at a reasonable level depending on variety of factors like entry price, relation of the price to some moving average etc. etc. So calling that stupid and being equal to "offering a bargain" or giving Mr. Market "wrong information" is itself stupid at best.
I do not believe retail investors are like a "herd of stupid sheep"; please do not put words into my mouth.
I believe some retail investors attempted to "protect their positions at a reasonable level" by using the wrong tool, namely, a "stop loss" order (which provides a sale at any price) rather than, say, a "stop limit" order (which provides a sale with a price floor.) This was a bad decision on their part, which in the case of the "flash crash" led to them offering their stocks at bargain prices (like Accenture for a penny.) Simply using the right tool would have protected these investors from taking such severe portfolio hits.
I agree that his conclusion of instability is difficult and will offer the idea that of all the critiques of the stock market, instability is probably the least credible.
A stock market with round the clock, instantaneous transactions is one of the most efficient markets that exist. Instability would suggest that it frequently allows for inefficient outcomes. A stock market's prices are determined exactly by buyer demand and seller supply. It corrects itself in the event of freak transactions (like the flash crash) and if it doesn't correct itself, that means the supply and demand have changed and the price of stocks nearly instantly reflect those changes.
I do tend to think of financial markets as a giant computer whose function is to allocate capital in the economy, and which just happens to use a really weird architecture/computational model that incorporates human brains as computational nodes, sort of like The Matrix if they hadn't gone with the goofy energy-extraction motivation instead.
It's probably an analogy with limited applicability, but I do think it might be interesting to examine the computational properties of financial markets, try to see to what extent arbitrary programs can be encoded in them, etc.
It's actually an analogy with great applicability. Quite a bit of work has been done on this topic - see the work of Pennock at Yahoo. One result: completely efficient prediction market is a general purpose computer. Such an object also requires a ridiculous number of market participants, since it can solve NP complete problems.
One very popular result (major media attention) along these lines is this paper, which proves that pricing a certain type of CDO is an NP-complete problem:
(Side note: this paper really pisses me off because they did not cite ANY of the prediction market literature. It's hard to believe this was not on purpose.)
So when the HFTs got hold of the futures contracts that the Waddell & Reed algorithm was selling, they would've first looked for legit buyers to flip them too. And, not finding any legit buyers, they just started rapidly buying and selling amongst themselves to at least pick up some rebate money.
A couple of things:
1. E-mini futures don't give rebates
2. He seems to suggest that different algos from different companies were knowingly trying to collude with each other to get these nonexistent rebates. This seems pretty far-fetched.
And even if there were rebates (as there are on stocks) you can't make a profit just passing them back and forth because taking liquidity costs more than is rebated to the offerer. If liquidity hot potato were profitable then that would be the sole purpose of the stock market until the exchanges ran out of money, because it'd be coming straight out of their pockets.
I'm not saying it didn't happen, just that it's not something that a "rational" algorithm would do, because the one that's taking liquidity is losing money. Unless they're profiting for reasons other than rebates, that is, which is another small but nonzero possibility.
My favorite theory is a purely technical one. I don't have much supporting evidence beyond the linked site though:
"""In summary, quotes from NYSE began to queue, but because they were time stamped after exiting the queue, the delay was undetectable to systems processing those quotes. The delay was small enough to cause the NYSE bid to be just slightly higher than the lowest offer price from competing exchanges. This caused sell order flow to route to NYSE -- thus removing any buying power that existed on other exchanges. When these sell orders arrived at NYSE, the actual bid price was lower because new lower quotes were still waiting to exit a queue for dissemination."""
Because market makers purchased them at a discount with the hope of selling them for a small profit sometime later in the day. That's what market makers do.
Trying to understand why an algorithm made a trade at a certain price is practically impossible. Assuming that these algorithms are profit seeking (and not utility seeking, as in trying to transfer risk), the short answer is: because they thought they had a statistical edge they could profit from in the future. Of course, this is a statistical edge. No algo is always right, or even near always right. The algos don't know who they're trading with. They see liquidity and they're willing to take it if they think it's a good price.
Now, you might ask, what is the higher purpose of all this trading, if all of it is profit seeking? That's pretty simple also. These trades help to determine the fair market price of a contract. The algos that consistently make the "right" trade survive ("right" defined as predicting future market price), and the ones that don't eventually get shut down. So even when the trading is non-utilitarian, there still is a greater purpose. That purpose is price discovery.
I think a bunch of message-passing Erlang processes seems like a more apt description. Any one of them can crash, and others will take over, if things are set up properly. Indeed, after a half an hour, things were mostly sorted out.
argh and it's a crappy analogy honestly.
There's no real deadlocks as far as cycles of dependencies on critical resources.
It's basically saying the stock market is a giant distributed system with a bunch of independent agents, which erm it is...
this analogy to threading just doesn't add anything, it's just the author's model for agents acting concurrently, threading is a low level OS thing not the fundamentals of concurrent action.
Were you actually intending to link directly to the comments section rather than to the top of the article? Am I the only one that finds that annoying?
Not to be pedantic but it's more like a piece of functional message passing software with many agents each with their own immutable state (the final record is the ticker tape and it's generally immutable).
If only all multithreaded programs behaved that well.
[edit: added parenthetical "(at the time)", thanks chr15.]