In the last few weeks the stock market has gone ballistic, with the Dow Jones Industrial Average swinging up and down by 400 points seemingly every day. Top-notch economists are scratching their heads; Wall Street traders are hysterical. Even the U.S. intelligence community is flummoxed. So those spies have decided to take matters — or markets — into their own hands.
Iarpa, the intelligence community’s way-out research division, thinks it can make sense of the stock market. But first, it needs some help. At a recent conference, Iarpa introduced a new program to develop tools to help analysts “quickly and accurately assess petabytes of complex anonymized financial data.”
The Markets Analysis and Testing of Contextual Hypotheses Enhancement System, or Matches, program(.pdf) would help spies analyze massive amounts of data to spot indicators of market behavior, find relationships between seemingly unrelated transactions across hundreds of global markets, and provide insight into specific events and general financial trends.
This might be tricky, to put it lightly. Iarpa acknowledges that analysts are bombarded with masses of “messy” financial data every day, from billions of transactions all over the globe. Right now the intelligence community doesn’t have many computational tools to sort through all that noise, unlike the financial industry, which enjoys a variety of “highly sophisticated technologies for market analysis.”
Iarpa is perhaps just slightly jealous.
So it’s looking for a novel approach, using fine-grained data sets, theory-based mathematical and behavioral models, and automated tools to “provide insights into the financial context of a specific event that has taken place somewhere in the world.” That means everything from the Greek debt crisis to the latest Ponzi scheme.
The agency will soon be on the look out for proposals from mathematicians, computer scientists, economists, financial engineers and statisticians, but it won’t just accept any old algorithm. Each proposed tool will be “tested against case studies provided by the government,” including real-life historical examples to establish “ground truth.”
And finally, Iarpa makes a point to emphasize the value of interdisciplinary teamwork. ”Remember,” it writes in the Matches program presentation, “you may be very accomplished, but can you do it all?”
We could probably ask Iarpa the very same question.
Posts
