The study, known as Aggregative Contingent Estimation, is designed to see whether the 17 agencies in the U.S. intelligence community can aggregate the judgment of its thousands of analysts — rather than rely on the expertise of just a few — to issue more accurate warnings to policy makers before and during major global events.
Participants in the project give their best guesses on whether the Free Syrian Army will gain control of Aleppo, for example, or whether Kim Jong Un will resign as the leader of North Korea before April 1, 2013, according to project websites…
The teams are trying to beat the average by using algorithms that, over time, give greater weight to the most successful crystal gazers among their participants and weed out the worst. Tetlock’s group, the Good Judgment Project, is in the lead so far.
“In year one, we beat the unweighted average by about 57%, which was big,” he said. A control group, run by Mitre Corp., averages scores without giving weight to participants who tally the best results.
Join the conversation as a VIP Member