Synthetic Prediction Markets with Algorithm Traders for Determining Experimental Reproducibility

C. Lee Giles
Sarah Rajtmajer
Anna Squicciarini

Sponsoring Agency
Army Research Office


The project will develop artificial prediction markets to evaluate the reproducibility of published research claims in the social and behavioral science literatures. Markets will be populated by artificial agents, trained and updated within human-expert prediction markets, but deployable offline. Artificial agents will represent atomic properties of relevant signals, including full text of scientific papers, metadata for specific papers, and metadata about the community and the field. Agents (“trader bots”) will learn trading patterns from subject matter experts engaged in prediction markets, but unlike their human counterparts, will have comprehensive, unbiased view on the existing literature and metadata. The project is funded through DARPA’s Systematizing Confidence in Open Research and Evidence (SCORE) program.

Research Area
Artificial Intelligence and Big Data
Social and Organizational Informatics