The rise and flaws of the mission machine: A historical critique of research success metrics since the 1960s.

Science policy systems depend on indicators to ensure that vast sums of money are spent in a transparent and accountable way and to help deliver national level policy objectives of government. In this article, we investigate why and how the key accounting system, the Frascati manual was developed and maintained over time, despite being increasingly at variance with contemporary science policy objectives. The manual has undergone several updates and revisions but remains based on a set of parameters originally designed to measure research activity as it was performed and understood over 60 years ago. We argue that the manual is ill-equipped to capture fundamental changes in research resulting from breakthroughs in multidisciplinary approaches, digital technologies, and the co-creation of knowledge in partnerships in recent decades, and poorly equipped to measure the impact of the social and human sciences. In conclusion, science policy today is served by a systematically biased accounting system: in essence, a flawed Mission Machine.