SpringerLink
Log in

Public Value Promises and Outcome Reporting in Advanced Research Projects Agency-Energy

  • Published:
Minerva Aims and scope Submit manuscript

Abstract

U.S. federal research funding is generally justified by promises of public benefits, but the specific natures and distribution of such benefits often remain vague and ambiguous. Furthermore, the metrics by which outcomes are reported often do not necessarily or strongly imply the achievement of public benefits. These ambiguities and discontinuities make it difficult to assess the public outcomes of federal research programs. This study maps the terms in which the purposes and the outcomes of Advanced Research Projects Agency-Energy (ARPA-E) -a relatively young and innovatively structured federal research agency-have been discussed and reported. I find that ARPA-E’s creation and funding have been justified with reference to a broad repertoire of economic, environmental, and national security values, but that the agency has been evaluated only through intermediate scientific and economic metrics and study of internal organizational structure. I suggest that these means of assessment elide ARPA-E’s lack of the financial scale, long time horizon, built-in customer, and radical vision which have been historically important to high-impact federal innovation, and I recommend the tracking of metrics more directly related to ARPA-E’s public value purposes as the agency grows older. This discussion illustrates the inadequacy of currently widespread metrics and conventional wisdom for the design and assessment of societally relevant research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. That is, roughly 10% of DARPA’s FY2019 budget, 1% of DoE’s, and 0.25% of the federal R&D budget (American Association for the Advancement of Sciences 2020; DARPA 2020; U.S. Department of Energy 2020).

  2. Different sources, and sometimes the same source, show that Rising Above the Gathering Storm was released by the National Research Council (NRC) in 2005 (NASEM, 2017, p. ix) and by the National Academies in 2007 (NASEM, 2017, p. 145). It appears that NRC released a preliminary version of the report in 2005, and that the National Academies released a final, public version in 2007.

References

  • Alic, John A., Lewis M. Branscomb, Harvey Brooks, Ashton B. Carter, and Gerald L. Epstein. 1992. Beyond spinoff: Military and commercial technologies in a changing world. Boston, MA: Harvard Business School Press.

    Google Scholar 

  • Alic, John A., Daniel Sarewitz, Charles Weiss, and William Bonvillian. 2010. A new strategy for energy innovation. Nature 466: 316–317.

    Article  Google Scholar 

  • America COMPETES Act, H.R. 2272, 110th Cong. 2007.

  • America COMPETES Act, S. 761, 110th Congress 2007.

  • America COMPETES Act of 2007, Pub. L. No. 110-69, 121 2007.

  • American Association for the Advancement of Science. 2020. Historical Trends in Federal R&D. Retrieved from https://www.aaas.org/programs/r-d-budget-and-policy/historical-trends-federal-rd.

  • American Institute of Physics 26 September. 2018. Final FY19 appropriations: DOE applied energy R&D. FYI: Science Policy News from AIP. Retrieved from https://www.aip.org/fyi/2018/final-fy19-appropriations-doe-applied-energy-rd.

  • ARPA-E. 17 May 2016. ARPA-E: The First Seven Years: A Sampling of Project Outcomes. Retrieved from http://arpa-e.energy.gov/sites/default/files/documents/files/Volume%201_ARPA-E_ImpactSheetCompilation_FINAL.pdf.

  • ARPA-E. 10 February 2017. ARPA-E Impacts: A Sampling of Project Outcomes, Volume II. Retrieved from https://arpa-e.energy.gov/sites/default/files/documents/files/Volume%202_ARPA-E_ImpactSheetCompilation_FINAL.pdf.

  • ARPA-E. 7 May 2018. ARPA-E Impacts: A Sampling of Project Outcomes, Volume III. Retrieved from https://arpa-e.energy.gov/sites/default/files/documents/files/ARPA-E-Impact-Book-Volume-3-Final-May10.pdf.

  • Azoulay, Pierre, Erica Fuchs, Anna P. Goldstein, and Michael Kearney. 2019. Funding breakthrough research: Promises and challenges of the ‘ARPA Model.’ In Innovation Policy and the Economy, vol. 19, eds. Josh Lerner and Scott Stern, 69–96. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Bonvillian, William B. 2018. DARPA and its ARPA-E and IARPA clones: a unique innovation organization model. Industrial and Corporate Change 275: 897–914.

    Article  Google Scholar 

  • Bonvillian, William B., and Richard Van Atta. 2011. ARPA-E and DARPA: Applying the DARPA model to energy innovation. Journal of Technology Transfer 36: 469–513.

    Article  Google Scholar 

  • Bornmann, Lutz. 2012. What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology 64(2): 217–233.

    Article  Google Scholar 

  • Bozeman, Barry. 2007. Public Values and Public Interest: Counterbalancing Economic Individualism. Washington, DC: Georgetown University Press.

    Google Scholar 

  • Bozeman, Barry, and Daniel Sarewitz. 2011. Public Value Map** and Science Policy Evaluation. Minerva 49(1): 1–23.

    Article  Google Scholar 

  • Brown, Nik. 2003. Hope against hype—Accountability in biopasts, presents and futures. Science Studies 16(2): 3–21.

    Google Scholar 

  • Bush, Vannevar. 25 July 1945. Science: the Endless Frontier, A Report to the President. Washington, DC: United States Government Printing Office.

  • Cook-Deegan, Robert. 2015. Has NIH lost its halo? Issues in Science and Technology 312. Retrieved from https://issues.org/has-nih-lost-its-halo/.

  • Crow, Michael M. 2011. Time to rethink the NIH. Nature 471: 569–571.

    Article  Google Scholar 

  • Crow, Michael M., and Barry Bozeman. 1998. Limited by Design: R&D Laboratories in the U.S. National Innovation System. New York, NY: Columbia University Press.

  • Defense Advanced Research Projects Agency. 2020. About Us: Budget. Retrieved from https://www.darpa.mil/about-us/budget.

  • Fuchs, Erica R.H.. 2009. Cloning DARPA successfully. Issues in Science and Technology Fall 2009: 65–70.

    Google Scholar 

  • H. Rept. No. 110-289, at 51 2007.

  • Hilgartner, Stephen. 2015. Capturing the imaginary: Vanguards, visions, and the synthetic biology revolution. In Science and Democracy: Making Knowledge and Making Power in the Biosciences and Beyond, eds. Stephen Hilgartner, Clark Miller, and Rob Hagendijk, 33–55. New York, NY: Routledge.

    Chapter  Google Scholar 

  • Majumdar, Arun. 15 June 2011. Statement of Dr. Arun Majumdar. In An Examination of DOE's Clean Technology Programs: Hearing Before the House Subcommittee on Energy and Environment, 112th Cong. 2011. Retrieved from https://republicans-science.house.gov/sites/republicans.science.house.gov/files/documents/hearings/061511_Majumdar.pdf.

  • Mansfield, Iain, and Geoffrey Owen 2020. Visions of ARPA: Embracing Risk, Transforming Technology. London, UK: Policy Exchange.

  • Mazzucato, Mariana. 2018a [2013]. The Entrepreneurial State: Debunking Public vs. Private Sector Myths. Westminster, UK: Penguin Books.

  • Mazzucato, Mariana. 2018b. Extracting Value through the Innovation Economy. In The Value of Everything: Making and Taking in the Global Economy, 189-228. New York, NY: PublicAffairs.

  • Meyer, Ryan. 2011. The Public Values Failures of Climate Science in the US. Minerva 49(1): 47–70.

    Article  Google Scholar 

  • National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press.

  • National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. 2007. Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, DC: The National Academies Press.

  • Papadakis, Maria. 1993. Patents and the Evaluation of R&D. In Evaluating R&D Methods: Impacts and Practice, eds. Barry Bozeman and Julia Melkers, 43–62. Boston, MA: Kluwer Academic Publishers.

    Google Scholar 

  • Polanyi, Michael. 1962. The Republic of Science. Minerva 1(1): 54–73.

    Article  Google Scholar 

  • Rentschler, Jun, and Morgan Bazilian. 2017. Policy monitor—Principles for designing effective fossil fuel subsidy reforms. Review of Environmental Economics and Policy 11(1): 138–155. https://doi.org/10.1093/reep/rew016.

    Article  Google Scholar 

  • Rosenberg, Nathan. 1994. Exploring the black box. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Rosenberg, Charles. 1976 [1961]. Science and Social Values in Nineteenth-Century America: A Case Study in the Growth of Scientific Institutions. In No Other Gods: On Science and American Social Thought, 135-152. Baltimore, MD: Johns Hopkins University Press.

  • Ruttan, Vernon W. 2001. Technology, Growth, and Development: An Induced Innovation Perspective. New York, NY: Oxford University Press.

    Google Scholar 

  • Ruttan, Vernon W. 2006. Is War Necessary for Economic Growth? Oxford, UK: Oxford University Press.

    Book  Google Scholar 

  • Sampat, Bhaven N. 2009. The dismal science, the crown jewel, and the endless frontier. In The New Economics of Technology Policy, ed. Dominique Foray, 148–162. Cheltenham, UK: Edward Elgar.

    Google Scholar 

  • Sarewitz, Daniel, and Roger A. Pielke Jr. 2007. The neglected heart of science policy: reconciling supply of and demand for science. Environmental Science & Policy 10: 5–16.

    Article  Google Scholar 

  • Seto, Karen C., Steven J. Davis, Ronald B. Mitchell, Eleanor C. Stokes, Gregory Unruh, and Diana Ürge-Vorsatz. 2016. Carbon lock-in: Types, causes, and policy implications. Annual Review of Environment and Resources 41: 425–452.

    Article  Google Scholar 

  • Stilgoe, Jack, Richard Owen, and Phil Macnaghten. 2013. Develo** a framework for responsible innovation. Research Policy 42(9): 1568–1580.

    Article  Google Scholar 

  • Stokes, Donald. 1997. Pasteur’s Quadrant. Washington, DC: Brookings Institution Press.

    Google Scholar 

  • Sturm, Christine. 2017. Inside the Energiewende: Policy and complexity in the German utility industry. Issues in Science and Technology 33(2). Retrieved from https://issues.org/inside-the-energiewende-policy-and-complexity-in-the-german-utility-industry/.

  • Toulmin, Stephen. 1964. The complexity of scientific choice: A stocktaking. Minerva 2(3): 343–359.

    Article  Google Scholar 

  • Toulmin, Stephen. 1966. The complexity of scientific choice II: Culture, overheads, or tertiary industry? Minerva 4(2): 155–169.

    Article  Google Scholar 

  • U.S. Department of Energy. 2020. Funding by Appropriation: FY 2020 Congressional Budget Justification. Retrieved from https://arpa-e.energy.gov/sites/default/files/ARPA-E%20FY%202020%20Budget%20Request.pdf.

  • U.S.C. § 16538.

  • Valdivia, Walter D. 2011. The Stakes in Bayh-Dole: Public Values Beyond the Pace of Innovation. Minerva 49(1): 25–46.

    Article  Google Scholar 

  • Weinberg, Alvin M. 1962. Criteria for scientific choice. Minerva 1(2): 158–171.

    Google Scholar 

  • Weinberg, Alvin M. 1964. Criteria for scientific choice II: The two cultures. Minerva 3(1): 3–14.

    Article  Google Scholar 

  • White House. 2012. National Bioeconomy Blueprint. Washington, DC: The White House.

  • Winner, Langdon. 1980. Do artifacts have politics? Daedalus 109(1): 121–136.

    Google Scholar 

Download references

Acknowledgments

I am grateful to Dan Sarewitz, Barry Bozeman, and Dave Guston for their support, guidance, and assistance in the framing and conduct of this research and the preparation of this manuscript. Any remaining mistakes are, of course, my own. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. 1841051. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. School for the Future of Innovation in Society at Arizona State University, 1120 South Cady Mall, Tempe, AZ, 85287-5603, USA

    John P. Nelson

Authors
  1. John P. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John P. Nelson.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nelson, J.P. Public Value Promises and Outcome Reporting in Advanced Research Projects Agency-Energy. Minerva 59, 493–513 (2021). https://doi.org/10.1007/s11024-021-09444-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11024-021-09444-7

Search

Navigation