OpenAI is developing a research program to assess the economic impacts of code generation models and is inviting collaboration with external researchers. Rapid advances in the capabilities of code-trained extensive language models (LLMs) have made it increasingly important to study their economic impacts on individuals, businesses, and society. Codex, an LLM developed by OpenAI wrapping GPT-3 in billions of lines of code publicly available on GitHub, has been shown to generate functionally correct code 28.8% of the time in a sample of test issues (Chen et al. to. 2021). ). This may have important implications for the future of encryption and the economics of the industries that depend on it. In this paper, we present a research agenda to assess the effects of Codex on economic factors of interest to policymakers, business and the public. We present a case for this research agenda highlighting the potentially broad applicability of code generation models to software development, the potential for other LLMs to create significant social and economic impact as model capabilities advance, and the value of using Codex to generate evidence and establish methodologies that may be applicable to research on the economic impacts of future models. We propose that academic and policy research focus on studying code generation models and other LLMs so that evidence on their economic impacts can be used to inform decision-making in three key areas: implementation policy, AI system design and public policy. To help guide this research, we outline six priority outcome areas within the scope of economic impacts that we intend to use Codex to study: productivity, employment, skills development, competition between firms, consumer prices, and economic inequality. For each area, we briefly discuss previous literature on the impacts of artificial intelligence on each of these outcomes, describe questions that we believe are key inputs for the three decision-making areas mentioned above, and provide research examples that could be pursued. Cape with Codex. To catalyze work building on this initial research agenda, we are announcing a call for expressions of interest from external researchers to collaborate with OpenAI researchers and clients to better measure the economic impacts of code generation models and other LLMs.
