Ali Farhadi is no tech rebel.
The 42-year-old computer scientist is a highly respected researcher, professor at the University of Washington and founder of a start-up acquired by Apple, where he worked until four months ago.
But Farhadi, who became executive director of the Allen Institute for aicalls for a “radical openness” to democratize research and development in a new wave of artificial intelligence that many believe is the most important technological breakthrough in decades.
The Allen Institute has launched an ambitious initiative to build a freely available ai alternative for tech giants like Google and startups like OpenAI. In an industrial process called open source, other researchers will be allowed to examine and use this new system and the data fed into it.
The stance taken by the Allen Institute, an influential nonprofit research center in Seattle, puts it squarely on one side of a fierce debate over how open or closed the new ai should be. Would the opening up of so-called generative ai, which powers chatbots like OpenAI’s ChatGPT and Google’s Bard, lead to more innovation and opportunities? Or would it open Pandora’s box of digital damage?
Definitions of what “open” means in the context of generative ai vary. Traditionally, software projects have open sourced the underlying “source” code for the programs. Anyone can then look at the code, spot errors, and make suggestions. There are rules that govern whether changes are made.
This is how the popular open source projects behind the widely used linux operating system, the Apache web server and the Firefox browser operation.
But generative ai technology involves more than code. ai models are trained and fine-tuned round after round of enormous amounts of data.
However well-intentioned, experts warn, the path the Allen Institute is taking is inherently risky.
“Decisions about opening up ai systems are irreversible and will likely be among the most consequential of our time,” said Aviv Ovadya, a researcher at Harvard’s Berkman Klein Center for Internet and Society. He believes international agreements are needed to determine what technology should not be disclosed publicly.
Generative ai is powerful but often unpredictable. You can instantly write emails, poetry, and term papers, and answer any question imaginable with human fluency. But he also has a disturbing tendency to make things up in what researchers call “hallucinations.”
The major chatbot makers (OpenAI and Google, backed by Microsoft) have kept their new technology closed, without revealing how their ai models are trained and tuned. Google, in particular, had a long history of publishing its research and sharing its ai software, but has increasingly kept its technology to itself as it developed Bard.
That approach, the companies say, reduces the risk that criminals will hijack the technology to further flood the Internet with misinformation and scams or engage in more dangerous behavior.
Supporters of open systems recognize the risks, but say having smarter people working to combat them is the best solution.
When Meta launched an ai model called LLaMA (Large Language Model Meta ai) this year, it created quite a stir. Farhadi praised Meta’s move, but believes it does not go far enough.
“Their approach is basically: I’ve made some magic. I’m not going to tell you what it is,” she said.
Farhadi proposes to reveal the technical details of the ai models, the data with which they were trained, the tuning that was performed and the tools used to evaluate their behavior.
The Allen Institute has taken a first step publishing a huge data set to train ai models. It is made up of publicly available data on the web, books, academic journals and computer code. The data set is curated to remove personally identifiable information and toxic language such as racist and obscene phrases.
In editing, judgmental decisions are made. Will removing some language considered toxic decrease a model’s ability to detect hate speech?
The Allen Institute’s trove of data is the largest open data set currently available, Farhadi said. Since its release in August, it has been downloaded more than 500,000 times on hugging facea site for open source ai resources and collaboration.
At the Allen Institute, the data set will be used for training and tuning a great generative ai programOLMo (Open Language Model), which will be released this year or early next year.
Large ai business models, Farhadi said, are “black box” technology. “We’re pushing for a glass box,” he said. “It opens up the whole thing and then we can talk about the behavior and partly explain what’s going on inside.”
Only a handful of core generative ai models of the size the Allen Institute has in mind are openly available. They include LLaMA from Meta and Falcon, a project backed by the Abu Dhabi government.
The Allen Institute seems like a logical home for a big ai project. “It is well-funded but operates with academic values and has a history of helping advance open science and artificial intelligence technology,” said Zachary Lipton, a computer scientist at Carnegie Mellon University.
The Allen Institute is working with others to advance its open vision. This year, the nonprofit Mozilla Foundation invest 30 million dollars in a new company, ai/” title=”” rel=”noopener noreferrer” target=”_blank”>Mozilla.aito create open source software that will initially focus on developing tools around open ai engines, such as the Allen Institute’s, to make them easier to use, monitor and deploy.
The Mozilla Foundation, founded in 2003 to promote the Internet remaining a global resource open to all, fears a greater concentration of technology and economic power.
“A small group of actors, all on the West Coast of the United States, are trying to block the generative ai space before it even really gets out the door,” said Mark Surman, president of the foundation.
Farhadi and his team have spent time trying to control the risks of their opening strategy. For example, they are working on ways to evaluate a model’s behavior in the training stage and then prevent certain actions such as racial discrimination and the manufacture of biological weapons.
Farhadi sees the security barriers on large chatbot models as Band-Aids that can be easily ripped off by savvy hackers. “My argument is that we should not allow that kind of knowledge to become codified in these models,” he said.
People will do bad things with this technology, Farhadi said, as they have with all powerful technologies. Society’s task, he added, is to better understand and manage risks. Openness, he maintains, is the best bet for finding security and sharing economic opportunities.
“Regulation alone will not solve this,” Farhadi said.
The Allen Institute effort faces some formidable obstacles. One of the main ones is that building and improving a large generative model requires a lot of computing power.
Farhadi and his colleagues say emerging software techniques are more efficient. Still, he estimates the Allen Institute initiative will require $1 billion in computing over the next two years. He has begun trying to drum up support from government agencies, private companies and tech philanthropists. But he declined to say whether he had secured sponsors or name them.
If successful, the biggest test will be fostering a lasting community that supports the project.
“You need an ecosystem of open players to really make a dent in the big players,” said Surman of the Mozilla Foundation. “And the challenge in that type of game is just patience and tenacity.”
