Companies today are incorporating artificial intelligence into every corner of their business. The trend is expected to continue until machine learning models are incorporated into most of the products and services we interact with every day.
As those models become a larger part of our lives, ensuring their integrity becomes more important. That’s the mission of Verta, a startup that grew out of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).
Verta’s platform helps companies deploy, monitor, and manage machine learning models securely and at scale. Engineers and data scientists can use Verta tools to track different versions of models, audit them for bias, test them before deployment, and monitor their performance in the real world.
“All we do is enable more products to be built with AI and do it safely,” says Verta founder and CEO Manasi Vartak SM ’14, PhD ’18. “We’re already seeing with ChatGPT how AI can be used to generate data, artifacts, whatever, that looks right but isn’t. There needs to be more governance and control over how AI is used, particularly for companies that provide AI solutions.”
Verta is currently working with large healthcare, financial, and insurance companies to help them understand and audit their model predictions and recommendations. It is also working with a number of high-growth technology companies looking to accelerate the deployment of new AI-enabled solutions while ensuring those solutions are used appropriately.
Vartak says the company has been able to reduce the time it takes customers to implement AI models by orders of magnitude while ensuring those models are explainable and fair, an especially important factor for companies in highly regulated industries.
Healthcare companies, for example, can use Verta to improve patient monitoring and treatment recommendations with AI technology. Such systems should be thoroughly screened for errors and bias before they are used in patients.
“Whether it’s bias, fairness or explainability, it goes back to our philosophy on governance and model management,” says Vartak. “We think of it as a pre-flight checklist – before a plane takes off, there are a number of checks you need to go through before your plane takes off. It is similar with AI models. You have to make sure that you’ve done your bias checks, you have to make sure that there’s some level of explainability, you have to make sure that your model is reproducible. We help with all of that.”
From project to product
Before coming to MIT, Vartak worked as a data scientist for a social media company. On one project, after spending weeks fine-tuning machine learning models that curated content to display in people’s feeds, he learned that a former employee had already done the same. Unfortunately, there was no record of what they did or how it affected the models.
For his PhD at MIT, Vartak decided to create tools to help data scientists develop, test, and iterate machine learning models. Working in the CSAIL Database Group, Vartak recruited a team of graduate students and participants in MIT’s Undergraduate Research Opportunity Program (UROP).
“Verta wouldn’t exist without my work at MIT and the MIT ecosystem,” says Vartak. “MIT brings together people at the forefront of technology and helps us build the next generation of tools.”
The team worked with data scientists in the CSAIL Alliances program to decide which features to build and iterate on based on feedback from early adopters. Vartak says the resulting project, called ModelDB, was the first open source model management system.
Vartak also took several business classes at the MIT Sloan School of Management during his PhD and worked with classmates at startups that recommended clothing and tracked health, spent countless hours at the Martin Trust Center for MIT Entrepreneurship, and participated in the accelerator for summer delta v downtown.
“What MIT allows you to do is take risks and fail in a safe environment,” says Vartak. “MIT allowed me those forays into entrepreneurship and showed me how to build products and find the first customers, so when Verta came along it was on a smaller scale.”
ModelDB helped data scientists train and track models, but Vartak quickly saw that the stakes were higher once the models were deployed at scale. At that point, trying to improve (or accidentally break) the models can have major implications for business and society. That idea led Vartak to start building Verta.
“At Verta, we help manage models, we help run models, and we make sure they work as expected, which we call model monitoring,” Vartak explains. “All of those pieces have their roots in MIT and my thesis work. Verta really evolved out of my PhD project at MIT.”
Verta’s platform helps companies deploy models faster, ensure they continue to perform as intended over time, and manage models for compliance and governance. Data scientists can use Verta to track different versions of models and understand how they were built, answering questions like how the data was used and what bias or explainability checks were performed. They can also examine them by running them through deployment checklists and security scans.
“Verta’s platform takes the data science model and adds half a dozen layers to it to turn it into something you can use to power, say, an entire recommendation system on your website,” says Vartak. “That includes optimizations for performance, scaling, and cycle time, which is how quickly you can take a model and turn it into a valuable product, as well as governance.”
Riding the wave of AI
Vartak says that large companies often use thousands of different models that influence almost every part of their operations.
“An insurance company, for example, will use models for everything from underwriting to claims, administrative processing, marketing and sales,” says Vartak. “So the diversity of models is really high, there’s a huge volume of them, and the level of scrutiny and compliance that companies need around these models is very high. They need to know things like: Did you use the data you were supposed to use? Who were the people who investigated it? Did you do explainability checks? Did you check for bias?
Vartak says that companies that don’t embrace AI will be left behind. Meanwhile, companies that drive AI to success will need well-defined processes to manage their ever-growing list of models.
“In the next 10 years, every device we interact with will have built-in intelligence, whether it’s a toaster or your email programs, and it will make your life much, much easier,” says Vartak. “What will enable that intelligence is better models and software, like Verta, that will help you integrate AI into all of these applications very quickly.”
