Shriyash Upadhyay and Etan Ginsberg, ai researchers at the University of Pennsylvania, believe that many large ai companies are sacrificing basic research for the sake of developing powerful and competitive ai models. The duo blames market dynamics: When companies raise substantial funds, most of it usually goes toward efforts to stay ahead of rivals rather than studying fundamentals.
“During our LLM research (at UPenn), we observed these worrying trends in the ai industry,” Upadhyay and Ginsberg told TechCrunch in an email interview. “The challenge is to make ai research profitable.”
Upadhyay and Ginsberg thought the best way to address this might be to found their own company, a company whose products benefit from interpretability. The company’s mission would naturally align with promoting interpretability research rather than capabilities research, they hypothesized, leading to more robust research.
That company, Martian, emerged quietly today with $9 million in funding from investors including NEA, Prosus Ventures, Carya Venture Partners, and General Catalyst. Profits go toward product development, conducting research on the models’ internal operations, and growing Martian’s team of 10 employees, Upadhyay and Ginsberg say.
Martian’s first product is a “model router,” a tool that receives a message destined for a large language model (LLM), for example GPT-4, and automatically routes it to the “best” LLM. By default, the router model chooses the LLM with the best uptime, skill set (for example, mathematical problem solving), and cost-performance ratio for the message in question.
“The way companies currently use LLMs is to choose a single LLM for each endpoint to which they send all their applications,” Upadhyay and Ginsberg said. “But within a task like creating a website, different models will be better suited to a specific request depending on the context the user specifies (what language, what features, how much they are willing to pay, etc.)… When using a team of models in an application, a company can achieve higher performance and lower cost than any LLM could achieve alone.”
There is some truth in that. Relying exclusively on a high-level LLM like GPT-4 can be cost-prohibitive for some, if not most, companies. The CEO of Permutable.ai, a market intelligence company, recently ai“>revealed It costs the company more than $1 million a year to process around 2 million items per day using OpenAI’s high-end models.
Not all tasks need the power of more expensive models, but it can be difficult to build a system that intelligently switches on the fly. That’s where Martian comes in and his ability to estimate the performance of a model without running it.
“Martian can target cheaper models in requests that perform similarly to more expensive models, and only target expensive models when necessary,” they added. “The Model Router indexes new models as they appear, incorporating them into applications without the need for friction or manual work.”
Now, Martian’s router model is not new technology. At least one other startup, Credal, provides an automatic model switching tool. Therefore, its rebound will depend on Martian’s price competitiveness and its ability to deliver in high-risk business scenarios.
Upadhyay and Ginsberg say there has already been some acceptance, even among “multi-billion dollar” companies.
“Building a truly effective router model is extremely difficult because it requires developing an understanding of how these models fundamentally work,” they said. “That’s the breakthrough we pioneered.”
