Words, data and algorithms are combined,
An article on LLMs, so divine.
A glimpse into a linguistic world,
Where the language machines are deployed.
It was a natural inclination to commission a long language model (LLM) like CHATGPT to create a poem that delves into the topic of long language models, and then to use that poem as an introductory piece for this article.
So how exactly did said poem come together in a neat package, complete with rhyming words and little snippets of clever sentences?
We went straight to the source: MIT Assistant Professor and CSAIL Principal Investigator Jacob Andreas, whose research focuses on advancing the field of natural language processing, both developing cutting-edge machine learning models and exploring of the potential of language as a means to enhance other forms of artificial intelligence. This includes pioneering work in areas such as using natural language to teach robots and leveraging language to enable computer vision systems to articulate the logic behind their decision-making processes. We probed Andreas regarding the mechanics, implications, and future prospects of the technology in question.
Q: Language is a rich ecosystem ripe with subtle nuances that humans use to communicate with one another: sarcasm, irony, and other forms of figurative language. There are numerous ways to convey meaning beyond the literal. Is it possible for large language models to understand the complexities of context? What does it mean for a model to achieve “learning in context”? Also, how do multilingual transformers process variations and dialects of different languages beyond English?
TO: When we think in linguistic contexts, these models are capable of reasoning about documents and chunks of text that are much, much longer than anything we’ve ever been able to construct before. But that’s just one kind of context. With humans, the production and comprehension of language takes place in a grounded context. For example, I know that I am sitting at this table. There are objects I can refer to, and the language models we have now typically can’t see any of that when interacting with a human user.
There is a larger social context that informs much of our language use to which these models are, at least not immediately, sensitive or aware. It is not clear how to give them information about the social context in which language generation and shaping takes place. Another important thing is the temporal context. We are filming this video at a particular moment in time when the particular facts are true. The models we have now were trained, again, on a snapshot of the internet that stopped at a certain point in time (for most of the models we have now, probably a couple of years ago) and they don’t know anything about what happened. Since then. They don’t even know when they are generating text. Figuring out how to provide all those different kinds of contexts is also an interesting question.
Perhaps one of the most surprising components here is this phenomenon called learning in context. If I take a small ML [machine learning] dataset and feeds it into the model, like a movie review and the star rating given to the movie by the critic, you give just a couple of examples of these things, the language models generate the ability to both generate reviews of plausible-sounding movies as predicting star ratings. More generally, if I have a machine learning problem, I have my inputs and my outputs. As you give the model one input, give it one more input, and ask it to predict the output, models can often do this very well.
This is a super cool and fundamentally different way of doing machine learning, where I have this big general purpose model that I can insert lots of little machine learning data sets into, and yet without having to train a new model at all, classifier or a generator or whatever is specialized for my particular task. This is actually something that we’ve been thinking about a lot in my group and in some collaborations with colleagues at Google, trying to understand exactly how this learning-in-context phenomenon actually happens.
Q: We like to believe that humans are (at least somewhat) in search of what is objectively and morally known to be true. The great linguistic models, perhaps with poorly defined or yet to be understood “moral compasses”, are not bound by the truth. Why do great linguistic models tend to hallucinate facts or assert inaccuracies for sure? Does that limit the utility for applications where accuracy of the facts is critical? Is there a leading theory on how we’ll figure this out?
TO: It is well documented that these models hallucinate facts, which are not always reliable. I recently asked ChatGPT to describe some of our group’s research. He mentioned five articles, four of which are not articles that actually exist, and one of which is an actual article that was written by a colleague of mine who lives in the UK, with whom I have never co-authored. Reality is still a big problem. Even beyond that, things that involve reasoning in a really general sense, things that involve complicated calculations, complicated inferences, still seem to be really difficult for these models. There may even be fundamental limitations of this transformer architecture, and I think a lot more modeling work is needed to improve things.
Why it happens is still partly an open question, but possibly, architecturally alone, there are reasons why it is difficult for these models to build coherent models of the world. They can do that a little bit. You can check them out with factual questions, trivia questions, and they get it right most of the time, maybe even more often than the average human user on the street. But unlike the average human user, it’s not really clear if there is anything living within this language model that corresponds to a belief about the state of the world. I think this is for both architectural reasons, that transformers obviously have nowhere to put that belief, and training data, that these models are trained on the internet, that it was written by a bunch of different people at different times. that they believe different things about the state of the world. So it’s hard to expect models to represent those things consistently.
Having said all that, I don’t think this is a fundamental limitation of neural language models or even more general language models in general, but it is something that is true about current language models. We’re already seeing models getting closer to being able to build representations of facts, representations of the state of the world, and I think there’s room for further improvement.
Q: The pace of progress from GPT-2 to GPT-3 to GPT-4 has been breakneck. What does the pace of the trajectory look like from here? Will it be exponential or an S-curve that will decrease in the short run? If so, are there limiting factors in terms of scale, compute, data, or architecture?
TO: Certainly in the short term, the thing that scares me the most has to do with these veracity and consistency issues that I was mentioning earlier, that even the best models we have today generate incorrect facts. They generate buggy code, and because of the way these models work, they do so in a way that is particularly difficult for humans to detect because the model output has all the correct surface statistics. When we think about code, it’s still an open question whether it’s actually less work for someone to write a function by hand or ask a language model to generate that function and then have the person review and verify that the implementation of that function it was really correct.
There is little danger in rushing to implement these tools right away, and we will end up in a world where everything is a bit worse, but where it is actually very difficult for people to reliably verify the results of these models. With that being said, these are issues that can be overcome. The pace at which things are moving especially, there’s a lot of room to address these issues of factuality and consistency and correctness of the generated code in the long run. These are really tools, tools that we can use to free ourselves as a society from a lot of unpleasant tasks, chores, or drudgery that have been difficult to automate, and that’s something to be excited about.
