MIT and Harvard researchers present language models trained in media diets that can predict public opinion

Traditional survey-based approaches to measuring public opinion have limitations, but public opinion reflects and influences the behavior of society. Questions about the extent to which AI can understand and adopt attitudes based on human language need to be explored. Answering these issues has become increasingly pressing as large language models are developed and more commonly used, thanks to recent work such as GPT3, PaLM, ChatGPT, Claude, and Bard.

Recent work from MIT and Harvard University follows in the footsteps of other recent advances in natural language processing software that summarizes large data sets to aid human decision making. They present a new method for investigating media diet models, which are modified language models that mimic the perspectives of subpopulations based on their consumption of certain media (such as Internet news, television broadcasts, or radio shows).

Predictive power, robustness to asking questions, effectiveness across all media types, and the presence of predictive cues after accounting for demographics are demonstrated for media diet models in public health and economic contexts. Additional analyzes show how sensitive they are to the level of attention people give to the news and how their impacts vary depending on the type of query made.

To anticipate how a subpopulation will respond to a survey question, the team uses a computer model that inputs a description of the subpopulation’s media diet and the question being asked. In silico public opinion models can be used if they can accurately forecast the results of human polls. Questions about public sentiment (such as “How do people feel about the pandemic?”) and scientific research on media effects (such as “How does media diet affect perceptions of the pandemic? “) could benefit from this approach.

There are three stages to developing a model for a media diet:

1. A language model is developed or used to predict missing words in a document. In this work, they mainly use BERT, a previously trained model.
2. Modifying the language model by training it on a media diet dataset includes content from various media outlets covering a given period of time. Researchers use television and radio to display transcripts and Internet news. This modification allows the model to take in new data while simultaneously updating its internal knowledge representations.
3. Ask these models questions to see if their response distributions reflect those of populations with different dietary patterns based on the media they consume. They analyze the responses to the survey questions by consulting the media diet model.

Researchers employ regression models in which (i) is used to predict (ii) to make public opinion forecasts. Survey data comes from state surveys on COVID-19 and consumer confidence. Finally, they employ the nearest neighbor method to track the source media diet data sets from which the forecasts for a specific survey question were derived.

The importance of diet research in the media is reinforced by three interconnected issues:

1. Selective exposure, or the broad systemic bias in which people gravitate toward information that is consistent with their prior ideas.
2. Echo chambers, where shared beliefs between like-minded people are amplified and strengthened by the chosen environment.
3. Filter bubbles, where content selection and recommendation algorithms display items based on users’ past activities, again reinforcing users’ worldviews.

Media diet models could be used to determine which groups are receiving the potentially most dangerous messages. They also provide a way to investigate the more nuanced effects of communications, such as variation in resonance caused by variations in word choice. While this has been investigated in controlled laboratory settings and, to a lesser extent, online, researchers focusing on media effects have been hampered by a lack of suitable tools.

The equipment these models will eventually be used to help solve real-world problems with a focus on people.

review the Paper. Don’t forget to join our 19k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

🚀 Check out 100 AI tools at AI Tools Club