Today’s students are terrible at separating true facts from misinformation online and on social media, many studies show. But it’s not because students aren’t good at critical thinking, maintains Mike Caulfield, a research scientist at the Center for an Informed Public at the University of Washington.
Instead, they just need a little guidance on how to deal with the deluge of text, images, and websites they encounter every day. And that guidance will be even more important as ChatGPT and other ai tools come onto the scene.
Caulfield, along with Stanford University professor emeritus of education Sam Wineburg, set out to create such a guide for students and anyone struggling to cope with today’s information landscape. The result is the book”Verified: How to Think Clearly, Be Deceived Less, and Make Better Decisions About What to Believe Online.”
One problem that students (and, for that matter, any of us) face, Caulfield maintains, is that people often approach information they find online with the same strategies for differentiating fact from fiction that worked well online. an earlier time, when most published material had gone through some level of research and verification.
“There wasn’t suddenly a massive decline in critical thinking,” Caulfield says. “People were just applying approaches to information on the Internet that weren’t really appropriate for the Internet, and people were applying some of these older approaches to the Internet that weren’t really applicable.”
EdSurge connected with Caulfield to talk about his strategies for managing today’s information avalanche and how new ai tools will impact educators’ efforts to teach information literacy.
Listen to the episode on Apple Podcasts, Cloudy, Spotify, Seamstress or wherever you listen to podcasts, or use the player on this page. Or read a partial transcript below, lightly edited for clarity.
EdSurge: In the book you argue that one of the most important things to do when categorizing information online is what you call “critical ignoring.” What is essential to ignore and why is it something you are highlighting?
Mike Caulfield: One of the main things you do when reading on the Internet is try to decide whether something is worth your attention or not. In fact, it’s probably the skill you apply the most because the Internet has virtually no filters. I mean, it’s filtered by algorithms and stuff, but compared to something like a traditional newspaper or a book or something, it’s relatively unfiltered, and you’re making the decision about what to read and what not to read, you’re constantly flipping through this kind of stuff. , and probably only a small fraction of the things online are worth your attention.
In traditional models, we have often taught students that the way to solve any problem is to give it deep critical attention. And of course this is disastrous on the Internet. If, for example, a student sees something that is Holocaust denialism and your advice to the student is: “Well, take an hour, engage deeply with this person’s arguments, follow the chains of thought, look at what they’re quoting” . I mean, that’s horrible, horrible advice.
Instead, look up the person who wrote what you’re reading and you’ll often be able to immediately see, “Oh, well, this person denies the Holocaust.” This person probably isn’t worth my time.’
It’s really difficult for academics to understand that: that the answer to every question is not simply to apply deep attention, but that attention is a limited resource.
The information is abundant. I have three or four years of reading behind me right now on my bookshelf if I didn’t dedicate more than my time to reading, right? So information is not scarcity. Your attention is scarcity, discovering what to apply your attention to.
If there is one thing we want to teach students, it is how to best choose where to invest their attention and time.
You have a lot of great metaphors in the book, and you argue that one problem is that people aren’t using the right kind of mental model to properly evaluate online information. How should people approach information online or on social media?
“It’s a little more like the world of verbal rumor… where information comes to you and you’re not quite sure what the origin is. And if you get a rumor, if someone says, ‘Oh, did you hear that Bob is suspected of embezzling money?’ Your first response is: ‘Where did you hear that?’
But somehow on the Internet, because it’s in print, because it looks so polished, everything has this kind of sheen of authority, people skip that step. Then we show them how to do it on the Internet through various techniques and quick searches.
He has developed what he calls the SIFT method for evaluating online information. What’s the elevator version of that?
The first thing is arrest. Stopping is a reminder that when you feel something is particularly compelling or interesting, you should stop and ask yourself if you know what you’re looking at. And that distinction is important. Many people think we mean stop and find out if this is true or not. And for us, that is not the first step. The first step is to ask yourself: ‘Do I know what I’m looking at?’ That’s where most people go wrong. Most people think, “Oh, well, I’m looking at a local newspaper.” And sometimes the truth is, no, it’s actually a partisan blog. Or they think, ‘Oh, I’m looking at a recent photograph from 2023.’ And it’s actually like, ‘No, you’re looking at a photograph from 2011, something that happened in Germany, not the United States.’ So the first thing is to stop and ask yourself: ‘Do you know what I’m looking at?’ ‘Do I know where it came from?’ ‘Do I know anything about this topic?’
The second is investigate the fountain. And we’re not talking about Pulitzer Prize-winning research here. We’re just talking about, ‘Is this a reporter or is he a comedian?’ Because that will make a difference in how you interpret breaking news. ‘Is this an academic work? Is this something else? Is this person a conspiracy theorist? Is this person in what we call a position of knowing through experience, of professional experience, of being a direct witness to something? Or is he a person who really doesn’t have a better idea of the situation than you and maybe isn’t worth your time?’
If you are looking at that source and it is not a substantially solid source, we ask that you go find anything else. One of the things we find with students is that they often seem tied to the first source they find. And what we’re trying to do with the ‘F’ in sifting, which is to find better coverage, is to step back for a second and ask yourself if what arrived at your door is not really the best source or a good enough source for you, go out and do a search and we’ll show you the techniques to find better information and get a source that really respects your time, that you can trust, that you’re in a position to know.
And then the final piece is trace – which means tracing statements, quotes and contexts back to the original source. And this isn’t always necessary, but one of the things we often found was that students would see a tweet, a post, or a TikTok that cited supposedly authoritative information. And they would just stop there and say, well, this says The New York Times said X. And it’s like, well, you can’t really do that. The person on TikTok who says, The New York Times said this, that’s not where you should stop. You have to go upstream. You have to go get that item.
Nowadays all the talk is about ChatGPT and other ai tools, and the regular Internet feels like old technology. How does ai change things?
A large language model (LLM) like ChatGPT does not think in any sense in which we normally define thinking. What it does is put together, for any given piece of text, including any question you ask, put together a model of the things that people would probably say in response to that text. And he is doing it statistically. It’s like autocomplete on your phone.
If you ask him something like: ‘What are the three reasons for the decline of the Roman Empire?’ analyzes the “decline of the Roman Empire” and “three reasons.” And you get predictive text like, Hey, in places where people talk about the decline of the Roman Empire, and they talk about reasons and they use this word “three,” what are some of the types of things? what people say? And it just does it on multiple levels. Therefore, it presents a quite convincing answer. It can be good for summarizing, where there is a lot of text to put together, a lot of text to extract from. But it has some flaws. And the biggest flaw is that it doesn’t really have communicative objectives. He really doesn’t know what he’s saying. He is not able to evaluate things like a human does.
And there are a couple of things wrong with that. Without understanding the goal of what you are doing, you can get sidetracked. And that’s not as big of a problem for experts in a field, because if you’re an expert in something, and you go to ChatGPT and you type something, you can immediately see, “Oh, actually, this is a useful summary.” ‘ Or, ‘Oh, no, there are things wrong with this.’ But it is not good for beginners.
And that is the problem. I think people have this backwards. People think, “Oh, ChatGPT will help a newbie be like an expert.” But actually, ChatGPT and LLM are good for experts because they can see when this is clearly talking nonsense.
One of the key points we have made throughout the book is that just because something seems authoritative is not enough. You have to ask, ‘Does this seem to make sense?’
ChatGPT makes it possible for anyone to look like they know what they’re talking about. And it gives a kind of surface that looks very impressive. That’s why it’s even more important that when you see something online you don’t say, ‘Oh, is this academic in tone? Does this have footnotes? Those things don’t make sense. Now, in the world of LLMs, anyone can write something that looks authoritative and has all the hallmarks of authoritative texts without knowing at all what they are talking about. And then you have to go somewhere else. You have to leave the page (to find out more about the source). And I think that makes these skills even more urgent.
