CSU's The Audit

Intro: Ever had that feeling, you know the one? The hair on the back of your neck stands up and you’re overcome by the eerie sensation that you’ve been in this exact situation before. It’s that creepy feeling of déjà vu, and it’s one that almost everybody experiences at some time or another.

But what is déjà vu? And why does it happen? Is it a psychic premonition? A message from the other side? A memory from a past life? While it’s long been the subject of horror movies and books, it turns out déjà vu isn’t really all that paranormal. In fact, it’s actually pretty normal, according to Colorado State University Psychology Professor Anne Cleary.

Cleary researches human memory; specifically, when that memory is just out of reach, such as with déjà vu or those moments when the word you want is right on the tip of your tongue. She recently spoke with The Audit about what actually happens when you experience this memory phenomenon.

Host: What drew you to studying déja vu, and what aspects of the phenomenon are you most interested in looking at?

Anne Cleary: I became interested in the study of déja vu around 20 years ago. I had been studying an aspect of memory known as familiarity-based recognition. In fact, I had a career grant called Familiarity-Based Recognition. The way that I had been studying that in the laboratory was to have people make familiarity judgments for different types of stimuli during incidents of recall failure. People couldn't recall anything specific that the stimulus might have reminded them of, but could feel a sense of familiarity, nonetheless.

I had been in the process of doing this when a researcher named Alan Brown published a paper in the journal Psychological Bulletin on the déja vu experience, and it was the first time any researcher had published an article for mainstream scientists, mainly cognitive scientists, but psychologists more broadly, on the phenomenon of déja vu. He brought it into the realm of science for the first time. In addition, it wasn't just a review of what had been done up until that point, and what had been done up until that point had basically been survey research or survey studies of people's déja vu experiences. No one had really attempted to study it in a laboratory setting yet or to induce it through experiments.

Host: Was that because it was seen more as an eerie supernatural thing then? What was the view of déja vu at that time?

Cleary: I think at that time probably a big reason scientists had steered clear of studying déja vu was precisely because it has this association with the paranormal. In fact, even in Alan Brown's review paper, even though it was a scientific journal, he acknowledged in parts that some of the surveys that he was pulling from were of a paranormal flavor. So, I think there was that element keeping it out of science.

This was the first time that a scientific paper on the subject had emerged in mainstream literature, and it wasn't just a review. It was also really a call to cognitive scientists to take up its study in the laboratory. So, in addition to reviewing what he could pull out of these various surveys that had been carried out over the decades, Alan Brown really pointed out a number of different ways that cognitive scientists could study déja vu in the laboratory.

I really took up that invitation because at the time I had been studying this similar aspect of memory, and immediately saw links between what I was already studying – people's sense of familiarity when they can't recall anything specific – and Alan Brown's call to scientists to take up the study of déja vu.

Host: It sounds like that's really the point where it made that jump from the paranormal of the unknown to really becoming legitimate.

Cleary: Yes, I see that paper as a critical catalyst for enabling scientists to take up this study. You didn't really see anyone, including myself, taking up its study in laboratory settings until that point in time. Then after that point, you see papers starting to emerge, including my own, over the years, studying déja vu in a scientific setting.

Host: What was the reaction that you got from other people in your field? Were they like, what are you doing? Were they excited about it?

Cleary: Both. There was some excitement, but I think there was some skepticism at first and also reluctance. In fact, after our initial studies on déja vu – which I thought were really neat studies that were the first to test an almost century old hypothesis of déja vu that had never been studied – there was skepticism by reviewers. It was quite difficult to get our first papers published because there was this skepticism that you could even study déja vu in a laboratory setting. Or that if you get participants to report feelings of déja vu in a laboratory setting, how do you know that's the real déja vu? How could you possibly induce that in a laboratory setting? It's too mysterious. It's too eerie. It can't be the real thing because it would be impossible to duplicate that. There was really this skepticism among a lot of people that it was something that could be studied.

Host: So how do you study déja vu? How do you trigger that in people?

Cleary: The approach that we've taken followed from my initial studies of familiarity-based recognition, where we would familiarize stimuli in the laboratory and then examine instances where people could recognize that something was familiar. For example, a face that had been familiarized in the lab, but they weren’t able to pinpoint why it's familiar or what the source of that familiarity was.

It wasn't too far of a leap to take that approach to studying déja vu. The approach we took follows from a number of suggestions that Alan Brown made in his initial review paper. One was something that was pulled from survey research. Across many survey studies across the decades, one trend that emerged was that the most common elicitor of déja vu is scenes or places. The other piece is a specific hypothesis that Alan put forward in his review paper and suggested could be studied in a lab but hadn't, what's known as the Gestalt Familiarity Hypothesis. It's the idea that maybe déja vu emerges because you enter a scene or a place, and even though you've never been in that specific scene or place before, it has the same layout, the same configuration of elements, like the placement of furniture, for example, relative to one another as another place that you've been in your past. You're just failing to recall that experience as the source of familiarity with this current scene.

That hypothesis had been out there in the literature but never tested. It’s conducive to being tested in a laboratory setting, particularly with the advent of virtual reality. What we could do now is create lots of different scenes, some of which have the same spatial arrangement on a grid but are otherwise distinct from one another. For example, we could create a bedroom scene that has the exact same spatial layout as a clothing store or a bowling alley that has the exact same spatial layout as a subway station, or a courtyard that has the exact same configuration as a museum scene.

Host: You also got a unique introduction to that format from a popular video game, right?

Cleary: Yes, when we first did this study, it was many years ago, before virtual reality as we know it today. So, we used an early 2000s game called The Sims. It's a life-simulation game that allows you to create houses and landscapes for your people to live in as part of the game. Often people will create elaborate mansions and gardens and sophisticated landscapes or even schools and museums. You can get really creative with this gaming platform. Some even put their virtual tours on YouTube. Even today if you look on YouTube, you can see people's creations of various Sims houses and structures.

We used the platform to create a number of scenes that were otherwise distinct from one another, but each were a pair that had the identical layout but were otherwise distinct. Then we essentially took people on virtual tours, much like The Sims game was designed to do, but in more of an experimental environment.

Host: And today it's a little more, as you mentioned, high tech, but uses the same concept, right?

Cleary: Yes, we've since upgraded our approach, as The Sims game doesn't work anymore on modern-day computers. In fact, I think we had to use a CD to get the game to run. And so, in order to allow other scientists to use the paradigm and replicate and extend the findings, we really needed to upgrade our approach.

This past year we published our upgraded version of the program, which is publicly available now on the open science framework so other researchers can pull from that to use it. It’s the same concept, though. I worked with a student from Emory University named Noah Okada. He was a computer science undergraduate then, now he’s a graduate student at Caltech. He basically recreated all of the scenes using the same idea but with novel scenes following from the same logic and approach.

Host: So, what did you find as you were doing experiments and seeing how people reacted to these very similar images?

Cleary: So, I come from a very long tradition in memory research of studying memory processes by giving people lists of things to remember and then giving them tests, like a recognition test. I have them do things like discriminate old from new or make familiarity judgments or try to recall something from earlier on in the experiment that a cue on a test might remind them of.

This was a virtual-reality version of that, where people would be basically zapped from one scene to the next. Today, our most common variant of this paradigm is to get pulled through a scene in a series of twists and turns, like you're on a tour of it in virtual reality. You're wearing a headset, and you can turn your head to look around and explore as you're being moved through the scene. You go through a number of these touring different scenes and exploring them. Then in a later phase of the experiment, it's the same thing. You're going through a number of different scenes. They're all unique, except later in the experiment, some of the scenes in a randomized order have the same spatial layout as a scene that you've already toured earlier in the experiment.

In the test phase, we prompt people. So, at the end of the tour we will typically ask things like, did you feel déja vu as you were in that scene, or how familiar do you find that scene. Finally, we ask, did this scene remind you of something specific, or did you think of a specific earlier scene that you think resembles this scene. It's interesting to listen to people as they're in the virtual-reality simulation talk about it. You can just hear it in their speech, they'll often say things like, “wow, I'm having déja vu, and I just have this feeling that I've been here before, but I know that I haven't.”

Right now, we're actually carrying out a study in which people are doing their thinking out loud because they love to talk when they're in virtual reality doing this. We're trying to see what linguistic elements seem to accompany déja vu and what we can glean from that.

Host: Let's talk a little bit about some of the misconceptions that people have about déja vu. There's a definite tie in people associate it with the spooky, paranormal experience. They think it's otherworldly, maybe even a form of premonition. Is there anything to that, or is Hollywood and our imaginations going a little wild?

Cleary: It's an interesting aspect of déja vu. Years ago, when I published my first papers on the Gestalt Familiarity Hypothesis of déja vu, I would get some pushback. Usually, it was from members of the public who had come across media reports of the articles and reached out either over email or by phone or sometimes by letter to tell me they thought that I was wrong. They felt that it can't just be a memory phenomenon because for them it's a precognition. I'm using air quotes as I say this because I wasn't familiar with that term “precognition.” I think it comes from the paranormal realm or paranormal literature, and to many people, this is a part of their déja vu experience.

At first, I was sort of dismissive of it as a scientist. I thought, it must just be a bias people have. They just have this pervasive belief that that's what déja vu is. But it kept happening, including at academic conferences. Graduate students and sometimes Ph.Ds. would come up to my students or to me after I'd given a talk and tell us that their experience is that it's this sense of prediction or predicting the future. For many people, it's not just a sense. They feel like they really did predict what was going to happen next.

We started to realize that this is a really common experience that people have. It doesn't seem like it's just a belief, but that when they have déja vu, it strongly feels like it's accompanied by a feeling of prediction, that you knew exactly what was going to happen next, and it happened in exactly that way. So, I became interested in trying to study that and, in fact, had been thinking about it for years. How would I go about studying that?

Eventually, I came up with this idea that maybe it too, is driven by memory. Maybe if a past situation dynamically unfolds in a very similar way to a current situation, maybe that unrecalled memory can contribute to the sense of having been in a place before that you know you haven't been. Maybe the way that an event dynamically unfolded over time in the past is leading you not only to feel a high sense of having been in this event before, but also a sense of knowing where it's going and what's going to happen next.

To develop a way to test that in the lab, basically I came up with the idea using The Sims virtual tour paradigm, where you're pulling people through each scene from a first-person perspective. What we can do is pull people through scenes in the study phase – what we call the encoding phase – in a series of twists and turns within each scene. Then, in the test phase, when they're being pulled through an identically configured scene, have that tour stop short of a turn that was taken earlier and see whether people could predict what the direction of the next turn should be in that test scene, even if they can't recall the earlier scene that's the source of that.

It seemed like a great way to test this hypothesis, that maybe it's a memory driven phenomenon that is allowing people to be able to predict the future in a sense, because it's really the past that's enabling it. Unfortunately, at the time when we ran that study, it didn't work or it didn't work insofar as people had no predictive ability that was grounded in memory, unless they recalled the earlier scene. Then they could predict what the direction should be on the basis of that recalled memory of a highly similar scene and what the direction of the turn was there. But our primary interest is in when people can't recall, can they just have this sense, like when they're having déja vu. Because if so, that would help to explain this mysterious sense of predicting the future.

But again, unfortunately, there was no predictive ability. We found our usual déja vu effects, but just déja vu wasn't associated with any actual predictive ability. So, I kind of sat on that study for at least a year or two, thinking of it as kind of a failure. Then one day it occurred to me that we hadn't asked them if they felt that they could predict the next turn. So, I went back to the lab and decided to test that. I ran the same study, but added a question about whether they feel as if they could predict the direction of the next turn. What we found was this enormous sense of prediction.

When people felt like they were having déja vu, they felt very strongly that they could predict the direction of that next turn, even though they could not. In a replication of our earlier study, there was no actual prediction. But what was fascinating was that there was a very, very strong feeling of prediction accompanying that sense of déja vu. That was our first step toward trying to understand this association between déja vu and feelings of premonition.

We've since replicated that basic, illusory sense of prediction many times. So, it's a very robust, large effect. What it suggests is that, while we don't fully understand why it's there yet, it's there. It's not every déja vu experience, so it's not as if people simply define déja vu as prediction. Roughly half of the déja vu reports in the lab are accompanied by this strong sense of being able to predict what's going to happen next.

Host: You're talking a lot about places. When I have déja vu, it's often a conversation.

Cleary: That is the second most common elicitor of déja vu, according to the survey research.

Host: I must admit, I feel like I knew they were going to say that. It's a really strong feeling that you knew that conversation was going to happen, but also that it's never happened before. Is there a reason why that happens differently – for some a place, others a conversation?

Cleary: So far in my lab, we've been focusing on places as the elicitor, and we haven't looked at conversations yet. I think primarily because of the logistics involved in trying to create that in something like virtual reality. It's not impossible.

My students and I have talked about the possibility of creating avatars that might talk to you and gesture, and could we somehow mimic that experience of feeling when you're talking to someone for the first time but you feel like they've said this before and you know where this conversation is going or what they're about to say next. I think you could design an experiment like that. It's just a little bit trickier because of the complexity compared to just putting elements of furniture and things on a grid in VR.

It would require this more sophisticated set of avatars who are gesturing and making facial expressions. But some day we may do that. I think it's a great idea and definitely a good next question. Then we could start to compare, are there phenomenologically different experiences when it's a place versus when it's a person or a conversation.

Host: It feels like most things have a purpose. Most things that we experience, there's a reason for it, even if we don't know what it is. So, why would déja vu be an important biological function?

Cleary: That's the million-dollar question. I used to approach the study of déja vu as if it's just an odd quirk of memory that's interesting to study, and we don't really know why it's there. In fact, I even thought of it years ago as perhaps just a side effect or residual aspect of other memory processes. Maybe it doesn't have an important aspect to itself. It’s just this lingering after effect. But I've changed my mind about that.

I now think that it probably does have a useful purpose and is there for a reason. Specifically, I think, and this is based on more recent evidence coming out of our lab, that it's not just a quirk. I think it's there to redirect our attention and have us focus in on our own memories and look for information from those memories that might be relevant to the current situation that we find ourselves in. Where that idea came from is interesting.

Prior to Noah Okada, the student I mentioned earlier, I had pretty much created all of the scenes that we were using in The Sims games. I spent a lot of time playing that game and constructing these scenes for research. So, I never got the chance to experience déja vu myself in the VR environment because every time I would go in to test the program or just explore, I knew every scene because I had created them. I knew which scenes mapped on to which others, and so it wasn't that interesting to me.

But when Noah first sent over the HTC VIVE program for me to test out in my lab to get ready to run here at CSU, I got to go through this program for the first time, not knowing what any of the scenes were or what any test scenes mapped onto because I hadn't created them. They were all new to me.

While I was in the test phase exploring the scenes that he had created, what I noticed was that I was very impressed with the creativity that went into creating these scenes. I was exploring them, really admiring, looking around, turning my head and oohing and awwing about the detail in the granite countertop and the bowl on top of it. The creative placement of different pieces of artwork and elements was very detailed, very high resolution, impressive VR environments. I was really enjoying being pulled through these scenes. They were novel to me, and it was very interesting to explore.

Suddenly, I'm in this beach resort scene that had little huts and tiki torches, and it has a boardwalk near the water, and I was overcome with this sense that I've been through this exact scene on this exact path before. I know I haven't, and I recognized it as a novel scene. So, I found that I had suddenly stopped exploring the scene. I had stopped taking in the details as I was being pulled through the rest of it and was instead racking my brain, searching my memory, trying to conjure up candidate scenes that I'd already been through from earlier, from my memory as possibilities for why I was having this intense feeling of familiarity that really felt like déja vu.

So, my attention turned inward. I was no longer exploring the scene. I was searching my mind for an explanation for why I was having this feeling. It happened to me several more times throughout the rest of the tours. But it really got me wondering if that’s what it feels like for participants in our studies. Because if it does, that might be a missing piece that hadn't occurred to us to explore yet. That when it happens, it switches your attention from being outward focused to being inward focused, searching your memory.

It got me thinking that might be the purpose of déja vu. It turns your attention inward and gets you searching your memory for something that might be relevant to the situation at hand. And if you think about it, that's probably a very useful purpose. It's sort of your brain telling you to stop because there's something here, and you really should figure out what it is that's familiar about this scene.

Host: I love the idea that it's actually your brain trying to give you a little kick to stop what you're doing and take note.

Cleary: Yeah, I love that. A little kick. That's kind of how I think of it now, that its purpose is to give you a little nudge to search your memory that might not otherwise occur. You might just keep exploring the scene, but the sense of familiarity is a sort of alert that there's something here that you've done before, and you should try to figure out what it is.

I think it's also telling us something important perhaps about how the mind works more generally. We don't really know a whole lot at this point in cognitive science about what causes our attention to flip inward when it was outward focused. In cognitive science, there have been decades and decades of studies on attention, telling us what in the environment can pull our attention outward.

We know that when something's novel, it'll tend to grab our attention outward. If something's salient in the environment, it'll tend to pull our attention outward toward it. But we don't know much about what in the environment might instead send our attention inward toward our own memories. I think déja vu is telling us something important there about how our attention gets redirected, sometimes involuntarily. Because if you think about it, déja vu is not really a voluntary form of cognition. It just happens to us, and when it does, it seems like it takes a front seat to whatever it was we were focusing on. It grabs our attention, holds it and redirects our focus. And that may be telling us something very important about how our mind refocuses.

Host: But when you have déja vu, it feels eerie. Is there an element to that that's important? Why do we have that spooky feeling that’s so palpable?

Cleary: People love using the word spooky to describe it. Even when I talk about déja vu and our research at scientific conferences, sometimes a scientist will raise their hand and ask about the “spooky” element.

It's interesting that we tend to remember it as if it were spooky or eerie when it happens. Some of my collaborators and I have been looking at what the emotional aspects are, the phenomenology, the subjective qualities of the experience itself. Is it really spooky? Is it negative or is it more positive?

We're starting to find evidence to suggest that when it does happen, at least in the moments that it happens. It's kind of positive that if we look at, for example, machine-learning approaches and something called natural-language processing to examine people's language when they answer survey questions about their déja vu experiences, they tend to have a positive slant to the descriptions of déja vu in the language that they're using.

It's an interesting question: Why is it so associated with the spooky? Is it because it has this paranormal sort of societal paranormal expectation around it or belief system around it? Or is there something else that leads us to feel like when we have déja vu, it's spooky. I don't have an answer to that yet, but we're doing a lot of studies right now, mainly with machine learning in collaboration with some folks in the computer science department here at CSU to try to start to get at that. Looking at people's language, if they talk out loud while they're in virtual reality. For example, can we then take the language itself and train machine-learning algorithms to differentiate when people do have déja vu versus when they don't? Are there linguistic qualities that seem to differentiate? And if so, are they sort of spooky qualities? Is their language indicating spookiness at all? Does the word "spooky" come up at all in their language? Those are good questions that I think we might be able to answer soon as we explore this.

Host: Well, thank you so much for taking the time to talk with you today. I really appreciate it.

Cleary: Thank you.

Host: That was CSU Psychology Professor Anne Cleary speaking about the memory phenomenon known as déja vu. I'm your host, Stacy Nick, and you're listening to CSU's The Audit.