The Stuff of Thought is the Stuff of Experience
In this study (), my colleagues and I addressed the question of how the contents of our thoughts ā concepts, ideas, beliefs ā are related to the physical world that we experience through our senses. This issue has long been debated among philosophers and cognitive scientists, a debate that can be traced back to the Ancient Greek philosophers Plato and Aristotle. According to Plato, humans are born with āinnate ideasā, which are mental templates for all the sorts of things that we can name or recognize. These innate ideas were the soulās memory of its prior existence in the āideal worldā. Plato argued that, since no two objects ā say, trees ā generate the exact same perceptual experience, it would be impossible for someone who comes across a particular tree for the first time to be able to recognize it as the same kind of thing as the trees they have encountered before unless they had an abstract template for the category "tree" somewhere in their minds. In the absence of such a template, any new object would be experienced as completely unique rather than a member of a kind. Innate ideas provided the abstract templates that allowed different things to be grouped together as a category. According to this view, as a child interacts with the world and comes across different things such as people, dogs, or trees, she would recognize each of them based on her innate, idealized template for that kind of thing. A similar view was later defended by the philosopher RenĆ©e Descartes, who argued for a sharp, qualitative distinction between the mortal sensing body and the immortal thinking soul.
Aristotle, on the other hand, argued that mental representations for natural kinds were not present from birth, but were learned from experience. By experiencing different trees, one gradually learns what is common to all of them (e.g., certain shapes, colors, sizes, etc.), and these commonalities, once they are abstracted away from particular experiences, provide the basis of the concept ātreeā. This view was further developed by the 17th century philosopher John Locke, who famously stated that the human mind is born as a blank slate that is written on by experience.
Today, researchers are still split with respect to this issue. Of course, no cognitive neuroscientist seriously believes in Platoās version of innate ideas or in Lockeās notion of tabula rasa, but a version of the debate persists. Some researchers argue that very basic natural kinds such as āanimalsā, āplantsā, or ātoolsā have become encoded in our genome via natural selection over the course of human evolution, and that these a priori categories provide a basis for learning new concepts. They believe that concepts are represented in the brain in some kind of āsymbolicā form, that is, the concept representation itself does not contain information about the sensory experiences through which they were learned. Just as computers represent information about images, sounds, or keystrokes using a symbolic digital code (i.e., the 0s and 1s in computer binary code), concepts would represent information about the world using a symbolic code that contains no information about visual, tactile, auditory, or other sensory experiences. Perceptual experiences of different trees would all be associated with the same conceptual symbol in the brain, thus allowing each of them to be recognized as an instance of the kind ātreeā; however, the conceptual symbol for ātreeā would contain no information about the sensory properties of those experiences. Because representations of this type would encode primarily information about category definitions, we call them ātaxonomicā representations.
This view is opposed by researchers who argue that concepts are formed through gradual generalization over sensory experiences, and that they simply encode information about which features of experience (e.g., colors, shapes, textures, sizes, sounds, locations, smells, etc.) must be re-activated when the concept is recalled. This view, known as āembodiedā or āsituatedā cognition, states that a concept is nothing but the condensed, abstracted record of the perceptual and emotional experiences that led to its formation. If that is true, then to think of the concept āappleā means to re-activate the sensory and affective representations that we learned to associate with that word, such as its typical shape, its typical color, its taste, its weight, and any emotions we have associated with it. There would be no symbolic code for concept representation that is dissociated from sensory and emotional information; the neural code for concepts would be a composite of the neural codes used for sensory and emotional representation.
More recently, some researchers have also proposed that, for word concepts (i.e., concepts that correspond to the meaning of a word, such as ādogā, āphoneā, or āinfinityā) one way in which the brain may represent them is by keeping track of how often specific words appear together during natural language use. The word ābirthdayā, for example, often occurs close to the words āyearsā, āoldā, āpartyā, ācakeā, etc., in conversation or in text. If the brain keeps tabs on how often a word co-occurs with every other word, it could, in principle, use that information to represent its meaning. In fact, that is how computer applications such as Google or Alexa represent word meaning. They can tell, for example, that āwolfā is more similar to ādogā than to ācatā, and that āpeelingā goes with āorangeā but not with āwatermelonā. This type of information is called ādistributionalā, because it is based on how a word is ādistributedā among the others in large bodies of text.
In our study, we attempted to find out how much of each type of information (taxonomic, experiential, and distributional) is encoded in the neural representations of word concepts. Our experimental approach is based on the fact that concepts can be more or less related to each other (e.g. āappleā is more related to āorangeā than it is to ābaseballā). However, the degree to which two concepts are related depends on the type of information used to define the relationship. For instance, based on perceptual information alone, āplateā is more related to āfrisbeeā than it is to āspatulaā, even though āplateā and āspatulaā are typically thought of as ākitchenwareā and āfrisbeeā as a toy. And based on co-occurrence information alone, āplateā is more closely related to ātableā than it is to either āspatulaā or āfrisbeeā. If we can determine how related, or how similar, these concepts are to each other according to the brain, we can learn something about the kinds of information that are encoded in their neural representations. Thus, we set out to record the patterns of neural activation in the brain that correspond to different word concepts and determine their āsimilarity structureā, i.e., how similar they are to each other.
We used a neuroimaging technique known as functional magnetic resonance imaging (fMRI) to measure neural activity throughout the brain while participants read each word on the screen. We did that for 524 different word concepts belonging to several categories, including animals, plants, tools, vehicles, places, human occupations, social events, natural events, and abstract concepts (across 2 separate experiments). This figure illustrates how we used the brain activation patterns induced by the words to determine the similarity structure of concept representations: