ŗŚĮϹŁĶų

header-logo
GettyImages-650160957-hero

The Stuff of Thought is the Stuff of Experience

In this study (), my colleagues and I addressed the question of how the contents of our thoughts – concepts, ideas, beliefs – are related to the physical world that we experience through our senses. This issue has long been debated among philosophers and cognitive scientists, a debate that can be traced back to the Ancient Greek philosophers Plato and Aristotle. According to Plato, humans are born with ā€œinnate ideasā€, which are mental templates for all the sorts of things that we can name or recognize. These innate ideas were the soul’s memory of its prior existence in the ā€œideal worldā€. Plato argued that, since no two objects – say, trees – generate the exact same perceptual experience, it would be impossible for someone who comes across a particular tree for the first time to be able to recognize it as the same kind of thing as the trees they have encountered before unless they had an abstract template for the category "tree" somewhere in their minds. In the absence of such a template, any new object would be experienced as completely unique rather than a member of a kind. Innate ideas provided the abstract templates that allowed different things to be grouped together as a category. According to this view, as a child interacts with the world and comes across different things such as people, dogs, or trees, she would recognize each of them based on her innate, idealized template for that kind of thing. A similar view was later defended by the philosopher RenĆ©e Descartes, who argued for a sharp, qualitative distinction between the mortal sensing body and the immortal thinking soul.

Aristotle, on the other hand, argued that mental representations for natural kinds were not present from birth, but were learned from experience. By experiencing different trees, one gradually learns what is common to all of them (e.g., certain shapes, colors, sizes, etc.), and these commonalities, once they are abstracted away from particular experiences, provide the basis of the concept ā€œtreeā€. This view was further developed by the 17th century philosopher John Locke, who famously stated that the human mind is born as a blank slate that is written on by experience.

Today, researchers are still split with respect to this issue. Of course, no cognitive neuroscientist seriously believes in Plato’s version of innate ideas or in Locke’s notion of tabula rasa, but a version of the debate persists. Some researchers argue that very basic natural kinds such as ā€œanimalsā€, ā€œplantsā€, or ā€œtoolsā€ have become encoded in our genome via natural selection over the course of human evolution, and that these a priori categories provide a basis for learning new concepts. They believe that concepts are represented in the brain in some kind of ā€œsymbolicā€ form, that is, the concept representation itself does not contain information about the sensory experiences through which they were learned. Just as computers represent information about images, sounds, or keystrokes using a symbolic digital code (i.e., the 0s and 1s in computer binary code), concepts would represent information about the world using a symbolic code that contains no information about visual, tactile, auditory, or other sensory experiences. Perceptual experiences of different trees would all be associated with the same conceptual symbol in the brain, thus allowing each of them to be recognized as an instance of the kind ā€œtreeā€; however, the conceptual symbol for ā€œtreeā€ would contain no information about the sensory properties of those experiences. Because representations of this type would encode primarily information about category definitions, we call them ā€œtaxonomicā€ representations.

This view is opposed by researchers who argue that concepts are formed through gradual generalization over sensory experiences, and that they simply encode information about which features of experience (e.g., colors, shapes, textures, sizes, sounds, locations, smells, etc.) must be re-activated when the concept is recalled. This view, known as ā€œembodiedā€ or ā€œsituatedā€ cognition, states that a concept is nothing but the condensed, abstracted record of the perceptual and emotional experiences that led to its formation. If that is true, then to think of the concept ā€œappleā€ means to re-activate the sensory and affective representations that we learned to associate with that word, such as its typical shape, its typical color, its taste, its weight, and any emotions we have associated with it. There would be no symbolic code for concept representation that is dissociated from sensory and emotional information; the neural code for concepts would be a composite of the neural codes used for sensory and emotional representation.

More recently, some researchers have also proposed that, for word concepts (i.e., concepts that correspond to the meaning of a word, such as ā€œdogā€, ā€œphoneā€, or ā€œinfinityā€) one way in which the brain may represent them is by keeping track of how often specific words appear together during natural language use. The word ā€œbirthdayā€, for example, often occurs close to the words ā€œyearsā€, ā€œoldā€, ā€œpartyā€, ā€œcakeā€, etc., in conversation or in text. If the brain keeps tabs on how often a word co-occurs with every other word, it could, in principle, use that information to represent its meaning. In fact, that is how computer applications such as Google or Alexa represent word meaning. They can tell, for example, that ā€œwolfā€ is more similar to ā€œdogā€ than to ā€œcatā€, and that ā€œpeelingā€ goes with ā€œorangeā€ but not with ā€œwatermelonā€. This type of information is called ā€œdistributionalā€, because it is based on how a word is ā€œdistributedā€ among the others in large bodies of text.

Stuff of Thought Similarity StructuresIn our study, we attempted to find out how much of each type of information (taxonomic, experiential, and distributional) is encoded in the neural representations of word concepts. Our experimental approach is based on the fact that concepts can be more or less related to each other (e.g. ā€œappleā€ is more related to ā€œorangeā€ than it is to ā€œbaseballā€). However, the degree to which two concepts are related depends on the type of information used to define the relationship. For instance, based on perceptual information alone, ā€œplateā€ is more related to ā€œfrisbeeā€ than it is to ā€œspatulaā€, even though ā€œplateā€ and ā€œspatulaā€ are typically thought of as ā€œkitchenwareā€ and ā€œfrisbeeā€ as a toy. And based on co-occurrence information alone, ā€œplateā€ is more closely related to ā€œtableā€ than it is to either ā€œspatulaā€ or ā€œfrisbeeā€. If we can determine how related, or how similar, these concepts are to each other according to the brain, we can learn something about the kinds of information that are encoded in their neural representations. Thus, we set out to record the patterns of neural activation in the brain that correspond to different word concepts and determine their ā€œsimilarity structureā€, i.e., how similar they are to each other.

We used a neuroimaging technique known as functional magnetic resonance imaging (fMRI) to measure neural activity throughout the brain while participants read each word on the screen. We did that for 524 different word concepts belonging to several categories, including animals, plants, tools, vehicles, places, human occupations, social events, natural events, and abstract concepts (across 2 separate experiments). This figure illustrates how we used the brain activation patterns induced by the words to determine the similarity structure of concept representations: