Search publications

Reset filters Search by keyword

No publications found.

 

Statistical or Embodied? Comparing Colorseeing, Colorblind, Painters, and Large Language Models in Their Processing of Color Metaphors

Authors: Nadler EOGuilbeault DRingold SMWilliamson TRBellemare-Pepin ACom?a IMJerbi KNarayanan SAziz-Zadeh L


Affiliations

1 Department of Astronomy & Astrophysics, University of California San Diego.
2 Graduate School of Business, Stanford University.
3 Mrs. T.H. Chan Division of Occupational Science and Occupational Therapy, University of Southern California.
4 Brain and Creativity Institute, Dornsife College of Letters, Arts and Sciences, University of Southern California.
5 Brain, Language, and Behaviour Laboratory, Bristol Centre for Linguistics, University of the West of England.
6 Southmead Hospital, North Bristol NHS Trust.
7 Faculty of Linguistics, Philology, and Phonetics, University of Oxford.
8 Department of Music, Concordia University.
9 Department of Psychology, University of Montreal.
10 Google DeepMind.

Description

Can metaphorical reasoning involving embodied experience-such as color perception-be learned from the statistics of language alone? Recent work finds that colorblind individuals robustly understand and reason abstractly about color, implying that color associations in everyday language might contribute to the metaphorical understanding of color. However, it is unclear how much colorblind individuals' understanding of color is driven by language versus their limited (but no less embodied) visual experience. A more direct test of whether language supports the acquisition of humans' understanding of color is whether large language models (LLMs)-those trained purely on text with no visual experience-can nevertheless learn to generate consistent and coherent metaphorical responses about color. Here, we conduct preregistered surveys that compare colorseeing adults, colorblind adults, and LLMs in how they (1) associate colors to words that lack established color associations and (2) interpret conventional and novel color metaphors. Colorblind and colorseeing adults exhibited highly similar and replicable color associations with novel words and abstract concepts. Yet, while GPT (a popular LLM) also generated replicable color associations with impressive consistency, its associations departed considerably from colorseeing and colorblind participants. Moreover, GPT frequently failed to generate coherent responses about its own metaphorical color associations when asked to invert its color associations or explain novel color metaphors in context. Consistent with this view, painters who regularly work with color pigments were more likely than all other groups to understand novel color metaphors using embodied reasoning. Thus, embodied experience may play an important role in metaphorical reasoning about color and the generation of conceptual connections between embodied associations.


Keywords: CognitionColorEmbodimentLanguageLarge language modelsMachine learningMetaphor processing


Links

PubMed: https://pubmed.ncbi.nlm.nih.gov/40621800/

DOI: 10.1111/cogs.70083