top of page

From Vision to Multisensory Experience: dialogues between cognitive science and design

By Andréa de Paiva

The eye collaborates with the body and the other senses. One’s sense of reality is strengthened and articulated by this constant interaction. ” Pallasmaa, 2005, p.68 [1]

From Vision to Multisensory Experience: dialogues between cognitive science and design | NeuroArchitecture

In recent years, the intersection of neuroscience and architecture has generated discussions about how the spaces we inhabit influence our experiences and well-being. Each environment offers users a unique experience, formed by: (a) combination of its physical characteristics that are perceived through our senses; (b) its use and the possibilities for action that the environment offers, known as affordances; (c) the symbols and meanings that the environment represents for those who occupy it; (d) the unique interpretation that each individual makes of that space, based on their genetics, memories and emotions, among others. Such an experience, which is multisensory, contributes not only to our awareness of the environment in which we are, but also to our perception and awareness of what we experience in these spaces. The question that arises, then, is how does this happen?

To begin our discussion, I propose that we look at a notable event that happened in the past and which culminated in the development of NeuroArchitecture. The personal experience of renowned scientist Jonas Salk, in the 1950s, when he created the first polio vaccine, is considered a starting point for the birth of NeuroArchitecture [2]: at a time when his investigations were locked in the traditional environment from the laboratory, Salk took a sabbatical to spend at the Abbey of Assisi, in Italy. It was there that he had the insight that led him to create the vaccine. In other words, in that extremely sensorially and symbolically rich environment, completely different from a laboratory, which is a more closed space with an aseptic appearance, he found the appropriate mental state to create the vaccine that would save thousands of children in the following years. He, in turn, not only detected the change in his mental state, but also its relationship with the physical environment, which inspired him to support research into how the physical environment can affect our brain and our behavior. Cases like this, combined with all the research that has been emerging on perception and consciousness, make us question how much we are impacted by the multisensory environment in which we find ourselves.

With more than 90% of our time spent in built environments [3], the design of cities and buildings has evolved significantly over the last century. Technological advances, from steel structures to electric lamps, air conditioning, elevators and cars, have transformed our surroundings and the way we interact with them. Along with amenities, this evolution also brought challenges, such as the increase in urban chaos, noise pollution and environments disconnected from human nature.

In addition to our mental state, as we saw in Salk's example, the environments we occupy also impact our well-being. For example, studies show that exposure to nature and natural elements tends to positively affect behavior by decreasing stress levels [4, 5], supporting cognition restauration [6], and diminishing risks of developing mental disorders [7, 8], among others. Supported by these studies, concepts such as Biophilic Design emerge, which seeks to incorporate natural and nature-inspired elements into built environments, with a focus on improving general well-being.

The effects of the environment are not limited to the presence or absence of nature. Studies have shown that music and soundscapes can change a doctor's performance in surgery, as well as a patient's perception of pain and the passage of time [9]. There is also research that shows how natural light in a classroom can improve cognition, increasing levels of attention and learning [10]; and studies on the effects of the tactile experience of certain materials, in addition to research on the effects of our olfactory experience on the environment, among many other topics.

As interesting as it is to identify how different qualities of the environment influence the people who occupy it, we cannot be tempted to limit ourselves to studying only the individual qualities and not the whole that arises from the combination of parts [11]. Understanding how different sensory stimuli interact when combined in the same environment is crucial. The McGurk Effect, for example, demonstrates how congruent information from multiple senses can enhance perception, while incongruence can disrupt it.

Applying this to architecture, congruent multisensory cues, including visual, sound and olfactory elements, can create more immersive and wellbeing-friendly atmospheres. Imagine seeing, in your living room at home, a beautiful wooden floor and, when taking off your shoes and walking barefoot across it, feeling the coldness and hardness of an (incongruous) porcelain tile instead of the more welcoming temperature and texture from wood (congruent).

The presentation of information from multiple sensory modalities in the environment can impact both behavior and perception. It has been demonstrated that multisensory stimuli, when congruent, decrease simple reaction times and facilitate stimulus detection, suggesting a convergence and integration of these signals in the brain, as advocated by Barry Stein and Alex Meredith [12, 13]. On the other hand, incongruity can reduce neuronal responses, making it difficult for us to interpret the environment and its stimuli, which can have detrimental effects on behavioral and cognitive performance.

The congruence of multisensory stimuli does not only concern their physical qualities. Semantics—that is, meaning—also plays a key role in determining whether multisensory stimuli are congruent or incongruent. When we enter an environment that has tables and chairs and a blackboard hanging on the wall, we immediately make a connection with the idea of "classroom". If a bed is also present in this same environment, this object - incongruous with the idea of a classroom - generates noise in the immediate interpretation we make. However, the idea of a classroom is formed based on each individual's experience. Therefore, what a person recognizes as a classroom does not necessarily encompass the same physical characteristics that another person from a different culture might consider as such. This means that our memories also influence perception, generating faster responses in the presence of cues that are semantically congruent with what we know [14].

Hospital design, which has been changing recently, is making good use of this knowledge about the relationship between memory and perception. Instead of just offering cold environments with an excessively aseptic appearance as before, hospitals are increasingly looking like real hotels. The resulting atmosphere positively impacts the experience of users, especially patients and companions, not only by being more welcoming, but by creating a greater bond with memories associated with past experiences in hotels, which are normally more positive and pleasant than the associated memories experiences in clinical and hospital environments.

Studies on the congruence of multisensory cues in architecture are not limited to hospital design. There is a lot of research on the consumer experience in stores and restaurants. For example, studies have shown that playing French music in a wine store led to French wine outselling German wine and playing German music led to German wine outselling French wine [15].

“Every touching experience of architecture is multi-sensory; qualities of space, matter and scale are measured equally by the eye, ear, nose, skin, tongue, skeleton and muscle. Architecture strengthens the existential experience, one’s sense of being in the world, and this is essentially a strengthened experience of self. Instead of mere vision, or the five classical senses, architecture involves several realms of sensory experience which interact and fuse into each other.” (Pallasmaa, 2005, p.68 [1])

Unfortunately, our culture that is extremely attached to vision ends up influencing architecture and encouraging projects that are visually magnificent, that generate beautiful photos for a magazine or for Instagram, but that do not necessarily generate a more human and welcoming experience from the point of view of the user who Explore the environment with your entire body. Dialogues between cognitive science and architecture have the potential to help architects and designers understand more deeply the relationship between people and spaces, encouraging them to incorporate this more human and multi-sensory point of view into their projects. Encouraging such interdisciplinary dialogues helps instigate a transformation in the approach of design professionals, favoring a more holistic view of spaces and their users and transcending mere visual aesthetics.

Was this article useful? Leave us a comment and follow NeuroAU

on social media!😊


[1] Pallasmaa, J. (2005) The Eyes of the Skin: Architecture and the Senses. John Wiley and Sons

[2] Eberhard, J.P., 2009. Brain Landscape: the Coexistence of Neuro-science and Architecture. University Press, Oxford.

​​[3] WELL Building Standard (2016)

[4] Ulrich, R. S. (1983). Aesthetic and affective response to natural environment. In I. Altman & J. F. Wohlwill (Eds.), Behavior and the natural environment (pp. 85–125). New York: Springer. Retrieved from http://link​.springer​.com/chapter/10.1007/978-1-4613-3539-9_4

[5] Ulrich, R. S., Simons, R. F., Losito, B. D., Fiorito, E., Miles, M. A., & Zelson, M. (1991). Stress recovery during exposure to natural and urban environments. Journal of Environmental Psychology, 11(3), 201–230. doi:10​.1016/S0272-4944(05)80184-7. 

[6] Kaplan, S (1995) The restorative benefits of nature: Toward an integrative framework. Journal of Environmental Psychology. Volume 15, Issue 3, Pages 169-182

[7] Engemann, K., Pedersen, C. B., Arge, L., Tsirogiannis, C., Mortensen, P. B., & Svenning, J. C. (2019). Residential green space in childhood is associated with lower risk of psychiatric disorders from adolescence into adulthood. Proceedings of the National Academy of Sciences of the United States of America, 116(11), 5188–5193.

[8] Krabbendam, L., & van Os, J. (2005). Schizophrenia and urbanicity: a major environmental influence--conditional on genetic risk. Schizophrenia bulletin, 31(4), 795–799.

[9] Spence, C., Keller, S. (2019) Medicine’s Melodies: On the Costs & Benefits of Music, Soundscapes, & Noise in Healthcare Settings. Music & Medicine. Volume 11, Issue 4. Pages 211 – 225 

[10] Shishegar, N.,  Boubekri, M. (2016) Natural Light and Productivity: Analyzing the Impacts of Daylighting on Students’ and Workers’ Health and Alertness. Conference: International Conference on "Health, Biological and Life Science" (HBLS-16)At: Istanbul, Turkey

[11] Chatterjee, A. (2024) The Neuro-Architecture Triad: A Cheat Code for Designers. Psychology Today.

[12] Stein, B.E., Meredith, M.A. (1993) The Merging of the Senses. MIT Press.

[13] Wu, C., Wick, F., Pomplun, M. (2014) Guidance of visual attention by semantic information in real-world scenes. Front. Psychol., 06 February 2014 |

[14] Laurienti, P., Kraft, R., Maldjian, J., Burdette, J., Wallace, M. (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res (2004) 158: 405–414

[15] North, A., Hargreaves, D., McKendrick, J. (1999) The Influence of In-Store Music on Wine Selections. Journal of Applied Psychology 84(2):271-276

109 visualizações


bottom of page