Many years ago a colleague of mine noticed that we, humans, are walking miracles – we can cross a street while at the same time talking on the phone, chewing gum, drinking water, smelling and sensing the fresh spring weather, measuring the distance and speed of the cars, turning and stopping when needed and managing hundred of other external factors. Since we know our brains can process one piece of information at a time (and thus it is better to be less distracted and more focused), how do we manage all this mess? Lets explore how we are influenced by different type of sensors – hearing, taste, visual, touch and smell.

In one experiment researchers showed the audience a video with a person saying the sound “ga”. In the middle of the video, researchers turned off original voice and started playing “ba” sound, with no interruption, almost like the person would naturally switch the sound. It turns out that if participants of the experiment had eyes closed, they could easily recognize the switch of the sound, but if the participants had the eyes opened, they say that the sound is unchanged or changed to “da”, something middle between what their eyes see (persons lips continue to show “ga”) and what they hear (sound “da”). Basically, our brains have to make a decision and it becomes a compromise, adjusted by what the eyes see. To make things more complicated, another study shows that if someone speaks on a video with a sound switched off, the audience’s brain area responsible for processing sounds is as active, as the actual sound was produced.

Researchers say that senses pass three stages: conversion of external information into code, routing the code through processing (every sense has different processor) with some supervision of brain region called thalamus and finally merging all information into perception. During the merging process, we analyze and interpret the inputs, sometimes changing the data, as in the case of the sound “ga” giving some weight to what eyes see and what ears hear. As a result, two people can get the same input, but come with different conclusions.

What is outstanding, different senses combined seems to produce a better learning and a longer memory. According to psychologist Richard Mayer presentation at Harvard, both learning outcomes: retention (recall or recognize) and transfer (evaluate or use the material in a new situation) can be enhanced with using multimedia – several forms of the message. He concluded that we learn better from words (both spoken and written) and pictures (animation, video, illustrations, photos) than the words alone. In the more detailed study he estimated the following principles:

Coherence: we learn more deeply when irrelevant material (words, pictures, sounds) is excluded rather than included.
Signaling: people learn more deeply when cues are added that highlight the main ideas and organization of the material. For instance, your learning should increase just be cause this text was written in bold.
Redundancy: we learn more deeply from animation and narration than from animation, narration, and on-screen text. In other words we should remember and learn from movies better if we watch without subtitles.
Spacial proximity: we learn more deeply when corresponding printed words and graphics are placed near rather than far from each other on the page or screen.
Simultaneous: we learn more deeply when corresponding graphics and narration are presented simultaneously rather than successively.
Segmenting: we learn more deeply when a narrated animation is presented in learner-paced segments than as a continuous unit. Thus it is better to present one slide than allow to press the button continue for the next slide rather than keep going continuously.
Pre-training: we learn more deeply from a narrated animation when they have had training in the names and characteristics of the main concepts.
Modality / Preference: People learn more deeply from graphics and narration than from graphics and on-screen text. In other words, some methods are more meaningful than others.
Personalization: we learn more deeply when words are in conversational style rather than formal style. Thus, just adding words yours, you increases the learning.
Voice: we learn more deeply when the narration is spoken in a standard-accented human voice than a machine voice.

The significance of multimedia could be illustrated with another research, where three groups were placed in different rooms and in one room people received one sensual information (for instance hearing), another room was presented with other sense (visual) and the third room with two senses combined (hearing and visual). The memory was always better for the combined senses. In other experiment multisensory presentations generated +50% more creative solutions on problem-solving. Another study shows that touch combined with visual information increases the learning by +30% comparing to touch alone. All this leads to an interesting conclusion – the more our brain has to “work” in processing various pieces of information, the better is the learning, but not any kind of work and not all the work is equally effective.

In another study people, who saw a movie with added popcorn smell in the theater remembered 2 times more comparing with the ones that had no smell. However, when testing different memories, the effect was not the same. Odour has the most effect when recalling emotional details of the memory and when the smell is relevant (principle of redundancy), but has little effect on declarative type of memory.

The phenomena of smell to recall memories is called the Proust Effect. This effect was described in the book “Remembrance of Things Past”, written by Marcel Proust. Smells bring back emotions, because smell stimulates the amygdala, the brain part which is responsible for emotional experiences.

It is nothing new that some companies are using the smells to grow the sales. In one experiment the machine with a scent of chocolate increase the sales of chocolate by +60%, in another experiment a scent of waffle increased the ice-cream sales by +50%. In another experiment, clothing store sales increased when women’s department was sprayed with the smell of vanilla and men department was sprayed with the rose maroc fragrance. Researches again emphasize that smell has to be relevant (coherence principle).