Imagine being able to see the spatiotemporal structure of any song that crosses your ears. Sensory blending of this sort falls into the realm of synesthesia. There are, in fact, a select few people, synesthesics, who do not require imagination to conceive the above; they experience it as part of their reality!
Such is a very powerful notion. Underpinning synesthesia is sensory crossover, a fundamental feature of our neural architecture. The brain integrates stimuli from all available senses, and by doing so, constructs reality. Synesthesia is thus a feat that most of us are capable of. However, and for whatever reason, our innate synesthetic abilities tend to remain dormant. Need this be the rule? On this note, I am curious: Can immersive technology, such as augmented reality (AR) and virtual reality (VR), be leveraged to summon the synesthetic capabilities that predominantly lie latent?
Pursuantly, I have prototyped a virtual reality experience in which music is chromatically visualized in an information-rich manner. The twelve chromatic tones in Western music’s ‘alphabet’ are mapped along the violet-to-red spectrum that spans the range of human vision. Via an immersive experience of this nature, it becomes possible to see the spatiotemporal structure of music. Moreover, as tones are equated with color, songs are endowed with a sort of ‘synesthetic constancy’, which may be of profound advantage in the realm of music education.
A link to the aforementioned experience is here: A SYNESTHETIC MUSICAL IMMERSION.