Synaesthesia is a neurological trait in which the different sensory modalities communicate with each other or crossover involuntary, and it is also used as a device for literary or artistic concepts.. Marshall McLuhan’s contribution to the discourse of a sensorium demonstrates that the phenomenon of synaesthesia; is a relationship between media, culture and our senses.

Finding a suitable media search index proved more difficult than expected. For now using Internet search engines proves successful and with interesting results on a technical and visual level. We are using Chromatik as images can be searched by colour proportion and keywords. As indexed pictures are obtained from FlickR API, Chromatik takes benefit from available metadata (location, tags, license) to improve the browsing. We use information from the emotion recognition result by Synesketch as search criteria for appropriate images within Chromatik, a searchable index to the image hoster Flickr: the colours associated with the strongest emotion and the words with largest contribution to the emotional evaluation of each phrase. We pick two of these colours at random and two keywords for the image search

A project we have expanded upon, Synesketch is an open-source software library made for digital artists. Their code is a medium of words, emotions, and visuals. The Synesketch approach centres on the idea that affective computer systems should be able to communicate and influence or evoke human emotions. Mapping emotion to color encounters problems of subjectivity and cultural constructs. The Synesktech color palettes try to avoid subjectivity by using the results of research by Pos and Green-Armytage, which come from Ekman’s emotional types. They studied how people from different cultural backgrounds associated colours with emotional types and no statistically significant differences were found between study groups.

We used the graphical programming environment Processing to create the visual design output of kaleidOk. By following the Geometry, Textures & Shaders with Processing tutorial, custom shapes and images are placed onto geometry and feed information to animate the imagery. Aspects of the audio analysis using TarsosDSP information were applied to the motion of the imagery to improve aesthetics and to act as an experiment for further research in this area.