Somatic Drawing Tool: 3D Body Sheet as Material for Articulating Synaesthetic Experiences
2026. To appear at ACM CHI 2026.
We present a participatory co-design cycle that studied synaesthesia, a cross-sensory phenomenon where one stimulus automatically triggers another perception. We worked with a community of synaesthetes to develop and evaluate an interactive, immersive technology to depict and communicate syaesthetic experience. By working with qualitative methods to understand experience, we accomodate the highly individual nature and support otherwise limited sensory vocabulary and lack of shared understanding of the condition. The first study involved a series of participatory workshops that explored text, creative tools, interactive technologies, and sensory materials in conveying the synaesthetic experience. Insights from these workshops informed the development of the Somatic Drawing Tool, a VR drawing system that allows users to depict their experiences. Within the tool, we deployed a the 3D Body Sheet, an immersive extension of techniques drawn from soma design. We engaged users in an initial evaluation of the system and took on board their feedback in a subsequent cycle of iterative development. This resulted in a bespoke system that could be personalised to diverse types of individual synaesthetic experience. This enables nuanced articulation, providing a supportive environment for externalising and reflecting sensations. These approaches for capturing the experiential nature of embodied sensory perception has potential to be applied in broader research in cross-modal and multisensory interaction.