Project Overview

This interactive application was developed as part of a medical awareness campaign for exhibition use. Designed to simulate the perceptual distortions caused by schizophrenia, the experience combines augmented reality visuals, 3D characters, and audio with a Kinect-based mirror setup. The goal was to create a thought-provoking, immersive demonstration that helps users better understand the subjective reality of those living with severe mental disorders.

Solution

When a visitor steps in front of the screen, Kinect detects their presence and initiates one of several predefined AR effects. A 3D character appears behind or beside the user’s reflection and interacts visually and audibly, creating the illusion of a non-existent threat — a typical symptom of schizophrenia.

To ensure accuracy and immersion, a configuration mode allows operators to:

• Adjust the lighting and character position for each effect
• Define trigger points and calibrate the camera angle
• Preview and fine-tune alignment for various booth setups

Photos of users during the experience are saved for post-processing or review, enhancing engagement and offering potential follow-up materials.

Impact

The simulation served as a powerful awareness tool, giving users a brief, impactful glimpse into the daily challenges faced by people with schizophrenia. Its mirror-based interaction helped ground the experience in realism, leaving a lasting impression on exhibition visitors.

Technical Details

• Engine: Unity
• Technologies: Kinect v2, Kinect SDK, MS-SDK
• Platform: Windows 10 x64
• 3D Content: Modeled and optimized in Blender

Core Features:

• Real-time mirror simulation with AR character effects
• Configurable lighting, character positioning, and triggers
• Voice playback for immersive feedback
• User photo capture system

This project demonstrates how interactive technology can be used not just for entertainment or engagement, but for empathy-driven education and awareness.