Dynamic desktop graphics sonification (alpha)Created on: 2016-01
by: Marek Samec
This patch is part of my doctoral thesis project. The pixels under the mouse curor are sonified. The patch examines possibility to sonify the desktop graphics. The The mouse movement is projected to the sound characteristics (X-frequency), (Y - simple stereo panning). The colors alter the timbre of the resultant signal (RGB levels of pixels under the mouse pointer influence the amplitude of waveforms for every color.

Known glitches:

Y position is incorrectly reflected to the GEM window, X position is ok. I need to dig into this if this is gemmouse issue or something else.

Sound clipping, not much time to fix this right know.

Lots of other small things

White pixels sound is too loud and clips the output
Platforms & version
Pure Data 0.42-extended