ScratchWorx VJ/DJ table
2003 · Developer
Two performers at the console, gooseneck controllers between them, pixel figures projected on the screen behind and the interface on a laptop to the right.
A multi-media DJ/VJ mixing console for live performance by three or more people at once. All the digital material is mixed the way a DJ or VJ would, but through hardware controllers rather than the usual interface. This project gave me the perfect opportunity to make software tangible. Hence my master thesis, Gaming with feeling.
Waag Society provided the underlying software, KeyWorx, and afterwards took the prototype to a next level themselves, with Bas van Abel.
What was asked for
Waag Society had KeyWorx, an abstract VJ/DJ application with every media type and input device you could think of, and filters and settings enough to combine them endlessly. The task was to make that usable by teenagers, for an educational programme called Creative Learning, and specifically for Montessori students.
The complexity was in the interface as much as the media: hexagons, octagons and opening spirals with dots that could be dragged and interconnected, with every KeyWorx client logging in to the same server so that all of them together made one performance in image and sound.
My part was the physical interfacing: connecting what a person did with their hands to what the software did in response.
How it was put together
Three GUI consoles, one realizer, one server, and a protocol holding them together.
Each console talked to the AMUSE server over an XML socket, and AMUSE broadcast every message to everyone logged into that space: the other two consoles and the realizer. So a change one performer made appeared on the other two screens as well as in the output. The hardware sensors went straight into the realizer, which handled the MIDI and pushed it back out to AMUSE over RTP, which also meant an external realizer could join a session from elsewhere.
In those days getting a sensor reading into a computer meant the USB port and an i-CubeX, four sensors per client.
Media files lived on the ScratchWorx server rather than travelling with the performance, so nothing stalled waiting for a download. When the realizer started it sent its media list up to AMUSE, which pushed it out to the consoles to fill their libraries.
The Flash side was a small API over that socket, with the vocabulary the instrument needed: log in, enter a space, join the resource and patch channels, fetch the file list, make and delete connections between modules, set an attribute on a filter. A ping-pong kept the socket alive, because AMUSE would drop it on a timeout.
Who built it
Bas van Abel, Roel Creemers, Deneva Goins, Daan Jobsis, Dirk van de Mortel and Annemieke Vliegen.