codeblue : a bluetooth interactive dance club system

Ravi Jain 12/2/03 1

CodeBLUE: A Bluetooth Interactive Dance Club System

Dennis Hromin, Michael Chladil, Natalie Vanatta, David Naumann,

Susanne Wetzel

Dept of Computer Science,

Stevens Inst of Technology

Farooq Anjum, Ravi Jain*

Applied Research,

Telcordia Technologies

*Current affiliation: DoCoMo USA Labs

[email protected]

Ravi Jain 12/2/03 2

Outline

• Motivation and Background

• System design

• Implementation

• Lessons learned

• Conclusions

Ravi Jain 12/2/03 3

Motivation

• Collaborative music creation– One of the most powerful forms of human

communication

• Can Bluetooth be used to enable music collaboration for untrained participants?– What are the strengths and limitations of

Bluetooth for such an application?

Ravi Jain 12/2/03 4

Approach

• Allow participants to transform simple dance movements into musical modifications

• Why Bluetooth?– Low cost

– Unlicensed ISM band

– Low power

– Small size

– Cool name …

Ravi Jain 12/2/03 5

Requirements

• Unobtrusive– Small size, low weight, easily wearable– About the size of a club wristband

• Rugged– Mosh pits

• Long battery life to minimize maintenance• Very low cost

– Ideally pennies, but dollars may be acceptable

• Responsive

Ravi Jain 12/2/03 6

System design

Sensor

Access Point

Single-board Computer

codeBLUE DJPC

Club Sound SystemClub Light System

MIDI Converter

Light Synthesizer Music Synthesizer

OriginalMusic Modified Music

Slave

Master

Ravi Jain 12/2/03 7

Outline


• System design

• Implementation

• Lessons learned

• Conclusions

Ravi Jain 12/2/03 8

CodeBLUE SensorsSensor Parameter Method Range Size (mm)

G-force Single-axis Acceleration

Piezo-electric

+/- 2.5 G 56x38x19

Close Proximity IR reflection

80 cm 13x40x30

Bend Flex angle Piezo-resistive

0-130o 110x7x0.5

Touch-Mini

Contact Pressure

Force-sensitive resistor

100g-10Kg

19 (circle) x0.5

Hot Temp Zener -14-100o C 6x10x8

Light Ambient light

Photo-resistor

130o

acceptance

6x10x5

Ravi Jain 12/2/03 9

Wireless sensor module

Body Sensor

PIC Microcontroller

UART Interface

Baseband Controller

RF Transceiver

Bluetooth Module

Up to 5 sensors are supported

364 bytes RAM, 8 kB ROM

Bluetooth 1 Mbps ACL data link

Ravi Jain 12/2/03 10

Protocol stack

• For the wireless link, L2CAP only is used• Higher layers of Bluetooth & IP not used to save memory & time

• Data sent to AP consists of sensor ID plus 7-bit digitized sensor value

Baseband / RF

LMP

HCI

L2CAP

Presentation Layer

Sensor Data Acquisition Application

Ethernet

IP

UDP

Presentation Layer

MIDI Applications

Baseband / RF Ethernet

IP

UDP

LMP

HCI

L2CAP

Bluetooth-to-IP Conversion Layer


DJ Console and Controller

• Data Filter algorithms must keep output within acceptable ranges• Must be smooth yet obvious to participants• Must be flexible enough to give the DJ creative room• Must be accessible via a simple, intuitive GUI• Fertile area for future research!

Configurator

Data Filter MIDI Player

MIDI Song Files

Sensor data


MIDI

• What is Musical Instrument Digital Interface (MIDI)?– binary data files containing instructions that tell

electronic instruments how to play a song– much like musical scores

• Why MIDI?– Much more efficient representation

• PCM: 10 MB/minute of stereo audio sampled at 44.1 kHz• MIDI: 10 kB/minute

– Can easily modify instructions in real time


Outline


• System design

• Implementation

• Lessons learned

• Conclusions


Related work• The Brain Opera, Machover 1996, 2000

– Not wireless

• The Interactive Dance Club, Synesthesia 1998– Not wireless

• Cybershoe, Paradiso 1998– Similar but limited to shoe, possible scalability issues, higher cost

• CosTune, Nishimoto 2001– Not a collaboration on a single song

• Music in Motion, Ng 2000, 2001– Interprets movement captured by a video camera

• Web of Life, Center for Art and Media, Karlsruhe, permanent display– Most similar, video images change in response to users hand movements


Discussion

• System Cost– Sensor module:

• $500 for Bluetooth (Yr 2000), $100 for the rest• Expected cost in volume is ~$5

– Access point• About $1000 as implemented (with Linux single-board PC)• Can be easily reduced to ~$100

– DJ Console and converter• Standard PC with MIDI player: ~$1000

– Sound and light hardware• MIDI converter (MidiMan), Dance Synth (E’Mu Mo’Phatt), Audio

Mixer (Mackie), Lighting Controller (DMX)• ~$4000


Discussion

• Total Effort: 24 staff-months• Usability

– Sensor module• 7 cm x 5 cm x 3 cm (without the sensor itself) • About 100 g• Size can be reduced (at least) 4x by one-chip

Bluetooth module and Li-Ion polymer battery

– DJ GUI– Data Filter algorithms


Wireless technology• codeBLUE requirements

– differ slightly from those generally considered in the literature• Voice, video, web browsing, m-commerce transactions• Needs very low bandwidth (~2 kbps) but also low latency (< 100 ms)

– Low-cost, small client size, low-power, rugged– Low maintenance cost and complexity

• Bluetooth is adequate– Designed as a cable replacement– Low bandwidth, small size, low latency– Reasonable spectral efficiency– Easy serial interface


Bluetooth Issues for codeBLUE

• User module cost still too high• Installation requires cabling to AP

– Expensive, cumbersome, unsightly, too fixed

– Power to AP could be avoided by batteries but currently consumption is too high

– Using wireless backhaul is problematic• Bluetooth scatternet requires complex scheduling and may not

meet time constraints

• 802.11b backhaul raises cost and interference issues


Other Bluetooth Issues

• Spatial Capacity– Max of seven active slaves per master (~10m – 30m

radius)

– Can put slaves into PARK, HOLD etc but this is complex

• The protocol stack is needlessly complex (for this application, at least)– Would prefer native IP running on a simple link layer

• Security


What about good old 802.11b instead?

• Cost still an issue• Power consumption is worse

– 180-450 mA versus ~70 mA

• Low spectral efficiency for small data packets• Hard to get delay guarantees (especially if shared)• Larger real-estate requirements, with large parallel

bus• Cabling issues still arise


Bluetooth vs 802.11b for codeBLUE

Attribute Bluetooth 802.11b

User module cost Slightly high Too high

Power OK Too high

Spectral efficiency OK Low

Delay OK OK, but not if shared

Spatial capacity Low OK

Cabling Problematic Problematic

Protocol Complex Simple

Security Not clear Not clear


Conclusions• codeBLUE was demonstrated twice to a live audience

– Stevens Inst of Technology annual Technogenesis Award, 2001

• Sensor technology is more than adequate• Bluetooth technology is adequate

– Better than 802.11b– but not ideal

• Ideal: – low-cost, low-power, multi-hop wireless telemetry network with – simple, efficient, IP-based, secure protocols and– rugged, disposable sensor clients


Further work• Needed

– A protocol stack that provides IP, and • marries the simplicity of 802.11b with

• the low power, size and spectral efficiency of Bluetooth

– A low-cost, rugged wireless sensor module for human-scale measurements

– Simple, configurable wireless security


Work in Progress

• In progress: case study for research in Bluetooth security – DJ PC application rewritten in Java to allow real-time

plug-in of 3rd party software to allow the DJ more choices

– But plug-ins open security risks– Rely on Java and .NET code access security

mechanism for runtime checks– But are these necessary and sufficient?– Use static analysis [Banerjee02,Banerjee02a]


Wireless sensor module:Hardware implementation

codeblue : a bluetooth interactive dance club system

Documents