audio networking: the forgotten wireless technologyanil.recoil.org/papers/2005-ieee-audio.pdf ·...

6
1536-1268/05/$20.00 © 2005 IEEE Published by the IEEE CS and IEEE ComSoc PERVASIVE computing 55 Audio Networking: The Forgotten Wireless Technology M odern computing and commu- nication devices offer a wide range of wireless communica- tion protocols to transmit data, such as infrared, Bluetooth, and Wi-Fi. However, one technology has fallen off the radar in recent years, even though it’s more ubiquitous with lower power requirements. Audio networking uses audible sounds as a low- bandwidth data channel and can enhance, for example, smart phone 1 usability. In that domain, audio networking offers both short-range communication with nearby devices as well as longer-range data transfer by introducing audio data packets into ongoing telephone conversations. More- over, applying audio network- ing to smart phone applications doesn’t change the fundamen- tal devices people use to com- municate. Instead, it can exploit the existing pro- grammable interfaces that modern smart phones offer to augment modes of communication that people already know and use. In this article, we’ll review various modulation schemes we’ve worked with previously, covering how to transfer data to nearby smart phones as well as usability and security issues. We’ll con- sider audio networking as a mechanism for intro- ducing data packets into ongoing mobile phone calls. We’ll also discuss some real-world prob- lems reported with telephone conferencing and apply audio-networking techniques to them in a case study application. Audio networking modulation At an abstract level, audio networking is a sim- ple concept: transmitters modulate data in a suit- able fashion and play the resulting audio data using the host machine’s speakers. (For more information on audio networking, see the related sidebar.) Receivers listen (via microphones) and demodulate incoming signals to recover the trans- mitted information. However, choosing from the plethora of audio data modulation schemes is more difficult. Each has different characteristics—bit rate, transmis- sion range, and what the data sounds like to humans. The latter characteristic, although as important as the first two, isn’t one that design- ers of traditional wireless networking technolo- gies have to face. In previous work, 2 we experimented with four audio data transmission schemes: dual-tone multifrequency (DTMF), 3 used in touch-tone phones; on-off keying (OOK) 4 modulated over a vari- ety of audible carrier frequencies; inaudible data transmission; and melodic data transmission. Here, we briefly summarize our results, giving an overview of various audio modulation schemes and the transmission speeds they facilitate. Conventional modulation schemes Using standard sound hardware (internal lap- top speakers and microphones and a pair of US$10 Harman/kardon HK 206 desktop speak- Audio networking can leverage existing smart phone components to transmit data reliably with increased simplicity and less power than more high-profile wireless communication protocols such as Bluetooth or infrared. WIRELESS COMMUNICATION Anil Madhavapeddy, David Scott, and Alastair Tse University of Cambridge Richard Sharp Intel Research Cambridge

Upload: hoangdat

Post on 10-Mar-2019

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

1536-1268/05/$20.00 © 2005 IEEE ■ Published by the IEEE CS and IEEE ComSoc PERVASIVEcomputing 55

Audio Networking: The Forgotten WirelessTechnology

Modern computing and commu-nication devices offer a widerange of wireless communica-tion protocols to transmit data,such as infrared, Bluetooth,

and Wi-Fi. However, one technology has fallenoff the radar in recent years, even though it’smore ubiquitous with lower power requirements.Audio networking uses audible sounds as a low-bandwidth data channel and can enhance, forexample, smart phone1 usability. In that domain,audio networking offers both short-range

communication with nearbydevices as well as longer-rangedata transfer by introducingaudio data packets into ongoingtelephone conversations. More-over, applying audio network-ing to smart phone applicationsdoesn’t change the fundamen-tal devices people use to com-

municate. Instead, it can exploit the existing pro-grammable interfaces that modern smart phonesoffer to augment modes of communication thatpeople already know and use.

In this article, we’ll review various modulationschemes we’ve worked with previously, coveringhow to transfer data to nearby smart phones aswell as usability and security issues. We’ll con-sider audio networking as a mechanism for intro-ducing data packets into ongoing mobile phonecalls. We’ll also discuss some real-world prob-lems reported with telephone conferencing andapply audio-networking techniques to them in acase study application.

Audio networking modulationAt an abstract level, audio networking is a sim-

ple concept: transmitters modulate data in a suit-able fashion and play the resulting audio datausing the host machine’s speakers. (For moreinformation on audio networking, see the relatedsidebar.) Receivers listen (via microphones) anddemodulate incoming signals to recover the trans-mitted information.

However, choosing from the plethora of audiodata modulation schemes is more difficult. Eachhas different characteristics—bit rate, transmis-sion range, and what the data sounds like tohumans. The latter characteristic, although asimportant as the first two, isn’t one that design-ers of traditional wireless networking technolo-gies have to face.

In previous work,2 we experimented with fouraudio data transmission schemes:

• dual-tone multifrequency (DTMF),3 used intouch-tone phones;

• on-off keying (OOK)4 modulated over a vari-ety of audible carrier frequencies;

• inaudible data transmission; and• melodic data transmission.

Here, we briefly summarize our results, givingan overview of various audio modulation schemesand the transmission speeds they facilitate.

Conventional modulation schemesUsing standard sound hardware (internal lap-

top speakers and microphones and a pair ofUS$10 Harman/kardon HK 206 desktop speak-

Audio networking can leverage existing smart phone components totransmit data reliably with increased simplicity and less power thanmore high-profile wireless communication protocols such as Bluetooth or infrared.

W I R E L E S S C O M M U N I C A T I O N

Anil Madhavapeddy, David Scott,and Alastair TseUniversity of Cambridge

Richard SharpIntel Research Cambridge

Page 2: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

56 PERVASIVEcomputing www.computer.org/pervasive

W I R E L E S S C O M M U N I C A T I O N

ers), we achieved a bit rate of 20 bpsusing DTMF across a three-meter dis-tance with a 0.006 percent symbol errorrate (that is, receivers decoded 0.006 per-cent of DTMF tones incorrectly). Wefound that our OOK coding schemeworked better in short-range transmis-sion (less than 1 meter) because youdon’t have to worry about pulse reflec-tions at low amplitudes. Using OOKmodulated over a 10-kHz carrier, weachieved a bit rate of 251 bit/s across 30cm with a bit error rate of 4.4 � 10–5.

Inaudible data transmissionWe implemented a variant of OOK

modulated over a 21.2-kHz carrier. Wechose this frequency because it’s greaterthan the maximum frequency of humanhearing and less than the Nyquist limit ofstandard sound cards (which operate at44.1 kHz).

Our experiments with inaudible datatransmission involved broadcasting using

a bit rate of 8 bps across 3.4 meters in anoffice with two occupants. While 8 bpsis undoubtedly a low transmission rate,it suffices for broadcasting small identi-fiers (for example, room-grained loca-tion beacons2,5).

The experimental setup consisted of aDell desktop with onboard sound (IntelAC97 Audio Codec) transmitting datathrough Harman/kardon HK 206 speak-ers. The receiver was a Sony Vaio PCG-Z600LEK laptop. The office’s occupantsgenerated ambient noise by continuingwith their normal tasks (which includedtyping, bursts of conversation, walkingaround the room, moving objects andpapers, answering the telephone, anddrinking coffee). Under these conditions,95 percent of the 172 16-bit identifierstransmitted were received correctly.

Melodic data transmission The audible property of audio net-

working is of course a double-edged

sword—it makes users aware of datatransmission between their devices, but itcan also become aggravating to hear it forlong data transmissions. To make short-range data transmission more pleasantfor users, we designed a simple melodicdata transmission scheme. Althoughunlikely to receive critical acclaim, these“amusing little ditties” nevertheless soundsurprisingly pleasant.

Our melodic data transmission schemeassumes the existence of a set of four car-rier frequencies. Playing a tone at one ofthese carrier frequencies for a prespeci-fied duration signals a two-bit value.Notably, instead of activating two of thesefrequencies simultaneously as touch-tonephones using DTMF do, we only activateone carrier frequency at a time.

At any given time, we choose fournotes from the C major (Ionian) scale asour carrier frequencies. We constructeda melody by varying the carrier fre-quencies over time according to a pre-

T ransmitting data over the phone network is clearly not a new

idea in itself; modems have done this for decades. Additionally,

our approach doesn’t focus on achieving maximum bandwidth from

this channel, but by multiplexing data into an existing audio channel,

we focus on addressing spontaneous interaction between users.

Others have explored audio networking in the context of device-

to-device and human-to-device communication.1,2 Our contribu-

tion is to explore how to integrate audio networking with the

telephone network to exploit preexisting, popular lines of commu-

nication that have evolved over the past century. Mobile phones

have always had audio capability, but only with the rise of program-

mable smart phones have we been able to take advantage of it for

data transmission.

Our telephone-conferencing application is inspired by the AT&T

Laboratories’ Broadband Phone (www.uk.research.att.com/

bphone), which is a VoIP (voice over IP) phone with a large display

and thin-client access to a suite of communications-orientated PC

applications. As many workers already have a Broadband Phone’s

constituent parts (a PC, telephone, and Internet access) on their

desks, we aim to provide users with the Broadband Phone experi-

ence using this existing hardware.

Researchers have explored computer and telephone convergence

by installing applications on over 7,000 PCs integrated with a cus-

tom corporate PBX (private branch exchange).3 They reported an

“overwhelmingly positive” reaction to their enhanced telephony

prototype, with 94 percent of the initial users recommending the

system to their colleagues. Clearly, user demand exists for the more

effective control over telephony that modern programmable smart

phones can provide.

Other researchers have commented on modern telephony inter-

faces’ complexity.4 Today’s myriad wireless networking protocols

only increase this complexity, and our work on exploiting existing

phone audio interfaces aims to give users simpler alternatives to

wireless radio protocols.

REFERENCES

1. V. Gerasimov and W. Bender, “Things That Talk: Using Sound forDevice-to-Device and Device-to-Human Communication,” IBM SystemsJ., vol. 39, nos. 3–4, 2000, pp. 530–546.

2. A. Nakayama et al., “Rich Communication with Audio-Controlled Net-work Robot. Proposal of ‘Audio-MotionMedia,’” Proc. 11th IEEE Int’lWorkshop Robot and Human Interactive Communication, IEEE Press, 2002,pp. 548–553.

3. J. Cadiz et al., “Exploring PC-Telephone Convergence with the EnhancedTelephony Prototype,” Proc. 2004 Conf. Human Factors in Computing Sys-tems, ACM Press, 2004, pp. 215–222.

4. N. Griffeth, “Making a Simple Interface Complex: Interactions amongTelephone Features,” Proc. Conf. Companion on Human Factors in Com-puting Systems, ACM Press, 1996, pp. 244–245.

Related Work in Audio Networking

Page 3: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

defined sequence previously known toboth the transmitter and the receiver. Weemployed a musician to choose carrierfrequencies that, irrespective of the databeing transmitted, result in pleasant-sounding melodies. We call a particularsequence of carrier frequencies a theme.Figure 1 shows an example of a themethat follows a chord sequence commonlyfound in baroque music. Each bar con-tains a group of four carrier frequencies.The nth carrier frequency (that is, the nthcrotchet) of each bar encodes the two-bit number n. Our current implementa-tion changes carrier frequencies everyeight notes (that is, every 16 bits). We’vemade available for download audio filesthat encode data packets in variousthemes (see http://anil.recoil.org/projects/telephony.html).

We prototyped our melodic data trans-mission scheme by encoding data asmonophonic mobile phone ringtones.(We achieved 83 bps, the fastest that themobile phone could play ringtones, withan upper limit of 0.001 percent bit-errorrate, with the phone held 2 cm from themicrophone.) However, we programmedour mobile phones to play the ringtonesat seven notes per second (a transmissionrate of 14 bps). We observed that al-though both receiver and phones arecapable of faster transmission, increas-ing the ringtones’ tempo beyond eight ornine notes per second makes them lesspleasant-sounding because the ear does-n’t have enough time to pick out themelody. Because we wish to emphasizethe data encoding’s melodic nature, wesee 14 bps as the upper limit of this en-coding technique.

Secure, short-range audionetworking

Spontaneous interaction between mo-bile users includes swapping contactdetails such as telephone numbers, home

addresses, or Internet URLs. In modernsmart phones, the most common proto-col used for this is the Bluetooth OBEX(object exchange) protocol.6 From a userperspective, this procedure has severalsteps that make it time-consuming anderror prone:

• Users must perform a Bluetooth devicediscovery to locate the other smartphone (which takes up to 10 secondsin a noise-free environment and, inpractice, can take much longer).

• The user must identify the recipientfrom the discovered list of phone names(which, unless they’ve been changedfrom the default, typically consist of alist of generic manufacturer names).

• The phone must establish a Bluetoothconnection and transmit the smallamount of contact information.

Audio networking offers a dramati-cally simpler model. First, a smart phoneencodes the desired contact informationinto an audible message, then you hold thetwo phones close together, allowing thedata to transmit at a low amplitude. Thiseliminates the complexity of Bluetoothdevice discovery, instead using close phys-ical proximity to form a low-bandwidthdata channel and also to act as a simpleauthentication mechanism (because anattacker without sophisticated listeningequipment would have to get very closeto listen to the transmission).

This simplicity also affects the secu-rity model presented to smart phoneusers—rather than requiring them tounderstand Bluetooth’s intricacies, weexploit the data’s audible nature. Byadjusting the output volume, two par-ties can ensure a suitable level of privacyfor their information exchange.

To exchange higher-bandwidth data,you can still use audio as a local chan-nel to bootstrap encrypted Bluetooth

connections. Currently, to create an en-crypted Bluetooth connection, both mo-bile phones must perform device dis-covery6 and also pair the two devices byentering a matching PIN number intoboth phones. The Bluetooth protocoluses this PIN to generate a session keyfor the encryption, and the PIN’s ran-domness decides the resulting encryp-tion’s strength. Unfortunately, smartphones depend on users to generate thePIN, typically leading to choices such as“1111,” “1234,” or other predictablesequences that brute-force dictionaryattacks can easily decode.

Audio networking provides a solutionto creating a spontaneous yet secure Blue-tooth connection between smart phones.A smart phone encodes its BluetoothBD_ADDR (a unique network addressthat every Bluetooth device has) and anautomatically generated random PIN asa short audio packet. As before, by hold-ing the two phones close to each other,the transmitting phone can send the(BD_ADDR, PIN) packet as low-ampli-tude audio.

This type of conversation faces twomain threats:

• The attacker could somehow learn thesession PIN and decode the rest of theconversation.

• The attacker could somehow fool thereceiving phone into hearing a PIN thatthe attacker has transmitted (alsoknown as a man-in-the-middle attack).

When using audio networking aswe’ve described, the first threat wouldrequire the attacker to possess sophis-ticated listening equipment to detectthe low-volume audio between thetwo mobile phones (in addition toBluetooth-sniffing capability). More-over, the directional speakers andmicrophones typically found in smart

JULY–SEPTEMBER 2005 PERVASIVEcomputing 57

44

C F Dm G Em AmFigure 1. Notes corresponding to carrierfrequencies for the “baroque” theme’sfirst six bars.

Page 4: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

phones don’t broadcast audio widely. For the second attack, if the attacker

generated a noise loud enough to con-vince the listening phone to use the fakeinformation, both of the real users wouldhear it and become suspicious. Audioalone has this property; in a purely Blue-tooth exchange, users wouldn’t be ableto detect a high-gain Bluetooth trans-mission by themselves.

Inserting audio data into callsBecause mobile phone audio channels

are band-limited to only 3 kHz, weadopt a data transmission scheme thatuses frequencies similar to DTMF (seetable 1). Touch-tone phones already usethese frequencies for functions such asdialing numbers, so they’re guaranteedto have good frequency response acrossmost phone channels. Although DTMFmandates minimum intertonal lengthsof at least 40 ms, we discovered that 10ms still achieves robust signal decoding.Each DTMF tone encodes 4 bits of data,resulting in a data transmission rate of40 bps—sufficient to robustly transmitsmall chunks of information such as GPScoordinates in a few seconds.

We managed to keep our telephone

data-transmission mechanism surpris-ingly simple. In the spirit of distraction-free computing,7 we deliberately avoidrequiring a MAC (media access control)layer for our audible packets—unlike, forexample, Ethernet, which senses channelavailability and randomly backs off whena collision occurs. Instead, we rely on thesocial-flow control between call partici-pants, allowing them to manually sched-ule audio data transmission as part of their normal conversation. Conse-quently, call participants are never sur-prised by the sudden interjection of datapackets during a conversation.

Figure 2 illustrates a snippet of tele-phone conversation between two hypo-thetical users, Alice and Bob. Alice phonesBob to find out where he is. Bob tells herhis location (“the Castle pub”), but Aliceisn’t sure exactly where this is. To resolvethe problem, Bob informs Alice that he’sabout to transmit his GPS coordinates toAlice, waits for her consent, selects hissmart phone’s audio transmission appli-cation, and sends the burst of data to her.Alice’s smart phone intercepts this audiotransmission, decodes it, and launches theGPS application to tell her how far thereceived location is from her current posi-

tion. They then continue the phone calland agree to meet.

Alternatively, Bob could have read outthe GPS coordinates to Alice, who wouldtype them into her smart phone manu-ally—a tedious and error-prone process.Or, Bob could have encoded the datainto an SMS (short message service) mes-sage and sent it out-of-band to Alice.However, most mobile phone networksdon’t guarantee the timely delivery ofSMS messages, and Alice might havewaited a long time for the small amountof data she required. Additionally, theycould have logged into an Internet-basedinstant-messaging application overGPRS (general packet radio service) andused that to exchange the data. How-ever, this requires Alice to contact Bobtwice—once using his mobile phonenumber and again using his IM address.Audio networking offers the advantageof allowing both voice and data to coex-ist on the same channel, so that you cansend data such as Bob’s GPS coordinatesduring the conversation.

Internet rendezvousEvery user of a mobile phone network

has a unique telephone number assignedto them that lets any other phone in theworld address them. This wide deploy-ment of phone numbers makes it themost ubiquitous naming system on theplanet, with the International Telecom-munication Union reporting the exis-tence of over 1.5 billion mobile phonesubscribers by mid-2004 (see www.theregister.co.uk/2004/12/10/itu_telecoms_report_2004). On a sociallevel, many tools have emerged to assistwith associating users with their tele-phone numbers, ranging from paper-

58 PERVASIVEcomputing www.computer.org/pervasive

W I R E L E S S C O M M U N I C A T I O N

Alice Bob"Hi Bob! Where are you?"

"I'm at the Castle pub.""I'm not sure how to get there."

"I'll send you my GPS location now.""Ok, go ahead."

"My phone map tells me it willtake me 20 minutes to join you." "Great! We'll still be here."

3-second audio data

Figure 2. An example of social-flow control scheduling audio data transmission as part of an ongoingmobile phone conversation.

TABLE 1Frequencies used for encoding 4-bit values for inline data transmission

across telephone lines.

1,209 Hz 1,336 Hz 1,477 Hz 1,633 Hz

697 Hz 1 2 3 4

770 Hz 5 6 7 8

852 Hz 9 10 11 12

941 Hz 13 14 15 16

Page 5: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

based (but regularly updated) phonebooks to mobile phones’ electronic per-sonal address books.

Many smart phone owners regularlyuse laptops or desktop PCs in their day-to-day activities. These PCs increasinglyhave high-bandwidth connectivity tothe Internet via broadband or leased-line connections. However, locatingother Internet users can still be difficultdue to the increasing deployment ofNetwork Address Translation (NAT)devices, firewalls, and other protec-tions. Services such as IM allow onlinechat but have fragmented naming suchthat members of competing servicescan’t talk to each other directly with-out using proxy applications.

We developed the concept of an Inter-net rendezvous service, which leveragesaudio networking to combine the advan-tages of the telephone network (global,end-to-end addressing scheme) and theInternet (high-bandwidth multiservicedata transmission). Figure 3 illustratesthe architecture to perform this integra-tion. We pair smart phones to the users’PCs via Bluetooth, with the PC acting asa headset via the Bluetooth Hands-FreeProfile (www.bluetooth.org/spec). UsingHFP, the PC can intercept the voice datagoing to and from the smart phone viaan isochronous SCO (Synchronous Con-nection-Oriented)6 link and control (forexample, dial numbers) using theRFCOMM (the Bluetooth equivalent ofa serial connection, complete with flow-control emulation) control connection.In our prototype software implementa-tion, we perform the signal processing ofaudio on the PC rather than on the smartphone. The software, called Bluegate,runs on Linux with the open-sourceBluez Bluetooth stack (www.bluez.org).It performs the appropriate pairing withthe smart phone and continuously scansvoice packets for audio data packetsencoded using the fast DTMF methoddescribed earlier. It can also encode fast

DTMF tones for transmission to thesmart phone.

Although this system works well as aresearch prototype, many PC-basedBluetooth stacks do not yet reliably sup-port SCO channels. We expect that thissituation will improve with time.

Enhanced telephone conferencingMost modern mobile phone networks

support telephone conferences. Telecon-ference participants often find it difficultto converse naturally because of the lackof social cues and body language foundin face-to-face conversation. We imple-mented a freely available telephone-con-ferencing application (download this athttp://anil.recoil.org/projects/telephony.html), which demonstrates the Internetrendezvous capability of audio net-working. We assume that participants inthe teleconference also have access to ahigh-bandwidth desktop PC (such as atwork) or a laptop (such as at a Wi-Fihotspot). Participants without access tosuch equipment can still take part in theteleconference using their mobile phone,but they won’t have access to the extrainformation provided by our telephone-conferencing software.

Our application supplies two mainfeatures that conventional telephoneconferences lack: a list of all participantsand a real-time display of who’s cur-rently talking. (User studies showed thatproviding these two features signifi-cantly improved participants’ experi-ence of telephone conferencing.8) Theonly channel the participants share isthe actual audio connection via the tele-phone network.

Suppose that Richard is the moderatorof a telephone conference. Once Anil andDave have dialed into the audio confer-ence bridge (in the usual manner), Richardclicks on his PC’s Create Conference but-ton. A brief one-second burst of audiodata is broadcast to the telephone confer-ence. This audio data encodes the IPaddress of an Internet-messaging service.Anil and Dave’s PCs intercept this audiotransmission, and their PC-based con-ferencing software initiates a connectionto the encoded IP address (see figure 4 foran example of how the screen nowlooks). When Anil speaks, his PC sends amessage to the Internet service indicatingthis, and the listening-conferencing appli-cations (on Richard and Dave’s comput-ers) update their screen to reflect the newstatus. A history window displays recentspeaker information, giving participantscontext about the flow of conversationover the conference.

We use audio networking only to auto-matically establish a common Internetserver. After that, all communicationbetween the software happens over thenetwork, minimizing the disruption toconversation caused by audio packets.Most modern teleconference bridges emitan audible sound when a participant joinsor leaves—we simply augment this exist-ing sound to encode data as well. Our cur-rent implementation of the Internet-mes-saging services uses the IRC (InternetRelay Chat) protocol, with a new channelcreated per telephone conference. Youcould use other IM protocols such as AIM(www.aim.com), Jabber (jabber.org), orSILC (Secure Internet Live Conferencing,s.ilcnet.org) to equally good effect.

JULY–SEPTEMBER 2005 PERVASIVEcomputing 59

Telephone

call

Audio communication Other communication (such as Internet or phone)

RFCOMM

SCO

Richard Dave

RFCOMM

SCO

InternetFigure 3. Architecture of smart phoneand PC/laptop integration via Bluetooth.

Page 6: Audio Networking: The Forgotten Wireless Technologyanil.recoil.org/papers/2005-ieee-audio.pdf · Using standard sound hardware (internal lap-top speakers and microphones and a pair

60 PERVASIVEcomputing www.computer.org/pervasive

W I R E L E S S C O M M U N I C A T I O N

Smart phones currently rely on com-plex, power-hungry protocols suchas Bluetooth to solve user require-ments for the spontaneous transfer

of small amounts of data. We’ve demon-strated that you can use audio to delivermany of the same benefits with a dramaticincrease in simplicity and ease of use.

People already make simultaneous useof telephone calls and Internet data trans-fers on an ad hoc basis—for example,sharing a URL by reading it out over thephone. By automatically inserting datainto telephone conversations, we hideURLs (and other control data requiredto bootstrap Internet connections) from

users. We see this as a conversational ana-log to the way hypertext hides URLsfrom readers, delivering similar benefits.Our telephone-conferencing applicationgoes further and broadcasts informationto provide participants with extra visualcues and context about the call.

We’re currently porting our researchprototype implementations to an appli-cation that you can install on SymbianSeries 60 phones (for example, Nokia6600 and 7610) and use on any mobilephone network. We plan to solicit feed-back and opinions from the large in-stalled user base of smart phone users tocontinue to develop the concepts pre-sented in this article.

REFERENCES 1. IEEE Pervasive Computing, special issue

on the smart phone, vol. 4, no. 2, 2005.

2, A. Madhavapeddy, D. Scott, and R. Sharp,“Context Aware Computing With Sound,”Proc. 5th Int’l Conf. Ubiquitous Comput-ing (UbiComp 03), LNCS 2864, Springer-Verlag, 2003, pp. 315–332.

3, Recommendation Q,23: Technical Featuresof Push Button Telephone Sets, Int’l Tele-communications Union, 1989.

4. V. Gerasimov and W. Bender, “Things ThatTalk: Using Sound for Device-to-Device andDevice-to-Human Communication,” IBM Sys-tems J., vol. 39, nos. 3–4, 2000, pp. 530–546.

5. G. Borriello et al., “Walrus: Wireless AcousticLocation with Room-Level Resolution UsingUltrasound,” Proc. 3rd Int’l Conf. MobileSystems, Applications, and Services (MobiSys05), ACM Press, 2005, pp. 191–203.

6. D. Gratton, Bluetooth Profiles, PrenticeHall PTR, 2003.

7. D. Garlan et al., “Project Aura: Toward Dis-traction-Free Pervasive Computing,” IEEEPervasive Computing, vol. 1, no. 2, 2002,pp. 22–31.

8. R. Colburn et al., Graphical Enhancementsfor Voice Only Conference Calls, tech.report MSR-TR-2001-95, Microsoft Re-search, 2001.

Figure 4. Screenshot of a telephone-conferencing prototype during a conference. Richard is currently talking(indicated by the red square around hisface). Anil has recently interjected a fewwords, and David is simply listening in.

the AUTHORS

Anil Madhavapeddy is a PhD student at the University of Cambridge ComputerLaboratory in the Systems Research Group and a developer on the secure OpenBSDoperating system. His research interests include networked systems security andubiquitous computing. He received his BEng in information systems engineeringfrom Imperial College London. Contact him at Computer Laboratory, Univ. of Cam-bridge, 15 JJ Thomson Ave., Cambridge, CB3 0FD, UK; [email protected].

Richard Sharp is a senior researcher at Intel Research. His research interests includeubiquitous computing, security, programming language design, and compilerimplementation. His recent research projects include investigating programminglanguages for multicore network processors and improving the security of complexWeb applications. He received his PhD in computer science from the University ofCambridge Computer Laboratory. Contact him at Intel Research Cambridge, 15 JJThomson Ave., Cambridge, CB3 0FD, UK; [email protected].

David Scott is a member of the technical staff at Fraser Research. His research inter-ests include computer security, ubiquitous computing, and networking. Previouslyat the AT&T Laboratories in Cambridge, he built part of a high-performance CORBA

middleware platform. He received his PhD in engineering from the University ofCambridge. Contact him at Fraser Research, 182 Nassau St., Ste. 301, Princeton, NJ08542; [email protected].

Alastair Tse is a PhD student at the University of Cambridge Computer Laboratoryin the Digital Technology Group. His research interests include wireless network pro-tocols, location systems, and ubiquitous computing. He received his BEng in com-puter engineering from the University of New South Wales, Sydney, Australia. Con-tact him at the Digital Technology Group, Univ. of Cambridge, 15 JJ Thomson Ave.,Cambridge, CB3 0FD, UK; [email protected].