android, arduino, and the headphone jack
DESCRIPTION
There are quite a few 3rd party devices that attach to a smartphone's 3.5 mm headphone jack, Square's Credit Card reader, being just one of them. In this session we will reveal the magic behind this cool innovative trick. Come to learn how it was done and how your mobile app too, could take advantage of the headphone jack. Hijacking power and bandwidth from the mobile phone's audio interface. Come to see and learn how mobile app take advantage of a smartphone's headphone jack. We will use "phone to phone" as well as Arduino to Android examples to demonstrate and explain this cool and innovative communication channel, and you may even pick up a couple DSP (Digital Signal Processing) basics along the way.TRANSCRIPT
![Page 1: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/1.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
“ Can you hear me now ? ”
Using a phone’s 3.5mm audio jack for data exchange
![Page 2: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/2.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 3: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/3.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 4: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/4.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 5: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/5.jpg)
Tip Ring Ring Sleeve
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 6: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/6.jpg)
build an embedded system that captures, encodes, and transmits sensor data
the theory behind audio encoding / decoding
build an Android app that receives, decode, and displays the sensor data
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 7: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/7.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 8: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/8.jpg)
Android Ground
iPhone Ground
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 9: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/9.jpg)
Transferring a Message
• Encode the Message (e.g. ASCII or Morse Code)
• Convert encoded message into audio signals [D➜A]
• Send (Play) the audio signals
• Receive (Listen/Record) audio signals
• Interpret (Filter/Transform) audio signals [A➜D]
• Decode digital signal into original message Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 10: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/10.jpg)
Amplitude
Period (time required for one complete cycle)
Frequency (number of cycles per second)Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 11: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/11.jpg)
FSK Frequency Shift Keying
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 12: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/12.jpg)
ASK Amplitude Shift Keying
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 13: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/13.jpg)
PSK Phase Shift Keying
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 14: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/14.jpg)
The Sender
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 15: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/15.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 16: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/16.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 17: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/17.jpg)
Arduino on the Web http://arduino.cc
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 18: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/18.jpg)
Getting Started with Arduino (Make: Projects)
by Massimo Banzi, Co-founder of the Arduino project
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 19: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/19.jpg)
const int ledPin = 13;const int outPin = 11;const int freq = 2000;const int duration = 2000;const int pause = 1000;
void setup() { pinMode(ledPin, OUTPUT); pinMode(outPin, OUTPUT);
}
void loop() { digitalWrite(ledPin, HIGH); tone(outPin, freq); delay(duration); noTone(outPin); digitalWrite(ledPin, LOW); delay(pause);
}Main
Lo
op
S
etu
p I
nit
iali
zati
on
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 20: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/20.jpg)
Demo
Arduino + external Speakerplays 2000 Hz sound for 2 seconds
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 21: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/21.jpg)
Port 11
Gnd
10K
2K 2K
1uF
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 22: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/22.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 23: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/23.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 24: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/24.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 25: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/25.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 26: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/26.jpg)
Transferring a Message
• Encode the Message (e.g. ASCII or Morse Code)
• Convert encoded message into audio signals [D➜A]
• Send (Play) the audio signals
• Receive (Listen/Record) audio signals
• Interpret (Filter/Transform) audio signals [A➜D]
• Decode digital signal into original message
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 27: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/27.jpg)
![Page 28: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/28.jpg)
ASCII - Encoding
H e l l o Wo r l d !
0x48 0x65 0x6C 0x6C 0x6F 0x20 0x57 0x6F 0x72 0x6C 0x64 0x21
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 29: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/29.jpg)
frequency = ((value -k)*M + offset ) * frequency_resolution
E.g.: ‘ A’ = 0x41 = 65 = value1st char in range: 45 = k
frequency bins per Symbol: 5 = M0Hz .. 5200 Hz* unusable: 60 = offset
44100 Hz / 512 Samples = 86.13.. = frequency_resolution((65 - 45)*5 + 60)*86.13 = 13780.8 Hz
* related to specific hardware
Mapping ASCII to Frequency
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 30: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/30.jpg)
Usable Frequency Range depends on the Hardware
![Page 31: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/31.jpg)
![Page 32: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/32.jpg)
const int ledPin = 13; // built-in LEDconst int outPin = 11; // PWM Port outconst int inPin = 0; // Light Sensor
const char STX = '?'; // Start Token const char ETX = '@'; // End Token
const unsigned int SAMPLE_RATE = 44100; // Hzconst int SAMPLES = 512 ; // Num of Samples (128,256,512,1024)const int REPEAT = 3; // prolong the signalconst int OFFSET = 60; // 60 un-usable bin on the spectrum's low end
const double FREQ_RES = (double) SAMPLE_RATE / SAMPLES; // 86.13 frq per binconst int DURATION = (int) REPEAT * (1000 / FREQ_RES); // about 35msconst int ENC_ETX = encodeAscii(ETX); // pre-calc for later use in loop
unsigned int frq[6]; // global frequency array, changes every run 1 of 3
Initialization
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 33: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/33.jpg)
void setup() { pinMode( ledPin, OUTPUT ); pinMode( outPin, OUTPUT ); pinMode( inPin, INPUT ); Serial.begin( 9600 ); // debugging on
}
/** * Encode the message into an frequency array * Wrap the message into Start and End tags */void encodeInt(int m) {
char message[5]; itoa( m,message,10 );
frq[0] = encodeAscii( STX ); for ( int i=0; i<5; i++ ) {
if ('\0'==message[i]) { // replace '\0' marker w/ ETX frq[i+1] = encodeAscii( ETX ); break;
} frq[i+1] = encodeAscii( message[i] );
}}
2 of 3
Setup and Encoding a message
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 34: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/34.jpg)
unsigned int encodeAscii(char c) { return (unsigned int) ((5 * (c-45) + OFFSET) * FREQ_RES);
}
void loop() { const int brightness = analogRead(inPin); // reads value 0-1023 encodeInt(brightness); // into global frq[] digitalWrite(ledPin, HIGH); int i=0; do {
tone(outPin, frq[i]); delay(DURATION); noTone(outPin);
} while (frq[i++] != ENC_ETX); digitalWrite(ledPin, LOW); delay(500); Serial.println(brightness);
} 3 of 3
Encoding a single char and main loop
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 35: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/35.jpg)
Demo
Arduino + external Speaker + TRRS-Plugplays LightSensor value Message every .5 seconds
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 36: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/36.jpg)
The Receiver
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 37: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/37.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 38: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/38.jpg)
• The total number of times the signal is sampled in one second is defined as the sampling frequency. Only 44.100 Hz is guaranteed on all Android Devices
• Nyquist–Shannon sampling theorem: The sampling frequency should be at least twice the highest frequency contained in the signal.
I.e. Signals from the Arduino board should be in the 0..22,050 Hz range.
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 39: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/39.jpg)
At a 44,100 Hz sample rate, 512 samples are taken in 11.6 ms
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 40: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/40.jpg)
We took 512 samples in 11.6 ms
But we need to know the frequency the Arduino played.
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 41: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/41.jpg)
FFT (Fast Fourier Transform)
FFT is an effective algorithm to convert signals from time domain to the frequency domain.
We use the 512 samples in the audio buffer as input; and the FFT algorithm returns a complex array, allowing us to calculate the magnitude of 512 frequency ranges.
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 42: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/42.jpg)
• Sampling Rate 44,100 Hz
• Effective Frequency Range 0..22,050 Hz (...Nyquist–Shannon)
• 512 Samples
• Frequency Range is split into 256 ranges
• The Frequency Resoultion 22,050/256 = 861.52 Hz
We input 512 samples and as a return we know which of the 256 frequency ranges had the strongest signal.
So we won’t know the exact frequency the Arduino played, but we know, what’s called the frequency bucket or frequency bin.
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 43: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/43.jpg)
Text .. Light sensor value 0..1023
ProgressBar %
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 44: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/44.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 45: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/45.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 46: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/46.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 47: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/47.jpg)
Demo
Arduino + Android + external Speaker + TRRS-PlugLightSensor value encoded into Audio SignalsAudio Signal decoded into Text and Progress-bar updated every .5 seconds
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 48: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/48.jpg)
Summary
We built an embedded system that captured sensor dataEncoded the data into ASCII and then into frequenciesGenerated tones for a predefined duration
On an Android Phone, we received the Audio Signal via TRRS CableDecoded the signal back into ASCII and then NumbersDisplayed the Text and animated a progress bar every .5s
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
![Page 49: Android, Arduino, and the Headphone Jack](https://reader038.vdocuments.us/reader038/viewer/2022102520/5552dd75b4c90532498b4cb8/html5/thumbnails/49.jpg)
Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production. All rights reserved.
Thanks for coming
http://wolfpaulus.com