sensor fusion between car and smartphone
DESCRIPTION
Droidcon London 2013 presentation about car sensor accessTRANSCRIPT
Thanks for the memory
Car offers
● Power (no need to worry about battery drain)● Fixed position (if in car kit)● Sensors
● More sensors → more exciting applications!● Sensor fusion: the use of multiple sensors so
that they compensate each other's weaknesses
Relevant phone sensors
● GPS – if the phone is placed in the car so that it “sees” the satellites
● Accelerometer – shaking, vibration, violent movements like aggressive braking or turns
● Gyroscope – turn detection, support sensor for the accelerometer
● Compass (magnetometer) – direction (not sensitive to drift), magnetic signature
Relevant car sensors
● Speed – driving style, distance (if integrated by time)
● RPM – driving style● Built-in GPS – not subject to incorrect
placement within the car● Large number of car diagnostic indicators –
remote diagnostic → not the subject of this presentation
Obtaining car sensor data
● OBDII (like OBD-2, On-board diagnostic)● Widely deployed (mandatory in the EU since
2001/2004)● Developed for diagnostic, not for obtaining
traffic data● Nevertheless, used for a variety of applications
like fleet management
(MirrorLink)
● Emerging standard for car infotainment system-phone connection
● Developed by Car Connectivity Consortium (CCC), http://www.mirrorlink.com/
● Supported by some recent phones and cars● Even the specification costs 150 USD● Not used in the experiments in this presentation
ELM327
● ELM327 is a popular microcontroller for OBD2 access. Developed by ELM Electronics cloned by many
● On the car side, it talks OBD2 to the car's on-board computer● On the terminal side, it offers a friendly serial interface with AT
commands reminiscent of the Hayes modem command set● Actual devices often offer USB or Bluetooth interface (anything goes
that has serial line support)● Bluetooth is ideal for Android device connectivity
● Basic Bluetooth example programs like BluetoothChat can be used to communicate with the device almost without changes
● Example program: http://mylifewithandroid.blogspot.com/2013/03/data-capture-application-for-car-speed.html
ELM327 sequence
● ATD OK● ATZ ELM327 v1.4● ATE0 OK● ATM0 OK● ATL0 OK● ATH0 OK● ATS0 OK● ATSP0 OK● 0100 SEARCHING..
4100BE3EB811● 010D 410D00
● Set all to defaults● Reset all● Switch off echo● Don't store discovered protocols● Don't send LF after CR● Don't show OBD headers● Don't insert spaces into hex digits● Discover OBD protocol automatically● Query current data 0x00 (register support)
● Query current data 0x0D (speed, 0 km/h)
Cutting edge technology!
OBDII dongle
Precise fixing is crucial
First use case
● Dead reckoning-type navigation when GPS is not available
● E.g. in tunnels, parking houses, dense urban settings (urban valleys) ...
Sensors used
● Base coordinate p0 comes from the GPS or from earlier measurement
● Absolute value of velocity vector (v) comes from the speed sensor of the car
● Direction of velocity vector● Gyroscope● Compass
Gyroscope recap
● Measures rotation around 3 axes● More exactly: measures rotation speed (angular
velocity) around the axes
v x=Δ φ
Δ tΔ φ=v x Δ t
φ '=φ+ Δ φ
The problem of devices
● Newer phones are not always better● Particularly not their sensors
● It happens that a new sensor type – like gyroscope – is introduced in good quality ● Then the cost-cutting starts in later, “more
advanced” device types
Nexus S gyro
Car making 90 degree turns, device horizontal.Red: angular velocity, blue: aggregated rotation degree
Galaxy Nexus gyro
Car making 90 degree turns, device horizontal.Red: angular velocity, blue: aggregated rotation degree
Origin of the spikes: unknown
Only Nexus S measurements will be used in this presentation
The problem of drift
Compass
● Measures the device orientation wrt. the magnetic vector of the Earth
● This vector points toward the magnetic center of the Earth
● It has a component that points to the magnetic North pole – that's what we use for orientation
● Beware of the z component! (also called magnetic inclination). If the device is not held horizontally, the downward vector element influences the measurement
Car's magnetic fieldTurning around, out of car
Phone is inside the car, ignition on, car makes full circle
Find the offset
● Identify 3 points on the circle (use the gyroscope to figure out whether there was enough rotation)
● Calculate the center of the circle
● Subtract the offset
Move the circle to the centerCircle moved to the zero point of the coordinate system
(on the x-y plane)
Initial circle
Compass vs. gyroCompass Gyro
Compass is harder to calibrate and less exact but it has no drift
Tunnels
Tunnels
Effect of external magnetic fields
Not called Chain Bridge by chance ...
Parking houses
Parking houses
Urban valleys
Urban valleys
No GPS readingin the circle
Results
● Dead reckoning using speed (from car) and direction (from smartphone) provides reasonable results when GPS is not available
● Compass and gyroscope need to support each other to obtain sufficiently precise direction vector
● Compass is an extremely problematic sensor in car/urban environment that needs constant recalibration
Second use case: vibrations
● Goal: use the phone's accelerometer for something useful (car does not have any similar sensor)
● Separating motion acceleration: already discussed here: http://mylifewithandroid.blogspot.com/2012/05/better-motion-control-using.html
(can be used to classify driving habits)
● Vibration:
● Damaged road
● Vehicle faults (not discussed here)
bumpy_road
Raw vibration
Bumpy road section
After high-pass filter
Effects of slow acceleration changes (turns) are removed
Power
Bumpy road section
Luckily we have exact position ...
Speed bumps
Drivers slow down ...
Red: speed, blue: acceleration absolute value
Speed drops &vibration spikeat the same time:speed bump
Or they don't ...
Red: speed, blue: acceleration absolute value power
Not possible to detect
Results
● Localized road damages cannot always be recognized
● Damages that span longer distances may be detected more reliably
● In any case, with a large number of measurements, road damages can be mapped efficiently
Idea of a platform
● Available for Android applications● Discovers car connectivity options and sensors● Notifies applications about the connectivity
status and available sensors● Allows single or repeated sampling of sensor
data● Controlling car's systems?
Questions?