acoustic source localization with a vtol suav deployable ...€¦ · construct a microphone array...
TRANSCRIPT
| Mechanical Engineering
Introduction Methodology Results Conclusion
Acoustic Source Localization with a VTOLsUAV Deployable Module
Kory Olney
Department of Mechanical EngineeringUniversity of South Florida
October 18, 2018
Kory Olney 1/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Overview1 Introduction
BackgroundObjectivesContributions
2 MethodologyMathematical ModelsHardware
Localization ModulesUAS Design
Software3 Results
ExperimentPerformance MetricsError Analysis
4 ConclusionSummaryFuture WorkAcknowledgments
Kory Olney 2/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Background I
The recent increase in availability of low cost small unmannedaircraft system (sUAS) has led to new opportunities
Sensors are becoming smaller and less expensive whileproviding more computational power
Many open source contributions in software and hardwareoriginating from the hobbyist radio control (RC) market
Kory Olney 3/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Background II - sUAS
Useful in applications where humans cannot safely go
Use cases for sUAS
Industrial inspection
Shipping and delivery
Agriculture
Cinematography
Military
And many more...
Kory Olney 4/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Background III - Source Localization
Applied throughout the acoustic and electromagneticspectrum
Beamforming
Wifi, GPS
Radio communications
Seismology
Acoustic range
Speaker localization
Critical infrastructure security
Military and law enforcement
Kory Olney 5/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Objectives
Construct a low-cost prototype sUAS with a payload capacityof at least 4 lb
Construct a microphone array that can addresses constraintsof the sUAS problem
Develop a data acquisition unit with DSP methods to performdirection of arrival (DOA) estimation in real time
Kory Olney 6/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Contributions
Developed code for 8 channel synchronous sampling
Designed and fabricated prototype direction finding module
Designed and fabricated a sUAS that can host the module
Applied DSP methods to the acoustic noise and direction ofarrival problems
Kory Olney 7/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Mathematical Models
Signal Modelx1(t) = s1(t) + n1(t)
where x1(t), s1(t), and n1(t) are real, stationary randomprocesses and s1(t) is assumed to be uncorrelated with n1(t)
E{n1(t)} = 0
Kory Olney 8/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Array Time Difference of Arrival (TDOA) Model
x1(t) = s1(t) + n1(t)
x2(t) = s1(t − D1) + n2(t)
...
xm(t) = s1(t − Dm) + nm(t)
D1 through Dm are thetime of arrival delaysbetween channels whichare to be estimated
Kory Olney 9/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Cross Correlation - Continuous
Rxy (τ) =
∫ ∞−∞
x∗(t)y(t + τ)dt
Rxy (τ) =1
T
∫ T
0x(t)y(t + τ)dt
Rx1x2(τ) = Rs1s1(τ + D) + Rn1n2(τ)
* denotes complexconjugate
T represents theobservation interval
Kory Olney 10/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Cross Correlation - Discrete
Rxy [D] =∞∑
n=−∞x∗[n]y [n + D]
Rxy (r∆t) =1
N − r
N−r∑n=1
xnyn+r
r∆t ≈ τr = 0, 1, 2, ...,m withm < N
r represents the index ofthe sequence
Kory Olney 11/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Cross Power Spectrum
Sxy (f ) =
∫ ∞−∞
Rxy (t)e−j2πftdt Fourier transform of thecross correlation
Kory Olney 12/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Generalized Cross Correlation
H1
H2
x1
x2
y1
y2
Delay
Cross
Correlation
Figure: Filter Model
Sy1y2(f ) = H1(f )H2∗(f )Sx1x2(f )
Kory Olney 13/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Generalized Cross Correlation - PHAT
ψp(f ) =1
|Sx1x2(f )|= H1(f )H∗2 (f )
Ry1y2(τ) =
∫ ∞−∞
ψp(f )Sx1x2(f )e j2πftdf
Phase transform (PHAT)
Weighting function appliedfor whitening the signals
Normalizes cross powerspectrum
Sharpens cross correlationfor better resolution intime domain
y1 and y2 are signals x1and x2 respectively, afterfiltering
Kory Olney 14/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Discrete Implementation
hj =N−1∑b=0
y1by2j+b
for j = −(N − 1),−(N − 2), ...,−1, 0, 1, ..., (M − 2), (M − 1)
Ry1y2(i) = hi−(N−1)
τy1y2 = argmaxi
Ry1y2(i)
N is the number of elements in sequence y1
M is the number of elements in sequence y2
τ is the estimate of time delay
Kory Olney 15/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Mic Position and DOA Vector
Pi =
xiyizi
K =
kxkykz
=
sinθcosφsinθsinφcosθ
where Pi is the position of the ith microphone in the arraywith respect to an arbitrarily defined reference
K is the DOA vector
Kory Olney 16/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
TDOA Between Two Channels
τi ,j =1
c[(xj − xi )sinθcosφ+ (yj − yi )sinθsinφ+ (zj − zi )cosθ]
τi ,j =1
c[Pj − Pi]K
Given by projecting the position difference vector onto theDOA
Assumes wavefront propagates as a plane
Kory Olney 17/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Reference Free Position Difference Matrix
S = [P2 − P1, ...,PN − P1,P3 − P2, ...,PN − PN−1]T ∈ RN(N−1)
2×3
Calculating the positioning difference between eachcombination of microphones in the array
N is the number of sensors in the array
Kory Olney 18/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Reference Free TDOA Vector
~τ = [τ2,1, ..., τN,1, τ3,2, ..., τN,2, ..., τN,N−1]T ∈ RN(N−1)
2×1
TDOA between all combinations of microphones in the array
Kory Olney 19/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
TDOA Equation set
τ =1
cSK
c is the estimate of the speed of sound
Leads to a linear least squares solution
K = −c(STS)−1ST τ
Kory Olney 20/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Elevation and Azimuth Estimates
φ = tan−1(ky
kx)
θ = tan−1(
√k2x + k2y
kz)
Azimuth
Elevation
Kory Olney 21/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware Overview
Localization module
Microphones
Array Design
TI ADS1278
NI myRIO
Interface Board
sUAS Design
Frame
Powertrain
Autopilot
Tx Rx
Kory Olney 22/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - Localization Module - Microphones
Figure: Adafruit Silicon MEMSMicrophone Breakout-SPW2430
Low cost, lightweight,MEMS breakout board
Amplifies signals to linelevel, with max 1V peakto peak
Flat frequency responsefrom 100Hz to 10kHz
3 wire configuration: 3.3Vsupply, ground, DC output
0.67V DC bias
Kory Olney 23/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - Localization Module - Array Design
Figure: 8 Channel MicrophoneArray Design
8 analog microphones inspherical arrangement
3D printed and carbonfiber frame
Radius of 11.25 in
Single 24 pin femaleribbon cable
Kory Olney 24/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - Localization Module - TI ADS1278
Figure: ADS1278 EVM-PDKADC
Synchronous sampling of 8analog channels
SPI output
∆Σ configuration -Oversampling
Onboard or supplieddigital clock
Up to 144 ksps/channel
Built in 1st-order analoglowpass filters
Kory Olney 25/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - Localization Module - NI myRIO
Figure: National InstrumentsmyRIO Embedded System
Reconfigurable InputOutput - RIO
Xilinx FPGA and CPU
40 MHz onboard clockthat can simulate higherrates
Built-in accelerometer
34 pin MXP connections
6V to 16V power supply
Simple functionality withLabVIEW
Kory Olney 26/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - Localization Module - Interface Board
Figure: Top of Interface Boardwith 34 and 24 pin Connectors
Interface betweenADS1278, myRIO, andmicrophones
Custom designed printedcircuit board (PCB)
Power: 3.3V, 5V, frommyRIO and 1.8V regulatoronboard
Kory Olney 27/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - sUAS Design - sUAV Frame
Figure: sUAV Hexacopter Frame
Turnigy Talon from HobbyKing
Carbon fiber frame exceptfor assembly hardware andmotor mounts
Boom radius of 12.5 in
Kory Olney 28/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - sUAS Design - Powertrain
Figure: KDE 2315XF Motor
Brushless DC motor
2050 RPM/V - 30340Max RPM at 14.8V
Motor torque 0.0035ft-lbs/A
2.66A at 14.8V
Theoretical max thrustwith 8 in propellers 3.5lb
Kory Olney 29/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - sUAS Design - Autopilot
Figure: Pixhawk 2.1
Open source hardwaredesign
Designed specifically tofunction with Ardupilot -open source software
Large user base forsupport and debugging
Redundant IMUs
Intel Edison port
Kory Olney 30/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - sUAS Design - Tx Rx
Figure: Taranis QX7 Transmitter
16 Channels
Haptic vibration feedbacksystem
Runs on OpenTX
Reciever signal strengthindicator
Real time flight datalogging to microSD
Kory Olney 31/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Hardware - sUAS Design - Tx Rx
Figure: FrSKY X8R 2.4GHzReceiver
8 channel telemetryreceiver with smart bus
Full duplex
Daisy chainable for 16channels
Kory Olney 32/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
System Overview
Source
Mic Array Interface
Board ADC
myRIO
FPGA
SPI Parsing
CPU
DMA
FIFO
DOA Vector
DOA
Algorithm
Bandstop
Filter
Match
Pattern?
GCC-
PHAT
TDOA
(F)
(T)
Figure: General Data Flow Through System
Kory Olney 33/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Field Programmable Gate Array
Figure: FPGA Top Level
State machine
SCTL at 48MHz
24MHz modulator clockto ADC
8 channels of 24 bits eachat 93.75 kHz
U32 to I32
Target Scope FIFO toDMA FIFO
Kory Olney 34/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
CPU I
Figure: RT Top LevelKory Olney 35/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
CPU II
Producer-consumer loop
Monitors single channel for impulse detection
Trigger sends array to local FIFO for DOA estimate
User defined bandstop filter and trigger threshold
Kory Olney 36/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Data Collection
Recordings of 9 different calibers at Pasco County Range
Microphones were approximately 29 feet from the source
5 shots of each caliber were recorded with only ambient noise
Shots were then recorded with the sUAS motors operating
Kory Olney 37/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Gunshot Waveform 9mm
Vol
ts
0.035
-0.04
-0.035
-0.03
-0.025
-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
Time (s)0.430 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
Time Domain Channel 1
Figure: 9mm Time Domain
Shifted from DC Bias
≈ 0.07V P2P
≈ 3 ms in length
Kory Olney 38/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Dual Channel 9mm
Volts
0.79
0.64
0.65
0.66
0.67
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
Time (s)0.350.1 0.125 0.15 0.175 0.2 0.225 0.25 0.275 0.3 0.325
Waveform Plot 2 channels
Figure: Comparing 2 CH 9mm
Channels 1 and 4
≈ 0.35 ms TDOA
Kory Olney 39/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Propeller Noise
Volts
0.79
0.66
0.67
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
Time (s)0.1280 0.02 0.04 0.06 0.08 0.1 0.12
Mic 5
Mic 6
Mic 7
Mic 8
Mic 1
Mic 2
Mic 3
Mic 4
Waveform Plot 1-8
Figure: All Channels of Props
Channels 1 through 8
Varying DC bias betweenchannels
≈ 0.02 V P2P
Recorded at 50 % throttle
Kory Olney 40/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Propeller Noise Filtering
Ampl
itude
1
0
0.2
0.4
0.6
0.8
Frequency (Hz) 25000 250 500 750 1000 1250 1500 1750 2000 2250
FFT Channel 1Raw FFT
Figure: Frequency Plot of Prop Noise
Ampl
itude
1
0
0.2
0.4
0.6
0.8
Frequency (Hz) 25000 250 500 750 1000 1250 1500 1750 2000 2250
Filtered FFT Channel 1 Filtered FFT
Figure: Filtered Sequence FFTKory Olney 41/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
9mm FFT
Ampl
itude
1
0
0.2
0.4
0.6
0.8
Frequency (Hz) 40000 250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000 3250 3500 3750
FFT Channel 1Raw FFT
Figure: Extended Band FFT
Ampl
itude
1
0
0.2
0.4
0.6
0.8
Frequency (Hz) 6000 50 100 150 200 250 300 350 400 450 500 550
FFT Channel 1Raw FFT
Figure: Lower Band FFTKory Olney 42/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Frequency Response
Mag
nitu
de d
B5.0
-250.0
-200.0
-150.0
-100.0
-50.0
Frequency (Hz)3000.00.0 250.0 500.0 750.0 1000.0 1250.0 1500.0 1750.0 2000.0 2250.0 2500.0 2750.0
MagnitudeMagnitude response
Figure: Frequency Response of Bandstop Filter
Finite impulse response (FIR)Cutoff from 500Hz to 1.9kHzLinear phase responseMaximum 511 taps
Kory Olney 43/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
System Weight
Component Weight(g)sUAV 1520
Battery 416
Mic Array 200My PCB 45
ADC 36myRIO 235Housing 201
Module: 717Total: 2653
Kory Olney 44/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Impulse Detection Reliability
Currently cross correlating each new frame against a templatewavelet
Normalized, windowed, and filtered
A trigger for an impulse detection is set to true if the maxvalue of the cross correlation exceeds a user defined threshold
False negatives and false positives are a significant concern forthis system
Kory Olney 45/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Estimation Error
Initial tests show that the error for elevation and azimuth isapproximately +/- 7 %
Recently simulated with clapping and snapping
Future experiment soon to study the localization error
Demonstration...
Kory Olney 46/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Summary
Successfully demonstrated that a lightweight acoustic sourcelocalization module can function in real time onboard a sUAVwhile estimating both elevation and azimuth without motorsoperating
GCC-PHAT method is sufficient for TDOA estimates inmodestly noisy environments
Noise characteristics of the current sUAS significantly degradesignal quality
Higher quality mics are necessary for long range estimates
Low cost solution to serve a variety of potential engineeringchallenges
Driver for ADS1278 has potential for a variety of applications
Many improvements can be made
Kory Olney 47/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Future Work
Perform a thorough evaluation of the system performanceunder varying conditions to determine the maximum effectiverange for accurate DOA estimation
Improve the digital signal processing techniques to be morerobust and better tuned for specific applications
Acoustic classification i.e. caliber recognition
Incorporate a camera and machine vision techniques toattempt at precisely identifying the source of an acoustic event
Develop a single PCB that is smaller, lighter, and performs allof the necessary computations internally
Create a module that is designed specifically to comply with aCOTS sUAS
Kory Olney 48/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Acknowledgments
Dr. Robert H. Bishop for providing me the opportunity toundertake this research and for finding funding for theassistantship. Dr. Jonathan Gaines for providing guidancethroughout this process. Michael Guinn for helping me obtain theposition to make this all happen. SOFWERX for providing thefinancial support and lab space. Pasco County Sheriff’s Office forallowing me to use their outdoor range for testing.
I would also like to thank my family and friends that havesupported me throughout my entire graduate research experience.
Kory Olney 49/50
| Mechanical Engineering
Introduction Methodology Results Conclusion
Questions?
Thanks for your attention!
Kory Olney 50/50