![Page 1: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/1.jpg)
Team Sound Assist
Adam Seubert, Alex Inskeep
Client: Dr. James FrenzelMentor: Dr. Feng Li
Members: Adam and AlexClient: James Frenzel
Mentor: Feng Li
![Page 2: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/2.jpg)
Problem Statement
● Speech in noisy media can be difficult for the hearing impaired
● Action sounds can make it difficult to understand characters
2https://coub.com/view/f166n16
![Page 3: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/3.jpg)
Project goals
● To create an assistive listening device to increase the intelligibility of human speech in common
media, such as movies and television
● The device must increase the intelligibility of human speech
● The device must easily integrate with a home theater system
3
![Page 4: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/4.jpg)
Design Overview
Sound Assist DeviceAnalog Audio Stream Vocal Boosted Analog Audio Stream
4
![Page 5: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/5.jpg)
Technical Approach
DUET
● Blind Source Separation Algorithm
● Isolate and amplify the human voice
Filtering
● Separately Increase the volume of the human voice frequency range
● Remix with scalar values to achieve greater intelligibility
5
![Page 6: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/6.jpg)
Hardware Design Overview
ADCInput Signal
RaspberryPI
DAC Output to Headphones
6
RCA Jacks
RCA Jacks
Sound Card
![Page 7: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/7.jpg)
7
Device Overview
https://en.wikipedia.org/wiki/Raspberry_Pihttp://www.audioinjector.net/rpi-hat
![Page 8: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/8.jpg)
8
Use Case
![Page 9: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/9.jpg)
Software Design Overview
9
PyAudio PyAudioRCA Sampling
DUET
RCA Out
Filtering
![Page 10: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/10.jpg)
PyAudio
● Python package to help facilitate audio recording and playing
● Using a callback method to reduce sound lag
● Samples small amount of audio, processes, and plays back
10
![Page 11: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/11.jpg)
Filtering
● Apply a 3rd order bandpass filter to a copy of the
original sample
● Cut off frequencies are 1kHz and 5kHz
● Mix filtered signal and original with linear scalars
to increase vocal intelligibility
11S. Makino et al. (eds.), Blind Speech Separation, 217–241. © 2007 Springer
![Page 12: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/12.jpg)
Duet
● Using only 2 inputs, it can separate an arbitrary
number of sources so long as the phase-frequency
representations of each source do not overlap
● Creates a 3D weighted histogram of the
phase-frequency data and the peaks of the histogram
are the separable sources
12S. Makino et al. (eds.), Blind Speech Separation, 217–241. © 2007 Springer
![Page 14: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/14.jpg)
End Product
● Plug and Play experience
○ Scripts start all programs on powerup
○ No additional connections or UI needed
● Compact and powerful solution with future options
14
![Page 15: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/15.jpg)
Future Work options
● Portability
● Wireless (Bluetooth, wifi)
● Real time DUET
● Customizable user profile
● User controlled knob for isolated vocal gain
15
![Page 16: Team Sound Assistimages.shoutwiki.com/mindworks/a/ae/Sound_Assist... · Design Overview Sound Assist Device ... Technical Approach DUET Blind Source Separation Algorithm Isolate and](https://reader036.vdocuments.us/reader036/viewer/2022071022/5fd6d0019681cd1bdc77d515/html5/thumbnails/16.jpg)
Got Questions?
16