a user study to compare two conversational assistants ... · aids and implants can restore hearing...
TRANSCRIPT
Proceedings of the Eighth Workshop on Speech and Language Processing for Assistive Technologies, pages 1–8Minneapolis, Minnesota, USA c©2019 Association for Computational Linguistics
1
A user study to compare two conversational assistants designed for peoplewith hearing impairments
Anja VirkkunenAalto University, Finland
Kalle PalomakiAalto University, Finland
Juri LukkarilaAalto University, Finland
Mikko KurimoAalto University, Finland
Abstract
Participating in conversations can be difficultfor people with hearing loss, especially inacoustically challenging environments. Westudied the preferences the hearing impairedhave for a personal conversation assistantbased on automatic speech recognition (ASR)technology. We created two prototypes whichwere evaluated by hearing impaired test users.This paper qualitatively compares the twobased on the feedback obtained from the tests.The first prototype was a proof-of-concept sys-tem running real-time ASR on a laptop. Thesecond prototype was developed for a mobiledevice with the recognizer running on a sepa-rate server. In the mobile device, augmentedreality (AR) was used to help the hearing im-paired observe gestures and lip movements ofthe speaker simultaneously with the transcrip-tions. Several testers found the systems usefulenough to use in their daily lives, with majoritypreferring the mobile AR version. The biggestconcern of the testers was the accuracy of thetranscriptions and the lack of speaker identifi-cation.
1 Introduction
Hearing loss can make the participation in normalconversations an exhausting task, because peoplewith hearing impairments need to focus more onthe conversation to be able to keep up (Arlinger,2003). This can cause the deaf and hard of hearingto withdraw from social interactions, leading toisolation and poorer well-being (Arlinger, 2003).Having access to an automatic speech recognizer(ASR) designed to answer their needs could makeparticipation in everyday conversations consider-ably easier for them.
People with hearing impairment are a hetero-geneous group with significant variability in the
degree of hearing loss and its causes. Hearingaids and implants can restore hearing to a degree,but they struggle in noisy environments (Goehringet al., 2016). Many people with hearing impair-ments also refuse to use aids because they areperceived as uncomfortable or costly (Gates andMills, 2005). Professional human interpreters canhelp the deaf and hard of hearing with their nearreal-time transcription, but they require advancebooking and are also costly (Lasecki et al., 2017).
ASR has the potential to both function as a sup-port and a replacement to other solutions. Thestrengths of ASR include accessibility with littlecost, nearly real-time transcription and indepen-dence of costly human labour. Furthermore, it canbe helpful to anyone irrespective of their degreeof hearing loss. The weaknesses of ASR are inrobustness and the lack of support for speaker di-arization. And even though the accuracy of ASRhas improved to a level where it rivals human tran-scribers (Xiong et al., 2017), noisy environments,accented speakers, and far-field microphones re-main a challenge (Yu and Deng, 2015). Addition-ally, recognizing and conveying paralinguistic fea-tures like tone, pitch and gestures is difficult forautomatic systems.
The objective of our work is to study the pref-erences of the deaf and hard of hearing when us-ing ASR-based conversation assistants. We con-structed two pilot Finnish language ASR systemsfor two portable devices with different display op-tions. The first system is a standalone laptopASR that does not utilize network connection orvideo camera. The second system is a mobile de-vice wirelessly connected to an ASR server. Inthe mobile device the ASR transcript is shown inaugmented video stream next to the head of thespeaker. The purpose of this augmented reality
2
(AR) view is to reduce visual dispersion, whichin this case refers to the need of the user to switchattention between multiple visuals. These two se-tups were tested by deaf and hard of hearing users,who were then interviewed to find out their pref-erences. We review the results and compare thefeedback the systems received.
1.1 Previous work
Automatic speech recognition research focusingspecifically on helping people with hearing im-pairments has gone on at least since 1996 (Ro-bison and Jensema, 1996). The first assistiveASR system for the Finnish deaf and hard ofhearing was devised in 1997 (Karjalainen et al.,1997). Since then, helping the deaf and hardof hearing in their school environment has beena concern in many assistive ASR systems. Alot of this research focuses in providing real-timeASR-generated transcriptions of lectures (Wald,2006; Kheir and Way, 2007; Ranchal et al., 2013).Major effort is also dedicated to improving theASR aided learning experience in other ways,such as minimizing visual dispersion (Cavenderet al., 2009; Kushalnagar and Kushalnagar, 2014;Kushalnagar et al., 2010), comparing captioningand transcribing of online video lectures (Kushal-nagar et al., 2013), and using human editors to cor-rect ASR output (Wald, 2006). Our work focusesmore generally on helping people with hearing im-pairments in conversational situations, not just theschool setting.
Other notable applications include the system ofMatthews et al. (2006), where mobile phones wereused for delivering human-made transcriptions viatext messages. They showed transcriptions couldhelp people with hearing impairments, but lackedthe ASR component. The transcription table de-sign from Van Gelder et al. (2005) provided allmeeting participants with partial text support. Theaim there was to minimize the stigma on the deafor hard of hearing participant.
The idea to use AR has also been introducedbefore in the work of Mirzaei et al. (2012, 2014)and Suemitsu et al. (2015). The system of Mirzaeiet al. is similar to our mobile AR system, but itis developed for ultra mobile personal computersand has a text-to-speech component. In the workof Suemitsu et al. the focus is on reducing effectof noise with directional microphones and beam-forming. As a consequence, their system works
well only if the speaker is directly in front of theuser. Moreover, both of these works lack the userperspective because the focus is more on the sys-tem design.
2 Conversation Assistant
Our aim in building the two Conversation Assis-tant systems was to provide automatic transcrip-tions to people with hearing impairments in a use-ful format. A useful conversation assistant sys-tem can (1) recognize large-vocabulary continu-ous speech in real-time, (2) manage varying acous-tic environments with noise, and (3) present thetranscriptions in a clear manner. Achieving thefirst two requirements is possible with modernspeech recognition systems, however, their com-puting power and memory consumption pose lim-itations on the system design. The minimal solu-tion to the third requirement would be to just dis-play the recognition results on a screen, but look-ing at the screen would cause the user to miss non-verbal communication, like gestures.
We built two prototypes of the Conversation As-sistant system: one running on a laptop and one ona mobile device with augmented reality (AR) ca-pabilities. In both systems, Kaldi Speech Recogni-tion Toolkit (Povey et al., 2011) was used to buildthe speech recognition models. In addition, weused Gst-Kaldi, a GStreamer plugin, for handlingthe incoming audio. The source codes for both thelaptop version1 and the mobile AR version2 arepublished on Github.
2.1 Laptop
The laptop version was built for the first round ofuser tests, to find out the preferences of the deafand hard of hearing for the Conversation Assistantconcept in general. We therefore built a decidedlysimple system. It ran a Kaldi speech recognitionmodel locally on a laptop. The acoustic modelof the automatic speech recognizer was a feed-forward deep neural network, trained on multi-condition data to increase noise robustness. Therobustness to noise is important, because conver-sations are rarely had in silent environment. Thedata for training the acoustic model came from theSPEECON corpus (Iskra et al., 2002). The lan-guage model was trained using the Kielipankki3
1https://github.com/Esgrove/mastersthesis2https://github.com/aalto-speech/conversation-assistant3The Language Bank of Finland
3
(a) Laptop version. The speech recognition results are at thetop of the window. The speech being recognized currently isdisplayed at the bottom with orange background.
(b) Mobile AR version. The speech recognition results areplaced in speech bubbles next to the face of the speaker.
Figure 1: User interface screenshots from the software.
corpus and the lexicon was based on morphs (Vir-pioja et al., 2013) instead of words, because of themorphological complexity of Finnish. The speechrecognition model had a word error rate (WER)of 29.7 % when tested on (low-noise) broadcast
news data from the Finnish public broadcasterYLE (Lukkarila, 2017). The user interface (UI),shown in Figure 1a, was kept minimal with onlya record button and space for the transcription re-sults. A more detailed description of the system isgiven in (Lukkarila, 2017).
2.2 Mobile AR
The main objective of the mobile AR Conversa-tion Assistant was to move the laptop-based sys-tem to a mobile device platform and use AR tobring the transcriptions and a visual of the speakerclose to each other. With the latter we aimed toreduce the amount of non-verbal communicationthe user loses when switching between the speakerand the transcriptions. The reduced computationalcapabilities of mobile devices necessitated split-ting the system into a separate mobile AR applica-tion and a speech recognition server.
The mobile application was developed for theiOS platform using a 10.5” iPad Pro tablet fromApple as a test device. In the application, AR cam-era is used to provide a view of the speaker. Theface of the speaker is located locally on the de-vice with a face recognition algorithm provided aspart of iOS toolkits. When speech is recorded bythe device, the application sends the audio to theserver for recognition. The text transcriptions re-turned from the server are then placed in speechbubbles next to the detected face as shown in Fig-ure 1b. The bubbles follow the face of the speaker,if he or she moves in the screen.
The server side uses an open-source KaldiServer implementation (Alumae, 2014) based onGst-Kaldi. The server is split to a controllingmaster server unit and workers responsible for therecognition process. Each mobile device connect-ing to the server needs one worker to do the tran-scribing and connections between the two are han-dled by the master server. The master server andthe workers can be run on different machines andall the communication happens over the internet.This requires stable connections from all parties,including the mobile client, but on the upside, thesystem can be scaled up as much as needed.
In the speech recognizer, the acoustic modelwas a combination of time-delay and long short-term memory neural network trained with FinnishParliament Speech Corpus (Mansikkaniemi et al.,2017). This training data is significantly largerthan the one used for the laptop system, but in-
4
cludes little background noise. The languagemodel was trained with the same data as in the lap-top system, but utilizing a more recent subwordmodeling optimized for Kaldi’s weighted finitestate transducer architecture (Smit et al., 2017).The speech recognition model scored WER of18.56 % on the YLE data which is more thana 10 % absolute improvement over the model inthe laptop system (Mansikkaniemi et al., 2017).Even though these evaluations were not performedfor noisy tasks, we expect that the improvementis substantial enough to cover the lack of noise-robustness for our user tests. For a more de-tailed description of the system, see the work in(Virkkunen, 2018).
3 User tests
To evaluate the prototypes, we organized user testsfor both versions with deaf and hard of hearingparticipants. Our aim was to simulate noisy con-versational situations to see how much the Conver-sation Assistant could help the tester in followingthe conversation. In the user tests of the laptopversion, the participant had a conversation withand without the support of the Conversation As-sistant. In the mobile AR tests, the comparisonwas made between a text-only view similar to thelaptop version and the AR view seen in figure 1b.The contents of this section are further detailed in(Lukkarila, 2017) and (Virkkunen, 2018).
The test users were recruited with the help ofKuuloliitto, the association of the deaf and hard ofhearing in Finland. The number of participants forthe laptop and mobile AR versions were nine andtwelve, respectively. The sample size was limitedby the number of volunteers we were able to find.Each participant was also asked to give a writtenconsent and permission to record the test session.Six people took part in both tests so in total therewere 15 unique participants. The age of the testers,excluding two who refused to disclose their age,ranged from 15 to 84, with the median age be-ing 55. All except two participants were women.Two of the participants were deaf, but they couldcommunicate verbally. The rest had different de-grees of hearing loss and used either hearing aids,cochlear implants or both in their daily lives. Fourparticipants also had used ASR applications be-fore, for example personal voice assistants, auto-matic video captioning, the Google Translate ser-vice and note takers.
3.1 Test setup
The test setup simulated a conversation of twopeople in a noisy environment. The test adminis-trator and the participant would sit face-to-face at atable surrounded by loudspeakers playing a loopednoise recording from a busy cafe. The participanthad the laptop/mobile device in front of them andthey could freely alternate between following theapplication and their conversation partner. In thecase of mobile AR, the participant could choose tohold the device in their hands or place it on a tri-pod. The test was designed to last for one hour andthe feedback was collected using a questionnaire.The overall structure and content of the question-naires is the same between the two user tests, butsmall changes were made to the mobile AR ver-sion to reflect the changes in the system.
The test and the questionnaire had four sections:introduction, word explaining, conversation, anddebriefing. In the introduction the test participantwas familiarized with the test plan and the Conver-sation Assistant. The participant was also asked tofill in their background information in the ques-tionnaire. In the debriefing section, the participantwas asked to give overall feedback on the system.
The first task, word explaining, consisted of thetest administrator explaining words from a list tothe participant, who tried to guess the word inquestion. Halfway through the task, the partici-pant was asked to switch between the comparedmethods (without versus with Conversation Assis-tant or text view versus AR view). After finishingthe word list or running out of time, the participantgave feedback in the questionnaire. In the secondtask, the conversation, the test administrator con-versed with the test participant on a range of com-mon topics from hobbies to food and travel for 10-15 minutes. Switch between the compared meth-ods was made at the midpoint again. And after atime limit set for the task was reached, the partic-ipant was asked to give feedback on the question-naire.
3.2 Results
The questionnaire had questions with both writ-ten and numeric format. Question with writtenfeedback were concerned with the potential usecases of the system and its strengths and weak-nesses. The numeric questions assessed the per-ceived quality of the system and the preferences ofthe testers. The numeric questions further break
5
Mobile AR Laptop Mobile AR Laptop
1
2
3
4
5
6
7R
atin
g
Did the Conversation Assistant help you understand speech?
Text-only AR Laptop Text-only AR Laptop
1
2
3
4
5
6
7
Ra
ting
Was following the Conversation Assistant easy?
Mobile AR Laptop Mobile AR Laptop
1
2
3
4
5
6
7
Ra
ting
Was the speech recognition fast enough?
Mobile AR Laptop Mobile AR Laptop
1
2
3
4
5
6
7
Ra
ting
Were the speech recognition results accurate enough?
Figure 2: Rating results from task 1, word explanation (left side of each plot) and task 2, conversation (right sideof each plot). The range from one to seven is the same as in Figure 3.
down into binary choices, one multiple-choicequestion and rating questions. The rating ques-tions had a range from one (negative) to seven(positive). Questions with numeric format alsohad text fields where the testers could elaboratetheir choices if they wanted.
Several numeric results (Table 1, Figure 3 andtop-left of Figure 2) show that the participantsfound both Conversation Assistant systems use-ful. Majority of the participants would adopt aConversation Assistant system in their daily lives.Potential use cases and environments mentionedincluded daily conversations, meetings, museums,restaurants, live television, lectures and office.Speed and ease of following the output in bothsystems were also rated favorably with few excep-
Would you use an application like theConversation Assistant in your daily life?Yes: 83 % No: 17 %Which mode would you prefer?With AR view: 67 % Text-only view: 33 %Which one of the following options isbetter?Text appears faster(<1 sec), but containsmore mistakes: 67 %
Text appears slower(>1 sec), but containsless mistakes: 33 %
Table 1: Results to the binary questions asked in theuser tests of the mobile AR version.
tions in Figure 2. Those who disagreed said thatthe following suffered mostly from recognition er-rors. Another point raised was the movement andpositioning of the speech bubbles in the mobileAR version which felt distracting to some.
Speech recognition errors were the biggestproblem reducing the perceived utility of the sys-
Mobile AR Laptop
1
2
3
4
5
6
7
Ra
ting
Was the Conversation Assistant
helpful in the test situations?
Figure 3: Rating of the overall usefulness of the sys-tem. The range is from one (negative) to seven (pos-itive). In the box plots, the circle with the dot marksthe median. The bottom and top of the box correspondto 25th and 75th percentiles, respectively. The sam-ple is skewed if the median is not at the middle of thebox. The whiskers show the extreme data points thatare not considered outliers. The data points outside thewhiskers are marked with plus signs.
6
Mobile AR
-3
-2
-1
0
1
2
3
Ra
ting
How would you rate the AR mode in
comparison to the text-only mode?
Figure 4: In the user tests of the mobile AR version,the testers were asked to rate whether they preferredhaving AR view over the text-only view. The range isfrom minus three (much worse) to three (much better).
tem. The ratings are similar for the two systems,but the AR mobile has less negative ratings over-all. In Figure 2 it can be seen that ratings of the ac-curacy reflect the ratings of the helpfulness. Manyfelt the transcriptions helped them get an idea ofthe conversation, but that they could not solely relyon the transcriptions because of the errors. More-over, a couple of participants noted they used theConversation Assistant only a little because theycould use lip reading instead. They would needtranscriptions only in group conversations or incases where the face of the speaker cannot be seen.
Figure 4 shows that the user testers preferred theAR view over having text-only view similar to thelaptop version. Majority of the answers cited theability to use lip reading in AR view as the decisivefactor. Two participants thought the views wouldhave different use cases, for example the text-onlyview could be useful in meetings. One found thetext-only view cleaner and easier to follow than theAR view. Some testers also worried that pointingthe device camera at the conversation partner inthe mobile AR version would feel inappropriate.
Figure 5 shows the participants preferences forend-user devices. It is clear from the answers thatthe device needs to be mobile to be of any use.Most people would like to have the applicationon their smartphones, as it is a device most peo-ple own and carry around everywhere. Tablets andlaptops also get many votes, especially from work-ing age people. Smart glasses got votes from threecurious participants, though we anticipated more.We hypothesize the image most people have of
Smartphone Tablet Laptop Smart glasses PC
1
2
3
4
5
6
7
8
9
10
11
12
Nu
mb
er
of
pe
op
le
Figure 5: Which device the participants would like touse the Conversation Assistant on.
AR glasses is that they are bulky, impractical andstand out. This could explain the lack of appealthe glasses had among the participants.
4 Conclusions
We evaluated two prototypes of a conversationalassistant for the deaf and hard of hearing by usertests. The results show that it is already possible tobuild an assistive application the deaf and hard ofhearing find useful with current technology. In thewritten feedback many expressed the urgent needfor this type of application. Several people notedthey would download the mobile application if itwere available, despite its flaws. Majority of thetest users preferred the mobile AR version becauseit supported their use of lip reading. A couple ofparticipants saw potential use for both versions de-pending on the situation.
Accuracy of the transcriptions was the biggestissue in need of improvement according to the par-ticipants. Several testers also noted that groupconversations in noise are the most difficult to fol-low. For them, speaker diarization would be thefeature that would make the Conversation Assis-tant truly useful. Both systems also lack direct un-mediated eye contact which could be potentiallysolved with AR glasses. However, to be widelyadopted, the glasses would have to be unobtrusiveand light weight.
Acknowledgments
This work was funded by the Academy of Finlandas part of the project ”Conversation assistant forthe hearing impaired” (305503) and Kone Foun-dation. The writers would also like to thank KatriLeino, Tanel Alumae and Kuuloliitto for help andresources.
7
ReferencesTanel Alumae. 2014. Full-duplex speech-to-text sys-
tem for estonian. In Baltic HLT 2014, pages 3–10,Kaunas, Lithuania.
Stig Arlinger. 2003. Negative consequences of uncor-rected hearing loss-a review. International journalof audiology, 42:2S17–2S20.
Anna C Cavender, Jeffrey P Bigham, and Richard ELadner. 2009. Classinfocus: enabling improved vi-sual attention strategies for deaf and hard of hearingstudents. In Proceedings of the 11th internationalACM SIGACCESS conference on Computers and ac-cessibility, pages 67–74. ACM.
George A Gates and John H Mills. 2005. Presbycusis.The Lancet, 366(9491):1111–1120.
Tobias Goehring, Xin Yang, Jessica JM Monaghan,and Stefan Bleeck. 2016. Speech enhancement forhearing-impaired listeners using deep neural net-works with auditory-model based features. In 201624th European Signal Processing Conference (EU-SIPCO), pages 2300–2304.
Dorota J. Iskra, Beate Grosskopf, Krzysztof Marasek,Henk van den Heuvel, Frank Diehl, and AndreasKiessling. 2002. Speecon - speech databases forconsumer devices: Database specification and val-idation. In LREC.
Matti Karjalainen, Peter Boda, Panu Somervuo, andToomas Altosaar. 1997. Applications for thehearing-impaired: Evaluation of Finnish phonemerecognition methods. In Fifth European Conferenceon Speech Communication and Technology.
Richard Kheir and Thomas Way. 2007. Inclusion ofdeaf students in computer science classes using real-time speech transcription. In Proceedings of the12th Annual SIGCSE Conference on Innovation andTechnology in Computer Science Education, ITiCSE’07, pages 261–265. ACM.
Raja Kushalnagar and Poorna Kushalnagar. 2014. Col-laborative gaze cues and replay for deaf and hard ofhearing students. In Computers Helping People withSpecial Needs, pages 415–422, Cham. Springer In-ternational Publishing.
Raja S. Kushalnagar, Anna C. Cavender, and Jehan-Francois Paris. 2010. Multiple view perspectives:Improving inclusiveness and video compression inmainstream classroom recordings. In Proceedingsof the 12th International ACM SIGACCESS Confer-ence on Computers and Accessibility, ASSETS ’10,pages 123–130, New York, NY, USA. ACM.
Raja S. Kushalnagar, Walter S. Lasecki, and Jeffrey P.Bigham. 2013. Captions versus transcripts for on-line video content. In Proceedings of the 10th Inter-national Cross-Disciplinary Conference on Web Ac-cessibility, W4A ’13, pages 32:1–32:4, New York,NY, USA. ACM.
Walter S. Lasecki, Christopher D. Miller, IftekharNaim, Raja Kushalnagar, Adam Sadilek, DanielGildea, and Jeffrey P. Bigham. 2017. Scribe:Deep integration of human and machine intelligenceto caption speech in real time. Commun. ACM,60(9):93–100.
Juri Lukkarila. 2017. Developing a conversation assis-tant for the hearing impaired using automatic speechrecognition. MSc Thesis, Aalto University.
Andre Mansikkaniemi, Peter Smit, Mikko Kurimo,et al. 2017. Automatic construction of the Finnishparliament speech corpus. In INTERSPEECH2017–18th Annual Conference of the InternationalSpeech Communication Association.
Tara Matthews, Scott Carter, Carol Pai, Janette Fong,and Jennifer Mankoff. 2006. Scribe4me: Evaluatinga mobile sound transcription tool for the deaf. InUbiComp 2006: Ubiquitous Computing, pages 159–176. Springer.
Mohammad Reza Mirzaei, Seyed Ghorshi, and Mo-hammad Mortazavi. 2012. Combining augmentedreality and speech technologies to help deaf and hardof hearing people. In 2012 14th Symposium on Vir-tual and Augmented Reality, pages 174–181.
Mohammad Reza Mirzaei, Seyed Ghorshi, and Mo-hammad Mortazavi. 2014. Audio-visual speechrecognition techniques in augmented reality envi-ronments. The Visual Computer, 30(3):245–257.
Daniel Povey, Arnab Ghoshal, Gilles Boulianne, LukasBurget, Ondrej Glembek, Nagendra Goel, MirkoHannemann, Petr Motlicek, Yanmin Qian, PetrSchwarz, Jan Silovsky, Georg Stemmer, and KarelVesely. 2011. The Kaldi speech recognition toolkit.In IEEE 2011 Workshop on Automatic SpeechRecognition and Understanding. IEEE Signal Pro-cessing Society. IEEE Catalog No.: CFP11SRW-USB.
Rohit Ranchal, Teresa Taber-Doughty, Yiren Guo,Keith Bain, Heather Martin, J Paul Robinson, andBradley S Duerstock. 2013. Using speech recogni-tion for real-time captioning and lecture transcrip-tion in the classroom. IEEE Transactions on Learn-ing Technologies, 6(4):299–311.
Joseph Robison and Carl Jensema. 1996. Computerspeech recognition as an assistive device for deafand hard of hearing people. In Biennial Conferenceon Postsecondary Education for Persons Who AreDeaf or Hard of Hearing (7th, Knoxville, Tennessee,volume 948, page 154. ERIC.
Peter Smit, Sami Virpioja, and Mikko Kurimo.2017. Improved subword modeling for WFST-based speech recognition. In INTERSPEECH 2017– 18th Annual Conference of the InternationalSpeech Communication Association, pages 2551–2555, Stockholm, Sweden.
8
Kazuki Suemitsu, Keiichi Zempo, Koichi Mizutani,and Naoto Wakatsuki. 2015. Caption support sys-tem for complementary dialogical information us-ing see-through head mounted display. In 2015IEEE 4th Global Conference on Consumer Electron-ics (GCCE), pages 368–371.
Joris Van Gelder, Irene Van Peer, and Dzmitry Ali-akseyeu. 2005. Transcription table: Text supportduring meetings. In IFIP Conference on Human-Computer Interaction, pages 1002–1005. Springer.
Anja Virkkunen. 2018. Automatic speech recognitionfor the hearing impaired in an augmented reality ap-plication. MSc Thesis, Aalto University.
Sami Virpioja, Peter Smit, Stig-Arne Gronroos, andMikko Kurimo. 2013. Morfessor 2.0: Python im-plementation and extensions for Morfessor Baseline.Technical report, Aalto University.
Mike Wald. 2006. Captioning for deaf and hard ofhearing people by editing automatic speech recogni-tion in real time. In Computers Helping People withSpecial Needs, pages 683–690, Berlin, Heidelberg.Springer.
Wayne Xiong, Jasha Droppo, Xuedong Huang, FrankSeide, Michael L Seltzer, Andreas Stolcke, DongYu, and Geoffrey Zweig. 2017. Toward human par-ity in conversational speech recognition. IEEE/ACMTransactions on Audio, Speech and Language Pro-cessing (TASLP), 25(12):2410–2423.
Dong Yu and Li Deng. 2015. Automatic Speech Recog-nition: A Deep Learning Approach. Springer, Lon-don.