center for language and speech processing the johns hopkins university language and speech...
Post on 13-Jan-2016
226 Views
Preview:
TRANSCRIPT
Center for Language and Speech ProcessingThe Johns Hopkins University
Language and Speech Processing at Johns Hopkins University
March 5, 2010
The JHU Center for Language and Speech Processing
CLSP was established in 1992 with outside support to promote research and education in the science and technology of speech and language.
CLSP
Applied Math & StatisticsBiomedical Engineering
Cognitive Science
Electrical and ComputerEngineering
Computer Science
Human Language Tech.Center of Excellence
Speech and Language Faculty at JHU(does not list senior research staff, postdocs, students, …)
Electrical & Computer Eng Andreas Andreou Mounya Elhilali Hynek Hermansky Frederick Jelinek (Director) Damianos Karakos Sanjeev Khudanpur
Computer Science Chris Callison-Burch Jason Eisner David Yarowsky
Applied Math & Statistics Carey Priebe
Cognitive Science / Psychology Justin Halberda Geraldine Legendre Kyle Rawlins Paul Smolensky (Asst Dir) Colin Wilson
Biomedical Engineering Eric Young
Applied Physics Laboratory James Mayfield Christine Piatko
HLT Center of Excellence Kenneth Church Mark Dredze`
Aren Jansen Ben Van Durmé
CLSP Vision Statement
Understand how human language is used to communicate ideas/thoughts/information.
Develop technology for machine analysis, translation, and transformation of multilingual speech and text.
CLSP Mission Statement
1. Research Advance state of the art in our interdisciplinary field Focus on developing key algorithms and statistical models Focus on strategic languages, including low-resource languages
2. Education Attract the best students and train them to be leaders Offer full spectrum of courses Conduct annual international summer school at JHU
3. Outreach Be responsive to government and industry problems Serve as a “hub” for the HLT community Organize international summer research workshop at JHU Welcome short- and long-term visitors
Research: Primary Areas
Speech Recognition Acoustic processing Acoustic-phonetic modeling Pronunciation modeling Language modeling
Speech Applications Keyword spotting Spoken term detection Speaker verification Language identification
Speech Science Auditory physiology Neuromorphic signal processing
Natural Language Processing Morphological analysis Syntactic analysis (parsing) Information extraction Co-reference resolution
Machine Translation Low-resource languages Arabic and Chinese
Knowledge-Base Population Automatic content extraction Inference and learning
Machine Learning Small-sample learning Structured prediction Minimally supervised learning
Sponsored Research in Speech & Language in WSE is ≈ $2.5M/year
P Investigator Project Title (Granting Agency) Period Amount
Jelinek Investigation of Meaning Representation in Language Understanding (NSF) 10/05-01/10 $2.5 M
Jelinek Cross Cutting Research Workshops in Intelligent Information Systems (NSF) 09/07-08/11 $830 K
Eisner Finite-State Machine Learning on Strings and Sequences (NSF) 02/04-01/10 $500 K
Khudanpur Rosetta: An Analyst Co-pilot (DARPA/IBM) 10/05-04/11 $3.4 M
Eisner Learned Dynamic Prioritization (NSF) 09/10-08/14 $1.2 M
Hager Gesture Induction for Manipulative and Interactive Tasks (NSF) 02/06-01/10 $490 K
Smolensky* Unifying the Science of Language (NSF: IGERT) 05/06-04/11 $3.0* M
Vice Provost* Human Language Technology Center of Excellence (MPO) 01/07-01/17 $50* M
Andreou Energy Efficient Organic Semiconductor Circuits (DOE) 05/07-04/10 $660 K
Yarowsky Multi-Level Modeling of Language and Translation (NSF) 06/07-06/10 $400 K
Karakos Novel Approaches to Unsupervised Classification via ISPDTs (NSF) 09/07-08/10 $300 K
Callison-Burch DARPA Computer Science Study Group (DARPA) 02/08-02/09 $93 K
Jelinek Research Workshops in Intelligent Information Systems (Google) 06/08-05/10 $270 K
Khudanpur Self-Supervised Discriminative Training of Statistical Language Models (NSF)
09/08-08/09 $137 K
Khudanpur Self-Training for ASR in Low Resource Languages (BBN) 09/09-08/10 $101 K
CLSP Mission Statement
1. Research Advance state of the art in our interdisciplinary field Focus on developing key algorithms and statistical models Focus on strategic languages, including low-resource languages
2. Education Attract the best students and train them to be leaders Offer full spectrum of courses Conduct annual international summer school at JHU
3. Outreach Be responsive to government and industry problems Serve as a “hub” for the HLT community Organize international summer research workshop at JHU Welcome short- and long-term visitors
Education: Interdisciplinary Environment
Who and where PhD, MSE, and BS students from multiple depts. Shared interdisciplinary offices in CSEB Shared technical perspective and computing infrastructure
Coursework Interdisciplinary core curriculum (extends dept. requirements) Variety of other relevant courses (growing list, new plans) International 2-week summer school
Research Students do research from the start Students work with faculty from multiple departments and HLTCOE
Other learning Distinguished outside speaker every week Student speaker and town meeting every week Reading groups and conference travel
Sample Courses for an MSE in Human Language Technology
Course Number Course Title Instructor
CS 600.465 Natural Language Processing Eisner
CS 600.466 Information Retrieval and Web Agents Yarowsky
CS 600.425 Declarative Methods Eisner
AMS 550.732 Pattern Recognition Priebe
COG 050.320 Introduction to the Syntax of Natural Language Legendre
COG 050.325 Sound Structure in Natural Language Burzio
COG 050.825 Optimality Theory Smolensky
ECE 520.445 Introduction to Speech and Audio Processing Elhilali
ECE 520.447 Introduction to Information Theory and Coding Jelinek
ECE 520.651 Random Signal Analysis Khudanpur
ECE 520.666 Information Extraction from Speech and Text Jelinek
ECE 520.674 Information Theoretic Methods in Statistics Khudanpur
ECE 520.682 Computational Systems Neuroscience Elhilali
ECE 520.735 Sensory Information Processing Andreou
Education: Track Record
WSE has the 2nd largest university group in the U.S. working on Human Language Technology
38 PhDs awarded, many more MSEs
CLSP PhDs presently hold research/faculty positions at
Carnegie Mellon University U. of Massachusetts, Amherst Swarthmore College Michigan State University Hong Kong Polytechnic Univ. Bogazici University (Turkey) U. of Karlsruhe (Germany) Saarland University (Germany)
CLSP PhDs presently hold senior technical/research positions at
Apptek BBN Convergys e-Scription Fair Isaac Google (several) Microsoft (several) MITRE IBM (several) NSA (several) Nuance (several) SRI International
CLSP Mission Statement
1. Research Advance state of the art in our interdisciplinary field Focus on developing key algorithms and statistical models Focus on strategic languages, including low-resource languages
2. Education Attract the best students and train them to be leaders Offer full spectrum of courses Conduct annual international summer school at JHU
3. Outreach Serve as a “hub” for the HLT community Be responsive to government and industry problems Organize international summer research workshop at JHU Welcome short- and long-term visitors
JHU Summer Workshops in HLT:Integrating Research and Education
Organized by JHU on behalf of the Human Language Technology field 3 teams per summer (since 1995)
selected & refined from 25 proposals by “interactive peer review” each team comes to JHU for 8 weeks of intense collaborative research
Mixed teams of senior and student researchers Team ≈ 3 academics, 1 industry, 1 govt, 2-3 grad students, 2 undergrads 30+ participants 8 weeks 15 years More than 160 star students trained in HLT research (1998—2007)
Outcomes Numerous research breakthroughs New, long-term collaborations, tangible knowledge transfer Diverse expertise, research infrastructure, data resources
15
A Few of ManyWorkshop Accomplishments
A small sample of research results and their wider impact Statistical Machine Translation (1999)
GIZA++ is extensively used to build SMT systems even today MEAD Multilingual Multi-document Summarization (2001)
100s of worldwide users, active developers in the community SuperSID: High-level information for Speaker-ID (2002)
Major breakthrough in speaker recognition technology Factored Language Models (2002)
Improved ASR technology for conversational Arabic Moses Machine Translation Repository (2006)
The de facto standard in statistical machine translation More than 100 refereed publications
Detailed technical reports also available on CLSP web-site
Human Language TechnologyCenter of Excellence at JHU
• Long-term research mission: Automatically analyze a wide range of speech, text, and document images in multiple languages.
• Founded with government support in 2007• Has brought many new researchers and
research challenges into the CLSP community• Aggressively hiring the top new Ph.D.s nationally
Human Language TechnologyCenter of Excellence at JHU
Sponsor RDLeadership
ExecutiveDirector
AdministrativeStaff
Director ofResearch
Researchers
JHU Provost
SponsorResearchers
Whiting School of Engineering
Sponsor Technical Board
Center forLanguage and
Speech Processing
SecurityStaff
Prof. Andreas G. Andreou: Sensory Information Processing in Natural and Synthetic Systems
Research• Principles of sensory information
processing in biology.• Sensory communication.• Algorithms and processor architecture
design for energy efficient acoustic, speech and vision processing.
• Physics of sensing and computation.
ApplicationsAlgorithms for robust ASR• Robust acoustic feature
representation and dimensionality reduction
• Algorithms and architecture optimization for Chip Multi Processors (CMP) in Exascale systems
Multimodal scene analysis• Active and passive processing for
scene analysis (visual & auditory)• Acoustic and EM micro-Doppler
imagingBio-inspired systems• Energy efficient microsystems for
processing what and where in natural environments.
Prof. Chris Callison-Burch: Statistical Machine Translation
Research• Statistical machine translation• Syntactic translation models• Low resource languages• Data-driven paraphrasing• Evaluation measures, creation
of shared data resources
Prof. Kenneth Church:Human Language Technology (HLT) at Scale
Applications• Web search• Cloud computing• Language modeling• Text analysis• Spelling correction• Word-sense disambiguation• Terminology• Translation• Lexicography• Compression• Speech recognition and synthesis• OCR
Research• Speech Processing at Scale• Language Processing at Scale• Web Search at Scale• Mining Speech/Language with
Zero Linguistic Resources
Prof. Mark Dredze: Applications of Machine Learning to Real-World Text Processing
ApplicationsDomain adaptation• Extending NLP models to new
datasetsCross-domain learning• Applying NLP techniques to
languages with few resourcesKnowledge base population• Building large high precision
knowledge bases from textIntelligent email• Improved email clients by aiding
the user with artificial intelligence
Research• Adaptation of machine
learning algorithms between text domains
• Large scale information processing and learning
• Intelligent user interfaces for information management
Prof. Jason Eisner: Algorithms and Models for Language Processing
ApplicationsParsing sentence structure• Faster and more accurate algorithms• Unsupervised or cross-lingual
learningMachine translation• Model syntax, structure, word order• Combinatorial methods for
translation and for training modelsMorphology / phonology• Word spelling and pronunciation• Variant word forms (conjugation,
transliteration, misspelling, …)Information integration• Truth maintenance • Deductive databases• Reasoning from facts in text
Research• Novel algorithms for NLP • Bayesian statistical models
of linguistic structure• Machine learning
(structured prediction, novel training objectives)
• Declarative formalisms for grammars and algorithms
Prof. Mounya Elhilali: Reverse Engineering the Neurobiology of Speech and Audio Processing
Applications• Speech intelligibility in noise and
distortions• Auditory scene analysis and
speaker segregation• Speech enhancement• Hearing prostheses• Adaptive audio systems • Robotics and autonomous
systems• Object tracking in sensor
networks• Communication channels • Microphone Design
Research Goals• Information representation
and computational strategies employed by the brain
• Sound perception in distorted or complex acoustic environments
Prof. Hynek Hermansky: Robust Acoustic Speech Processing
ApplicationsSpeech recognition • what has been said?Speaker identification• who is speaking?Speaker verification• is the talker the one claimed to be?Language identification• which language is being used?Speech and audio coding• how to store/transmit the signal
efficiently?Enhancement of degraded speech• how to make noise or reverberated
speech easier listening to?
Technology• Proprietary techniques
based on temporal cues in the signal and on artificial neural net post-processing
• Emulations of auditory processing in biology
Prof. Aren Jansen:Knowledge-based Approaches to Speech Processing
ApplicationsNoise-Robust Speech Recognition• Invariance and efficiency through sparsityLow-Resource Speech Recognition• What can be done with little or no
transcribed training data?Spoken Term Detection and Discovery• “Google” for speech documents• Query-by-example vs. text queriesLarge-Scale Speech Processing • Scaling speech technology to massive
problem sizes
Research• Pursuit of more invariant representations of
speech• Unsupervised/semi-supervised learning of
speech units• Sparse representations and models• Computational models of human speech
perception
Prof. Frederick Jelinek:Statistical Speech Recognition and Machine Translation
ResearchStatistical aspects of Automatic Speech
Recognition (ASR)Language Modeling• Predicting next word given the pastReconstruction of ASR output• Create a grammatical sentence
preserving the speaker’s intended meaning
Rescoring of ASR output alternativesSearch algorithms for ASR and
Machine Translation
Interests• Statistical grammar and
parsing• Signals and systems• ASR treatment of out-of-
vocabulary words and phrases
• Machine translation
Prof. Damianos Karakos: Statistical Aspects of Speech and Language
ApplicationsSpeech recognition • Adaptation to the speech topic• Error corrective techniquesMachine translation• System combination• Language modelingDocument categorization• Automatic clustering into
meaningful categories• Detection of topics of interest
Technology• Data fusion and
dimensionality reduction for improved inference in text classification.
• Novel language modeling techniques for speech recognition.
Prof. Sanjeev Khudanpur:Statistical Modeling for Information Processing
ApplicationsAutomatic speech recognition• Domain and genre adaptation• Pronunciation variability modelingMachine translation (text & speech)• Output language word ordering• Context dependent translationMultimedia search and retrieval• Searching large speech archives• Content-based image/video searchRobotic minimally invasive surgery• Automated skill assessment• Automated surgical training
Basic Research• Stochastic Modeling of Signals
and Systems• Parameter Estimation• Model Structure Estimation• Information Theory and
Statistics
Prof. Benjamin Van Durme: Computational Semantics and Large-Scale Text Processing
ApplicationsKnowledge Acquisition• Enable “everyday” reasoning• Formal interpretation of generic
sentences (e.g., dictionary definitions)
“Deep” Information Extraction• Infer implicit relations• Semantic language modeling• Recognize higher order
modification of factoidsOrganizing Social Media• Dynamic clustering of authors,
documents, feeds
Research• Application of theoretical
semantics to problems in language technology
• Streaming algorithms for efficient processing of large text collections
Prof. David Yarowsky:Minimally Supervised Learning for Low-Resource Languages
ApplicationsMachine Translation• Translation discovery without
aligned bilingual text• Exploiting language universals and
language family relationshipsNatural Language Processing• Word sense disambiguation• Inflectional and derivational
morphologyInformation Extraction• Biographic fact extraction• Characterizing communicants• Informal genres
Basic Research• Cross-language information
projection• Cross-domain knowledge transfer• Co-training• Active learning and human
computation• Creative bootstrapping from multiple
knowledge sources
Prof. Kyle Rawlins: Formal & Computational Semantics
Prof. Paul Smolensky: Architecture of Universal Grammar
Prof. Colin Wilson: Theoretical, Experimental, & Computational Phonology
Prof. Geraldine Legendre: Syntax, Morphology, Acquisition
Linguistics and Human Language Processing
Prof. Justin Halberda: Word Learning in Children + a new professor …
Human Sentence Processing
Lots of Great Ph.D. Students:The Next Big Things!
top related