recent ccrma research on virtual musical instrumentsjos/pdf/askenfeltjos09.pdf · overview overview...
TRANSCRIPT
![Page 1: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/1.jpg)
Julius O. Smith III Askenfelt60, KTH, Sweden – 1 / 23
Recent CCRMA Research on Virtual Musical Instruments
Julius O. Smith IIICCRMA, Stanford University
Askenfelt60, KTH, Sweden
October 21, 2009
![Page 2: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/2.jpg)
Overview
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 2 / 23
![Page 3: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/3.jpg)
Recent Research on Virtual Musical Instruments at CCRMA
Overview
• CCRMA
• Outline
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 3 / 23
CCRMA building: The Knoll, Stanford University
![Page 4: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/4.jpg)
Outline
Overview
• CCRMA
• Outline
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 4 / 23
• Virtual Acoustic Guitar — Nelson Lee(Computer Science PhD student)
• Haptic Virtual Instruments — Ed Berdahl(Electrical Engineering PhD student)
• Virtual Harpsichord — Jack Perng(Physics PhD student)
• Acoustic Space Modeling — Consulting Professor JonathanAbel, Music PhD student Nick Bryan, EE graduate student TravisSkare, and others
• Special Issue on Virtual Analog Audio Effects & MusicalInstruments, for the IEEE Transactions on Audio, Speech, &Language Processing (ASLP) — edited by Valimaki, Fontana,Zolzer, & Smith
![Page 5: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/5.jpg)
Virtual Acoustic Guitar Models
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 5 / 23
![Page 6: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/6.jpg)
Coupled Strings Analysis and Synthesis
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 6 / 23
Submitted paper based on recent CCRMA/CS thesis by Nelson Lee :
“Analysis and Synthesis of Coupled Vibrating StringsUsing a Hybrid Modal-Waveguide Synthesis Model”
by Nelson Lee, Julius Smith, and Vesa Valimaki.Accepted for publication in the IEEE special issue onVirtual Analog Audio Effects and Musical Instruments,May 2010 (est.)
![Page 7: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/7.jpg)
Overview of Nelson Lee String Model
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 7 / 23
• String excitation (for commuted waveguide synthesis) is highpassfiltered to avoid exciting first N partials
• Lowest N partials are replaced by fourth-order resonators (whichcan independently beat and give two-stage decay)
• Similar to Balazs Bank formulation which adds second-orderresonators to existing partials of the filtered-delay-loop
• New analysis methods (in thesis) for estimating partialparameters, as well as other results
![Page 8: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/8.jpg)
Sound Examples of Individual Effects
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 8 / 23
From Nelson Lee’s thesis defense:
• Original waveform: (WAV) (MP3)• Simple lossless, reflectively terminated digital waveguide (DWG):
(WAV) (MP3)• Add loop filter: (WAV) (MP3)• Add interpolation filter: (WAV) (MP3)• Add excitation (ICMC07): (WAV) (MP3)• Add body response: (WAV) (MP3)• Add hybrid modal/waveguide model: (WAV) (MP3)• Exaggerate pitch glide due to tension modulation: (WAV) (MP3)
![Page 9: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/9.jpg)
Virtual Acoustic Guitar Sound Examples
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 9 / 23
More Nelson Lee examples:
• Original 1: (WAV) (MP3)• Synthesized 1: (WAV) (MP3)
• Original 2: (WAV) (MP3)• Synthesized 2: (WAV) (MP3)
• Original 3: (WAV) (MP3)• Synthesized 3: (WAV) (MP3)
• Original 4: (WAV) (MP3)• Synthesized 4: (WAV) (MP3)
• Original 5: (WAV) (MP3)• Synthesized 5: (WAV) (MP3)
• Original 6: (WAV) (MP3)• Synthesized 6: (WAV) (MP3)
• Synthesized Chord Demo: (WAV) (MP3)
![Page 10: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/10.jpg)
Haptic Feedback Control forVirtual Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 10 / 23
![Page 11: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/11.jpg)
Haptic Virtual Musical Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
• Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 11 / 23
CCRMA/EE PhD student Ed Berdahl is working on
Haptic Feedback Control for Virtual Instruments
Dissertation expected by the end of the year
Goals:
• Assist and/or augment gestures• Assist with accurate playing• Recent projects:
◦ Haptically plucked virtual string◦ Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/-HapticDrum/
![Page 12: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/12.jpg)
Virtual Harpsichord
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 12 / 23
![Page 13: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/13.jpg)
Harpsichord Modeling
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
• Harpsichord
• Harpsichord Jack
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 13 / 23
CCRMA/Physics PhD student Jack Perng is working on
1. Building a harpsichord jack and monochord (mostly done)2. Measuring position, velocity, etc. (getting started)3. Studying models of Giordano et al., Valimaki et al., and others
(well under way)4. Looking for ways to improve the model(s)
Prof. Tom Rossing collaborating
![Page 14: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/14.jpg)
Harpsichord Jack and Monochord
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
• Harpsichord
• Harpsichord Jack
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 14 / 23
![Page 15: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/15.jpg)
Acoustically Transparent andConfigurable Microphone Array
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 15 / 23
![Page 16: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/16.jpg)
Microphone Array
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
• Mic Array
• Mic Array Paper
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 16 / 23
• Adustable geometry (software calibrated)• Sixteen microphones (Countryman B6 Omni Lavalier):
◦ 2 mm diameter capsules◦ 1 mm diameter flexible mounting wire◦ Acoustically transparent over most of the audio band
![Page 17: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/17.jpg)
Upcoming Paper
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
• Mic Array
• Mic Array Paper
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 17 / 23
“A Configurable Microphone Array with Acoustically TransparentOmnidirectional Elements”
Jonathan Abel, Nicholas Bryan, Travis Skare, Patty Huang, DariusMostowfi, Miriam Kolar, and Julius Smith
AES-2009, New York
Current Application:
Recording and modeling acoustic properties of undergroundgalleries at pre-Inca archeological site Chavın de Huantar in Peru
![Page 18: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/18.jpg)
Special Issue of the IEEE ASLP
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 18 / 23
![Page 19: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/19.jpg)
IEEE ASLP Special Issue
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
• ASLP Special Issue
• Issue Overview
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 19 / 23
We have been working on a special issue devoted to
Virtual Analog Audio Effects and Musical Instruments
for the
IEEE Transactions on Audio, Speech, and LanguageProcessing (ASLP)
• Editors:
◦ Vesa Valimaki◦ Federico Fontana◦ Udo Zolzer◦ Julius Smith
• Eleven papers accepted so far, with more to come• Likely publication date: May 2010
![Page 20: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/20.jpg)
Accepted Papers on Virtual Musical Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
• ASLP Special Issue
• Issue Overview
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 20 / 23
• “Tubular Bells — A Physical and Algorithmic Model” byRabenstein, Koch, and Popp
• “A block-based physical modeling approach to the soundsynthesis of drums” by Marogna and Avanzini
• “A Virtual Model of Spring Reverberation” by Bilbao and Parker
• “Analysis and Synthesis of Coupled Vibrating Strings Using aHybrid Modal-Waveguide Synthesis Model” by Lee, Smith, andValimaki
Likely additional papers on
• virtual plucking methods for waveguide string models,• piano modeling via physically directed modal synthesis, and• another approach to spring reverb modeling
![Page 21: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/21.jpg)
Summary
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 21 / 23
![Page 22: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/22.jpg)
Summary
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
• Summary
• Conclusions
Julius O. Smith III Askenfelt60, KTH, Sweden – 22 / 23
A quick look at recent acoustic-modeling research at CCRMA:
• Coupled Strings Analysis and Synthesis — Nelson Lee (CS) —Fourth-order modes for low partials, waveguide model for upperpartials; new analysis techniques
• Haptic Virtual Instruments — Ed Berdahl (EE) — Real controllers(with force feedback) for virtual instruments
• Virtual Harpsichord — Jack Perng (Physics) — Monochord+jackmeasurements toward improved harpsichord synthesis models
• Microphone Array — Jonathan Abel et al. — Acousticallytransparent, configurable, software-calibrated microphone arrayfor sampling the 3D sound field
• Special Issue on Virtual Analog Audio Effects and MusicalInstruments — Vesa Valimaki et al., eds.
![Page 23: Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview Acoustic Guitar Models Haptic Instruments Harpsichord Models Microphone Array ASLP](https://reader033.vdocuments.us/reader033/viewer/2022051801/5ae0c3037f8b9a5a668dffe8/html5/thumbnails/23.jpg)
In Conclusion
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
• Summary
• Conclusions
Julius O. Smith III Askenfelt60, KTH, Sweden – 23 / 23
Happy Birthday Anders!