the clam framework
TRANSCRIPT
The CLAM framework
Developing Cross-Platform
Audio and Music Applications with the
CLAM Framework
ACM Multimedia '06
Xavier Amatriain (UCSB)
Pau Arum (UPF)
David Garcia (UPF)
Index
Introduction
Applications
Infrastructure
Repositories
Tools
Conclusions
Demo!
Started in October 2000
There are more than 250 C++ classes (50.000 loc), compiled under GNU/ Linux, Windows and OSX.
CLAM has been used for various audio projects: time-stretching, real-time sax synthesis, content analysis and Mpeg7 description, real-time audio effects, audio annotation...
It has also been used for educational purposes.
Factsheet
Highlights
Although it currently specializes in audio and music, it presents a metamodel that has proven valid for general multimedia.
Deals well with spectral-domain processing
Can process streams with different kinds of data types, even user-defined
Two different working modes: application framework and rapid-prototyping.
It is really Object-oriented and has been documented through a Pattern Language
It is efficient and can be used for real-time applications
Applications. Annotator
Applications. SMSTools
Applications. Network Editor
Applications. Others
CLAM Components
Infrastructure. CLAM Network model
The CLAM network is a graphical model of computation based on Dataflow Process Networks
Scheduling can be performed both statically and dynamically, depending on the particular application.
Infrastructure. Ports, Controls, and Configurations
Two kinds of connection mechanisms between Processing Objects:
Ports
Data is transmitted synchronously through FIFO queues
Controls
Events are transmitted asynchronously
Configurations, unlike Controls, hold parameters that produce expensive or structural changes in a Processing Object
They can only be set when the Processing Object is not running
Infrastructure. Processing
Infrastructure. Dynamic vs. Static Compositions
Processing Objects can be grouped using two different mechanisms:
Networks
Created dynamically at run-time
Strive for flexibility and understandability
Offer automatic flow and data management
Processing Composites
Created statically at compile-time
Strive for efficiency
The programmer is in charge of most flow management issues.
Repositories (black-box)
Ready-to-use processing classes (almost 150):
Analysis (FFT, spectral analysis, SMS analysis, Tonal Analysis, descriptor extraction...), Arithmetic Operators, Input/Output Processing Objects (Audio, AudioFile, MIDI, SDIF), Generators, Transformation, Synthesis
Ready-to-use data classes:
Audio, Spectrum, SpectralPeakArray, Fundamental, Frame, Segment, Descriptors...
Tools. Platform Abstraction
Audio I/O
MIDI I/O
Audio File I/O: wav, aiff, mp3, ogg...
ID3 tags.
SDIF File support
Support for OSC, JACK, LADSPA, SDIF, VST, ASIO...
Tools
XML
Any Processing Data or Configuration has automatic XML persistency.
GUI
Visualization module based on the Qt toolkit
Plus many ready-to-use graphical components (widgets)
Including debugging tools
External Open Source libraries
FFTW (FFT)
Xercesc & libxml (XML using DOM API)
FLTK and QT GUI toolkits
RtAudio, PortAudio or DirectX (for Windows audio)
Libsndfile, Ogg-Vorbis, libmad (mp3), id3lib, for handling audio files.
oscpack
libjack
CppUnit (testing framework, only used for development)
pthreads (multithreading on Windows)
Conclusions
CLAM is currently in release 0.91 and we plan on getting to 1.0 by the end of 2006.
But it already offers:
A powerful framework that can yield efficient and robust applications.
A prototyping tool for rapid development
A number of ready-to-use applications.
Demo
Visit us at....
www.clam.upf.es
Xavier Amatriain, Pau Arumi, David Garcia ...
Click to edit the title text format
Click to edit the outline text format
Second Outline Level
Third Outline Level
Fourth Outline Level
Fifth Outline Level
Sixth Outline Level
Seventh Outline Level
Eighth Outline Level
Ninth Outline Level