techshanghai2016 - dsp enhanced iot applications and development platform
TRANSCRIPT
DSP Enhanced IoT Applications and Development Platform
2 CEVA Proprietary Information
Not All IoT Devices Are Created Equal …
Connected
While IoT must be connected, it is not always smart !
Smart Vs.
smart home security device
3 CEVA Proprietary Information
IoT Layers – Connected vs. Smart Example: a truly smart home controller would …
Input Processing Smart Analysis Actions
Speech Speech recognition
Voice command Speaker verification
Turn on/off lights, window shades, appliances … Allow access to devices per user profiles
Sound Sound sensing
Is music playing ? Which music ? Alert noises ? (e.g. breaking glass)
Change light-bulb color per music type Call security company; notify owner
Vision Face recognition
Who is in the house ? Resident or guest ?
Set mood (color, music) per person or group Send message to owner about guests presence
Motion Motion detection
People moving in home ? Device being moved ?
Learn habits of tenants Alert about anomalies
Connectivity SW PHY Is Wi-Fi available ? Yes – use in-home Wi-Fi No – use LTE/MTC (preferred for security events)
DSP enables the creation of smart IoT devices !
4 CEVA Proprietary Information
DSP Powered Use-cases and Solutions
5 CEVA Proprietary Information
Smart IoT Platform – DSP Enabled
Connectivity
Sensing
CPU OS
Intelligence
DSP Based SW PHY Wi-Fi, Bluetooth, ZigBee, LTE (CAT0/ MTC/NB-LTE), PLC
DSP based intelligence Speech Recognition, Face Recognition, Depth mapping, Analytics, Learning
DSP based sensing Motion sensing, Sound sensing, Camera, Health
CEVA IoT solutions for Connectivity + Sensing + Intelligence
6 CEVA Proprietary Information
Challenges in IoT Connectivity
ANT+
Blu
etoo
th
LTE
DEC
T U
LE
NFC
PLC
Wi-F
i
ZigB
ee Z-
Wav
e O
ther
s
Wearables ● ● ● ● ● ●
Smart Devices ● ● ● ● ● ●
Smart Homes ● ● ● ● ● ● ● ● ●
Cars ● ● ● ●
Smart Cities ● ● ● ● ●
Industrial Internet ● ● ●
Multiple and constantly evolving communication standards
CEVA’s DSP based SW PHY allows multi-standard connectivity platform
7 CEVA Proprietary Information
CEVA-Dragonfly M2M Platform ► CEVA-Dragonfly is an integrated HW/SW Platform to speed up and lower
risk of SoC development for multi-mode M2M connectivity endpoints ► Includes all required M2M modem HW IPs and SW components to let you focus on
application integration
► Target applications include wearalone, wearables, smart grid, surveillance systems, asset tracking, remote monitoring systems, connected cars and smart utilities
► Supports every M2M standard including LTE MTC Cat-1, Cat-0 and Cat-M standards, Wi-Fi 802.11n/ah, PLC, 802.15.4g, ZigBee, GNSS, LTE-OTDOA, NB-IoT, LoRa, SIGFOX
8 CEVA Proprietary Information
► Based on CEVA-TeakLite-4 DSP + PHY and MAC hardware ► Enables most power efficient design ► Scalable for other connectivity standards ► TeakLite-4 DSP available for additional
functionality such as audio, voice, sensing
► Down to 500K gates ► Complete solution: DSP, PHY, MAC, HW, SW
► Less than 30mW for 1Mbps 802.11n 1X1 in 55nm
CEVA-WiFi Platform
Lower MAC
RF Interface
Wi-Fi Protocol Stack
PHY & MAC Hardware
Modem
MAC HW
Bus Interface
Upper MAC
AHB
CEVA-TeakLite-4
WiFi Direct
Application
9 CEVA Proprietary Information
► CEVA-Bluetooth ► Classic Bluetooth (2.1+EDR, 3.0) ► Low Energy Bluetooth (4.2) Single/Dual Mode ► CEVA-Bluetooth BB HW integrated with processor ► CEVA-Bluetooth Controller SW stack ► Customizable RF interface for 3rd party Radio
► Single Mode ► Reduced HW/SW footprint for low power & cost
► Dual Mode ► Addition of low energy protocol HW/SW to Classic BT
► Bluetooth 5.0 is just around the corner ► Audio over BLE, IPv6 over BLE, extended range and
much more
CEVA-Bluetooth Platform
Classic 4.n Link Controller
Classic 4.n Link Manager
Host Controller Interface
Hardware Abstraction Layer
RF Interface
Bluetooth Software Stack
HCI
Baseband Hardware
RF Interface
Classic 4.n HW Link Controller
Bus Interface
LE - Link Controller
LE HW Link Controller
Host Software and Profiles
AHB
10 CEVA Proprietary Information
Always-On Multiple Sensing
Low cost, low power sensors are common
However, low cost/power generally means noisier sensors
So where do DSPs fit in sensing? Need a lot of signal-cleaning (filtering, smoothing, calibrating, etc’) in order to extract meaningful data DSP requirements further increase with introduction of mics and biometric sensors Intelligent context awareness
Sensing in IoT is based on various and growing list of sensors IoT Sensors Evolution
GPS GNSS GNSS
BT (beacon)
Accelerometer Accelerometer Accelerometer
Magnetometer Magnetometer
Barometer
Context
Motion
Location
Gyroscope
Microphone(s)
Ultrasonic Image/ Gesture CMOS Sensor
Pow
er C
onsu
mpt
ion
& C
ost
2010… 2015…
$
Power consumption is key criteria: need <2mA for the complete always-on use case
Temperature
Barometer
Fingerprint
Heart Rate
Biometric
CMOS Sensor
Light Light Light
Temperature
11 CEVA Proprietary Information
Application Processor (AP)
Accelero. sensor
Gyroscope sensor
Compass sensor
Pressure sensor
Motion algorithms Sensor hub DSP
I2C
I2C/SPI
Android Applications Application Framework
Sensor HAL + sensor fusion algorithms Kernel
Application Processor (AP)
Accelero. sensor
Gyroscope sensor
Compass sensor
Pressure sensor
I2C
Android Applications Application Framework
Sensor HAL Kernel
Before Sensor fusion running on AP High power consumption (120 to 190mA) Effort to maintain sensor fusion in Android user space Motion sensor vendor dependent Repeated software porting effort for new motion sensors
Now Dedicated DSP to run sensor fusion SW Low power consumption (~1mA) for Always-on 10-axis fusion algorithm sensor independent Real time DSP environment for real time algorithms Host OS independent + Host offloading
Mandates offloading to a dedicated processor for low power !
Always-On Sensor-Fusion
12 CEVA Proprietary Information
Always-On Voice Activation
TL410 Voice activation Wake-Up
Indication
CPU/OS
Speech Recognition SW - Voice Trigger - Voice Command - Speaker verification Keyword, e.g. “OK Baidu”
CEVA-TL410 enables < 20uW DSP power consumption at 28nm for always-on voice trigger and command
13 CEVA Proprietary Information
DSP-Based Sensing
Sensor fusion Contextual awareness Motion gestures Indoor navigation
Voice trigger
Ultrasonic gestures
Face trigger
BLE
Multiple Sensing Technologies Using a Single DSP
DataMemory
MSS
TeakLite-4
ProgramMemoryDMA
(4 Ch.)
4 x TIMER(WDT)
Master
32 x GPIO(I2C) ICUPMU
SlaveTDM R/T(I2S)
D P
APB
AHB
Gesture/face/voice Trigger Indication
AP
Interrupt
MEMS Mic
MEMS Sensors
MIPI
Bluetooth Baseband Radio
Less than 150uW @ 28nm HPM for Always-on Sensor Fusion + Voice Trigger + Face Trigger + BLE
14 CEVA Proprietary Information
IoT Layers – Local Intelligence vs. Cloud DSP powered local intelligence enables:
Camera/microphone/other sensors raw data does not need to be sent to the cloud, only processed meta-data is being sent Increased privacy
Reduced data bandwidth, transfer overhead and processing latency to/from cloud lower on-device power + lower cost of cloud service
Efficient processing for scene analysis (sound/vision) with lower power than GP CPU/GPU Lower power consumption, longer battery life
Increased security by using multiple connectivity standards Using SW PHY allows switching from Wi-Fi to LTE for tamper-proof security actions
Local intelligence is key for smart IoT devices !
15 CEVA Proprietary Information
DSP-Based Audio Analytics Valuable Sound Classification & Analysis
Mobile, Wearable, Smart Home and Robot applications
Voice recognition and speaker identification Speaker separation through beamforming Environment sensing (e.g. cinema, train) Emotion detection
Security applications can alert upon: Glass breaking Baby crying Keyword (virtual red button) Aggression
Extensive computational power enables a standalone (cloudless) audio analysis Efficient processing of large amounts of data
Handcrafted data cache and DMA 128-bit data bandwidth Dual load/store units
Enables intensive noise reduction required for accurate analysis Lowest-power on its category
CEVA-TL421 as an Audio Analyzer
Advanced contextual awareness includes intelligent sound analysis and requires a powerful programmable audio/voice processor
16 CEVA Proprietary Information
DSP-Based Vision Analytics Computer Vision & Video Analytics
Markets: Mobile, Wearable, Smart Home, Smart Cities, Security & Surveillance
Dedicated computer vision engine for complex analytics applications
4th generation computer vision DSP Up to 1.2GHz @ 28nm HPM Combines fixed and floating-point math Up to 4096-bit vector processing per cycle
Efficient processing of large amounts of data Innovative DMA mechanism capable of transpose, crop and checkered data on the fly 512-bit bandwidth
Fully equipped with SW framework, library and tools for easy SW porting and optimization
CEVA-XM4 Computer Vision Engine
Example Applications Human & object recognition Face recognition, gesture recognition Tracking based on feature and pattern matching Emotion detection Image and video enhancement for complex outdoor conditions
Growing need for sophisticated digital analysis on Camera calls for dedicated CV programmable processors
17 CEVA Proprietary Information
CEVA IoT Platform Enablers
DSP enables smart IoT devices !
CEVA-XM4
Computer Vision, Video Analytics
Edge Processing
Shared Memory
Interconnect
LCD
CEVA-TL421 Audio Analytics
Edge Processing
I2C I2S Ext Mem PMU
L1
UART MIPI Boot ROM
CODEC
sensors
L1 L1 L1
LTE Cat-0 PHY + MAC
CEVA-TL410 Sensor
Processing, Always-on
Radio Radio* Radio
L1 L1
Bluetooth PHY + MAC
Wi-Fi PHY + MAC
CPU / MCU L1 L1
DSP Intelligence DSP Sensing DSP based connectivity (SW PHY)
18 CEVA Proprietary Information
CEVA ‘Smart & Connected’ IoT Development Platform
19 CEVA Proprietary Information
Intelligence
The ‘Smart & Connected’ (IoT) Strategy
*Source: IDC
Total IoT units expected to grow at 18% CAGR to reach 28 billion units by 2020*
Wi-Fi, Bluetooth, LTE & Narrowband LTE (NB-LTE)
Camera : Face recognition , AR, VR ,ADAS Microphone/sensors: Always on, Noise
suppression, sensor hub
1. 2.
Connectivity CPU
20 CEVA Proprietary Information
‘Smart & Connected’ Development Platform
Availability: Now, directly from CEVA
21 CEVA Proprietary Information
CEVA-TeakLite-4 Processor The “Brains” of the Platform
High-performance Low-Power 32-bit DSP, Scalable, Extendible
Optimized for audio, voice and general control/DSP applications
CEVA’s fourth generation TeakLite architecture Leverages > 100 design wins, > 4.5 billion devices in the market, > 150 audio/voice SW modules, > 30 active ecosystem partners
Low die size and cost 90K gates in its smallest form
Designed for low power TeakLite-4 development chip made in collaboration with Brite and fabricated at SMIC 55nm LP, running at 500MHz
22 CEVA Proprietary Information
CEVA-TeakLite DSP Family
TeakLite-II
CEVA-TL3211
CEVA-TL3210 TeakLite-II Architecture
(16-bit)
TeakLite-III Architecture
(32-bit)
TeakLite-4 Architecture Framework (32-bit)
Feat
ures
& P
erfo
rman
ce
CEVA-TL411 CEVA-TL410
CEVA-TL421 CEVA-TL420
Future TL4 Cores
Available Roadmap All TeakLite cores are
assembly level compatible
Deeply Embedded
Cache Enabled
Dual 16x16 Single 32x32
Quad 16x16 Dual 32x32
Multiple configurations
for optimal application
needs
23 CEVA Proprietary Information
Complete TeakLite-4 Subsystem Included in TeakLite-4 Silicon
Fully integrated subsystem, comprised of main system peripherals, seamlessly connected to TeakLite-4 DSP
Complete with drivers and reference design
Shorter TTM and reduced risk for customer SoC
DataMemory
MSS
TeakLite-4
ProgramMemoryDMA
(4 Ch.)
4 x TIMER(WDT, PDM)
Master
32 x GPIO(I2C) ICUPMU
TDM R/T(I2S)Slave
D P
APB
DSP CoreSubsystem
AHB
24 CEVA Proprietary Information
More than 150 Audio/Voice SW Modules Always-on &
NUI RTOS Voice Noise Reduction
& Echo Cancellation
Audio Post-processing
Dolby HD-Audio DTS HD-Audio
Sensory TrulyHandsFree; Sensory User Defined Trigger; Sensory Speaker Verification; Rubidium Voice Trigger; Malaspina Labs Voice Trigger; Cywee Motion Sensing; Visidon Face Detection; IVT BT Stack
FreeRTOS; Express Logic ThreadX; ENEA OSEck; Quadros RTXC; uT-Kernel; CMX-RTX; Mentor Graphics Nucleus
G.723; G.729; G.728; G.729.1; G.711; G.722; G.726; G.727; G.168; G.161; iLBC; AMR-NB; HR; FR; EFR; AMR-WB; EVRC; EVRC-B; QCELP; SILK (32-bit); Opus; AMBE Vocoders; EVS
NXP SW Life Vibes Voice Experience; Alango Voice Comm. Package; Dimagic Flexbeam; Cypher Voice Isolation; Malaspina Labs Noise Reduction; Waves MAXX Voice; Waves MAXX Speech
MP3; MP3Pro; Ogg Vorbis; FLAC; MPEG4 AAC LC; HE AAC V1; HE AAC V2; HE-AAC V2 5.1 Ch; MPEG4 AAC BSAC; WMA; RealAudio; SBC; CELT; DRA
DTS/SRS TruMedia HD; DTS/SRS StudioSound 3D; DTS Neo:6; Dolby ProLogic IIx; Dolby Mobile 3+; Dolby DS1; Dirac Speaker Correction; AM3D Zirene; AM3D Sound/3D; Waves MAXX Audio; Arkamys MobiSound; Qsound MicroQ
Dolby TrueHD; Dolby Digital Plus; Dolby Digital decoder (AC3); Dolby Digital encoder (DDCE); Dolby MS10; Dolby MS11; Dolby Volume
Master Audio; High Resolution; Low Bit Rate; Extended Surround (ES); DTS 96/24; DTS Digital Surround; DTS Transcoder; DTS M6; DTS M8
Major cost reduction and time-to-market advantage
25 CEVA Proprietary Information
TeakLite-4 DSP Library
Bit-accurate C (Visual Studio) and TeakLite-4 assembly functions for easy integration
Supports both 16-bit and 32-bit precision
Supports all TeakLite-4 cores and configurations (audio, voice, compatibility)
Total of 74 functions including full source code Fully integrated with SDT - NO ADDITIONAL COSTS INVOLVED!
Autocorrelation Cross-correlation Convolution (Block FIR) Block LMS Filter Delayed LMS Filter Decimation Interpolation Symmetric FIR Single Sample FIR Complex IIR (Biquad)
Filter Functions
Division Square Root Inverse Square Root Log Power Cosine Sine Tangent Arctangent
Math Functions
Bit-Reverse Permutation Complex FFT Real FFT Complex Inverse FFT Real Inverse FFT
Fourier Transformations
Addition Subtraction Dot Product Maximum / Minimum Multiplication Shift
Vector Operations
26 CEVA Proprietary Information
Profiler – Function Graph
Profiler – Cache Performance
Profiler – Function Info
Profiler – Code Coverage
IDE / Debugger View
CEVA-ToolBox™ Software Development Environment
Highly focused on two main objectives:
Performance of generated code, especially cycle count and code size
User experience, with emphasize on ease of programming, user interface and automation tools
Build Optimizer
27 CEVA Proprietary Information
Android Connectivity & CPU Offloading CEVA AMF (Android Multimedia Framework)
AMF is an Android based software framework for offloading audio/voice tasks to the DSP
CEVA AMF defines a HW and SW interface between the CPU running Android and single or multiple CEVA DSPs
Multimedia workloads (e.g. voice trigger, sound sensing, audio playback) are abstracted on the CPU and executed on the DSP with better performance and reduced power > 5-9x lower power for audio/voice applications
Multimedia tasks may be “tunneled” for saving data transfer overhead to and from the CPU
CEVA’s AMF Android support: > KitKat, Lollipop – now > Marshmallow – in 2016
CPU
CEVA Link Driver CEVA Link Driver
DSP/RTOS
The CEVA AMF (Android Multimedia Framework) Seamlessly Bridges the Gap Between Android OS and CEVA Platforms !
28 CEVA Proprietary Information
CEVA ‘Smart & Connected’ Dev Platform
Analog Mic 2
Analog Mic 1
Line out
Line in
Digital mics
Ethernet
USB/UART RF I/F
PCIe
User switches and dip-switches
Power
Color LCD
Arduino shield connectors & GPIOs
TeakLite-4 Silicon (DSP, DMA, TDM, I2S, I2C, ICU, Timers) RTOS
JTAG port
ARM Cortex-A9 x 2 Linux OS User area FPGA
+
29 CEVA Proprietary Information
CEVA ‘Smart & Connected’ Dev Platform Real-time Power Measurement and Tuning
► Platform includes the ability to measure power separately for:
1. TeakLite-4 DSP 2. On-chip memory 3. On-chip peripherals (DMA, TDM …)
► On-board real-time power measurement allows the user to modify code and observe power differences application power tuning and optimization !
30 CEVA Proprietary Information
CEVA ‘Smart & Connected’ Dev Platform
Availability: Now, directly from CEVA
31 CEVA Proprietary Information
Summary
Connectivity/Sensing/Intelligence IP (HW/SW) + Dev. Platform = DSP Enhanced IoT Applications
CEVA-XM4
Computer Vision, Video Analytics
Edge Processing
Shared Memory
Interconnect
LCD
CEVA-TL421 Audio Analytics
Edge Processing
I2C I2S Ext Mem PMU
L1
UART MIPI Boot ROM
CODEC
sensors
L1 L1 L1
LTE Cat-0 PHY + MAC
CEVA-TL410 Sensor
Processing, Always-on
Radio Radio* Radio
L1 L1
Bluetooth PHY + MAC
Wi-Fi PHY + MAC
CPU / MCU L1 L1
DSP Intelligence DSP Sensing DSP based connectivity (SW PHY)
+
THANK YOU