Autonomous Cars:Radar, Lidar, Stereo Cameras

1 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

CAMERAS FOR AUTONOMOUS DRIVING

2 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Role of Cameras in Automotives

Camera module market ~$25B in 2015, expected to double by 2025 (Yole)Machine Vision integration with multiple sensors for ADAS and partial autonomous drivingDisadvantages of cameras• Environmental conditions can introduce problems• Difficulty in detecting non- illuminated and varying lighting conditions• Computer vision limitations for reliable detection

3 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

CMOS image sensors for Automotive are quite different from Consumer Electronics

Source: Yole

Improved Low light sensitivity by larger pixel sizeLower resolution (it’s not about Mega pixels)Fast response time (significantly faster than smartphone cameras

Packaging Differences• Embedded logic in image sensor package• Ceramic packages for higher functional safety requirements

4 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

ChallengesFast image acquisitionStereo camera for depth and distance of the object in image planeHigh resolution AND high speed

• Resolution: greater than 1k x 1k pixels, aim for 4k x 4k• Frame rate: 60 fps and higher for shorter reaction time

Zoom into image area for a greater detail Efficient image processing for real-time analysisWider field of view than radar or laser systems

5 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Cameras in Current Cars (Bosch)

Bosch‘s mono and stereo camera system: smallest currently available in the market. Has a 50-degree horizontal field of view for 50m distance.

Bosch’s stereo camera system

6 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Cameras in Current Cars (Panasonic)

Stacked CMOS imaging chip and processing electronics in one packageCompact enough to fit within rearview mirror assembly or behind the windshieldLow cost high resolution cameras, but limited speedHigh speed cameras limited resolutionCircuits for efficient image processing for real-time analysisCan be implemented with multiple units for stereo vision

Panasonic CMOS 34227 Sensor

7 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Prior Work at GT: 1st CIS DEMONSTRATOR on LOW COST 3D GLASS PACKAGE

100um Thin Glass Substrate

Wafer Level Camera on Top Side

Thin Logic Emulator IC on bottom side

Solder reflow at chip level

Passed initial tests at Georgia Tech

8 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Thermal Control Using Glass PackageGlass limits lateral thermal spreading by designJunction temperatures can be reduced by vertical copper viasHeating of image sensor with IC stacking or silicon interposers

Without copper With copper

Glass

38C 58C 48C 50C

49.1C 49.4C 49.1C 49.4C

Silicon10mm x 10mm Test Chipon Glass or Si Interposer

9 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

RADAR

10 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Why RADAR in NAE?

• Above functions Require radar sensors due to their robustness in varying environmental conditions like rain, dust, or sunlight

(J. Hasch, "Driving towards 2020: Automotive radar technology trends," Microwaves for Intelligent Mobility (ICMIM), 2015 IEEE MTT-S International Conference on, Heidelberg, 2015, pp. 1-4.)

Roadmap for driver assistance functions

• Short, Medium and Long Range RADAR Modules

Critical to Fully Autonomous Driving

11IEEE-CPMT Workshop – Autonomous Cars

Automotive RADAR – Brief Intro

• Basic Architecture and Requirements- Frequency Modulated Continuous Wave (FMCW) Doppler Radar- 76 GHz – 81 GHz- Severe Environment Conditions- Long Term Reliability

Many challenges to device and design !!

TxRx

fr

Freq

uenc

y

fb

fd

fb fd

fr

Transmit FMCW Signal76-81 GHz

Compare Received Signal with Transmitted Signal to Extract

Distance and Velocity

PROF. JOHN CRESSLER

12 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Automotive RADAR Evolution

1999 Mercedes-Benz • First manufacturer to use radar

for autonomous cruise control (ACC) system in S-class

Active Components Evolution

Pack

agin

g Ev

olut

ion

GaAs Technology

• Discrete semiconductor components MMIC blocks or even complete

transceiver circuits

Silicon-based SiGe technology• Improved RF performance at high freq.• Destined to be the mainstream

semiconductor technology millimeter-wave

(J. Hasch, E. Topak, R. Schnabel, T. Zwick, R. Weigel and C. Waldschmidt, "Millimeter-Wave Technology for Automotive Radar Sensors in the 77 GHz Frequency Band," in IEEE Transactions on Microwave Theory and Techniques, vol. 60, no. 3, pp. 845-860, March 2012.)

Chip on board wire bonding

Flip chip mounting on substrate

eWLB (embedded wafer level

BGA) package

Evolution of SiGe Device Technology

Courtesy of P. Chevalier

13 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Challenges in Automotive RADARPackage Challenges

Low losses and reflections chip to substrateTransitionThermal dissipationHigh Reliability

Low cost

• Generation of sufficient output power at high frequency

• LNA demonstration

Device Challenges

• No LNA: High NF → Low SNR → Lowmaximum detecting range of system

77GHz 4-channel automotive radar transceiver chip specification

(H. P. Forstner et al., "A 77GHz 4-channel automotive radar transceiver in SiGe," Radio Frequency Integrated Circuits Symposium, 2008. RFIC 2008. IEEE, Atlanta, GA, 2008, pp. 233-236. )

14 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Status of Automotive RADAR Products

Bosch LRR3 Sensor• SiGe MMICs instead of Gunn oscillators and discrete mixer diodes • Size and package complexity reduction

Bosch LRR3 SensorBosch LRR2 Sensor

15 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Latest Automotive RADAR Products

Bosch Mid Range Radar (MRR) Sensor• Frequency band 76-77 GHz• Distance range up to 160 meters • Integrates two electronic boards and STMicroelectronics devices• RF board with Hybrid PTFE/FR4 substrate and equipped with planar antennas• Infineon 77GHZ SiGe Monolithic Microwave Integrated Circuits (MMIC) used as High-Freqency

transmitter and receiver

16 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Recent Automotive RADAR Packaging Technologies Infineon RRN7745P & RTN7735P eWLB Fan-out Package - 77GHz Radar Dies

The radar receiver die is packaged in an eWLB (embeddedWafer Level BGA) package, Fan-Out technology fromInfineon.

Back view

Die3mm x 3mm

Package cross-section

17IEEE-CPMT Workshop – Autonomous Cars

Proposed High Linearity Low Noise RX

Block Diagram

One of the first Radar modules to integrate LNA in front endLeading Edge SiGe Receiver

Employ Advanced SiGe Node

Use Novel Circuit Design

Incorporate High Gain High Linearity

LNA

Current Design Phase

LNA layout modificationPROF. JOHN CRESSLER

18IEEE-CPMT Workshop – Autonomous Cars

Proposed High-Linearity LNA

IBM 9HP 90nm technology3 cascaded cascode stagesFully differential to improve CMRRInput transformer balun:

- Provide RF ESD protection and differential signal

Inter-stage matching with transformersVariable gain

- Improve input P1dB

Area: 0.6mm x 1 mmVCC: 2.5V

IN

OUT

VCC

VB1_13

VB2_13

VB1_2

VB2_2

GND

GND

GND

PROF. JOHN CRESSLER

19IEEE-CPMT Workshop – Autonomous Cars

LNA Initial Design and SimulationTransistor Sizing: 6um, 8um, 10um Simulation Results

S-param

NF

P1dB

Gain 17.3 dBNF 6.221 dB3dB Freq 56GHz-94GHzDC Power 91.5 mWInput P1dB -14.5 dBm

PROF. JOHN CRESSLER

IEEE-CPMT Workshop – Autonomous CarsSlide 20

CONFIDENTIAL

GT Program FocusThe research objective is to design and demonstrate an ultra miniaturized low costintegrated 77GHz radar & high speed camera module applying the most advancedpanel-level glass fan-out (PGFO) packaging technologies to the following parameters:

77 GHz Radar Chip

Processor

Image sensor

Antenna Incident light

Glass

Cover glass

Active area Camera ModuleRadar Module

Properties Objectives Prior Art

Integrated Module

Package type GFO FO-WLP/ CeramicCost 500 um

RadarNoise Figure 6-7 dB > 20 dB

Gain 15 dB < 15 dBAntennas Integrated in package Integrated in PCB

CameraData Rate ~ 1 Gbps 1 ms

21 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

LIDAR

22 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Why Lidar for Automotive ?

Cameras and radar cannot ensure 100 % safetyRadars provide no object detectionCameras depend on environmental conditions.Lidar enables high precision detection in real time Time of Flight lasers in Lidar are the most accurate for real time and long range detection.

23 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Basics of Lidar

Fired Laser Pulses

Reflected Laser Pulses

Starting a timer when the pulse goes out and stopping by the reflected pulse

24 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Challenges

Cost reduction (see figure)Miniaturization for easy integration in car bodyHigh aperture angle – number of channels Reliability by application of solid state lidar

25 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Lidar Products

Velodyne lidar system VLP-16• Range: 100 m, • Power consumption: ~8 W, • Weight: 830 grams, • Footprint:~Ø103 mm x 72 mm,• 16 channels, ~300,000

points/sec, • 360° horizontal field of view,

30° vertical field• Accuracy: +/- 3 cm (typical)• Rotating mirror inside

assembly,• Laser: Class 1 – 903 nm

wavelength

26 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Lidar Products

Quanergy Solid-state LIDAR system• Field of view is 120 degrees both horizontally and vertically. • The minimum range is 10 cm, and the maximum range is at least

150 m at 8 percent reflectivity. • At 100 meters, the distance accuracy is +/- 5 cm, and the

minimum spot size is just 9 cm.

Small (9 cm x 6 cm x 6 cm), no moving parts

27 | IEEE-CPMT Workshop – Autonomous Cars Prof. Rao R. Tummala

Solid State Lidar

Uses an optical phased array as a transmitter (no micro mirrors), which steers laser pulses by shifting the phase.