adas high bandwidth imaging implementation strategies · adas high bandwidth imaging implementation...

©2017MIPIAlliance,Inc.

Shiou Mei Huang, Automotive ApplicationsMayank Mangla, ADAS Imaging ArchitectTexas Instruments

ADAS High Bandwidth Imaging Implementation Strategies


AutomotiveImagingApplications

2

CompiledwithGoogleSearchImages

FrontCam

SV/Park Assist

Radar/Lidar

DMS

Fusion

CMS

SmartRearCam


Challenges• HighBandwidthRequirements

– Fasterframerate– Highersensorresolution– Highdynamicrange(multipleexposuresperpixel)

• Multi-CameraSystems– Numerousimage/videoformats– Singleprocessortoprocessvideoinputsfromall

cameras– Needtodistinguishamongstvideostreams– Multi-modalfusion

www.ti.com/ADAS

• SoC DesignConstraints– Package:Minimalpincount– Routing:Signalintegrity– Lowercableusage

• AutomotiveQuality&Safety– Needtomaintainsignalintegrityfor

interfacedistance>20ft


ExampleSolutions• FPDLinkDeserializer

– Quadhubchip– EachcameraconnectedthroughFPD-Linkcable– VCtagforindividualcamerastreams– Mux4camerasintoasingleCSI-2℠ stream

• VisionProcessor– RxmodulefullycompliantwithMIPICSI-2℠ spec1.0

• Future:32VirtualChannels– Upto6.0Gbps throughput– HWsupporttoparseCSI-2℠ streams– Highspeed,multichannelimageprocessor– Visionaccelerators:HWIPs+ProgrammableCore

4

www.ti.com/ADAS

DISPLAY


WhyMIPICSI-2℠inAutomotive

5

www.ti.com/ADAS

§ Lowpower

§ Highspeedtransfers

§ Lowpincounto Flexibilityinchoosing1/2/3/4lane(s)

§ StandardizedvsProprietaryprotocolso Easytointerfaceawiderangeoftransmitters

andreceivers

§ Virtualchannels§ Allowsconnectionsofmultipledevicestothe

samebus

§ Errordetectionandrecoveryo Synccodeso ECCcodeso 16-bitsCRCChecksum

§ Safetyo Safetyrelatedmeta-dataalongwithpixeldata


MIPICSI-2℠ LaneScalability

6

www.ti.com/ADAS

RECEIVER

Ø Inhighbandwidthscenarios,CSI-2℠ canbeconfiguredtosplitbytestreaminto1/2/3/4datalanes

Ø OnTx sidethestreamisdistributedacrossmultiplelanes.OnRxsideincomingpacketsaremergedintoasinglestream

Ø Eachlaneoperatesindependently,includingSoT frame,StartPacketcodeandEndPacketcode


VirtualChannelsandDataTypes

7

www.ti.com/ADAS

Ø EachCSI-2℠ packethasa“DataIdentifier”field,whichspecifiesthepayloadDataType(DT)andVirtualChannel(VC)numberitcarries

Ø Differentdatatypesfromdifferentsourcescanbemergedintoasinglestream

Ø CSI-2℠ RxusesDTandVCfieldstodistinguishamongstdifferentpacketsandprocessthemaccordingly

Ø Examples:• Pixeldatavsembeddeddata• Datafromdifferentcamerasmultiplexedintoasingle

CSI-2℠ stream• Multi-exposureWDRwhereL/M/Spixelsarecomingin

asinglestream.


Example– SurroundView

8

www.ti.com/ADAS

Ø 4xFullHDCameras– Throughputrequirement>3GbpsØ EachcamerasendsahighresolutionRAWstreamatahigh

framerateØ SurroundViewApplication(SRV)

4:1Deserializer Hub(DS90UB96x)• Synchronizesvideoinputsacrossmultiplecameras• Aggregatescamerastreamsfromfoursidesofthevehicle• TagseachcamerastreamwithuniqueVCandmetadata• Multiplexesintoasinglestream• SendsoverCSI-2℠ interfacetothehostprocessor

HostProcessor(TDAx)• Receivestheincomingstream• Parseseachpacket,identifiesthecamerastreambasedonVC• Obtains4independentvideostreaminseparatebufferqueues• AppliescomplexSRVimageprocessingalgorithms


Example– SurroundView

9

www.ti.com/ADAS


SurroundViewConfiguration

10

www.ti.com/ADAS


SurroundViewSolution

11

www.ti.com/ADAS

ChallengesØ HighDataRate– Upto3GbpsØ HighPinCount– TosupportmultiplecamerasØ LongChannelLength– Cameraslocatedfarawayfromthe

processorØ Sync– Allcamerasmustbesynchronizedtoeliminatemotion

artifacts

MIPICSI-2℠ totherescueØ 4-laneinterface forHighSpeedDataTransferØ SupportforVirtualChannelsandDataTypes

• Allowsmultiplecamerastreamstobemuxed intoasingleCSI-2℠ stream

• ProcessorcanidentifyaCSI-2℠ packetusingVCandDTinfo

Ø CSI-2℠ enabledSerDes chipsallowlongchannellengthoverFPDLink• Actas4-camerahub• Implementssyncacrossallcameras


SurroundViewFlow

12

www.ti.com/ADAS


Example– Multi-ModalFusion

13

www.ti.com/ADAS

Lighting Rain

SnowFog/Smog

Night

Dirt

PassiveSensorCamera

ActiveSensorRadar,Lidar,Ultrasound

InfrastructureHigh-precisionMapV2XCommunication

MidandLongRangeAdaptiveCruiseControl,Emergencybraking,Fully/HighlyAutonomousDriving

ShortRangeandUltraShortRangeBlindSpot,CollisionAvoidance,LaneChangeAssist,PedestrianDetection,ParkAssist

ProximitySensorsOccupantDetection,GestureRecognition,DriverMonitoring


FusionSolution

14

www.ti.com/ADAS

ChallengesØ HighDataRate– 1Gbps/Camera+2.4Gbps/RadarØ HighPINCount– TosupportlargenumberofdevicesØ ImagingFormatVariations– Needastandardprotocolforimaging

andnonimagingdevices

MIPICSI-2℠ totherescueØ 4-Laneinterface forHighSpeedDataTransfer.Ø AbilitytoMUXCamera/Radar/LIDARØ ProcessorcanidentifythesourceusingVCandDTØ CSI-2℠ specificationeasilyextendstononimagingdevices

likeRadar.Ø NodesignchangeneededattheprocessorRx

interfaceØ Serializer andDeserializer chipsforFPDlink

transmissioncanprocessRadarstreamexactlylikecamera.


Fusion– SLAM(SimultaneousLocalizationandMapping)

15

www.ti.com/ADAS

6x-10xCameras

6x-10xRadar

1x-4xLIDARs

8x-12xUltrasonic

GPSIMU

SensorProcessing

SensorProcessing

SensorProcessing

SensorProcessing

SensorProcessing

Perception

Perception

Perception

Perception

Perception

FUSION• SensorFusion• Localization• Mapping

Maps

PLANNINGANDCONTROL• Pathplanning• Motionplanning• Vehiclecontrols

• Acceleration• Brake• Steering

FEEDBACKTODRIVER• Visualization/Display• Warnings

Drivermonitoring

Thermal/IR


SummaryMIPICSI-2℠ isgainingpopularityinAutomotiveindustrybecauseofthefollowingbenefits

• HighBandwidth:Total6.0Gbps.Allows4x1080pcamerasonasingleProcessor• Lowpincount: Scalablefrom6- 12• MultiCamera/MultiModalArchitecture:EfficientuseofVirtualChannelandDataTypes• Quality:SerDes solutionsconvertsimagesignalfromdigitaltoanalogform.FPDlinkcabletransportsanalog

signaloverlongdistancewithoutdegradation• Safety:TDAxx processorshasadvancedcapabilityforerrordetection,recovery,andembedsafetydata

alongwithpixeldata• Future:32virtualchannels

WithmoreImagesensors,RadarandotherdevicesadoptingCSI-2℠,thetrendisexpectedtocontinueandgainmomentumintheyearstocome.

16

www.ti.com/ADAS

adas high bandwidth imaging implementation strategies · adas high bandwidth imaging implementation...

Documents