autonomous helicopter flight - eecs at uc berkeleypabbeel/cs287-fa...§ white helicopter is inferred...
TRANSCRIPT
![Page 1: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/1.jpg)
AutonomousHelicopterFlight
PieterAbbeelUCBerkeleyEECS
![Page 2: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/2.jpg)
![Page 3: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/3.jpg)
n Unstable
n Nonlinear
n Complicateddynamicsn Airflow
n Coupling
n Bladedynamics
n Noisyes>matesofposi>on,orienta>on,velocity,angularrate(andperhapsbladeandenginespeed)
ChallengesinHelicopterControl
![Page 4: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/4.jpg)
n Justafewexamples:n Bagnell&Schneider,2001;
n LaCivita,Papageorgiou,Messner&Kanade,2002;
n Ng,Kim,Jordan&Sastry2004a(2001);Ngetal.,2004b;
n Roberts,Corke&Buskey,2003;
n Saripalli,Montgomery&Sukhatme,2003;
n Shim,Chung,Kim&Sastry,2003;
n Dohertyetal.,2004;
n Gavrilets,Mar>nos,MeWlerandFeron,2002.
n Varyingcontroltechniques:inner/outerloopPIDwithhandorautoma>ctuning,H1,LQR,…
SuccessStories:HoverandForwardFlight
![Page 5: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/5.jpg)
[Ng,Coates,Tse,etal,2004]
![Page 6: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/6.jpg)
AlanSzabo–SundayattheLake
![Page 7: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/7.jpg)
OneofourfirstaWemptsatautonomousflips[usingsimilarmethodstowhatworkedforihover]
Targettrajectory:me>culouslyhand-engineeredModel:from(commonlyused)frequencysweepsdata
![Page 8: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/8.jpg)
n Hover/sta>onaryflightregimes:
n RestrictaWen>ontospecificflightregime
n Extensivedatacollec>on=collectcontrolinputs,posi>on,orienta>on,velocity,angularrate
n Buildmodel+model-basedcontroller
à Successfulautonomousflight.
n Aggressiveflightmaneuvers---addi>onalchallenges:
n Taskdescrip7on:Whatisthetargettrajectory?
n Dynamicsmodel:Howtoobtainaccuratemodel?
Sta>onaryvs.AggressiveFlight
![Page 9: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/9.jpg)
n Gavrilets,Mar>nos,MeWlerandFeron,2002n 3maneuvers:split-S,snapaxialroll,stall-turn
n Key:Expertengineeringofcontrollersalerhumanpilotdemonstra>ons
Aggressive,Non-Sta>onaryRegimes
![Page 10: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/10.jpg)
SundayinOpenLoop
![Page 11: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/11.jpg)
n Ourwork:n Key:Automa>cengineeringofcontrollersalerhumanpilot
demonstra>onsthroughmachinelearning
n Widerangeofaggressivemaneuvers
n Maneuversinrapidsuccession
Aggressive,Non-Sta>onaryRegimes
![Page 12: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/12.jpg)
n Learningatargettrajectory
n Learningadynamicsmodel
n Autonomousflightresults
LearningDynamicManeuvers
![Page 13: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/13.jpg)
n Difficulttospecifybyhand:n Requiredformat:posi>on+orienta>onover>me
n Needstosa>sfyhelicopterdynamics
n Oursolu>on:n Collectdemonstra>onsofdesiredmaneuvers
n Challenge:extractacleantargettrajectoryfrommanysubop>mal/noisydemonstra>ons
TargetTrajectory
Abbeel,Coates,Ng,IJRR2010
![Page 14: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/14.jpg)
ExpertDemonstra>ons
![Page 15: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/15.jpg)
• HMM-likegenera>vemodel– DynamicsmodelusedasHMMtransi>onmodel
– Demosareobserva>onsofhiddentrajectory
• Problem:howdowealignobserva>onstohiddentrajectory?
LearningaTrajectory
Demo1
Demo2
Hidden
Abbeel,Coates,Ng,IJRR2010
![Page 16: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/16.jpg)
n DynamicTimeWarping(Needleman&Wunsch1970,Sakoe&Chiba,1978)
n ExtendedKalmanfilter/smoother
LearningaTrajectory
Demo1
Demo2
Hidden
Abbeel,Coates,Ng,IJRR2010
![Page 17: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/17.jpg)
Results:Time-AlignedDemonstra>ons§ Whitehelicopterisinferred“intended”trajectory.
![Page 18: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/18.jpg)
Results:Loops
Evenwithoutpriorknowledge,theinferredtrajectoryismuchclosertoanidealloop.
Abbeel,Coates,Ng,IJRR2010
![Page 19: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/19.jpg)
n Learningatargettrajectory
n Learningadynamicsmodel
n Autonomousflightresults
LearningDynamicManeuvers
![Page 20: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/20.jpg)
StandardModelingApproach
Abbeel,Coates,Ng,IJRR2010
3Gerror!
![Page 21: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/21.jpg)
KeyObserva>on
Errorsobservedinthe“baseline”modelareclearlyconsistentaleraligningdemonstra>ons.
Abbeel,Coates,Ng,IJRR2010
![Page 22: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/22.jpg)
n Ifweflythesametrajectoryrepeatedly,errorsareconsistentover>meoncewealignthedata.
n Therearemanyunmodeledvariablesthatwecan’texpectourmodeltocaptureaccurately.
n Air(!),actuatordelays,etc.
n Ifweflythesametrajectoryrepeatedly,thehiddenvariablestendtobethesameeach>me.
~musclememoryforhumanpilots
KeyObserva>on
Abbeel,Coates,Ng,IJRR2010
![Page 23: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/23.jpg)
n Learnlocally-weightedmodelfromaligneddemonstra>ons
n Sincedataisalignedin>me,wecanweightby!metoexploitrepeatabilityofunmodeledvariables.
n Formodelat>met:
n Obtainamodelforeach>metintothemaneuverbyrunningweightedregressionforeach>met
Trajectory-SpecificLocalModels
Abbeel,Coates,Ng,IJRR2010
![Page 24: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/24.jpg)
n Learningatargettrajectory
n Learningadynamicsmodel
n Autonomousflightresults
LearningDynamicManeuvers
Abbeel,Coates,Ng,IJRR2010
![Page 25: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/25.jpg)
ExperimentalSetup
Microstrain3DM-GX1@333HzRPMsensor@20-30Hz
Sonar
OxoardCameras1280x960@20HzExtendedKalmanFilterRHDDPcontroller
Controls@20Hz
“Posi>on”
3-axismagnetometer,accelerometer,gyroscope
(“Orienta>on”)
Abbeel,Coates,Quigley,Ng,NIPS2007
![Page 26: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/26.jpg)
1. Collectsweepstobuildabaselinedynamicsmodel
2. Ourexpertpilotdemonstratestheairshowseveral>mes.
3. Learnatargettrajectory.
4. Learnadynamicsmodel.
5. Findtheop>malcontrolpolicyforlearnedtargetanddynamicsmodel.
6. Autonomouslyflytheairshow
7. Learnanimproveddynamicsmodel.Gobacktostep4.
àLearntoflynewmaneuversin<1hour.
ExperimentalProcedure
Abbeel,Coates,Ng,IJRR2010
![Page 27: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/27.jpg)
Results:AutonomousAirshow
![Page 28: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/28.jpg)
Results:FlightAccuracy
![Page 29: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/29.jpg)
AutonomousAutorota>onFlights
Abbeel,Coates,Hunter,Ng,ISER2008
![Page 30: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/30.jpg)
Chaos[“flip/roll”parameterizedbyyawrate]
![Page 31: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/31.jpg)
![Page 32: Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred](https://reader035.vdocuments.us/reader035/viewer/2022070108/60280b77aca95d3b4340a1dc/html5/thumbnails/32.jpg)
ThankYou