development and implementation of a high-level command system and compact user interface for...
TRANSCRIPT
![Page 1: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/1.jpg)
Development and Implementation of a High-Level Command System and
Compact User Interface for Nonholonomic Robots
Hani M. Sallum
Masters Thesis Defense
May 4, 2005
![Page 2: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/2.jpg)
2
Outline• Overview and Goals• Development
– Control System
– Data Analysis and Mapping
– Graphical User Interface
• Results
![Page 3: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/3.jpg)
3
Overview
This work details the design and development of a goal-based user-interface for unmanned ground vehicles which is maximally simple to operate, yet imparts ample information (data and commands) between the operator and the UGV.
![Page 4: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/4.jpg)
4
Typical UGV Usage
![Page 5: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/5.jpg)
5
Other UGV Issues• Multi-person crews• Proprietary Operator Control Units
(OCU’s) for each UGV
Is there a way to have local users control UGV’s without the operational overhead currently required?
![Page 6: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/6.jpg)
6
Current State of the Art
![Page 7: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/7.jpg)
7
Why UA/GV’s?• Over the last two decades there has been a dramatic
increase in the complexity/availability of manufactured electronics.
• As a result, the capital cost of robotic systems in general has decreased, making them more feasible to implement.
Example: N.A.S.A. Mars Pathfinder/Sojourner system was built largely out of commercially available (OTS) parts (sensors, motors, radios, etc.)1.
• Additionally, the capacity and functionality of devices such as PDA’s and cellular phone has increased as well.
1. N.A.S.A., Mech. Eng. Magazine, Kodak
![Page 8: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/8.jpg)
8
Motivation:
Considering the ubiquity of PDA’s, smartphones, etc., is it possible to develop a method of using these devices as a form of common O.C.U.?
Q: Do custom O.C.U.’s need to be developed when commercial technology is evolving so rapidly?
![Page 9: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/9.jpg)
9
Goals:
1. Develop a control system for a UGV • Automates low-level control tasks
2. Develop a method of rendering sensor data into maps of the UGV’s environment
3. Design a GUI which runs on a commercially available PDA
![Page 10: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/10.jpg)
10
Hardware: RobotiRobot B21R Mobile
Research Robot (nonholonomic)
Camera
Sonar
Laser Rangefinder
![Page 11: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/11.jpg)
11
Definition of Nonholonomic
Unable to move independently in all possible degrees of freedom.
Example: Cars have 3 degrees
of freedom (x, y, ), but
can not move in x or alone.
![Page 12: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/12.jpg)
12
Hardware: PDA
Hewlett-Packard iPAQ
802.11/BluetoothAntenn
a240x320 Color Screen
Touch Sensitive
Windows CE Operating System
![Page 13: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/13.jpg)
13
Navigation Control System
Two aspects of the navigation process:
•Target Approach
•Obstacle Avoidance
Multimodal Controller
Separate control laws depending on the desired operation of the robot.
![Page 14: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/14.jpg)
14
Proven Method• Schema Architecture [Chang et al.]
• Discrete shifts between control modes
• Straightforward to implement
• “chattering” between modes
![Page 15: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/15.jpg)
15
Proposed Method• Fuzzy Control [Wang, Tanaka, Griffin]
• Gradual shifts between control modes
• More complicated controller
• Smoother trajectory through state-space
![Page 16: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/16.jpg)
16
Fuzzy Controller
Sensor Data
Target Approach
Mode
Obstacle Avoidance
Mode
Fuzzy Blending
{K, {K,
Fuzzy Control Signals {v,
![Page 17: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/17.jpg)
17
Target ApproachControl of Turning Velocity
• Final orientation unconstrained
• Implement a proportional controller driving the robot heading to a setpoint equal to the current bearing of the target (i.e. DEV 0)
• Produce APP
• Saturate the controller at the max allowable turning speed
• Use high proportional gain to approximate an arc-line path
![Page 18: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/18.jpg)
18
Target ApproachControl of Forward Velocity
• Final position close to target
• Implement a proportional controller to scale the forward velocity, based on the robot’s distance to the target coordinates (i.e. DTAR 0)
• Produce APP
• Saturate the controller at APP =1 (scale to the maximum allowable forward speed)
![Page 19: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/19.jpg)
19
Obstacle AvoidanceControl of Turning Velocity
• Implement a proportional controller driving the robot heading to a setpoint 90º away from the nearest obstacle (i.e. OBS ±90º)
• Produce AVOID
![Page 20: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/20.jpg)
20
Obstacle AvoidanceControl of Forward Velocity
• Implement a proportional controller to reduce (scale down) the forward velocity when nearing an obstacle
• Produce AVOID
![Page 21: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/21.jpg)
21
Obstacle Avoidance• Forward Control
• Inner threshold elliptical to avoid being stuck to obstacles:
![Page 22: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/22.jpg)
22
Final Control LawTurning Control:• Blend the target approach and obstacle avoidance control
signals using a weighted sum:
WAPPAPP + WAVOIDAVOID = FUZZY
• Determine weights using membership functions based on the robot’s distance to the nearest obstacle.
![Page 23: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/23.jpg)
23
Final Control LawForward Control:• Blend the target approach and obstacle avoidance control
signals by multiplying the maximum forward velocity by the scaling factors produced by each control mode.
KAPPKAVOIDvMAX = vFUZZY
![Page 24: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/24.jpg)
24
Outline• Overview and Goals• Development
– Control System
– Data Analysis and Mapping
– Graphical User Interface
• Results
![Page 25: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/25.jpg)
25
Data Analysis and Mapping
• Render data from the laser rangefinder into significant features of the environment:
“Fiducial Points”e.g. corners, ends of walls, etc.
• Use these fiducial points to generate primitive geometries (line segments) which represent the robot’s environment.
![Page 26: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/26.jpg)
26
Distillation Process
RAW DATA
Object Detection
Segment Detection
Finding Intersections
FIDUCIAL POINTS
Categorizing Points
Line Fitting
Finding Fiducial Points
![Page 27: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/27.jpg)
27
Why Find Fiducial Points?Laser Rangefinder Data
![Page 28: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/28.jpg)
28
Object DetectionRange vs. Bearing
(used by Crowley [1985])
![Page 29: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/29.jpg)
29
Object Detection
![Page 30: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/30.jpg)
30
Object Detection
![Page 31: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/31.jpg)
31
Segment DetectionRecursive Line Splitting Method used by Crowley [1985], B.K.
Ghosh et al. [2000]
![Page 32: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/32.jpg)
32
Proposed Threshold FunctionCREL: Relative Threshold CABS: Absolute Max Threshold
Segment Detection
![Page 33: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/33.jpg)
33
Line FittingUse perpendicular offset least-squares line fitting
![Page 34: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/34.jpg)
34
Finding Intersections
![Page 35: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/35.jpg)
35
Categorization
• Each fiducial point is either interior to an object, or at the end of an object.
• Fiducial points at the ends of objects are either occluded or unoccluded.
![Page 36: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/36.jpg)
36
DistillationFinding Fiducial Points
![Page 37: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/37.jpg)
37
Mapping
• Fiducial Points provide a clear interpretation of what is currently visible to the robot
• Provide a way to add qualitative information about previously observed data to local maps Global map
![Page 38: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/38.jpg)
38
Creating a Global Map
Global Map:Occupancy Evidence Grid [Martin, Moravec, 1996] based on laser rangefinder data collection.
![Page 39: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/39.jpg)
39
Creating a Global Map
Global Map:Occupancy Evidence Grid [Martin, Moravec, 1996] based on laser rangefinder data collection.
![Page 40: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/40.jpg)
40
Local MappingMap Image:• Sample section of global map for qualitative a
priori information about local area.• Overlay map primitives.
![Page 41: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/41.jpg)
41
Local MappingExample:
Local map with and without a prior information.
![Page 42: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/42.jpg)
42
Vision MappingVision Map:• Transform map primitives to perspective frame and
overlay camera image of local area.
![Page 43: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/43.jpg)
43
Vision MappingFind common geometries for defining vertical and
horizontal sight lines.
![Page 44: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/44.jpg)
44
GUI• Serve web content from the robot to the
iPAQ
• Use image-based linking (HTML standard) to allow map images to be interactive on the iPAQ
• Use web content to call CGI scripts onboard the robot, which run navigation programs on the robot
![Page 45: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/45.jpg)
45
GUIMain Map/Command
Screen
![Page 46: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/46.jpg)
46
GUILong-Range
Map/Command ScreenClose-Range
Map/Command Screen
![Page 47: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/47.jpg)
47
GUIRotation
Map/Command ScreenVision
Map/Command Screen
![Page 48: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/48.jpg)
48
Results
• GUI: Main Map/Command Screen
![Page 49: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/49.jpg)
49
Results
• GUI: Rotation Map/Command Screen
![Page 50: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/50.jpg)
50
Results
• GUI: Vision/Command Screen
![Page 51: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/51.jpg)
51
Results
• Obstacle Avoidance
![Page 52: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/52.jpg)
52
![Page 53: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/53.jpg)
53
![Page 54: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/54.jpg)
54
![Page 55: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/55.jpg)
55
Conclusions
• The infrastructure exists for implementing an OCU on a PDA• OTS devices
• Networking/web standards
• Fuzzy logic methods can be applied to mobile robot control• Obstacle Avoidance
• Economic Path Generation
• Variable thresholds can be used for more robust range data interpretation
• Object detection based on incident angle
• Segment detection based on two parameters
• Fusion of data can impart more information to the operator• Occupancy information and fiducial points
• Fiducial points and visual data
![Page 56: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/56.jpg)
56
Future Work
• Use fiducial points to implement Simultaneous Localization and Mapping
• Address control system limitations
• Streamline/upgrade web content and programming for the GUI
![Page 57: Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense](https://reader035.vdocuments.us/reader035/viewer/2022062720/56649f075503460f94c1cf93/html5/thumbnails/57.jpg)
57
Thanks To:
•My Committee:•Professor Baillieul
•Professor Wang
•Professor Dupont
•ARL/MURI
•Colleagues in IML
•Family and Friends
•AME Staff
•Professor Baillieul