the australian centre for robotic vision (acrv)
TRANSCRIPT
Juxi Leitner arc centre of excellence for robotic vision
queensland university of technology<[email protected]> http://Juxi.net
robotic vision an australian vision for robotics
ACRV @ UC Berkeley
http://roboticvision.org/2
tinyurl.com/QUTRobotics
Brisbane, Australia
http://roboticvision.org/
SIFRslide
http://roboticvision.org/
http://roboticvision.org/
tinyurl.com/QUTRobotics
roboticvision.org
http://roboticvision.org/
10 96%eyedesigns
animalshave eyes
Nature Reviews | Neuroscience
Protostomes
Bilateria
Ecdysozoa
Lophotrochozoa
~580 Mya
~550 Mya
~530 Mya
~500 Mya
~430 Mya
Deuterostomes
Chordates*
Craniates*
Vertebrates*
Gnathostomes*
Arthropods
Annelids
Molluscs
Hemichordates
Echinoderms
Cephalochordates*
Tunicates*
Myxiniformes*
Petromyzoniformes*
Last fossil jawless fish
Stages of interest in vertebrate eye evolution Cambrian
Mya 600 550 500 450 400 0
2 1 3 4 5 6
Lampreys
Jawed vertebrates
Hagfish
Lancelets
Sea squirts
Ocellus
Eye patch
ProtostomeAn animal belonging to the protostome super-phylum, which is characterized by its members’ embryonic development, in which the first opening (the blastopore) becomes the mouth (protostome is Greek for ‘first mouth’). All protostomes are invertebrates.
DeuterostomeAn animal belonging to the deuterostome super-phylum of the animal kingdom, which is characterized by its members’ embryonic development, in which the first opening (the blastopore) becomes the anus (deuterostome is Greek for ‘second mouth’). In addition to the chordate phylum (which includeds vertebrates), the other two main phyla are the echinoderm phylum and the hemichordate phylum.
ChordateAn animal belonging to the chordate phylum, which comprises vertebrates, tunicates and cephalochordates. These animals are characterized by the presence of a notochord, a dorsal-nerve cord and pharyngeal slits or pouches.
AgnathanA jawless fish within the chordate phylum (agnatha is Greek for ‘no jaw’). The two extant groups are hagfish and lampreys.
GnathostomeThe jawed vertebrates (gnathostome is Greek for ‘jaw mouth’), comprising fish and tetrapods (including birds and mammals).
craniates, hagfish have the most basal body-plan. They possess neither jaws nor vertebrae and are therefore usually regarded not as vertebrates but rather as a sister group. The vertebrates comprise an early jaw-less (agnathan) division, of which the only living examples are lampreys, and a later jawed division, the gnathostomes, which includes fish and tetrapods.
Controversy has long surrounded the interrelation-ship between hagfish, lampreys and jawed vertebrates. BOX 1 summarizes current views, and in FIG. 1 we show hagfish diverging either before the divergence of lam-preys or else after lampreys separated from the line that would become the jawed vertebrates.
Not only has extensive gene duplication occurred throughout the evolution of animals22, but in addition it is widely accepted that two rounds of whole-genome duplication occurred early in vertebrate evolu-tion23–29; most likely, one duplication occurred before the agnathans split from the vertebrate line and one occurred after (FIG. 1; for reviews, see REFS 30–32). It is also clear that the vertebrate organizer, which deter-mines the body plan of developing embryos, arose in early chordates33–35. These genetic developments are likely to have been of crucial importance in early vertebrate evolution, but they are beyond the scope of this Review.
Figure 1 | The origin of vertebrates. The evolution of jawed vertebrates is illustrated against an approximate time-scale of millions of years ago (Mya). The taxa considered in this Review are indicated with an asterisk and are accompanied by schematics and diagrams of the ‘eye’ region. The earliest chordates, represented by extant cephalochordates and tunicates, are thought to have appeared around 550 Mya. Jawless craniates (agnathans) were present in the early Cambrian, by 525 Mya, and a time of 530 Mya has been indicated for their presumed first appearance. As elaborated on in BOX 1, there is considerable controversy as to whether myxiniformes (solely represented by extant hagfish) diverged before or after the separation of lampreys from jawed vertebrates (shown as dashed black and grey lines). Numerous lines of jawless fish evolved between 500 and 430 Mya ago, although none have survived to the present day. The first jawed vertebrate arose around 430 Mya, and this line is represented today by cartilagenous fish, bony fish and tetrapods. Six ‘stages of interest’ in vertebrate eye evolution correspond to the time intervals between the divergence of important surviving taxa. This diagram does not include the evolutionary changes that have occurred in the last 400 million years. The presented timeline is based primarily on evidence from the fossil record; see REFS 2,13,15,17,18,144,160–163. The schematics are modified, with permission, from REF. 11 (1996) Oxford University Press (lancelet, sea squirt, hagfish and lamprey) and REF. 164 (2004) Academic Press (jawed vertebrate). The eye images are reproduced, with permission, from the following references: lancelet, REF. 165 BIODIDAC (1996) University of California Museum of Paleontology; sea squirt, REF. 63 (2006) Blackwell Publishing; hagfish, REF. 166 (2006) Australian Museum. Lamprey and jawed vertebrate eye images are courtesy of G. Westhoff and S. P. Collin).
R E V I E W S
NATURE REVIEWS | NEUROSCIENCE VOLUME 8 | DECEMBER 2007 | 961
http://roboticvision.org/
Australian Centre for Robotic Vision
Robust Vision
Vision and Ac/on
Seman/c Representa/ons
Algorithms and Architecture
Visual Learning
Dalle Molle Institute for AI (IDSIA)
Work
Juxi Leitner
PhD Informatics / Intelligent Systems
MSc Space Robotics & Automation
BSc Information & Software Engineering
Intelligent (Space) Robots European Space Agency (ESA)
Erasmus Intelligent Systems
Work (Humanoid) Robot VisionInstituto Superior Técnico (IST)
Mobility Intelligent Space Systems Laboratory
About Me
Current Robotic Vision and Actions
h9p://Juxi.net
Queensland University of Technology (QUT)
coordinationeye-hand
move to see
see to move
Vision and Action (VA)
http://Juxi.net/projects/VisionAndAction/
http://Juxi.net/projects
https://www.youtube.com/watch?v=w_qDH5tSe7g&list=PLOqMcGR_zUyH4IaYurBhB5D02TurBQOQW&index=2
manipulation for improved perception
http://roboticvision.org/
cameraslightfieldwork lead by Donald G. Dansereau
Deblurring3D scenes yield complex blur Generalized deconvolution for 6DOF motion
http://roboticvision.org/
propertiesdetectingvisually[Dansereau, Singh, Leitner, ICRA2016]
http://Juxi.net/projects/squishy/
http://roboticvision.org/
http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign
http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign
ObjectDetection
ObjectLocalisation
ObjectIdentification
Shelf Localisation
Moveto Bin
Grasp Selection
Grasp Execution
systemdesign
ObjectDetection
ObjectLocalisation
ObjectIdentification
Shelf Localisation
perception
Kinect2 (once)
RealSense (every attempt)
KinectFusion
2D regions(CNN)
point cloud
segmentation
grasp point detection
http://Juxi.net/projects/AmazonPickingChallenge/
http://roboticvision.org/
score
http://roboticvision.org/
ARC Centre of Excellence for Robotic Vision roboticvision.org
fruit picking
https://www.youtube.com/watch?v=8rq4iSTsg68
tinyurl.com/QUTLunaRoo
LUNAROO: CREATING A HOPPING LUNAR SCIENCE PAYLOAD
tinyurl.com/QUTLunaRoo STUDENT SUMMER PROJECT
ROB LEE, TIM HOJNIKSKP-1
tinyurl.com/QUTLunaRoo STUDENT SUMMER PROJECT
ROB LEE, TIM HOJNIK
Robotic Vision Testbed
[Zhang et al, ACRA 2015, IROS 2016]
understanding limita/ons of deep nets, reinforcement learning & transfer of knowledge
deep learning visual control
http://roboticvision.org/
http://roboticvision.org/
http://Juxi.net/special-issue/ijrr-deep-learning-2016/
IJRR Special Issue
R:SS WS(what) scep/cs?
deeper understanding
tool (how to use it?)
RV vs CV
data driven models
for listeningthank [email protected] http://Juxi.net/projects