IST-1999-29017 – OMNIVIEWSIST-1999-29017 – OMNIVIEWSOmnidirectional Visual SystemOmnidirectional Visual System
Final Review
September 27-28, 2001
Lisbon
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Agenda of the meetingAgenda of the meetingFirst DayFirst Day
Agenda of the meetingAgenda of the meetingFirst DayFirst Day
14:30 Reviewers’ private meeting15:00 Welcome Jose' Santos-Victor15:10 Goal of Review Pekka Karp15:20 Omniviews Main Achievements Giulio Sandini16:00 Coffee Break16:30 Mirror design principles Branislav Micusik16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco17:10 Demos Introduction17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino18:00 Transmission Demo Pedro Soares, Giulio Sandini18:30 End of first Day 20:30 Dinner
14:30 Reviewers’ private meeting15:00 Welcome Jose' Santos-Victor15:10 Goal of Review Pekka Karp15:20 Omniviews Main Achievements Giulio Sandini16:00 Coffee Break16:30 Mirror design principles Branislav Micusik16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco17:10 Demos Introduction17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino18:00 Transmission Demo Pedro Soares, Giulio Sandini18:30 End of first Day 20:30 Dinner
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Agenda of the meetingAgenda of the meetingSecond DaySecond Day
Agenda of the meetingAgenda of the meetingSecond DaySecond Day
9:00 Navigation Demo José Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini10:10 Reviewer’s Private Meeting (with coffee)11:10 Preliminary Evaluation Report11:40 End of meeting
9:00 Navigation Demo José Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini10:10 Reviewer’s Private Meeting (with coffee)11:10 Preliminary Evaluation Report11:40 End of meeting
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Main FactsMain FactsMain FactsMain Facts
Consortium:DIST - University of Genova - GenovaCMP - Czech Technical University in PragueVISLAB - Instituto Superior Técnico - Lisbon
Consortium:DIST - University of Genova - GenovaCMP - Czech Technical University in PragueVISLAB - Instituto Superior Técnico - Lisbon
Project Start and Duration:September 1st 2000 – One year
Project Start and Duration:September 1st 2000 – One year
Funding: 100 K€Funding: 100 K€
Open-scheme, Assessment Phase ProjectOpen-scheme, Assessment Phase Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project’s Main ObjectiveProject’s Main ObjectiveProject’s Main ObjectiveProject’s Main Objective
The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas.
In particular our intention is to design and realize a low-cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images.
The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas.
In particular our intention is to design and realize a low-cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images.
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Key TechnologiesKey TechnologiesKey TechnologiesKey Technologies
Retina-like visual sensorRetina-like visual sensor
Omnidirectional MirrorsOmnidirectional Mirrors
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Specific Objectives of Specific Objectives of Assessment PhaseAssessment Phase
Specific Objectives of Specific Objectives of Assessment PhaseAssessment Phase
Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained.
Demonstrate its utility in key application areas If successful present a follow-up proposal
Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained.
Demonstrate its utility in key application areas If successful present a follow-up proposal
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
MethodologiesMethodologiesMethodologiesMethodologies
1. Use the currently available SVAVISCA camera for initial experiments
2. Design and simulate mirror using SVAVISCA camera
3. Realize the OMNIVIEWS mirror for the current sensor
4. Demonstrate the mirror in key applications
1. Use the currently available SVAVISCA camera for initial experiments
2. Design and simulate mirror using SVAVISCA camera
3. Realize the OMNIVIEWS mirror for the current sensor
4. Demonstrate the mirror in key applications
Simulated image:SVAVISCA cameraHyperbolic mirror
Simulated image:SVAVISCA cameraHyperbolic mirror
SVAVISCAPixel layout
SVAVISCAPixel layout
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Mirror’s design principleMirror’s design principleMirror’s design principleMirror’s design principle
1. Uniform Cylindrical Projection
2. Direct read-out through log-polar mapping
1. Uniform Cylindrical Projection
2. Direct read-out through log-polar mapping
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
OMNIVIEWS MirrorOMNIVIEWS MirrorOMNIVIEWS MirrorOMNIVIEWS Mirror
Mirror’s ProfileMirror’s Profile
Experimental Set-up and test imagesExperimental Set-up and test images
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Assessment CriteriaAssessment CriteriaAssessment CriteriaAssessment Criteria
1. Direct read-out of panoramic images
2. Frame rate
3. Resolution and layout of the sensor
4. Mirror profile and size
5. Lens characteristics
6. Camera cost
7. Image quality
1. Direct read-out of panoramic images
2. Frame rate
3. Resolution and layout of the sensor
4. Mirror profile and size
5. Lens characteristics
6. Camera cost
7. Image quality
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC1: Direct read-out of AC1: Direct read-out of panoramic imagespanoramic images
AC1: Direct read-out of AC1: Direct read-out of panoramic imagespanoramic images
Direct read-out from OMNIVIEWS:About 30,000 read-out operations
Direct read-out from OMNIVIEWS:About 30,000 read-out operations
Image Obtained from a conventional camera:About 1.8 M operations required:882,000 read-outs (30 times more)882,000 additions30,000 divisions
Image Obtained from a conventional camera:About 1.8 M operations required:882,000 read-outs (30 times more)882,000 additions30,000 divisions
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC2: Frame rateAC2: Frame rateAC2: Frame rateAC2: Frame rate
Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s.
More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA)
Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s.
More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA)
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC3: Resolution and Layout of AC3: Resolution and Layout of the sensorthe sensor
AC3: Resolution and Layout of AC3: Resolution and Layout of the sensorthe sensor
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC4: Mirror Profile and SizeAC4: Mirror Profile and SizeAC4: Mirror Profile and SizeAC4: Mirror Profile and Size
Mirror profile and size meets the original plan (6 cm.).Mirror profile and size meets the original plan (6 cm.).
Furthermore:1) New technology for mirror realization2) Mirror’s design tool of general utility3) Design and realization of “mixed-mirror”4) Overall size can be reduced
Furthermore:1) New technology for mirror realization2) Mirror’s design tool of general utility3) Design and realization of “mixed-mirror”4) Overall size can be reduced
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC5: Lens characteristicsAC5: Lens characteristicsAC5: Lens characteristicsAC5: Lens characteristics
Standard C-Mount lenses have been used for all experiments and demos
Standard C-Mount lenses have been used for all experiments and demos
No difficulties in principle are envisaged for the design of smaller size lenses (possibly including the mirror).
No difficulties in principle are envisaged for the design of smaller size lenses (possibly including the mirror).
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC6: Camera CostsAC6: Camera CostsAC6: Camera CostsAC6: Camera Costs
Cost of obtaining panoramic images is zero in our case
Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology.
The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size.
Cost of obtaining panoramic images is zero in our case
Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology.
The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size.
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
AC7: Image qualityAC7: Image qualityAC7: Image qualityAC7: Image quality
Topology of images meets the quality criteria Evaluation of numerical approximations Three demonstrations:
1) Surveillance (two parts)2) Navigation3) Image Transmission
Further processing experiments:1) Localization using Agam fiducials2) 3D reconstruction
Topology of images meets the quality criteria Evaluation of numerical approximations Three demonstrations:
1) Surveillance (two parts)2) Navigation3) Image Transmission
Further processing experiments:1) Localization using Agam fiducials2) 3D reconstruction
The project will be successful if we demonstrate that it is possible to create virtual images by simple reading out the pixels from the proposed sensor and to use such images in the aimed applications ….
The project will be successful if we demonstrate that it is possible to create virtual images by simple reading out the pixels from the proposed sensor and to use such images in the aimed applications ….
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project
Additional remarksAdditional remarksAdditional remarksAdditional remarks
1. Software mirror-design tools have been developed
2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”)
3. 10 scientific papers have been published
4. Plans for the future are clearer.
1. Software mirror-design tools have been developed
2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”)
3. 10 scientific papers have been published
4. Plans for the future are clearer.