multimedia research in e-education veljko milutinović, fellow of the ieee an overview of the...

Multimedia ResearchMultimedia Researchinin

E-EducationE-Education

Veljko Milutinović, Fellow of the IEEE

An Overview of the Ongoing Projects

http://galeb.etf.bg.ac.yu/~vm

e-mail: [email protected]

of 79

Major R&D BottlenecksMajor R&D Bottlenecks

• Integrated Educational SystemsIntegrated Educational Systems

• Concept Understanding and Interactive CooperationConcept Understanding and Interactive Cooperation

• Intelligent Search and Retrieval for Educational PurposesIntelligent Search and Retrieval for Educational Purposes

• Innovative ApplicationsInnovative Applications

of 79

U. of California at BerkeleyU. of California at Berkeley

• Integrated Systems: Intel MES GrantIntegrated Systems: Intel MES Grant• Concept Understanding: WISEConcept Understanding: WISE• Intelligent Search: GISTIntelligent Search: GIST• Innovative Applications: BMRCInnovative Applications: BMRC

of 79

MITMIT

• Integrated Systems: Microsoft I-CampusIntegrated Systems: Microsoft I-Campus

• Concept Understanding: OXYGENConcept Understanding: OXYGEN

• Intelligent Search: NCAMIntelligent Search: NCAM

• Innovative Applications: NRENInnovative Applications: NREN

of 79

Stanford UniversityStanford University

• Integrated Systems: Challenge 2000 MultimediaIntegrated Systems: Challenge 2000 Multimedia

• Concept Understanding: Media XConcept Understanding: Media X

• Intelligent Search: CBIRIntelligent Search: CBIR

• Innovative Applications: SUMMIT, CERAS, MM Collab, ShakespeareInnovative Applications: SUMMIT, CERAS, MM Collab, Shakespeare

of 79

Project Name (University)Project Name (University)

• Project Leader(s)Project Leader(s)• Project URLProject URL• Project Essence in ASCIIProject Essence in ASCII• Project Essence in JPEG/MPEGProject Essence in JPEG/MPEG

of 79

Intel MES Grant (Berkeley)Intel MES Grant (Berkeley)

• James Demmel

• http://www.intel.com/pressroom/archive/releases/CO081897.HTMAA

• RISC approach to MESRISC approach to MES

of 79

WISE (Berkeley)WISE (Berkeley)

• Eric BaumgartnerEric Baumgartner

• http://wise.berkeley.edu/pages/intro/wiseIntro01.html

• Risk approach to K12Risk approach to K12

of 79

GIST (Berkeley)GIST (Berkeley)

• Joe HellersteinJoe Hellerstein

• http://now.cs.berkeley.edu/AMhttp://now.cs.berkeley.edu/AM

• Risk approach to generalized search tree for secondary and Risk approach to generalized search tree for secondary and multimedia storage; supports any lookup over that data multimedia storage; supports any lookup over that data

of

79

BMRC (Berkeley)BMRC (Berkeley)

• Larry RoweLarry Rowe

• http://www-plateau.cs.berkeley.edu/http://www-plateau.cs.berkeley.edu/

• Risk approach to contents and technology managementRisk approach to contents and technology management

of

79

Microsoft I-Campus (MIT)Microsoft I-Campus (MIT)

• Thomas L. Magnanti

• http://web.mit.edu/newsoffice/tt/1999/oct06/microsoft.htmlhttp://web.mit.edu/newsoffice/tt/1999/oct06/microsoft.html

• Risk approach to $25 millionRisk approach to $25 million

of

79

OXYGEN (MIT)OXYGEN (MIT)

• Anant Agarval, John Ancorn, Krste Asanovic, Rodney Brooks, …Anant Agarval, John Ancorn, Krste Asanovic, Rodney Brooks, …• http://oxygen.lcs.mit.edu/http://oxygen.lcs.mit.edu/• Risk approach to bringing abundant computation, multimedia, Risk approach to bringing abundant computation, multimedia,

and communication naturally into people's lives, through an and communication naturally into people's lives, through an infrastructure of mobile and stationary devices connected by a infrastructure of mobile and stationary devices connected by a self-configuring networkself-configuring network

of

79

NCAM (MIT)NCAM (MIT)

• Geoff FreedGeoff Freed

• http://www.ncddr.org/du/researchexchange/v06n03/multi.htmlhttp://www.ncddr.org/du/researchexchange/v06n03/multi.html

• Risk approach to accessible multimedia and distant educationRisk approach to accessible multimedia and distant education

of

79

NREN (MIT)NREN (MIT)

• MIT, ARPA, DOE, NASA, NSFMIT, ARPA, DOE, NASA, NSF

• http://www.ccic.gov/pubs/blue94/section.3.2.htmlhttp://www.ccic.gov/pubs/blue94/section.3.2.html

• Risk approach to gigabit communications infrastructure for Risk approach to gigabit communications infrastructure for e-research and e-educatione-research and e-education

of

79

Challenge 2000 MM (Stanford)Challenge 2000 MM (Stanford)

• Shari Golan, Barbara Means, Bill PenuelShari Golan, Barbara Means, Bill Penuel

• http://ctl.sri.com/projects/displayProject.jsp?Nick=ch2000mmhttp://ctl.sri.com/projects/displayProject.jsp?Nick=ch2000mm

• Risk approach to the next decade challenges in MESRisk approach to the next decade challenges in MES

of

79

Media X (Stanford)Media X (Stanford)

• John PerryJohn Perry

• http://mediax.stanford.edu/about/education.htmlhttp://mediax.stanford.edu/about/education.html

• Risk approach to multimedia courses in interdisciplinary major Risk approach to multimedia courses in interdisciplinary major undergraduate and graduate programsundergraduate and graduate programs

of

79

CBIR (Stanford)CBIR (Stanford)

• Jia Li, James Z. Vang, Gio WiederholdJia Li, James Z. Vang, Gio Wiederhold

• http://www-db.stanford.edu/IMAGE/http://www-db.stanford.edu/IMAGE/

• Risk approach to contents based image retrieval: semantics-sensitive integrated matching for picture libraries, wavelet-based image indexing and searching

of

79

SUMMIT (Stanford)SUMMIT (Stanford)

• Parvati DevParvati Dev

• http://www-smi.stanford.edu/projects/summit.htmlhttp://www-smi.stanford.edu/projects/summit.html

• Risk approach to MES in medicineRisk approach to MES in medicine

Potentials of R&D in MESPotentials of R&D in MESthroughthrough

Italy/Serbia Cooperation Italy/Serbia Cooperation

An Overview of Belgrade University Projects

for High-Tech Computer Industry

in the USA and EU (IPSI)

of

79

Some of the Recent ProjectsSome of the Recent Projects

NCR, Compaq, SUN, Intel,

Comshare, Zycad, QSI, Virtual,

TechnologyConnect, BioPop, eT, MainStreetNetworks,

Salerno, Pisa, Siena, L’Aquila, ...

of

79

Response: IndustryResponse: Industry

of

79

Response:Response: Academia Academia

Flynn, M. J., Computer Architecture, Jones and Bartlett, USA (96)position 1 (12 citations)

Bartee, T. C., Computer Architecture and Logic Design, McGraw-Hill, USA (91)position 1 (2 citations)

Tabak, D., RISC Systems (RISC Processor Architecture), Wiley, USA (91)position 1s (6 citations)

Stallings, W., Reduced Instruction Set Computers (RISC Architecture), IEEE CS Press, Los Alamitos, California, USA (90)position 1s (3 citations)

Heudin, J. C., Panetto, C., RISC Architectures, Chapman-Hall, London, England (92)position 3s (2 citations)

van de Goor, A. J., Computer Architecture and Design, Addison Wesley, Reading, Massachusetts, USA (2nd printing, 91)position 4s (3 citations)

Tannenbaum, A., Structured Computer Organization (Advanced Computer Architecures), Prentice-Hall, USA (90)position 5s (4 citations)

Feldman, J. M., Retter, C. T., Computer Architecture, McGraw-Hill, USA (94)position 7s (2 citations)

Stallings, W., Computer Organization and Architecture, Prentice-Hall, USA (96)position 9s (3 citations)

Murray, W., Computer and Digital System Architecture, Prentice-Hall, USA (90)position >10s (2 citations)

Wilkinson, B., Computer Architecture, Prentice-Hall, USA (91)position >10 (2 citations)

Decegama, A., The Technology of Parallel Processing (Parallel Processing Architectures), Prentice-Hall, USA (90)position >10s (2 citations)

Baron, R. J., Higbie, L., Computer Architecture, Addison-Wesley, USA (92)position >10s (1 citation)

Tabak, D., Advanced Microprocessors (Microcomputer Architecture), McGraw-Hill, USA (95)position >10s (1 citation)

Zargham, M. R., Computer Architecture, Prentice-Hall, USA (96)position >10s (1 citation)

Hennessy, J. L., Patterson, D. A., Computer Architecture: A Quantitative Approach, Morgan-Kaufmann, USA (96)na (0 citations)

Hwang, K., Advanced Computer Architecture, McGraw-Hill, USA (93)na (0 citations)

Kain, K., Computer Architecture, Addison-Wesley, USA (95)na (0 citations)

of

79

N.B.N.B.

ERRORS

MADE

&

LESSONS

LEARNED

of

79

11

of

79

22

of

79

33

of

79

SummarySummary

The world’s best journals – IEEE (50):The world’s best journals – IEEE (50): A European record in ICTA European record in ICT

Books with 7 Nobel Laureates:Books with 7 Nobel Laureates:

Kenneth Wilson, Ohio (North-Holland)Kenneth Wilson, Ohio (North-Holland) Leon Cooper, Brown (Prentice-Hall)Leon Cooper, Brown (Prentice-Hall) Robert Richardson, Cornell (Kluwer-Academics)Robert Richardson, Cornell (Kluwer-Academics) Jerome Friedman, MIT (Kluwer-Academics)Jerome Friedman, MIT (Kluwer-Academics) Herb Simon, CMU (IOS)Herb Simon, CMU (IOS) Arno Penzias, AT&T (IOS)Arno Penzias, AT&T (IOS) Harold Kroto, England (IOS)Harold Kroto, England (IOS)

of

79

Multimedia Internet GalleryMultimedia Internet Gallery

Funding:Funding: Fraunhofer IPSI, Darmstadt, GermanyFraunhofer IPSI, Darmstadt, Germany

Implementation:Implementation: IPSI, Belgrade, SerbiaIPSI, Belgrade, Serbia

Project Termination Date:Project Termination Date: August 31, 2003.August 31, 2003.

Users:Users: Germany, Serbia, USA, Canada,…Germany, Serbia, USA, Canada,…

Demo:Demo: www.ipsi.co.yuwww.ipsi.co.yu

of

79

IPSI Belgrade, Ltd.IPSI Belgrade, Ltd.

3D@3D MULTIMEDIA 3D@3D MULTIMEDIA ARTSHOP GALLERYARTSHOP GALLERY

[email protected], [email protected]://galeb.etf.bg.ac.yu/vm, http://www.ipsi.co.yu

of

79

AuthorsAuthors

Milutinovic Veljko

Toskov IvanVujovic Ivana

Skundric Nikola Milutinovic Darko

Stojanovski Aleksandar Nikezic Gavro

Anucojic Goran

Radakovic MiroslavMarinkovic Ivan

of

79

Introduction – IPSI BelgradeIntroduction – IPSI Belgrade

IPSI BelgradeIPSI Belgrade is a company jointly founded by German and Serbian capital

Partners:

• IPSI Fraunhofer, Darmstadt, Germany

• Telecom Italia Learning Services, Italy

• NYU, School of Continuous Professional Studies, USA

• Purdue University, School of Technology, USA

of

79


- Multimedia Workspaces of the Future- Multimedia Applications for the Web- Environments for Cooperative Working and Learning- Virtual Information and Knowledge Environments- Mobile Interactive Media- Open Adaptive Information Management Systems- Publication Engineering and Technology- Hardware Design and Operating Systems- Networks and WWW- Semantic Web and Datamining

of

79


Products:

• Advanced Multimedia Virtual Gallery• Tools for B2B Matchmaking on the Web• Web security for P2P, The injection cache, The STS cache, Genetic Search with Spatial/Temporal Mutations, Customer Browser Satisfaction Web Search, Browser Acceleration, Technology Transfer, Testing Infrastructure for EBI, Distant Web Educating Machine, e-Tourism, …

of

79

MMAG: Problem StatementMMAG: Problem Statement

- Creating Web based art gallery with “look and feel” of the real world exhibitions

- Visitor moves through the gallery by “walking with options”

- 2D on 3D and 3D on 3D, with multimedia options

of

79

Existing SolutionsExisting Solutions

- Musee national des Arts asiatiques http://www.museeguimet.fr/tour-guimet/index.html

- Web Server of the Galleria degli Uffizi in Florence http://www.uffizi.firenze.it

- The Distributed Interactive Virtual Environment (DIVE) http://www.sics.se/dive/

- The Web3D Repository http://www.web3d.org/vrml/artgal.htm

of

79

Proposed SolutionProposed Solution

- Virtual reality gallery: Multimedia In Action

- Advanced search capabilities: Dream Search

- Artist’s criteria room generation: Do It By Yourself

of

79

Why is it better?Why is it better?

- Dynamically generated/exploitable gallery- Dynamically generated/exploitable gallery

- Content based search engine- Content based search engine

- User satisfaction - User satisfaction

of

79

Conditions and AssumptionsConditions and Assumptions

- PC- Internet connection- Internet Explorer 5.0 or higher - Netscape 7.0 - Cortona VRML plug-in for IE- Basic multimedia tools and standards

of

79

Analysis and ImplementationAnalysis and Implementation

- Application is written in ASP.NET using C# as code-behind, and ADO.NET for database access.- Database server is SQL Server 2000.- Communication with the database is entirely made through XML (using SQLXML3.0 framework).

- Queries are made in XPath, while adding, changing and deleting of the records is done through UpdateGrams.

- Application is optimized for Internet Explorer 5.0 or higher, at the 1024x768 screen resolution. Netscape 7.0 or higher is also supported.

- 3D gallery is completely generated on the server side (dynamically) using VRML.

of

79

Track 1Track 1

Track Requirements:Track Requirements:• To stand as the integrative part for the other two tracksTo stand as the integrative part for the other two tracks• To provide:To provide:

– User interactionUser interaction– Database connectivity (database independent)Database connectivity (database independent)– Search functions Search functions

(simple and advanced using Track3 output)(simple and advanced using Track3 output)– Information brokering between artists and buyersInformation brokering between artists and buyers– Administration toolsAdministration tools– Artworks management tools, etc.Artworks management tools, etc.– Thin client (3D scene generation on server side)Thin client (3D scene generation on server side)

of

79

Track 1Track 1

Development Tools:Development Tools:• Application server platform:Application server platform:

– Windows XP ProfessionalWindows XP Professional– IIS 5.1IIS 5.1– MS SQL Server 2000MS SQL Server 2000

• Development platform:Development platform:– ASP.NETASP.NET– C# as code-behind.C# as code-behind.

• Communication with the underlying database:Communication with the underlying database:– XML & XSD using XPath queries (DB independent)XML & XSD using XPath queries (DB independent)– Currently using SQLXML3.0 add-on for ADO.NETCurrently using SQLXML3.0 add-on for ADO.NET

of

79

Track 1Track 1

Database Design:Database Design:

of

79

Track 1Track 1

Administrator Tools:Administrator Tools:

• Separate entry point:Separate entry point:

http://<server_address>/artshop/adminhttp://<server_address>/artshop/admin

of

79

Track 1Track 1

Users & Exhibitors:Users & Exhibitors:• Entry point:Entry point:

http://<server_address>/artshop/index.htmhttp://<server_address>/artshop/index.htm

of

79

Track 1Track 1

Interesting Details:Interesting Details:

• Native XML DBMS under development at IPSI Fraunhofer• Practical testing of the XML/XPath database access• Dynamic addition (to the system) of new multimedia types• 3D view of search results

of

79

Track 1Track 1

Interesting Details:Interesting Details:• Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting

and possibly useful idea and possibly useful idea (user just has to set one XML file containing local field mapping, and one XSD to map the (user just has to set one XML file containing local field mapping, and one XSD to map the database fields to the pre-defined scheme)database fields to the pre-defined scheme)

• Cons:Cons:– XPath queries are lot less powerful then standard SQL queriesXPath queries are lot less powerful then standard SQL queries– Inherently, loss of speed Inherently, loss of speed

(one complex SQL query had to be simulated with couple of XPath queries and additional (one complex SQL query had to be simulated with couple of XPath queries and additional processing in the code).processing in the code).

– For now, SQLXML3.0 does not support complete For now, SQLXML3.0 does not support complete XPath standardXPath standard..

of

79

Track 1Track 1

Errors Made:Errors Made:

• Initially, content analysis, picture processing, and adding data to database were completely Initially, content analysis, picture processing, and adding data to database were completely separated (as specified in the contract), separated (as specified in the contract), with the idea of later (partial) integration.with the idea of later (partial) integration.

• Turned out to be a bad ideaTurned out to be a bad idea(required a lot of intervention(required a lot of interventionfrom the ArtShop system administrator when adding artworks).from the ArtShop system administrator when adding artworks).

of

79

Track 1Track 1

Lessons Learned:Lessons Learned:• Problem solved by complete integration of forementioned tasks Problem solved by complete integration of forementioned tasks

into the one system process which monitors input directory, automatically schedules picture into the one system process which monitors input directory, automatically schedules picture processing and content analysis, and takes care of updating of all necessary fields in all processing and content analysis, and takes care of updating of all necessary fields in all required databases.required databases.

• With that, we achieved maximum automation, With that, we achieved maximum automation, reduced time needed for artwork addition, reduced time needed for artwork addition, and reduced amount of data transferred through the Internet and reduced amount of data transferred through the Internet (between the administrator’s machine and the application host).(between the administrator’s machine and the application host).

of

79

Track 2Track 2

Image-Content-Oriented SearchImage-Content-Oriented SearchTrack requirements:Track requirements:

• Images used for extracting objects are artistic paintings• Image analyses• Extraction of the features• Create XML file for each image• Fetch the database with the features

of

79

Track 2Track 2

Algorithm:Algorithm:Open the picture

Load parameters, input and output directory

Determine the filter value

Put the picture into the reduced matrix

Determine histogram

Create objects

Merge objects into bigger objects

Create sorted array of objects

Create database objects, prepare them, and put them in XML file and tables in database

of

79

Track 2Track 2

Determine histogram:Determine histogram: Create general histogram

Remove zero values

Sort histogram

Remove redundancies

Sort histogram

Refresh matrix

of

79

Track 2Track 2

Regions forRegions for processing matrix: processing matrix:

P8

P4P5

P5P6P3

P3

P4P5

P3

P8

P8P8

P8

the first colon the last row

the last colon

the rest of the matrix

the last element in the last row

of

79

Creating Creating objects:objects:

Any pixel left in the current region?

p8 = the current pixel

call matrix.unite_pixels method

p8.oi == 0 (it doesn’t belong to any object)

create new object

Any neighbour pixel left?

p8.oi != px.oi (don’t belong to the same object)

take the next pixel

take the next neighbour pixel

px = the neighbour pixel

true

true

true

false

true

false

false

false

of

79

Merge Merge objects objects into into bigger bigger objects:objects:

Is Picture_Objects list empty?

q==true (are there any objects inside Border which colors after translation are the same as the color of Core object)

q = true; Edge[0] = Picture_Objects[0]

i < number of objects inside Edge list

Find all neighbour objects, which colors after translation are the same as the color of Core object, and put them into Border

Move all pixels from objects inside Edge to Core object

Remove objects that are inside Edge from Picture_Objects list

Is Border empty

q=false; add Core object to big list; Remove pixels belonging to Core object from Picture_Objects

false

true

true

Take ith object

i = i+1

Move all objects inside Border to Edge

false

true

false

true

false

of

79

Track 2Track 2

Tools used in development:Tools used in development:• C# programming language – the chosen tool

– Advantage: Includes the best properties from other programming languages (C++, Java, Visual Basic)

– Disadvantage: slower processing speed than C++, which is not necessary in this application

• C++ - the best alternative tool

– Advantage: faster processing speed (unnecessary)

– Disadvantage: more complicated code, 50% of all bugs due to use of pointers

of

79

Track 2Track 2

Original picture:Original picture:

of

79

Track 2Track 2

Picture after applyingPicture after applyinghistogram values:histogram values:

of

79

Track 2Track 2

Picture representedPicture representedthrough extracted objects:through extracted objects:

of

79

Track 2Track 23D HSL space => 1D histogram3D HSL space => 1D histogram

Index of histogram array Hue Saturation Luminance Description

0 any any <=30 black

1 any any >=lummax white

2 any <20 >30<30+lum

the darkest grey

... … … … …

ilmax-1 any <20 >=30+(ilmax-3)*lum<30+(ilmax-2)*lum

the lightest grey

ilmax <=hue/2>=240-hue/2

>20<20+1*sat

>30<30+lum

the darkest red with the smallest saturation

ilmax+1 <=hue/2>=240-hue/2

>=20+1*sat< 20+2*sat

>30<30+lum

the darkest red with smaller saturation

… … … … …

ilmax+isat-2 <=hue/2>=240-hue/2

>=20+(isat-3)*sat< 20 +(isat-2)*sat

>30<30+lum

the darkest red with the biggest saturation

ilmax+1*(isat-1) <=hue/2>=240-hue/2

>20<20+1*sat

>=30+lum< 30+2*lum

darker red with the smallest saturation

ilmax+1*(isat-1)+1 <=hue/2>=240-hue/2

>=20+1*sat< 20+2*sat

>=30+lum< 30+2*lum

darker red with smaller saturation

… … … … …

ilmax+1*(isat-1)+isat-2 <=hue/2>=240-hue/2


>=30+lum< 30+2*lum

darker red with the biggest saturation

ilmax+2*(isat-1) <=hue/2>=240-hue/2

>20<20+1*sat

>=30+2*lum< 30+3*lum

dark red with the smallest saturation

of

79

Track 2Track 23D HSL space => 1D histogram3D HSL space => 1D histogram

ilmax+2*(isat-1)+1 <=hue/2>=240-hue/2

>=20+1*sat< 20+2*sat

>=30+2*lum< 30+3*lum

dark red with smaller saturation

… … … … …

ilmax+(ilmax-3)*(isat-1)+1 <=hue/2>=240-hue/2

>=20+1*sat< 20+2*sat

>=30+(ilmax-3)*lum< 30+(ilmax-2)*lum

the brightest red with smaller saturation

… … … … …

ilmax+(ilmax-3)*(isat-1)+(isat-2) <=hue/2>=240-hue/2



the brightest red with the biggest saturation

ilmax+1*(ilmax-2)*(isat-1) > hue/2< 3*hue/2

>20<20+1*sat

>30<30+lum

the darkest orange-red with the smallest saturation

ilmax+1*(ilmax-2)*(isat-1)+1 > hue/2< 3*hue/2

>=20+1*sat< 20+2*sat

>30<30+lum

the darkest orange-red with smaller saturation

… … … … …

ilmax+1*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2)

> hue/2< 3*hue/2



the brightest orange-red with the biggest saturation

ilmax+2*(ilmax-2)*(isat-1) >=3*hue/2< 5*hue/2

>20<20+1*sat

>30<30+lum

the darkest red-orange with the smallest saturation

ilmax+2*(ilmax-2)*(isat-1)+1 >=3*hue/2< 5*hue/2

>=20+1*sat< 20+2*sat

>30<30+lum

the darkest red-orange with smaller saturation

… … … … …

ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+1 >=(2*ihmax-1)*hue/2<(2*ihmax+1)*hue/2

>=20+1*sat< 20+2*sat

>30<30+lum

the darkest magenta-red with smaller saturation

… … … … …

ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2)

>=(2*ihmax-1)*hue/2<(2*ihmax+1)*hue/2



the brightest magenta-red with the biggest saturation

of

79

Track 2Track 2

Lessons learned:Lessons learned:

• It is impossible to extract objects using only colors as a criterion• It is impossible to extract objects,

even using textures, edges, different transformations as criteria• Semantics should be used in segmentation• Colors are the most important features in artistic paintings

of

79

Track 3Track 3

Track requirements:Track requirements:• Possibility of moving through 3D galleries

• Automatic generation of 3D galleries based on user’s query

• Manual generation of 3D galleries

• User interface for image zooming

• Application for image processing

of

79

Track 3Track 3

Underlying algorithms:Underlying algorithms:

• Dynamic creation of gallery

• Creation of static galleries

• Algorithm for picture zooming

• Algorithm for picture processing

of

79

Track 3Track 3

Creation of galleries:Creation of galleries:

• Validation of the created gallery

• Forming VRML files depending on users query

• Determining the number of pictures in the gallery

• Drawing a 2D floorplan based on the 3D gallery

• “Forest fire” algorithm for filling the floorplan with color

of

79

Track 3Track 3

Picture processing:Picture processing:

• Loading image into memory

• Clone image into different-size copies

• Filtering of copies

• Parting of copies

of

79

Track 3Track 3

Development tools:Development tools:

• C# in .NET Framework for programming image processing

• Macromedia Dreamweaver for programming zoom tool

• VRML Pad v2.0

of

79

Track 3Track 3

Flowchart:Flowchart:

of

79

Track 3Track 3

Creation of galleries:Creation of galleries:

• Making files based on users data

• Putting data on serverso it can be available for artists

• Artist chooses which gallery he/she will be using for exhibition

• User can move through 3D world

• Selecting the textures for gallery

• Selecting the starting position of the user

of

79

Track 3Track 3

NF filter details:NF filter details:

• If the new picture is smaller, every pixel is one pixel of the old picture.

• If the new picture is bigger, pixels are calculated based on the pixels surrounding the current.

of

79

Track 3Track 3

Errors made:Errors made:

• Requests were not precise, so there was a gap at the end of the project between wanted and done

• Better results could be done with better using of ASP and XML

of

79

Track 3Track 3

Lessons learned:Lessons learned:

• Every member of the team gets a part where his experience is dominating

• More planning at the start reduces a lot of work later

• Good communication between programmers can save a lot of time

of

79

DemoDemo

of

79

Future PlansFuture Plans

• Improving the existing 3D dynamic gallery

• Improving search engine capabilities

• Improving feature extraction algorithms and objects recognition

of

79

Future Track 1Future Track 1

3D Multimedia Showroom Environment:3D Multimedia Showroom Environment:

• Implementing a generalized Web based 3D Multimedia Showroom Environment

• Exhibiting various MM data types: images, 3D objects, videos, audio, etc.

• Set of MM data types should be extendable

of

79


MM Object Feature Extraction:MM Object Feature Extraction:

• Implementing algorithms and software components for extracting features from MM data types (images, videos, 3D objects), in order to enable content based search

• System should be extendable (“plug-in”)

of

79


Semantic Abstraction of MM Feature Spaces:Semantic Abstraction of MM Feature Spaces:

• Developing methods and SW components which derive mapping from extracted features of MM objects to semantic concepts

• Using intelligent classification algorithms (Neural Networks, Fuzzy Classifier)

• Developing semantic query engine (answering questions, which could previously only be answered by humans)

of

79

UsabilityUsability

• Art GalleriesArt Galleries

• MuseumsMuseums

• Exhibition FairsExhibition Fairs

of

79

Instead of ConclusionInstead of Conclusion

IPSI Belgrade, [email protected]://www.ipsi.co.yu

of

79

http://galeb.etf.bg.ac.yu/~vm/

e-mail: [email protected]

multimedia research in e-education veljko milutinović, fellow of the ieee an overview of the...

Documents

mes slide

berkeley integrated

wise intelligent search

bmrc slide

multimedia integrated

k12 risk approach

data risk approach

eduam risk approach