PowerPoint Presentation

SKETCH4MATCH CONTENT-BASED IMAGE RETRIEVAL SYSTEM USING SKETCHESGEETHU P THAFSA HASSANHONEY MERRIN SAMSHIBIJA KGROUP IVTABLE OF CONTENTSINTRODUCTIONCURRENT SITUATIONPROPOSED SOLUTIONALGORITHM USEDPURPOSEGLOBAL STRUCTURE AND SUBSYSTEMFUNCTIONAL REQUIREMENTSPERFORMANCE MATRICESFEATURE EXTRACTIONUSE CASE DIAGRAMDATA FLOW DIAGRAMADVANTAGESSBIR VS TEXT-BASED IMAGE RETRIEVALRESEARCH CHALLENGESAPPLICATIONSFUTURE GROWTHCONCLUSION

INTRODUCTIONThe sketch based image retrieval is one of most popular, rising research areas of the digital image processing.

Goal of SBIR is to extract visual content like colour , text, or shape. Introduces design based on a free hand sketch

Making search more efficient hereby

Test results show that the sketch based system allows users an intuitive access to search-tools.

In todays corporate world huge data has to be managed, processed and stored

Text based search. Keywords! Is this efficient?

Therefore it is to develop a SBIR system, which can retrieve using sketches in frequently used databases.

Drawing area to sketch the required image.

Matched images to sketch are retrieved.

CURRENT SITUATIONIn earlier days, image retrieval from large image database can be done by following ways.

Automatic image annotation and retrieval using cross media relevance modelsConcept based query expansionQuery system bridging the semantic gap for large image databasesOntology-based query expansion widget for information retrievalDetecting image purpose in world-wide web documentsPROPOSED SOLUTIONRelevance feedback is an interactive process that starts with normal CBIR.

The user input a query, and then the system extracts the image features and measure the distance with images in the database.

An initial retrieval list is then generated.

This process can be iterated many times until the user find the desired images.InputQueryFeatureExtractionSimilarityMeasureRetrievalResultFind allImages?UsersFeedbackQueryupdateFinalRetrievalResultALGORITHM USEDIn this system an efficient image retrieval algorithm based on CCM (Colour Co-occurrence Matrix) is proposed.

The CCM for each pixel of an image is found using the Hue Saturation Value (HSV) of the pixel and then compared with CCM of the images in the database and the images are retrievedHSV COLOUR MODEL:

HSV colour model stands for Hue Saturation Value colour model.

This model describes colours in terms of their shades and brightness (Luminance).

This model offers a more intuitive representation of relationship between colours. COLOUR CO-OCCURRENCE METRIX(CRM):

A co-occurrence matrix is a matrix that is defined over an image to be the distribution of co-occurring values at a given offset. Value of an image is originally the grey scale value of a specified pixel. In our case we take the values to

The co-occurrence matrix is mainly used to measure the texture of the image and hence it is used for texture analysis.

Query ImageExtract HSV Formulate CCMQuery ImageExtract HSV Formulate CCMCompare and MatchRetrieved ImageSBIR using HSV modelPURPOSEThe goal of this paper is to develop a SBIR search engine, which with free hand sketch content can be retrieved.

The most important task is to bridge the gap between the free hand sketch and the picture.

Introducing this system into search engines makes corporate world and even other users bit more efficient in retrieval of data effectively.

GLOBAL STRUCTURE AND SUBSYSTEMGLOBAL STRUCTURE OF A SYSTEM

The system was designed for databases containing relatively simple images, but even in such cases large differences can occur among images in file size or resolution.In addition, some images may be noisier, the extent and direction of illumination may vary(fig a) and so the feature vectors cannot be effectively compared. In order to avoid it, a multistep pre-processing mechanism precedes the generation of descriptors.

The retrieval has to be robust in contrast of illumination and difference of point of view. Displaying SubsystemPreprocessing SubsystemFeature Vector generating subsystem Retrieval subsystemDatabase ManagementsubsystemFeature vectorimagePreprocessed imageResultStock index2. THE PREPROCESSING SUBSYSTEMThe system was designed for databases containing relatively simple images, but even in such cases large differences can occur among images in the size or resolution.

In addition, some images may be noisier, the extent and direction of illumination may vary, and so the feature vectors cannot be effectively compared.

In order to avoid it, a multistep pre-processing mechanism precedes the generation of descriptors.Preprocessing subsystemInputOutputImageProcessed ImageThe system is for databases containing simple imagesFUNCTIONAL REQUIREMENTS LOGIN MODULE

2. ADMIN MODULE

3. USERS MODULE

PERFORMANCE METRICESSBIR is essentially an information retrieval problem. Two of the most popular evaluation measures are the ,

PRECISION The precision measures the proportion of the total images retrieved which are relevant to the query.

precision = number of relevant images retrieved Total retrieved

RECALL

The recall measure is defined as the fraction of the all relevant images.

Recall = Total number of relevant images Number of relevant images retrieved

High precision means that less irrelevant images are returned or more relevant images are retrieved.

high recall indicates that few relevant images are missed

FEATURE EXTRACTIONCOLOUR

Colour is the most extensively used visual content for image retrieval. The most common primary colours in computing are red, green and blue (e.g. colours used in a monitor). Usually colours are defined in three dimensional colour spaces. In image retrieval systems, colour histogram is the most commonly used feature representation. The colour histogram describes the proportion of pixels of each colour in an image with simple and computationally effective manner.

TEXTURE

Texture refers to the visual patterns with properties of homogeneity that do not result from the presence of a single colour or intensity. It is that innate property of all surfaces that describes visual patterns such as; clouds, leaves, bricks, fabric, etc. It contains information about the structural arrangement of surfaces and their relationship to the surrounding environment. Texture properties include coarseness, contrast, directionality, regularity and roughness

SHAPE

Shape is an important criterion for matching objects based on their profile and physical structure. Shape does not refer to the shape of an image but to the shape of a particular region that is being sought out. Shape features can represent spatial information that is not represented by colour or texture. It contains all the geometrical information of an object in the image which does not change generally change even when orientation or location of the object are changed.

RESULTS OF FEATURE EXTRACTION:

USE CASE DIAGRAMA Use case diagram is a list of steps, typically defining interactions between a role (known in UML as an "actor") and a system, to achieve a goal.

The actor can be a human or an external system.

The purpose of use case is to present overview of the functionality provided by the system in terms of actors, their goals and any dependencies between those use cases.

Admin Data baseSystemIndex data base imagesLoad search images to bufferSelect algorithm for searchSearch database for similar imagesUser

SystemUpload/search imagesSketch imagesRetrieve imagesQuery updateData baseDATA FLOW DIAGRAMIndexing and searchingCalculate resultDisplayImageUserData baseFeatureExtractionImageDatabaseCompute similarityMeasureVisualization

ADVANTAGESMake convenient to retrieve data or images based on sketches so that even illiterates , who do not know to write text can also make use of system effectively.

Introducing this system into search engines makes corporate world and even other users bit more efficient in retrieval of data effectively.

Visual features, such as colour, texture, and shape information, of images are extracted automatically

Similarities of images are based on the distances between features

SBIR VS TEXT-BASED IMAGE RETRIVEL SYSTEMImage retrieval algorithms roughly belong to two categories:

Text-based approaches The text-based approaches associate keywords with each stored image.

Content-based methodsHere retrieval of images is guided by providing a query image or a sketch generated by a user.

In SBIR, each image that is stored in the database has its features extracted and compared to the features of the query image. It involves two steps:

FEATURE EXTRACTION

The first step in the process is to extract image features to a distinguishable extent.

MATCHING

The second step involves matching these features to yield a result that is visually similar.

HOW TO DESCRIBE THIS IMAGE ?

WHAT IS THIS IMAGE?

SKYRIVERCOCONUT TREES

GiraffeText based image retrieval systemImage database

Image databaseCBIRSystem

RESEARCH CHALLANGESThe implementation of SBIR systems raises several research challenges. Some of these are:

Understanding image users needs and information-seeking behaviour. Investigating new user interfaces for annotating, browsing, and searching based on image content .New tools for marking/annotating images (and their regions) Matching query and stored images in a way that reflects human similarity judgmentsProviding compact storage for large image databases Efficiently accessing stored images by content

APPLICATIONSMEDICAL APPLICATIONSThree large domains can instantly take advantage of SBIR techniques: TeachingResearchDiagnostics

BIODIVERSITY INFORMATION SYSTEMSIdeally, Biodiversity Information Systems (BIS) should help researchers to enhance or complete their knowledge and understanding about species and their habitats by combining textual, image content-based, and geographical queries. 3. DIGITAL LIBRARIES

One example is the digital museum of butterflies aimed at building a digital collection of Taiwanese butterflies.

This digital library includes a module responsible for content-based image retrieval based on colour, texture, and patterns. FUTURE GROWTHWEB ORIENTEDTo better organize and retrieve the almost unlimited information, web based search engines are highly desired.Such solutions exists for text based information's

HIGH DIMENSIONAL INDEXINGMost currently existing research prototype systems only handle hundreds or at most thousands of images.

CONCLUSIONThe area of SBIR is a hybrid research area that requires knowledge of both computer vision and of database systems.

The technology is exciting but immature.

Among the objectives of this paper ,two main aspects were taken into account.The retrieval process has to be highly interactive.The robustness of the method.

Frequently updated shared image database and the regular comparison of system performances would be of great benefit to the SBIR community.

The field appears to be generating interesting and valid results, even though it has so far led to few commercial applications.

Agencies concerned with technology transfer or dissemination of best practice in fields which could potentially benefit from SBIR .