Competition Monitor for RetailersOverview, June 2008

Market Intelligence System

www.datamine.gr

Company profile

datamine ltdDatamine was founded in January '05 and is based in Athens, Greece. Having as primary activity the

design & implementation of integrated business solutions using cutting edge I.T. and data modeling

techniques, Datamine is dedicated in redefining the role of information technology within the modern,

customer-centric enterprise through innovative data-driven applications

Solutions & ServicesDatamine provides state-of-the-art I.T. solutions with embedded Statistical & Data Mining logic, answering

the demanding management information needs of competitive corporations. Solutions range from

analytical CRM, data modeling & business intelligence to loyalty, campaign management and executive

information systems

Human ResourcesDatamine’s team demonstrates sound scientific background and extensive professional experience in

statistical analysis, data mining, software architecture, data warehousing and B.I. within telecom and

banking environments

Major Customers Partners

www.datamine.gr

Contents

Market Intelligence SystemAn overview of the solution

Business Value Long term benefits of the solution

Architecture overviewThe key components of the solution

Market Search & Query engineDescription of functionality on market search, browsing and querying

Crawler specificationCrawler execution planning, editions/ variations depending on embedded processing model

Rule Engine & AlertingDescription and examples on the rule engine

ReportingHigh-level description of reporting capabilities

Implementation roadmapPackaging, task summary and estimated effort

www.datamine.gr

Market Intelligence System – Overview

Datamine’s Market Intelligence Solution

Market

Intelligence

database

The

web

Management, Business Analysts

Market

Synch

engine

Crawler

engine

Market Manager

Monitoring, analysis and administration

Integrated systems

Online retailer

Intranet applications

Owned Portals, e-

commerce applications

Obtain Market overview & insight, trends, competitors

pricing policies, KPIs

Define business rules, alerts, synchronization settings

The proposed solution performs the following set of tasks:

1. Automatically collects product price information from a configurable set of web sites

2. Maintains a historical market database

3. Enables powerful price comparisons, reporting and analytics

4. Provides advanced tools such as alerting, conditional synchronization of external systems

5. Provides a framework for management & administration (execution settings, control and monitoring)

www.datamine.gr

Benefits

A wider range of benefits result from the proper operation of the system:

- Market responsiveness: Become more competitive on a day-to-day level – flexible to

instantly response to selected price-related triggers.

- Monitor the market: Minimize the effort required to monitor the market, standardize

monitoring process, eliminate related sources of errors, missing info. KPIs, targets, baselines

and limits may ‘automate’ the monitoring process

- Obtain market insight: Analyze & understand the market through reporting and analytics,

custom analysis and queries

- Understand competition: Monitor and analyze competitors pricing strategies. Examine

trends, cycling behaviors, special offerings

- Market database: smooth operation of the system will build a rich, historical market database

with a wealth of information useful in several marketing activities

www.datamine.gr

Market Intelligence System – Overview

Crawler

Raw input

handler

Market

Intelligence

database1 2

5

Web Services/

API

Market Manager

Monitoring, analysis, adminCustom Apps

The

web

Text Modeler

Text structure, &

classification

Reporting

Services

Advanced

Analytics

Request

handler

Search &

Compare

functionality

Administration

Execution

settings,

configuration

manager

3

4

8

6

7

Retailer -

application

layer

Datamine’s

Market

Intelligence

infrastructure

Product

catalogue

Synchronizer

in/outbound

synchronization

Retailer

Intranet

applications

portals, e-

commerce

applications

Integrated

systemsData sources

www.datamine.gr

Market Search & Query Engine

Powerful search & query engine against the underlying market database. Enables efficient market

query and navigation. Authorized users will be able to search products and pricing info:

1. Across providers (competitors) for user defined time intervals

2. Search using structured product information, categories, price limits, product hierarchies

3. Search using tags as automatically generated and maintained by text modeler component,

on descriptive product information (free text)

Examples:

- Get all products in category <laptop computers> for which <Competitor A> is on average

more than <5%> cheaper within the last <6-months>

- Get all products in category <home cinema> with at least 3 price reduction events during the

last 12 months

- Get competitors with average price in category <electronics> between <150> and <300>.

Drill through for each competitor to examine the detailed product listing.

- Get detailed price change history for products with code like ‘XXXX%’ for <competitor A>

Authorized users can manage Market Queries (save, edit, status change)

Export functionality in MS Excel or flat files is provided

www.datamine.gr

Crawler Execution planning

Execution manager enables the configuration of parameters which define how and when crawler is

executed. Configuration is feasible in several levels. The most significant are:

- List of target sites: define the targets to by analyzed (set of web-sites, portals etc)

- List of target products: define the set of primary products for which to gather competition

data (by reference and/or through keywords)

- Frequency of execution: define how often and under what conditions the process is

executed

- Logging options/ level of detail: define the level of detailed to be logged in the database

- Alerting options: define rules in order to trigger execution alerts to subscribed users

Specialized reporting for monitoring the smooth execution of the crawler, is included in the solution.

This involves a set of action analyzers, providing detailed information on the performance &

effectiveness of the crawler

www.datamine.gr

Crawler: Key-Characteristics

The proposed crawler component ensures performance, reliability and value maximization of

the information collected. The most significant characteristics of the crawler engine are:

- Price validation against private or category pricing history, with alerting if price is outside ± n

standard deviations. This alerts for erroneous data collected and prevents from loading into

the database.

- Reference matching: product names and categories against a valid and rated product

catalogue and owned metadata as generated by the system itself

- Similarity measures: similarity-based matching enabling efficient handling of product name

variations etc.

- Product/ category matching score: indicating if the product is matched and with what

confidence. information on the level of product matching achieved will be available for

reporting, searching, alerting and publishing procedures

- Detailed history of products, prices and information retrieved for each competitor (target site)

- Reporting and analysis of product identification performance

www.datamine.gr

Crawler editions

Three major crawler variations are offered :

Target-Optimized Crawler: this takes advantage of the specific structure of each target site in order

to perform fast, accurate data extraction (with minimized navigation attempts, post processing and

noise reduction operations)

- Benefits: performance, implementation cost, minimized volume of data transferred/ handled

- Limitations: less flexible – must be pre-configured for each specific target site. In case of

target site structural changes, re-configuration must be performed.

Unattended Crawler: this variation required limited initial information on the structure of the target

site. The crawler identifies the pieces of information to be extracted through a full-reference search

(exhaustive site search until the product identification for reference product listing is complete)

- Benefits: flexible – no need to pre-configure site. In case of target site structural changes, re-

configuration is not needed.

- Limitations: performance downgrade, increased implementation cost, larger volumes of data

transferred/ handled.

Hybrid Crawler: this combines the benefits of the two versions. Supports but does not require initial

target pre-configuration.

www.datamine.gr

Rule engine & Alerting

Enables the definition & management of custom-alerts based on price-related triggers. Authorized

user may define rules to alert marketing when:

- Order changing price modification: a change in the price of a registered product, setting a

competitor in a different rank based on price offered.

- Distance changing price modification: a change in the price of a registered product altering

the distance between your offering and market’s best

- Sudden, significant price change : Price for any registered product is significantly changed

by a competitor> indicates a special offering> can be also verified through text modeller

automated text mining

- Massive product price change: competitor A changed prices in a large set of products in

category A, B or C

Alert execution monitoring is feasible through specialized reporting

www.datamine.gr

Reporting & Analytics

Efficiently monitor the market across competitors, product hierarchies, against time, through

powerful, standardized parameterized reporting:

- Market trends report: time series of price aggregates, per product categories, competitor

price statistics against time dimension

- Competitor pricing strategy analyzer: detailed price variations, across products, against

time, for the selected competitor

- Benchmarking reports: comparison figures on competitor positioning, against the product

hierarchy

Dynamic reporting (a drag-n-drop model) can be delivered via OLAP components. This advanced

mode enables power users to define specific, specialized view on the underlying market. Advanced

features such as drill down and drill through are also supported in this option.

Report export in any standard office format is supported for both static & dynamic reporting

Integration with an existing data mart/ warehouse or Business Intelligence infrastructure is offered as

an additional package of services

www.datamine.gr

Crawler & IP tracking: proposed solution

Web

Crawler Host

Server

Target Web

Server

ISP Activity Logging

Web Proxy 1

Web Proxy 2

Web Proxy 4

Web Proxy 3

Web Proxy 5

Web Proxy 6

Web Proxy N

Encrypted

connection

(Active SSH Tunnel)

George Krasadakis Head of Engineering & Product Developmentg.krasadakis@datamine.gr

datamine ltdDecision Support Systems

22 Ethnikis Antistasis avenue15232 Chalandri

Athens, Greece

T: (+30) 210 6899960F: (+30) 210 6899968

info@datamine.grhttp://www.datamine.gr

Contact