27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Monitoring of Aggregation Levels Monitoring of Aggregation Levels in Distributed Component Based in Distributed Component Based Data Production SystemsData Production Systems
BTW 2003, Leipzig, 27.02.2003
Anja SchanzenbergerGfK Marketing Services, NürnbergUniversity of Middlesex, London
Colin Tully, Dave LawrenceUniversity of Middlesex, London
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
2 Application
3 Monitoring of Aggregation Levels
1 Application Area
Agenda
The General Business of GfK Marketing Services The Basic Idea of Data Production System
The Planning, Controlling and Monitoring System
Single Record Tracking The Tubing System Reconstructing Aggregation Levels
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Application Area1
4
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
The GfK Group: Key Features
Total revenue
Anticipated EUR 568 million in 2002; previous year: EUR 506 million
Increase on the previous year: +12%
Employees More than 4,800 full-time staff 70% of which abroad
Network
Over 130 subsidiaries, branches and participations in 50 countries on five continents
Services
Integrated systems using standardised instruments throughout Europe and beyond
5
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Four Complementary Business Divisions
Consumer Tracking
Consumer and retail panel based Business Information Solutions for manufacturers and retailers for consumer packaged goods and service companies
In interview and panel based audience and readership measurement and consumer response testing for TV, print, radio and Internet
Media
Non-Food Tracking
Retail panelbased marketing
information for manufacturers and
retailers in consumer technology industries
Interview and test market based support
information for new product development
and brand management across a
wide range of industries
Ad Hoc Research
6
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
GfK Business Divisions
Consumer Tracking16.4%
12.1%Media
Non-Food Tracking23.6%
41.5%Ad Hoc Research
Other6.4%
Share of total
performance
7
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Non-Food Tracking: Key Services
Information services in 44 countries on marketing, sales, logistics in retail and industry for companies operating in consumer technology markets.
Direct access to databases and/or transmission of standardized analyses to support, monitor and manage short, medium and long term decisions on product and pricing policy, advertising, distribution, sales and logistics.
Key services
The advantage for clients
Consumer Tracking Media
Ad Hoc Research
Non-Food Tracking
Non-Food Tracking
Retail panel
periodical monitoring
periodical monitoring
Market leader in the regions Europe and Asia and Pacific as well as in the Arab countries; together with partner NPD Intelect, market leader in North America.
Positioning
Information services on consumer durables, in particular for the consumer electronics, photographic, information technology, telecommunications, software, domestic appliances and equipment markets
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Application2
9
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
StarTrack Working Areas
Data Warehouse(Extrapolation, Reports)
Data - IN Data - Preparation
DWH
Retailers Clients
Creating value through knowledge
IDAS
MDM
10
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Data Production System
General InterfaceManager (GIM)
Data receipt
Separation
Identification(WebTAS)
Central IDASoutput pool
Local Output
Planning – Controlling – Monitoring System
Local client
Local server Central server
DWH Projectionsystem
Mainframe
11
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
PCMS Dimensions
current state • predefined process steps
• manual state checking
• manual error tracking
envisioned state
• dynamic production process configuration
• production planning and monitoring
• proactive error handling
Data Data Production Production
SystemSystem
PLANNING
MONITORING CONTROLING
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Monitoring of Aggregation Levels3
13
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Definitions
aggregation
separation
(disaggregation)
aggregation levels
input
many data sets
output
many data sets
output
one data set
input
one data set
instruction
aggregate functions
multiple groupings
14
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Aspects to the Monitoring of Aggregation Levels
Summaries after significant process steps summaries of operating figures
Single Record Tracking tracking of single retailer items up to the
customer report simulation of planned production cycles
(ETL-Tools)
15
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Example - Single Record Tracking
component X
Item AR: Vobis – DP: CW 04/2002-sales volume: 6Item BR: Vobis – DP: CW 04/2002-sales volume: 9
Item AR: Vobis – DP: CW 05/2002-sales volume: 4
Item AR: Vobis – RP: Jan 2002-sales volume: 10
Item BR: Vobis – RP: CW 04/2002-sales volume: 9
R: retailer CW: calendar weekDP: delivery periodRP: reporting period
pool A pool B
16
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Strategies of Tracing Aggregation Levels
Tubing SystemTubing System
the complete workflow cycle error situations
17
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Characteristics of Monitoring
TYPE CHARACTERISTIC
amount of data static / non static
aggregation policies known / unknown
job parameters known / unknown
storage requirements
minimal to maximal
18
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Possibilities to reconstruct Aggregation Levels (1)
component X
Static Volumes of Data
-all items
-all retailers
-all delivery
periods
instruction:
SELECT...WHEREDP1=CW 6, DP2=CW 7DP3=CW 8, DP4=CW 9GROUP BY Vobis, item
pool A pool B
item,retailer: Vobisreporting period: Feb/2002
DP: delivery periodCW: calendar week
job parameters:itemretailerreporting period
19
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
(1) Static Volumes of Data
Advantages no additional storage required historically stored data allows stepwise tracking
possibilities
Disadvantages historically stored requires increased storage
facilities this approach is only significant for a small (historical
stored) quantity of data all job parameters are required increasing the quantity of data in storage slows
down the control system as well as the controlled system
requires additional administration effort
20
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Possibility (2)
component X
-all items
-all retailers
-all delivery periods
pool A pool B
item,retailer: Vobisreporting period: Feb/2002
job parameters:job_iditemretailerreporting period
Single Record Logging
log:
timestampjob parameters records of A: -item -retailer -delivery period -facts -price
21
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
(2) Single Record Logging
Advantages no policies needed no static volumes of data
Disadvantages additional job parameters are needed at least twice the storage requirement additional administration effort slowdown of systems
22
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Possibility (3)
Advantages storage requirement (approach 3) <
storage requirement (approach 2) no policies needed no static data volumes
Disadvantages no deleting of records, but new attribute values
for the same records additional administration effort slowdown of systems
Primary Key Logging most important attributes job parameters needed logging: item, retailer, delivery period reduction at GfK: ~1/5
23
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Possibility (4)
component X
Data Evaluation
pool A1 pool B1
pool A2 pool B2
processing time
tracking time
all
all
item,retailer: Vobisreporting-period: Feb/2002
item,retailer: Vobisreporting-period: Feb/2002
instruction
instruction
job parameters:itemretailerreporting period
24
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
(4) Data Evaluation
Advantages no additional logging no additional storage required alterations of records allowed no static data volumes
Disadvantages policies are needed program extension job parameters are needed only an imprecise estimate
(processing time <> tracking time) double execution time of component
25
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Conclusion (I)
1. Static Volumes of Data environments: (historical) static data volumes least logging effort best approach, but often not applicable
2. Single Record Logging environments: min. 2*storage required and slowdown
acceptable suitable when gathered amount of data >> processed
amount of data (e.g. ad-hoc reports)
3. Primary Key Logging environments: less manipulations acceptable deleting of records is not allowed additional logging effort
26
27. February 2003GfK Group Monitoring of Aggregation Levels. Anja SchanzenbergerNon-Food Tracking
Conclusion (II)
… more [email protected]
4. Data Evaluation environments: level of impreciseness acceptable no additional logging effort no additional implementation work no additional storage required system load increases -> recommended for slack times