Innovative data collection methods for road freight transport statistics

Page 2

Road Transport Statistics in Austria

EU Regulation 70/2012 provides a general legal and methodological framework for the different national surveys (territoriality principle)

Stratified quarterly sample comprises 6,500 vehicles per quarter out of a total of about 72,000 registered. Meets quality criteria described in the regulation Original sample size of 26000 vehicle weeks per year significantly reduced (since 2006)

Local units, operating vehicles and drawn for the sample use paper based questionnaires or an electronic questionnaire (11% in 2009)

Efforts to complete the questionnaire have been reduced. 27.3 minutes on average to complete a questionnaire 150 work weeks per annum for the Austrian economy …

Main task for completing the questionnaires consists of collecting and preparing the information within their companies.

Page 3

Current situation

Limited relevance of data for Austria’s traffic & transport planners due to sample errors on county or smaller traffic cell levels

Insufficient precision of reported tonne-km as a result of automated imputation of the distance travelled between origin and

destination of a journey using distance matrices, ignoring potential detours in between.

Possible over-estimation of empty trips assuming that distances between places of unloading and subsequent places of loading

are empty trips

Assumed under reporting of trips respondents assumed to minimize efforts, and report a vehicle to be out-of-order during

the sample week.

Limited accuracy of cargo types i Respondents have limited information regarding cargo moved („mixed cargo“) Assignment of goods to NST/R classification by respondents leads to incoherent results

Quality concerns regarding road transport data

Consortium

Page 4

Gebrüder Weiss GmbH

Paradigma Unternehmensberatung GmbH

Wirtschaftsuniversität Wien – Inst. f. Transportwirtschaft und Logistik

Petschl Transporte Österreich GmbH & Co KG

Austrian Institute of Technology – Department Mobility

Process knowledge and experience of transport companies

Technology, data management und electronic data exchange

Methods and legal environment of road freight statistics in Europe

Page 5

Goals and Results of the Project

Project Goals Project deliverables

Further reduction of the respondents efforts through automation

Return to a larger sample to meet national requirements

Increase of data quality and actuality

Reduction of required ressources for preparation and processing of the data

Prototypic and fully functional implementation of the connection between data (from companies) to the XML-Interface

Test the applicability of automatic data collection technologies and algorithms to obtain precise measurements.

Legal, economic and methodical evaluation of the results with respect to the road freight transport statistic

Research Objectives

Prove the technical feasibility building such a working prototype develop a sufficiently generic standard interface

Assess the organizational impact on the respondents obtain empirical information on the benefits as well as potential issues

Use information about goods from transport booking data, to infer cargo types (NST/R) Train a Bayes algorithm, part of the KNIME data mining software to classify goods using

free text

Use GPS location to measure distances travelled and to infer load/unload events. Obtain route information from GPS readings – infer events and compare with order

information

Obtain experience with the technical and economic challenges to implement the standard interface industry software as well as individually developed software

International coordination

Page 7

EUROSTAT International harmonization of electronic data exchange Potential requirements for national RF surveys (data

collection) Standards and architectures

Identify similar European initiatives Identification of similar projects in Europe Learn from experience and best practice Implications of easier data exchange for survey design and

sampling

Software providers (ERP, TMS) Statistical microdata exporter as standard software

component Economies of scale targeting the EU marketplace Talks with SAP, Navision, TransIT, Sauer, …

Page 8

International responses

EUROSTAT: Joint approach of data collection Metadata – enrichment as early as possible Joint, comparable Method (for consolidation) Collection of fuel use (CO²-emission)

CBS (NL) is a guide for InnoRFDat-X Sustains good contacts with SW-industry for realisation of

XML-based reports Uses algorithms for ease of input (goods classification, route

validation)

Experiences of the Ministère du Développement Durable (FR)

Hesitant Respondents Heavily fragmented company landscape with litte IT use No strategy for integration of SW-industry

Page 9

International responses

Trafik Analys (Sweden) At the moment with traditional questionnaires only with

manual recording Interested in InnoRFDat-X approach

Kraftfahrt-Bundesamt Flensburg (Germany) Survey: Increasing use of standardised software but lack of

citical mass of one provider Search for alternative data sources for groups of goods, origin

& destination

Danske Statistik (Denmark) Legal requirement to replace paper as used medium by the

end of 2012 -> Web-questionnaire with manual input Attempts on using TMS/ERP-data

Target Solution Architecture

Data interfaces are in the public domain and provided to the Software- and System developers

Different information entities are consolidated according to the data model specifications and business logic

Completion of missing data and corrections are performed by the respondent

Based on the transfer-format the specific required structure for the respective country questionaire is generated

Page 11

Stakeholders & expected benefits

Respondents Carriers Freight Forwarders Companies with own

fleet

Users Public sector decision

makers Interest groups General public

Producers National Statistics

Institutes Ministries of

Transportation

Reduce cost No need to collect data

manually Reduction of paper-based work No follow-up calls from NSI‘s

Increase of data quality Increased accuracy Increased coverage

Lower production cost Less correction, completion

efforts Reduce, eliminate paper based

work

Page 12

Information elements collected

4 XML-based interface specifications are provided

FleetMasterData

Data on lorries and trailers (capacity, axles, odometer, age, license, etc.)

FleetStatusData

Information on specific lorries at certain times (driven distance, fuel usage, etc.)

ConsignmentData

order related data containing information on goods, packaging, origin and destination …

PositionData

GPS readings, country/ZIP codes, activity (loading, border crossing)

The information model

0 or more journeys performed during the observation period per vehicle can be reported.

0 to 3 trailers per journey can likewise be reported as well as 0 or more different shipments per journey.

0 or more combined transports per journey can be reported.

0 or more transit countries (international journeys) per journey can be reported (not shown in the diagram).

0 or more shipments per journey may be reported; each shipment can be associated with 0 or more containers.

A shipment can comprise 1 or more commodities; each commodity can be classified to be a dangerous good if applicable.

class PIM Ov erv iew

query::journey

query::journey::trailer

query::journey::shipment

query::journey::shipment::commodity

query::journey::shipment::commodity::dangerous

query::journey::shipment::container

query::journey::combinedTransport

query::motorVehicleNotification

query::motorVehicle

query

0..2

0..*

0..3

0..1

1..*0..* 0..*

1..*

1

1..*response

1

Page 14

Data Collection Service

Protoytped process deployed to move data from IT systems to eQuest

Insert your own text here

Questionnaires as XML files are generated by the eQuest system run by Statistics Austria.

These Questionnaires contain the selection criterias for the data export.

The data export extracts the information out of the ERP/TMS-systems and saves them as XML-files (4 predefined formats).

These files are uploaded to the „SGVS-Konsole“ web-application.

The respondent can now revise the data. The web-application generates a

„completed questionnaire“ and uploads this to the eQuest system.

In the eQuest system the report is finished.

Questionnairewith data

XML-Question-naire

Selection criteria

Data export

ExportedXML data

ERP/TSM

DatabaseAccess

API

Web application „SGVS Console“

eQuest Web-application

Web-Application

Page 15

Web-application

Page 16

Validation Rules (Excerpt)

Every lorry or articulated vehicle mentioned in the NSI’s questionnaire must have an entry in the company’s fleet management system.

Odometer readings at the beginning and the end of the reporting period must be available, where the latter has to be greater or equal than the former. If multiple odometer readings are available over time, the sequence of readings must be non-decreasing.

Every shipment must have been allocated to one or more sections of a journey.

If events and activities such as load, unload are reported or inferred from the position data (see below), corresponding sections of journey’s have to be reported as well.

Reported sections and journeys of a given vehicle must not overlap

Automatic classification of goods

Page 18

Official statistics

Respondents

InnoRFDat-X

In compliance with national regulations all transported goods are classified by the NST/R

Hauliers and forwarding agents often use free text in their operative data. (i.e. 10 ldm bathtubs, granite, etc.)

Assignment to NST/R classification is conducted manually.

Development of a model for automatic classification according to NST/R categories for all free texts.

Experiences Assignment of goods to NST/R classification through

respondents leads to incoherent results (DE) Hauliers provide free texts, classification is done by NSO (NL)

Autoteile , Coils Stahl und Blech 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche max . 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche Überbreite 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Coils 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)kompl . Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Leitschienen 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)nahtlose Stahlrohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 12,2 m 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile lt . Beilage 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Sonderfahrt , Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stabstahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl ( S320GD+Z275MB) , 2 Coil 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Vg . 1/7 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahlbleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)

Page 19

Automatic classification - experiences

Expect improvements when trained using respondents texts

Correct categorization

Pervasive use of product codes in one case;

Transport provider has insufficient information („45 parcels“)

Insufficiently discriminating

Find a balance between „rote learning“ and the capability to correctly classify new descriptions

Imprecisions

Based on an algorithm trained on classification texts and applied to a sample of 1000 cargo descriptions from transport orders

Summary

Model deployed is capable to automatically classify cargo Sufficient precision achieved after training with respondent

specific datasets Less effort required from the respondents Quality improvements as a consequence of consistent

classification

Implementation aspects Improve „training phase“ using both descriptions and pre-

classifications of a sample of respondents Encourage consigners to provide more descriptive data of their

goods Provide functions to manually override misclassifications

Page 20

Page 21

Route & Event Detection

Protoytped process deployed to generate trips using GPS data

Objective

Position data file containing the geographic details of every tour where

a tour starts with the loading of an empty lorry

ends with the unloading of the last cargo

Two-Step Heuristics applied

First to detect all stops in the data file

Second to eliminate non-loading/unloading stops

Detect Stops

Speed gradient

Geographic change

Spatial distance

Classify StopsDistance to point of loading

Mandatory rest period

Distance to motorway services, etc.

•Stop-Position•Stop-Duration

•Timestamp•GPS-Coordinates

Loading stopResting stop

Loading point order info:(ZIP-Code)

Page 22

Route & Event Detection: Prototype

Implementation architecture

Implementation

GPS data input (via ad hoc XML file)

Consignment data input using interface definition

Linkage between GPS Data and consignment data via postcodes (geoname.org)

Result fed into the Position.Data interface definition

Event detection performance

Heuristics gave 100% Recall but only 28% Precision

Heuristics + Order details (ZIP-Codes) gave 100% Recall and 85% Precision

Page 25

Live Test Run Experiences / Feedback

Use of unfamiliar terms which did not always correspond with specific business practices

Usability problems with handling the web-service prototype (method to complex)

Interface was tested successfully and is usable

Inconsistencies and errors in the application were corrected

Journeys could be reconstructed based on position data and associated with certain orders

Automatic classification of transported goods possible

Page 26

TMS Software penetration in Austria

Page 26

32

15

14

40

12

14

Sauer Bespoke SWHypersoft, -sped No TMS Helpten COSwareC-Logistic Transporeon (?)

Methodology

Contacts to 55 larger companies, o.w. 29 responded and provided information

Significant proportion has outsourced transport

Respondents represented 7,3 % of all trucks registered in Austria (SGVS 2007, Q4)

5.230 / 72.000 = 7,3 %

Lessons learned

The collection and use of operational available at transport companies to produce road traffic statistics is feasible Standardized interfaces can be cost-effectively implemented

Research implications A large set of position data should reveal patterns of

loading/unloading locations to better determine between rest and load/unload stops

Respondent specific training sets is expected to increase the precision of goods classification

Generation of data from ERP/TMS-system would allow for continuous reporting of transport activities.

Recommended changes to the legal framework New regulations to mandate the collection and production

methodologies Development towards a mode-integrative approach Obligation/encouragement of NSI‘s to utilize available technologies for

the collection of raw data

The way forward

Solution is not restricted to the Austria territory. The choice of application architecture and scope has been guided by

the vision of a European rather than only a national application.

A Europe wide adoption of our methodology is expected to significantly increase both quality as well as the quantity of the data collected by the member states. applicability of transport statistics all over Europe will be enhanced comparability of transport statistics across countries will be improved.

Piloted process has shown the potential to substantially reduce the administrative burden on reporting companies

It is also expected to further raise the efficiency in the statistical production process leading to cost reductions and time savings.