![Page 1: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/1.jpg)
DG DIGIT / ISA Programme
D03.02.03 – Monthly Evaluation Report M3
Action 4.2.4 European Federated Interoperability Repository
Specific Contract 63 within Framework Contract DI/07171 – Lot 2
![Page 2: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/2.jpg)
D03.02.03 – Monthly Evaluation Report
05/01/2015 Page i
Document Metadata
Property Value
Release date 2014-12-08
Status For review
Version 0.03
Authors Joan Bremers – PwC EU Services
Thibaut Knop– PwC EU Services
Reviewed by Pieter Breyne – PwC EU Services
Approved by
Document History
Version Date Description Action
0.01 2014-12-01 Initial draft Creation
0.02 2014-12-08 Internal review Review
0.03 2014-12-08 Internal review Review
![Page 3: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/3.jpg)
D03.02.03 – Monthly Evaluation Report
05/01/2015 Page ii
Table of Contents
1 INTRODUCTION ................................................................................... 1
1.1 CONTEXT........................................................................................... 1 1.2 OBJECTIVE ......................................................................................... 1 1.3 SCOPE .............................................................................................. 1 1.4 STRUCTURE ........................................................................................ 2 1.5 GLOSSARY ......................................................................................... 2
2 WEB PERFORMANCE METRICS .............................................................. 3
2.1 UNIQUE VISITORS (‘USERS’ IN GA) ........................................................... 3 2.2 TOTAL VISITS ..................................................................................... 4 2.3 TOTAL VISITS PER LOCATION ................................................................... 4 2.4 TOTAL PAGE VIEWS - CATALOGUE .............................................................. 7 2.5 TOTAL PAGE VIEWS – MOST VIEWED INTEROPERABILITY SOLUTIONS ...................... 8 2.6 PAGE VIEWS PER VISIT .......................................................................... 10 2.7 AVERAGE VISIT DURATION ...................................................................... 11 2.8 PAGE VIEWS PER VISIT AND AVERAGE VISIT DURATION PER CHANNEL SOURCE .......... 11 2.9 BOUNCE RATE .................................................................................... 12 2.10 NEW VS. RETURNING VISITOR ................................................................. 13 2.11 METADATA PAGE VIEWS ......................................................................... 13 2.12 TOP REFERRING SEARCH TERMS ............................................................... 14 2.13 TOP SEARCH TERMS (INTERNAL SEARCH) ..................................................... 15 2.14 EXITS TO A WEBSITE OF A FEDERATION PARTNER ........................................... 17 2.15 DOWNLOADS ..................................................................................... 17
3 GOALS ................................................................................................ 19
3.1 GOAL 1: EXTERNAL SITE/SEARCH ENGINE -> CATALOGUE -> ASSET PAGE .............. 19 3.2 GOAL 2 – EXTERNAL SITE/SEARCH ENGINE -> CATALOGUE -> ASSET PAGE ->
DOWNLOAD/OUTBOUND LINK ............................................................................ 20 3.3 GOAL 3 - EXTERNAL SITE/SEARCH ENGINE -> ASSET PAGE ................................ 20 3.4 GOAL 4 – EXTERNAL SITE/SEARCH ENGINE -> ASSET PAGE -> DOWNLOAD/OUTBOUND
LINK 21 3.5 GOAL 5 – CATALOGUE -> ASSET PAGE ....................................................... 21 3.6 GOAL 6 – CATALOGUE -> ASSET PAGE -> DOWNLOAD/OUTBOUND LINK ............... 22 3.7 GOAL 7 – ASSET PAGE -> DOWNLOAD/OUTBOUND LINK .................................. 23
4 CONCLUSION ..................................................................................... 24
4.1 IMPORTANCE OF EFIR FOR THE PLATFORM ................................................... 24 4.2 REAL DIFFERENCE BETWEEN “VISITORS” AND “USERS” ..................................... 24 4.3 KEY QUESTIONS .................................................................................. 24
4.3.1 WHAT are the most popular paths to downloading an interoperability
solution (so-called asset release on Joinup)? ............................................. 24 4.3.2 WHICH assets, asset releases and repositories are the most popular
(most visited, with the highest number of downloads) and WHY? ................. 24 4.3.3 HOW long and HOW often do people browse the repository? .......... 25 4.3.4 WHAT is the geographic distribution of the users of EFIR?.............. 25 4.3.5 HOW do people experience the visit to the catalogue and the
catalogue itself? ..................................................................................... 25 4.3.6 HOW do people use the search functionalities of EFIR (focusing on
the advanced search)?............................................................................ 25 4.3.7 HOW do people experience the addition of content on EFIR
(interoperability solutions, projects and repositories)? ................................ 25
ANNEX 1 – IN-PAGE ANALYTICS ............................................................... 26
![Page 4: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/4.jpg)
D03.02.03 – Monthly Evaluation Report
05/01/2015 Page iii
List of Tables
Table 1 - Glossary ......................................................................................... 2
List of Figures
Figure 1 - Unique visitors ................................................................................ 3 Figure 2 - Detailed comparison of EFIR and EFIR Engaged Users ......................... 3 Figure 3 - Total visits / Sessions ...................................................................... 4 Figure 4 - Europe and World map .................................................................... 4 Figure 5 - Top 5 non-EU countries per visit ....................................................... 5 Figure 6 - Detailed report of performance per EU country ................................... 6 Figure 7 - Catalogue usage comparison ............................................................ 7 Figure 8 - Catalogue usage ............................................................................. 7 Figure 9 - Top 25 interoperability solutions ....................................................... 9 Figure 10 - Top 10 federated repositories ........................................................ 10 Figure 11 - Top 10 projects ............................................................................ 10 Figure 12 - Pages per visit comparison ............................................................ 10 Figure 13 - Average visit duration comparison .................................................. 11 Figure 14 - In-depth channel source analysis ................................................... 12 Figure 15 - Bounce rate ................................................................................. 12 Figure 16 - Percentage of new visits................................................................ 13 Figure 17 - New vs. Returning visitor average .................................................. 13 Figure 18 - Metadata page view comparison .................................................... 14 Figure 19 - Top 10 keywords .......................................................................... 14 Figure 20 - SEO results from Google Webmaster Tools ...................................... 15 Figure 21 - Internal search usage overview ...................................................... 16 Figure 22 - Top 10 internal search terms ......................................................... 16 Figure 23 - Advanced searches ....................................................................... 17 Figure 24 - Total number of Events (Downloads & Outbound links) ..................... 17 Figure 25 - Top 10 Outbound Links ................................................................. 17 Figure 26 - Top 25 downloads ........................................................................ 18 Figure 27 - EFIR Goal 1 ................................................................................. 19 Figure 28 - EFIR Goal 2 ................................................................................. 20 Figure 29 - EFIR Goal 3 ................................................................................. 20 Figure 30 - EFIR Goal 4 ................................................................................. 21 Figure 31 - EFIR Goal 5 ................................................................................. 22 Figure 32 - EFIR Goal 6 ................................................................................. 22 Figure 33 - EFIR Goal 7 ................................................................................. 23 Figure 34 - In-page analytics for Catalogue of Interoperability Solutions.............. 26
![Page 5: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/5.jpg)
D03.02.03 – Monthly Evaluation Report
Page 1
1 INTRODUCTION
1.1 Context
The report is prepared in the context of Action 4.2.4 European Federated
Interoperability Repository. A key to federating further national and local repositories
or standardisation bodies into the EFIR will be their representation on the Joinup
platform. The solutions will need to be described in a high quality, informative manner
and be easily re-usable.
Therefore, a formal evaluation of the performance of the Joinup platform will be
performed on a monthly basis, focusing on the EFIR activities. During the overall
project, from September till January 2014, a report will be published each month,
describing the performance of EFIR on Joinup during the reporting month, based on
the gathered metrics defined in Task 03. This report covers the month of November
2014.
1.2 Objective
The objective of these evaluation reports is to provide the necessary data-based
evidence to the stakeholders of EFIR – both the interoperability solution providers
and re-users – upon which the value and future sustainability of the Repository can
be assessed.
To do so, the performance evaluation provided in the context of this project aims at
answering the following key questions:
1. WHAT are the most popular paths to downloading an interoperability solution
(so-called asset release on Joinup)?
2. WHICH interoperability solutions and repositories are the most popular (most
visited, with the highest number of downloads) and WHY?
3. HOW long and HOW often do people browse the repository?
4. WHAT is the geographic distribution of the users of EFIR?
5. HOW do people experience the visit to the catalogue and the catalogue itself?
6. HOW do people use the search functionalities of EFIR (focusing on the
advanced search)?
7. HOW do people experience the addition of content on EFIR (interoperability
solutions, projects and repositories)?
1.3 Scope
The scope of this Evaluation report is the evaluation of the EFIR on the Joinup
platform. The other functionalities of the Joinup platform are out of scope for this
evaluation. Assets that do not qualify as interoperability solutions are also out of
scope.
![Page 6: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/6.jpg)
D03.02.03 – Monthly Evaluation Report
Page 2
1.4 Structure
The remainder of this deliverable is structured as follows:
In chapter 2 we present the web performance metrics;
In chapter 3 we present the results of the pre-defined goals; and
In chapter 4 we provide a conclusion on the results.
1.5 Glossary
Throughout this report, we will use the following terms defined by Google Analytics:
Table 1 - Glossary
Session A session (previously a visit) consists of a series of pageviews that a
single visitor makes during a period of activity. A session/visit ends
after the visitor closes the browser, clears cookies, or is inactive for
30 minutes.
User A user (previously a unique visitor) is someone making a unique visit.
Unique
visit
A unique visit is defined per browser and per computer. Technically,
a unique visit is defined by Google Analytics using the cookie _utma.
Following this definition:
Closing or changing the browser/computer increases the number of
unique visits
Clearing the cookies increases the number of unique visits
Search
exit
The number of searches made immediately before leaving the site.
This means the user ran a search, but did not click on any of the
search results, and left the website instead.
Federated
repository
A federated repository is an existing repository of interoperability
solutions and open-source software that exchanges information
about its interoperability solutions and open-source software projects
with the Joinup platform, creating a pan-European federation of
repositories of interoperability solutions. In consequence, the Joinup
platform provides “federated” access to these repositories from a
single point of access, giving more visibility to the external repository
and its solutions.
Project In the context of the Joinup platform, a project is interoperability
solution or open-source software that is hosted on Joinup (not
federated through a repository). Joinup offers a special kind of
community around open-source software or interoperability solution.
It provides a set of communication (blogs, wikis, forums, documents)
and collaboration tools for collaborative development (source code
repository, issue tracker, release management tool).
Google Analytics has recently modified the terminology related to visits. More
specifically, old-called ‘visits’ are now called sessions. Besides, old-called ‘unique
visitors’ are now called users. Those terms will be equally used throughout the
monthly evaluation report.
![Page 7: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/7.jpg)
D03.02.03 – Monthly Evaluation Report
Page 3
2 WEB PERFORMANCE METRICS
In the context of this report, all metrics below have been gathered for the period
from November 1st, till November 30th.
In order to keep the metrics as accurate as possible, we excluded all data coming
from Zaventem, so as not to count the visits and page views of the PwC consultants
working on the Joinup platform.
2.1 Unique visitors (‘Users’ in GA)
Figure 1 presents the monthly comparison (day by day) of the number of unique
visitors on the segments EFIR and EFIR Engaged Users for the month of November.
Figure 1 - Unique visitors
As shown in Figure 1, engaged users represent only about 17.2% of the visitors on
EFIR, which is a decrease with regards to last month (0.8%).
On Sunday 9 November, Joinup reached a peak of 872 unique visitors, which
represents roughly an increase of 79% regarding normal weekdays. However, it
appears that the traffic generated on that day led the users to spam content mainly
about retransmission of US Football games.
The EFIR segment represents 34.51% of all traffic on Joinup, while the EFIR Engaged
User represents 5.95% of all traffic on Joinup. A detailed comparison of both
segments is presented in Figure 2.
Figure 2 - Detailed comparison of EFIR and EFIR Engaged Users
![Page 8: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/8.jpg)
D03.02.03 – Monthly Evaluation Report
Page 4
2.2 Total Visits
Figure 3 presents the November report (day by day) of visits (so-called ‘Sessions’ in
GA) on the EFIR segment.
Figure 3 - Total visits / Sessions
Figure 3 shows that the total number of visits including EFIR pages is quite stable
over the month of November 2014: about 550 visits per day on weekdays and 150
per day on weekends, except for the 9th of November during which we reached a
peak of 996 sessions.
2.3 Total Visits per location
In this subsection, we present the monthly reporting on the location from which visits
to the EFIR segment came from. In Figure 4 you will find the maps of the locations
of visits on EFIR. The darkest blue represents a large number of visits; the lightest
blue represents a small number of visits. Figures 4, 5 and 6 represent statistics over
the month of November.
Figure 4 - Europe and World map
In order to get a view on the usage of EFIR outside of the border of the European
Union, we present in Figure 5 a detailed report of the top 5 non-EU countries in
term of visit to the EFIR segment.
![Page 9: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/9.jpg)
D03.02.03 – Monthly Evaluation Report
Page 5
Figure 5 - Top 5 non-EU countries per visit
In order to get a view on the usage of the EFIR platform within the European Union,
you will find in Figure 6 detailed reporting on the performance of EFIR in the
different European countries.
![Page 10: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/10.jpg)
D03.02.03 – Monthly Evaluation Report
Page 6
Figure 6 - Detailed report of performance per EU country
![Page 11: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/11.jpg)
D03.02.03 – Monthly Evaluation Report
Page 7
2.4 Total page views - Catalogue
In Figure 7 we present a view on how many times each of the catalogue pages has
been consulted during the month of November 2014, for the EFIR segment (in blue)
compared with the EFIR Engaged User segment (in green).
Figure 7 - Catalogue usage comparison
We observed that despite the large difference in overall number of visitors between
EFIR and EFIR Engaged Users (presented in subsection 2.1), the difference in the
number of catalogue page views is significantly less important. This means that the
catalogue pages are heavily used by Engaged Users.
In Figure 8 you will find the daily comparison of aggregated catalogues page views.
Figure 8 - Catalogue usage
Detailed in-Page analytics of the Catalogue of Interoperability Solutions (Figure 34)
are presented in Annex 1.
![Page 12: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/12.jpg)
D03.02.03 – Monthly Evaluation Report
Page 8
2.5 Total page views – most viewed interoperability solutions
Below we present the most popular interoperability solutions (Figure 9), federated
repositories (Figure 10), and projects (Figure 11), for the EFIR segment.
![Page 13: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/13.jpg)
D03.02.03 – Monthly Evaluation Report
Page 9
Figure 9 - Top 25 interoperability solutions
![Page 14: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/14.jpg)
D03.02.03 – Monthly Evaluation Report
Page 10
Figure 10 - Top 10 federated repositories
Figure 11 - Top 10 projects
2.6 Page views per visit
In Figure 12 we present the monthly comparison (day by day) of the depth of visits
(number of pages viewed per visit) for each segment.
Figure 12 - Pages per visit comparison
Over the month of November, the number of page viewed per visit stay more or less
constant, for both the EFIR and the engaged users segment, with respectively 4,32
et 15,73 pages viewed per visit, which represents a slight increase with regards to
![Page 15: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/15.jpg)
D03.02.03 – Monthly Evaluation Report
Page 11
the previous months. The bounce rate for the EFIR segment is 46.48%, meaning that
roughly the half of the users from the EFIR segment leave the website after the first
page.
2.7 Average visit duration
In Figure 13 we present a monthly comparison (daily numbers) of the length of the
visits for the EFIR and the EFIR Engaged Users segments.
Figure 13 - Average visit duration comparison
Over the month of November, the average visit duration has remained more or less
constant, with 4:40 minutes for overall EFIR users and 19:59 minutes for Engaged
Users. It represents an increase of 53’ and 1:45 respectively with regards to the
month of October.
2.8 Page views per visit and average visit duration per channel source
In Figure 14 we present the number of visits, the number of pages viewed per visit
and the average duration of a visit, grouped by channel source (the source of the
visit) for the 5 main sources of traffic for EFIR.
![Page 16: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/16.jpg)
D03.02.03 – Monthly Evaluation Report
Page 12
Figure 14 - In-depth channel source analysis
The channel source (direct) means that the visitor directly entered Joinup’s URL in
the browser.
Just as it was the case until July, a large amount of traffic is coming from agid.gov.it,
which also partly explains the high traffic received from Italy.
2.9 Bounce rate
Figure 15 presents an analysis of the bouncing rate on the EFIR segment for the
month of November. By bouncing we understand visitors coming on an EFIR page
and leaving the website without doing anything else.
Figure 15 - Bounce rate
![Page 17: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/17.jpg)
D03.02.03 – Monthly Evaluation Report
Page 13
The bounce rate has remained constant around 46.48%, which is within the range of
the industry average for a content website (40-60%).
2.10 New vs. Returning visitor
In Figure 16 we present the evolution of the percentage of new visits for the month
of November.
Figure 16 - Percentage of new visits
As shown in the above figures, the percentage of new visits remains fairly constant
over the period and is only slightly higher during weekends.
Figure 17 presents the monthly average of new versus returning visitors.
Figure 17 - New vs. Returning visitor average
2.11 Metadata page views
Figure 18 presents a comparison of the number of page view for the metadata pages
of EFIR (metadata export functionality).
![Page 18: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/18.jpg)
D03.02.03 – Monthly Evaluation Report
Page 14
Figure 18 - Metadata page view comparison
Over the period of November 2014, almost no visitor has taken advantage of this
service.
2.12 Top referring search terms
In Figure 19 we present the top 10 most popular keywords driving traffic to EFIR for
the month of November.
Figure 19 - Top 10 keywords
(Not provided) means that Google Analytics was not able to gather the queried
keywords from the search engine. In most cases this is due to the cookie policy of
the visitor or its browser.
In order to provide a more detailed view of the search keywords, we set up the
Google Webmaster Tools for Joinup and started gathering Google Search data, which
is presented in Figure 20.
![Page 19: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/19.jpg)
D03.02.03 – Monthly Evaluation Report
Page 15
Figure 20 - SEO results from Google Webmaster Tools
‘Impressions’ are the number of times Joinup appeared in the results of the query in
Google Search (for example Joinup appeared 2,500 times in the results on a Google
search due to people searching semic on Google). One important remark is that one
search can often result in several impressions (if I search “joinup” the first 3 results
lead to Joinup and thus will be counted as 3 impressions).
‘Average position’ represents the position in which Joinup appeared in the results of
the query. One important note is that we are tracking https://joinup.europa.eu, and
as a result Google will not count the average position if the result is
http://joinup.europa.eu (most of the time the ‘https’ is first, or second just behind
the ‘http’).
‘CTR’ is the ‘Click Through Rate’, meaning the percentage of people who clicked on
an impression and accessed Joinup.
It is important to note that we cannot distinguish EFIR and non-EFIR visits in the SEO
results, and that these results only take in consideration Google searches. It is
nonetheless a good way of representing the most popular keywords leading to Joinup
from a public search engine.
2.13 Top search terms (internal search)
In Figure 21 we present an overview of the internal search engine usage over the
month of November. It shows that on average 13.14% of EFIR visitors are using the
internal search tool.
![Page 20: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/20.jpg)
D03.02.03 – Monthly Evaluation Report
Page 16
Figure 21 - Internal search usage overview
In Figure 22 we present detailed statistics on the top 10 internal search terms.
Figure 22 - Top 10 internal search terms
Due to the limitations of Google Analytics on internal searches on Joinup, the
following should be noted:
- “bs_current_version:true” is the default search of the catalogue page, to
display the top results (most recent);
- “sm_facterbuilder_facet_node_type:“facet_node_type:facet_24” is the
search filter activated when a user limits the search to Interoperability
Solutions;
- Tid:16106 is the search filter activated when a user limits the search to Tools;
- Tid:16104 is the search filter activated when a user limits the search to
Framework;
- Ss_asset_language:en is the search activated when a user limits the search
to language English.
Figure 23 presents the advanced searches for the reporting period. Very few visitors
(0.19%) used the advanced search capabilities.
![Page 21: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/21.jpg)
D03.02.03 – Monthly Evaluation Report
Page 17
Figure 23 - Advanced searches
2.14 Exits to a website of a federation partner
Figure 24 presents the total number of events during the month of November. We
defined downloads and clicks on outbound links (hyperlinks to partner websites) as
events in Google Analytics.
Note that due to issue ISAICP-1720, most of outbound links to federation partners
websites are not tracked. However, this issue is intended to be fix by Mid-December.
Figure 24 - Total number of Events (Downloads & Outbound links)
In Figure 25 we present the 10 most popular destinations for outbound links on
Joinup.
Figure 25 - Top 10 Outbound Links
In November, EFIR referred 2,646 visits to the websites of partners and publishers,
thus providing added value and incentive for these publishers to share their solutions
on Joinup. Note that the peak of events on the 9th of November is explained by an
increasing in the number of clicks on spam outbound links that were inserted by spam
users. The content type where those links appeared have been deleted.
2.15 Downloads
Figure 26 presents the 25 most popular downloads (interoperability solutions) of
the month of November.
![Page 22: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/22.jpg)
D03.02.03 – Monthly Evaluation Report
Page 18
Figure 26 - Top 25 downloads
![Page 23: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/23.jpg)
D03.02.03 – Monthly Evaluation Report
Page 19
3 GOALS
In the Evaluation Plan we identified 7 behaviour goals in order to get an answer on
some of the evaluation questions.
It is important to note the following about the goals denomination:
- By catalogue, we understand any of the 3 catalogue pages:
o https://joinup.ec.europa.eu/interoperability/search?solr_search=1&fil
ters=%21bs_current_version%3Afalse
o https://joinup.ec.europa.eu/catalogue/repository
o https://joinup.ec.europa.eu/project/all
- By asset page, we understand any of the following page categories:
o /asset/…
o /asset_release/…
o /software/…
o /project/…
3.1 Goal 1: external site/search engine -> catalogue -> asset page
In Figure 27 we present the metrics about users who complete the pre-defined goal:
the user arrives on Joinup on a catalogue page via an external site or search engine,
and goes from that catalogue page to consulting an asset page (after searching or
browsing through the assets on the catalogue).
Figure 27 - EFIR Goal 1
The number of visitors demonstrating the behaviour of goal 1 is quite limited.
However, considering the average number of pages viewed per visit (13.51) and the
average duration of each visit (00:14:42), we can conclude that these visitors have
really engaged with EFIR. With regards to the past months, the average session
duration for the users completing the Goal 1 is 3 minutes longer, and 2 pages are
additionally visited in average.
![Page 24: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/24.jpg)
D03.02.03 – Monthly Evaluation Report
Page 20
3.2 Goal 2 – external site/search engine -> catalogue -> asset page ->
Download/Outbound link
In Figure 28 we present the metrics about users who complete the pre-defined goal:
the user arrives on Joinup via an external site or search engine and lands directly on
a catalogue page, then browses to an asset page and downloads an interoperability
solution or clicks on a link towards a publisher’s website.
Figure 28 - EFIR Goal 2
As this goal is very restrictive, (the user has to land on a catalogue page), the number
of visitors is very limited. This is due to the fact that most interoperability solution
pages are easy to reach directly from an external search engine (as we will see in
Goal 3).
We note that very few visitors (18) have accomplished that goal in the past month,
which represents 50% more than in October.
3.3 Goal 3 - external site/search engine -> asset page
In Figure 29 we present the metrics about users who complete the pre-defined goal:
the user arrives on Joinup via an external site or search engine and lands directly on
an asset page.
Figure 29 - EFIR Goal 3
![Page 25: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/25.jpg)
D03.02.03 – Monthly Evaluation Report
Page 21
When we compare goal 1 and 3 regarding the number of visits (140 vs. 10,779) and
unique visitors (117 vs. 8,733) for the month of November, we can conclude that
most visitors are not arriving on Joinup through the catalogue page, but rather arrive
directly on an asset page. Based on this we can conclude that:
- This means that the asset pages are well described and thus easily findable
through a standard search engine (like Google or Bing);
- The fact that EFIR’s interoperability solutions are easily findable on standard
search engine means that they are visible on these standard search engines.
This means added promotion for our publishers, thanks to their good
description on Joinup and the platform’s high ranking in standard search
engines.
3.4 Goal 4 – external site/search engine -> asset page -> Download/Outbound link
In Figure 30 we present the metrics about users who complete the pre-defined goal:
the user arrives on Joinup via an external site or search engine, lands directly on an
asset page, and downloads an asset or software or clicks through to a link towards a
publisher’s website.
Figure 30 - EFIR Goal 4
As we can see, there is a large peak on the 9th of November. As seen previously, the
peak is explained by the important number of clicks on spam outbound links.
3.5 Goal 5 – catalogue -> asset page
In Figure 31 we present the metrics about users who complete the pre-defined goal:
the user arrives on a catalogue page (from anywhere, search engine, other sections
of Joinup,) and then goes directly to an asset page (after searching or browsing
through the assets on the catalogue).
![Page 26: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/26.jpg)
D03.02.03 – Monthly Evaluation Report
Page 22
Figure 31 - EFIR Goal 5
When we compare goal 1 and 5, we can conclude that most visitors using the
catalogue are doing so because they were already on Joinup (i.e. they did not land
on the catalogue, most likely they landed on a news item, the homepage, or a
community and then browsed internally to a catalogue page).
3.6 Goal 6 – catalogue -> asset page -> Download/Outbound link
In Figure 32 we present the metrics about users who complete the pre-defined goal:
the user arrives on a catalogue page (from anywhere, search engine, other sections
of Joinup …), then goes directly to an asset page (after searching or browsing through
the assets on the catalogue), and downloads an asset or software or clicks through
to a link of a publisher’s website.
Figure 32 - EFIR Goal 6
By comparing goal 2 and 6, we note that most visitors access the catalogue pages
from other locations on Joinup and not directly from an external source. This indicates
a low presence of the catalogue pages in public search engine.
By basing ourselves on the fact that the numbers of goal 7 and goal 3 are much
higher than goal 6, we can definitely conclude that asset pages are appearing more
often that catalogue pages while searching on public search engines.
![Page 27: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/27.jpg)
D03.02.03 – Monthly Evaluation Report
Page 23
3.7 Goal 7 – Asset page -> Download/Outbound link
In Figure 33 we present the metrics about users who complete the pre-defined goal:
the user arrives on an asset page (from anywhere, search engine, catalogue, other
sections of Joinup …), and downloads an asset or software or clicks through to a link
towards a publisher’s website.
Figure 33 - EFIR Goal 7
By comparing goal 6 and goal 7, we note that most visitors do not use the catalogue
in order to access the asset or software they wish to download.
![Page 28: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/28.jpg)
D03.02.03 – Monthly Evaluation Report
Page 24
4 CONCLUSION
4.1 Importance of EFIR for Joinup
Based on the gathered metrics, we can conclude the following about the importance
of EFIR for Joinup:
Overall EFIR represents 34,5% of all traffic on Joinup, meaning that the
published interoperability solutions are highly visible on Joinup;
The EFIR pages also have a significantly lower bounce rate than the average
Joinup pages (46.48% vs. 55.46%, taking in consideration that EFIR pages
are included in the general Joinup metric); and
EFIR’s visits in November were coming for about 68.09% from new visitors,
meaning that EFIR is successful in attracting new visitors, but is not really
successful in retaining and engaging users.
4.2 Real difference between “visitors” and “users”
We can distinguish a difference between EFIR’s “visitors” and EFIR’s “users”, as was
shown by the tremendous gap between the segment EFIR Engaged Users and the
generic EFIR segment:
- While representing only 17% of the EFIR visitors, engaged users represent
about 62% of all EFIR page views;
- Average visit time of Engaged Users is 19:59 minutes, compared to 4:14
minutes for the generic segment, and only 48 seconds after removing the
Engaged Users from the generic segment;
- Engaged users will browse on average 15.73 pages, while visitors in the
generic segment (after exclusion of the engaged users) will only browse an
average of 1.92 pages.
2255 visitors were engaged users in Novembers, and 47.94% of those were new
visitors, meaning that EFIR is successful in engaging new users.
4.3 Key questions
4.3.1 WHAT are the most popular paths to downloading an interoperability solution
(so-called asset release on Joinup)?
Most visitors arrive directly on an interoperability solution from a search engine,
thanks to their good ranking in the standard search engines. The catalogue pages
are the least preferred options to arrive on an interoperability solution.
4.3.2 WHICH interoperability solutions and repositories are the most popular (most
visited, with the highest number of downloads) and WHY?
The most popular assets, asset releases and repositories are presented in sub section
2.5.
![Page 29: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/29.jpg)
D03.02.03 – Monthly Evaluation Report
Page 25
Over the month of November, the most popular for consultation were:
- Interoperability Solutions: SD-DSS
- Federated repository: UN/CEFACT
- Project: circabc
Over the month of November, the most popular for downloads were:
- EUPL
In November, EFIR referred most traffic to:
- https://static.arhs-developments.com/dgmarkt-dss-rEEwAj4c/DSS-4.2.0/
4.3.3 HOW long and HOW often do people browse the repository?
On average the visit lasts 4:14 minutes and 4.32 pages are viewed per visit. However
as explained sub section 4.2, there is a large gap between visitors and users.
4.3.4 WHAT is the geographic distribution of the users of EFIR?
Out of the 13,086 visits to the repository in November, 8,815 were coming from a
Member State (67.3%). A detailed representation of the geographic distribution can
be found in Figure 6 - Detailed report of performance per EU country for the detail
for each Member State and in Figure 5 - Top 5 non-EU countries per visit for the 5
non-member state countries generating the most visits to the repository.
4.3.5 HOW do people experience the visit to the catalogue and the catalogue itself?
The catalogue is mostly used by recurring users (47%), and only a very limited
number of visitors actually land on the catalogue while visiting the repository. Visitors
going through the catalogue are also staying longer on the repository (19.59 minutes
on average) and browsing more pages (15.73).
4.3.6 HOW do people use the search functionalities of EFIR (focusing on the advanced
search)?
Most searches realized on Joinup are made by EFIR visitors. However this still only
represents about 13.14 % of all visits. This can be partly explained by the fact that
most visitors arrive directly on the asset or software page that they wish to visit, and
thus did not need to use the internal search functionality.
A very limited number of visitors used the advanced search capabilities.
4.3.7 HOW do people experience the addition of content on EFIR (interoperability
solutions, projects and repositories)?
To provide an answer to this question statistics about /node/add/* pages must be
collected. However those pages are currently not tracked by Google Analytics. In
consequence, no data can be gathered.
![Page 30: D03.02.03 – Monthly Evaluation Report M3 · 2017-10-03 · D03.02.03 – Monthly Evaluation Report Page 2 1.4 Structure The remainder of this deliverable is structured as follows:](https://reader034.vdocuments.us/reader034/viewer/2022042417/5f32b443099000637a5dbf0f/html5/thumbnails/30.jpg)
D03.02.03 – Monthly Evaluation Report
Page 26
ANNEX 1 – IN-PAGE ANALYTICS
Figure 34 - In-page analytics for Catalogue of Interoperability Solutions