Mass.gov: A Guide to Data-Informed Content OptimizationNathan James and Julia Gutierrez

@mediacurrent @MassGovDigital

| 2

Today’s Team

Nathan JamesAssociate Director,

Development

Julia GutierrezEngagement

Manager

@tnathanjames @jajahulia

| 3

About Mediacurrent

| 4

Who We Are

Mediacurrent is a full-service digital agency that

implements world class open source software

development, strategy, and design to achieve

defined goals for enterprise organizations

seeking a better return on investment.

| 5

About Massachusetts Digital Service

| 6

Who We Are

Massachusetts Digital Service is at the forefront of the

Massachusetts government's digital transformation. We take

an agile, constituent-centric approach to developing great

public services, and the products and services that we work on

touch the lives of millions of people in Massachusetts.

| 7

“Wow, that was just awesome”

| 8

Leverage the best technology and information to make people’s digital interactions with government: fast,

easy, and wicked awesome.

Massachusetts Digital Service Mission

| 9

Government services

| 10

Data matters

| 11

You can’t manage, much less improve, what you can’t measure.

Website usage

User verbatim feedback Application flow

Cross-domain Call center

In-person wait time

| 12

Today’s GoalsI. Demonstrate challenges and complexities of

empowering authors with data

II. Show some ways Mass.gov provides data to improve

content

III. Glimpse into the future at some possibilities

| 13

Not Your Run-of-the-mill Content Challenge

| 14

The scale of Mass.gov

9-11mUnique pageviews a

month

650Content authors and

editors

330Have logged in during

the last month

61,989Pieces of published

content

27Content types

3,616Services

| 15

After updating content, an author snoozes

content to revisit the score in 4 weeks.

| 16

| 17

Content performance experimentsTraffic

Findability

| 18

| 19

| 20

Data doesn’t flow: you have to make it move!

Custom Content API

ETL

ETL

ETL

ETL

Transformed

Data

Apache Superset

Score and Pageview Endpoint

FeedbackManager

API

| 21

Data doesn’t flow: you have to make it move!

Custom Content API

ETL

ETL

ETL

ETL

Transformed

Aggregated

Data

Apache Superset

Score and Pageview Endpoint

FeedbackManager

API

| 22

Data doesn’t flow: you have to make it move!

Custom Content API

ETL

ETL

ETL

ETL

Transformed

Aggregated

Data

Score and Pageview Endpoint

FeedbackManager

API

Apache Superset

| 23

Data doesn’t flow: you have to make it move!

Custom Content API

ETL

ETL

ETL

ETL

Transformed

Aggregated

Data

Score and Pageview Endpoint

FeedbackManager

API

Apache Superset

| 24

ServiceHow-to

Guide

List

Details Location

Service Family

| 25

Exploring Session Quality

Friction

&

Frustration

!!!

Lostness

&

Confusion

???

Credit

(or dings!)

Where It’s Due

| 26

Ejects per 1,000 Pageviews

- Return to Home

- Initiate Site Search

- Top Navigation

Interaction

Sample Indicators

Architecture

Movement

- Up

- Down

- Parallel

Action Rate

- Percentage of

sessions that

touched service

and “did the thing”

| 27

What we’ve learned and where we are going

| 28

Accuracy vs. scalability

Scalability

Acc

ura

cy

| 29

What’s wrong: our best indicators today

Scalability

Broken links

Acc

ura

cy

Grade level

No’s / 1,000

What’s wrong

| 30

What’s right: key performance indicators

Scalability

Acc

ura

cy

What’s right

| 31

What’s right: key performance indicators

Scalability

Acc

ura

cy

| 32

Measuring end-to-end service delivery

Mass.gov agency-configurable KPIsV1

V2

V3

Cross-domain agency-configurable KPIs

Omni-channel agency-configurable KPIs

| 33

Thanks!

| 34

Yes, We’re Hiring...

Booth #321

Booth #315

Friday, April 12, 2019

#DrupalContributions

Contribution Opportunities

9:00-18:00Room: 602

Mentored Contribution

First TimeContributor Workshop

9:00-12:00Room: 606

GeneralContributions

9:00-18:00Room: 6A

Join Us!

Thank you!

@mediacurrent

What did you think?Locate this session at the DrupalCon Seattle website:

http://seattle2019.drupal.org/schedule

Take the Survey!https://www.surveymonkey.com/r/DrupalConSeattle

@MassGovDigital