the big migration: how cerner moved from confluence 3.5 to 5.8

TODD STUCK DIRECTOR • CERNER

@JAMESTSTUCK

The Big MigrationHow Cerner Moved From Confluence 3.5 to 5.8 DC

KATE LOMAX PROGRAM COORDINATOR • CERNER

@KATELOMAX7

Shark Bear

Who would win?

vs

Culture clash can influence change initiatives

Shark Bear

Are you prepared?

vs

InnovativeRegulatedTech Savvy High Availability

About Cerner

Healthcare IT company with an advanced user

population

Healthcare industry requires FDA and ISO

compliance

Hosting line of business expects 99.999%

availability

A core value to the company and key driver

for customization

The Aftermath

The Upgrade

The JourneyBackground, goals, and planning tips for a major upgrade

Tips and tricks for a successful upgrade and no user down time

How to get your community to settle into the “new normal”

The Agenda

uCern Wiki Timeline

3.1.2 7,500 Cerner Associates

3.5 14,000 Cerner Associates

5.8 DC 24,000 Cerner Associates

2009 2013 2016

Today's Statistics

600K Wiki Pages

160K Controlled Pages

27M Page Views per Year

130K Registered Users

22 3rd party Plugins

15 Cerner

Customizations

uCern Wiki (Confluence) Today

Customizations Goals for Success Upgrade Planning Tips

The Upgrade Journey

Custom branding and theme so that clients and associates have a consistent Cerner experience

Custom banner visible only to associates so they know clients can view this content

Custom published workflow banners and exports to meet regulatory and legal compliance

Shared identity, avatar, and profile across uCern domain

Custom wiki book builder to group wiki pages into “books”

for view, export, and print.

Required Domain Expertise to Edit

3.5 Challenges 5.8 Opportunities

Modernization and Support

Long Load Times and an Increase in

Down Times

Increase in Performance and

Stability

Simpler Editing Experience

Security Vulnerabilities


The Upgrade Journey

Repeatable Upgrade Process Leverage TAM and 3rd

Party Experts Reduce TCO via Automation

Wiki Upgrade Goals for Success


The Upgrade Journey

This screenshot is just an example of piece of content that could be placed here. If there is text to accompany the image in this area, it should be as concise as possible. Please delete this box and text.

Use Agile, Set Milestones and Track Progress

Use Champions for Early Testing

Have A Marketing and Learning Plan



UAT Testing Phase2 (2 weeks)Use Champions for Early Testing


UAT Testing Phase1 (2 weeks)



Use Champions for Early Testing


No Down Time Upgrade Upgrade Tech Tips Automation

The Upgrade

3.5.17 Clustered

3.5.17 Standalone

4.3.7 Standalone

5.8.13 Standalone

.5h 2.5h 5.5h .5h

9 hours with NO down time…

5.8.13 Data Center

Our Upgrade Path

Load Balancer1

App Node1

App Node2

App Node3

Oracle DB1

Oracle Node2

Sync

Load Balancer2

App Node3

App Node2

Oracle DB2

wiki.ucern.com wikiupgrade.ucern.com

3) Take Nodes 2/3 out of load

2) Stop database synchronization

1) Hide the edit button for read-only

4) Upgrade Node2 & DB2

5) Test using wikiupgrade DNS

Confluence

3.5.17

Confluence

5.8.136) Upgrade Node3 for HA

7) Change wiki DNS to go live

How to Achieve No Down Time


The Upgrade

Upgrade Tech Tips

Bypass Interim 4.x Markup to XHtml and

Plugin Migration

Upgrade Steps:

1) Bypass interim markup to xhtml conversion by removing wikiToXhtmlMigration task from upgradeSubSystemContext.xml prior to 3.5 to 4.3 upgrade.

2) Bypass interim plugin upgrades by inserting plugins binaries directly into the database prior to 4.3 to 5.8 upgrade. *

3) After 5.8 upgrade, manually run the markup to xhtml conversion by re-inserting wikiToXhtmlMigration task and launch force-upgrade.action.

Time Savings: ~4 hours

Upgrade Tech Tips

Skip Automatic Search Indexing

Upgrade Steps:

1) Skip automatic search indexing during interim hops by setting this VM argument when starting Confluence:

-Dconfluence.skip.reindex=true

2) After the 5.8 upgrade, use admin console to manually kickoff the search re-indexing during uptime.


Upgrade Tech Tips

Bypass 5.x Recovery File Generation


Upgrade Steps:

1) Bypass 5.8 recovery file generation by setting this VM argument when starting Confluence:

-Dconfluence.upgrade.recovery.file.enabled=false

Upgrade Tech Tips

Custom Scripts for User Macros, Workflow,

and Page Content

Custom Scripts:

1) User macros weren’t compatible. Manually converted user macros into 5.8 syntax in non-prod. Custom post upgrade job to update macros in the Bandana table.

2) Comala workflow definitions changed from global to space level. Custom post upgrade job using Comala workflow API to map spaces to new definitions.

3) Page content conversion caused wiki pages to render incorrectly. Custom post upgrade job to convert embedded PDF “!example.pdf!” to the viewFile macro.

Upgrade Tech Tips

Use Quick Restore DB Utility for Dev/Test

Flashback Steps on Oracle Database:

1) After creating a restore point “BEFORE_UPGRADE”, run the upgrade and restore back to previous version in less than 30 minutes.

> shutdown immediate;

> startup mount;

> FLASHBACK DATABASE TO RESTORE POINT "BEFORE_UPGRADE";

> shutdown immediate;

> startup mount;

> alter database open resetlogs;


The Upgrade

Automated Builds and Unit Testing using

Maven

Automated Certification Testing using Selenium

Automated Deployments using

Chef

Automation and DevOps

Next Steps

The Aftermath

Stakeholder Reactions

Issues

CollaborationHelp DocVideos Classes

Preparing the Community

Next Steps

The Aftermath


Issues

Where is the Wiki Markup?ASSOCIATE POPULATION, CERNER

“ ”

Next Steps

The Aftermath


Issues

Enhancements

Protecting the now and the next

Preparing for Disaster Recovery

Looking to the next

Thank you!

TODD STUCK DIRECTOR • CERNER

@JAMESTSTUCK

KATE LOMAX PROGRAM COORDINATOR • CERNER

@KATELOMAX7

the big migration: how cerner moved from confluence 3.5 to 5.8

Technology