Medicare Siebel IP19 Upgrade Siebel CAB
Dave ThomasRobert PonderSubodh Patra
June 23, 2020
1
Department of Health and Human Services
• The United States government's principal agency for protecting the health of all Americans and providing essential human services, especially for those who are least able to help themselves
• Includes a number of very well known agencies such as CMS, FDA, CDC and many others
• Alex Azar is the Secretary of HHS
2
Centers for Medicare & Medicaid Services
• Administers the nation’s major healthcare programs including Medicare and Medicaid
• Seema Verma is the Administrator for CMS• Oversees 1T budget representing 26% of the Federal
Budget
3
National Government Services
• Experts in Federal Health Care• Partners with Federal Government to provide innovative healthcare
solutions• Providing solutions since 1966• Provides support for the CMS’ Siebel Fins Call Center solution
4
CMS Siebel Landscape• 10k+ Call Center users• Millions of external web users• Over 10 call centers across various states in the US• Nearly 40 Windows application servers• Oracle Supercluster database server• Fins and EAI inbound external facing custom application• Genesys CTI• Chat• Interfaces with a large number of mainframe systems• 24 x 7 uptime• Real disaster recovery (DR)
5
Upgrade Details• Expected 17 / 18 would take some time to stabilize and
mature and planned project accordingly• Development upgrades performed in PPS lab• Extensive Siebel Administrator and Developer training
provided• Overcame large learning curve and this took time• Also hit some defects which took time• 4.5 months actual upgrade project• Extensive LoadRunner testing
6
Go Live Weekend• Very fast go live timeline
7
CMS Siebel Upgrade History• Siebel 7.5 -> 7.7 in 2004• Siebel 7.7 -> 8.1 in 2009• Siebel 8.1 -> 8.1.1.11 in 2014• Siebel 8.1.1.11 -> IP 15 in 2015• IP 15 –> IP 19.11 Live Jan 2020• IP 19.11 -> 20.6 Planned 2020• Oracle Database 19c Planned 2020
8
How Did the Upgrade Go?• Very minor changes needed for application to work
correctly • Customized web templates and manifest entries caused
issues had to revert some to OOTB• SBL-UIF-00251: No content found in template!
• 17.x WE1252 non Unicode issues finally moved to UTF-8• GW cluster issues dual network card (NIC)• Many hours spent with SMC working out server
configuration• Found a few issues with Gateway cluster and Siebel
server operations
9
Open UI Changes• Next record / next record set instead of last / first record• Require search object selected before search can be
performed• Handful to 10 +/- changes we make to OOTB Open UI• Save pending changes and Find control
10
Upgrade Tuning• Extensive upgrade tuning performed as part of this
project• Tuning for 19.11 upgrade took just a few hours plus a
few more hours for additional improvements• Final upgrade ran fast at around 4.5 hours considering
1B s_evt_act records• For other smaller Oracle databases have the process
running in just 45 minutes!
11
Pushing The Limits with IP 19/20
• Full use of migration application• Zero down time deployment and rollback• Parallel development with integration workspaces
12
Siebel IP 20 Upgrade Good to Know• Enable Siebel crash handler sqlnet.ora• S_DIAG• SQL patch• Tomcat settings• Hardware requirements• zkui.exe• Siebel Developer and Admin training• Turn off transaction logging• Seamless repository upgrade
13
Siebel Upgrade Best Practices
• Don’t attempt in place install or upgrade• Upgrade practice runs• Don’t wait on load testing until near end of project• Build new production and test there sooner instead of later• Server manager command line only
14
Questions & Answers
15
Bonus Content Follows
16
IP 20 Cheat SheetIP 16 and Before IP 20
Full compile Full publish
Siebns.dat ZK version-2 folder
Generate and edit lbconfig ZK does automatically via service discovery
Enterprise and server configuration wizards SMC profile create and deploy
SRF Runtime repository
SWT files SWT database CLOB
SWE AI
Choice of web server Apache Tomcat
Tools check out / check in Workspace checkpoint, submit and deliver
Cluster GW for HA A/P using MSCS Supported with 18.5 and is A/A and is not MSCS
Upgrade Continuous innovation
Can configure pointing to any database Can only configure if pointing to IP 17/18/19/20 database
17
IP 20 Cheat Sheet - IIIP 16 and Before IP 20
Siebel repository Design time and run time repositories
Multiple repositories Only one design time repository allowed
Database ODBC / TNS Now have JDBC SMC entry too
Configure Siebel server before upgrade Can’t configure Siebel server until after the upgrade
Server name win193 fine for installation Server name must be fully qualitied win193.pps.com
eapps.cfg SMC, server.xml and .properties files
genbscript full publish or genbscript
Web template SWE tags <div> HTML elements
Old C++ applet and view graphical editors New .js applet and view graphical editors added 17.4
Backup siebns.dat Copy entire version-2 folder
Full compile for each language ENU and ESN Single full publish with all languages given ENU,ESN
18
IP 20 Cheat Sheet - IIIIP 16 and Before IP 20
Edit LOVs Open editable workspace then edit LOVs
Dev2Prod full development repository Export / import runtime repository via Migration app
Start gateway and then Siebel server Start gateway, Tomcat and then Siebel server
Create custom object manager Same
Server manager commands Same except backup
Siebel server performance Same or better
SSO implemented with parameters SSO implemented with reverse proxy server
SWE HTTP by default AI HTTPS by default
Dev2Prod SMC Migration
http://win187/fins_enu/ https://win186.pps.com:9021/siebel/app/fins/enu
19
IP 20 Cheat Sheet - IVIP 16 and Before IP 20
srvrmgr.exe -g WIN191 … srvrmgr.exe -g WIN191.pps.com:9014 …
WE8MSWIN1252 AL32UTF8
Server install/patch server and swe Server install/patch server both server and AI same media
Run upgrade with Server ODBC Create own ODBC and edit siebel.cfg
Upgrep, merge, upgphys Upgrep, merge, full publish, upgphys
C:\Siebel\SWEApp\PUBLIC\files\custom C:\Siebel\AI\applicationcontainer\webapps\siebel\files\custom
Tools and Dedicated Client Web Tools and Thin Client
Siebel Gateway Name Server Siebel Gateway Registry
20
Great Example Vendor / Client Cooperation
• Regular weekly meetings with Oracle• Oracle TAM engaged• Oracle Support and service requests used extensively• Oracle held accountable with SRs being closely monitored for
progress
21