tfs at siemens healthcare - microsoft 1/track 3/tfs... · tfs at siemens healthcare best practices...
TRANSCRIPT
Unrestricted © Siemens AG 2014 All rights reserved. Answers for life.
TFS at Siemens Healthcare
Best practices and learning's ALM Days 2014 – Thomas Dasch & André Mainka
Page 2 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 2 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
syngo.via – Get the full picture.
TFS 2012 migration
About us
Large-scale considerations
Our experience
Agenda
Operate a huge build farm
Page 3 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 3 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
About us
Page 4 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 4 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
The syngo portfolio
Seamless integration across the entire imaging workflow
1 syngo.via can be used as a stand-alone
device or together with a variety of
syngo.via-based software options which are
medical devices in their own right
2 syngo.share is a medical device of ITH
icoserve technology for healthcare GmbH,
Innsbruck. syngo.share is currently
available only in various European
countries.
Page 5 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 5 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
syngo.via – Get the full picture.
The syngo.via1 3D routine and
advanced reading solution helps to
accelerate workflows across all
modalities.
It is efficient.
It helps to accelerate radiological
workflows and save valuable time.
It is flexible.
You can customize its applications
and layouts in line with your needs.
It is intelligent.
The software guides you through
the entire workflow.
syngo.via General Engine1
1 syngo.via can be used as a stand-alone device or together with a variety of syngo.via-based software options, which are medical devices in their own right. This product is not yet
commercially available in all countries. Due to regulatory reasons it’s future availability cannot be guaranteed. Please contact your local Siemens organization for further information.
Page 6 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 6 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our constraints
Large scale projects have to manage various challenges
Medial devices
Regulated environment
Global development
footprint
Thousands of active users
Million lines of code
Multiple product lines
Terabytes of data
Massive number of
builds
Myriad of tests
Page 7 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 7 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Drivers in large scale ALM
Performance Serviceability Scalability
Page 8 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 8 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our TFS history
Stepwise completing the ALM approach
2010 • TFS 2010
Infrastructure
2011 • Project
Management
• Backlog Planing
2012 • Build & Version
Control
• Automated Test Execution
2013 • Defect
Management in TFS
• TFS 2012
2014 • TFS 2013
• Test Management in TFS
Page 9 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 9 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
TFS 2012 migration
Page 10 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 10 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
TFS upgrade
Rollout in a large scale environment is challenging
Plan
Prepare
Test
Migrate
Monitor
Tradeoff
update cadence
vs. effort
Page 11 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 11 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
TFS 2010 to TFS 2012 upgrade
Upgrade is more than just running the installer
Precondition: TFS 2012 Update 2 made build 2010
infrastructure compatible
Prepare your migration,
e.g. by reducing the amount of data to be migrated
• Outdated build results
• Obsolete shelvesets
• Unnecessary labels
• Superfluous workspaces
• Locked files
Is your old build infrastructure compatible with the
new TFS backend?
Execute a bunch of scout tests after the migration
Page 12 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 12 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
TFS 2010 to TFS 2012 upgrade
Don‘t underestimate data migration
• Upgrade process longer than expected
• Huge amount of data increases the data migration
• Warehouse is recreated from scratch
• Upgrade schedules additional jobs
• Still a few bugs found
• QueryMerges API returns wrong results
• SymbolServer deletion does not work
• JobAgent crashes under certain conditions
• Builddefinitions compatibility with Visual Studio
versions
Page 13 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 13 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Microsoft communication strategy
Advertising surface vs. detailed change knowledge
New features are highly advertised and widely
discussed
More crucial for installed base is detailed information
about
• Stability fixes
• Performance fixes
• Scalability fixes
• Compatibility fixes
• Customization impact
http://blogs.msdn.com/b/visualstudioalm/archive/2011/09/20/
visual-studio-team-foundation-server-11-developer-preview-
what-s-new-for-team-foundation-server.aspx
http://blogs.msdn.com/b/bharry/archive/2012/03/23/keeping-
your-tfs-2010-up-to-date-as-of-march-2012.aspx
Page 14 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 14 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Large scale considerations
Page 15 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 15 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
• Build and test infrastructure on
site
• Version control proxy
Physical network topology
Thousands of users require an advanced configuration
• Load balanced application tiers
• Dedicated hardware for
reporting and analysis services
• SQL cluster with high
availability configuration
Page 16 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 16 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Server side customizations
Various extension points can be used to adapt the feature set
• German/European industrial relations law (BetrVG)
• Make reports anonymized
• Regulatory requires traceability
• Prevent to edit user display name
• Disable version control adapter
• Data warehouse crashes if amount of data is
extremely high
• Backend automation/aggregation Service
• E.g. accumulate story points into parent work
item
Page 17 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 17 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
SQL administration
SQL server is the mission critical resource
• Ensure that your backup procedure is operational
• Working with your amount of data
• Test the disaster: Restore backup
• Ensure that SQL configuration is optimized for
performance
• Monitor log data and free space
• Monitor Temp DB free space
• Monitor table sizes & row counts
http://blogs.msdn.com/b/granth/archive/2013/10/08/what-
does-a-well-maintained-team-foundation-server-look-
like.aspx
Page 18 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 18 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
TFS operations tasks
Regular maintenance keeps the system operational
• Deleted builds are just marked as deleted and need
to be destroyed
• Clean test attachments
• Delete shelvesets which remain by failed gated
checkin builds
• Delete unused workspaces
Page 19 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 19 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Operate a huge build farm
Page 20 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 20 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Real-time overview of running builds
Build explorer fulfills just a subset of the demands
View currently running builds
• Monitor queue time
• How many builds are enqueued?
• How long are builds already running?
Evaluate the history of a single build definition
• Check completed builds for irregularities
• How long does the build take in average?
• How often gets this build executed?
The overview gives you a first impression of
what’s going on in your system.
Page 21 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 21 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Keep track of different build definitions
Aggregate the latest results on a dashboard
No „out of the box“ feature for
• Monitoring staged integration gates
• Keeping track of multiple branches
Recommendations
• Aggregate the most important details
• Audio feedback to increase awareness
• Have an overview of all your builds
• Keep everything in view, focus on the essential
• Display the status of the last build on a dashboard
Page 22 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 22 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Analyze build details
Know exactly where the time is spent
• Fast real-time detail view for a single build
• Immediately see the context if the build has failed
• Which part of the build takes most time?
• Which area requires optimization?
• Does the load balancing of my tests work?
Customized from
http://buildexplorer.codeplex.com/
First focus on bottlenecks, before starting with
fine-tuning
Page 23 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 23 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Build optimization
Various areas impact quality and speed
Infrastructure (Server, Network, Agents, Controller)
Teambuild Test Framework /
Test Environment
Product and Tests
Page 24 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 24 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Hardware optimizations could be cheap
High performance setup requires fine tuning
Measure, measure, measure…
Identify physical bottlenecks:
• CPU, RAM, HDD, Network
Fine tune configurations
• Power options: high performance
• Disable Windows search indexing
• Exclude \builds, \symbols, \drop, team explorer
cache from anti-virus real time scanning
• Clean up the \builds and TEMP folders on build
agents to remove old workspaces
Page 25 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 25 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Compile time is a significant part of build cycle time
Fast Compilation is a key to optimize performance
Compilation should fully leverage all CPU cores
Does compile use an incremental approach?
Advanced optimizations
• Reduce dependency chains
• C++ compile on multiple processors
• C++ incremental compile
• C++ precompiled Header
• Optimize critical path
Learn how MsBuild works
Page 26 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 26 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Find a balance between testcoverage and execution time
Test execution does not scale on a single machine
• Fast feedback of Integration tests is mandatory
• Scale out especially for UI based or long running
tests
• Leverage TestController infrastructure (without lab)
to distribute tests across multiple test agents (aka
remote execution)
• Invoke directly from teambuild
http://msdn.microsoft.com/en-us/library/dd728093.aspx
Reuse/integrate existing infrastructure components
Page 27 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 27 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Fine-tune the build process
Improve and optimize – towards release pipeline
Fine-tune build definition
• Incremental get
• Label sources
• Drop build output
• Index sources
Customize build process for advanced configuration
• Dump important settings to buildlog
• Fail fast
• Remote test execution
• Drop log files only
• Reduce „noise“ in build logs
• Enforce checkin-policies by build
Page 28 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 28 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Cleanup build artifacts
Sophisticated Retention Policies needed
• Default retention policies are not sufficient
• „I like to keep 100 builds but cleanup the details“
• Less amount of build drops needed
• Delete oldest drops before drop location
exceeds disk space
• How to cleanup my custom shares?
• How to delete build artifacts for already deleted
builds?
• Note: Deleted builds vanish in build explorer but
remain in database
Page 29 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 29 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Keep track of the overall Build Farm
Monitor fundamental parameters and alert before you run into a downtime
• Impossible to manually keep track of many
machines
• „I like to know that my infrastructure is operational
without being called by my customers“
• Use monitoring tools like Nagios/Icinga
• Monitor build and test machines
• Keep track of handles, memory, disk space,
latency
http://www.nagios.org/ https://www.icinga.org/
.Net 4.5.1 installed
Page 30 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 30 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our experience
TFS Strengths and weaknesses
syngo.via WebViewer VA11A is not commercially available in all countries. Due to regulatory reasons its future availability cannot be guaranteed. Please contact your local Siemens organization for further details. Diagnostic
reading of images with a web browser requires a medical grade monitor. For iPhone and iPad country-specific laws may apply. Please refer to these laws before using for diagnostic reading/viewing.
Apple®, the iPhone®, the iPad®, iPod Touch® are trademarks of Apple Inc., registered in the U.S. and other countries. For iPhone, iPad, iPod Touch country specific laws may apply. Please refer to these laws before using .
Page 31 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 31 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our experience
TFS supports performance for large scale ALM
Performance
Strengths
• Reasonable response time
• Allows fine tuning at many places
• Supports efficient development
Weaknesses
• Some fundamental features missing out of the box
Page 32 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 32 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our experience
TFS supports maintainability for large scale ALM
Serviceability
Strengths
• Is highly customizable
• Customization allows to workaround issues
• Requires low administration efforts
Weaknesses
• Transparency about known issues
• Monitoring solution / administrative guidance
• Tight coupling of TFS to latest .Net framework
Page 33 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 33 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
Our experience
TFS supports scalability for large scale ALM
Scalability
Strengths
• Scales well for very large teams
• Fits for global development footprint
• Allows customized reuse of existing services
Weaknesses
• Does not work out of the box for large scale
• Some hard-constraints limit extensibility
Page 34 2014-02-24
Unrestricted © Siemens AG 2014 All rights reserved.
Page 34 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO
syngo – See results.