condor in flight at the hartford 2006 transformations condor week 2007 bob nordlund
TRANSCRIPT
Condor In Flight at The Hartford2006 Transformations
Condor Week 2007
Bob Nordlund
About The Hartford…
• Headquartered in Hartford, CT• Founded in 1810• Fortune 100• 31,000 employees worldwide• $26.5 billion revenues• $2.9 billion core earnings• $377.6 billion assets under management
The Hartford’s Businesses
• Property & Casualty• Auto, home, marine, workers compensation, etc.
• Retail Investment Products• Variable and fixed annuities, mutual funds, 529 college
savings plans
• Retirement Plans• 401(k), 403(b), 457
• Institutional Financial Solutions• Individual Life Insurance• Group Benefits• International
A Brief History…
• Exponential growth in risk modeling activity exceeded existing computing capabilities.
• Grid technology was identified as a possible solution.
• A pilot program with a commercial grid provider was started.
• In parallel, a ‘skunkworks’ Condor implementation was evaluated as an alternate approach.
• Condor was selected over the commercial provider.• Windows Support• Simple, Scalable, and Flexible• Active Community• Free
2006 Transformations
• GRAPE• Condor 6.8 Upgrade• Accounting and Usage Reporting• GridMC• User Community
GRAPE Challenge(Global Risk Analysis Projection Enhancement)
• Prepare comprehensive risk analysis report for all products for investment analysts.
• Run three orders of magnitude more projections through the system.
• Perform stochastic on stochastic on stochastic analysis.
• 2.4 billion individual simulations.
GRAPE IT Response
• Infrastructure• 500 dual-core, dual-CPU AMD servers
• 20 Racks
• Dedicated GigE switches
• Condor pool with HAD
• 4 filers with 8TB of storage
• Purchased, installed and configured in 6 weeks
• Application• Java-based liability model
• 75x performance gain
• Meta-scheduling application
• In-memory market scenario generation
Condor 6.8 Upgrade
• Dual version Condor MSI installation• Centralized configuration management• Multiple schedulers
• 6.8 shadow process overhead
• Load balancing scripts
• Scheduler coordination
• Accounting Groups• SOAP based submission• Mixed Linux Scheduler/Windows Execute• High Availability Central Managers• 1000+ desktops, notebooks, and workstations
Accounting and Usage Reporting
GridMC (Grid Management Console)
• Web-based view of the grid• Administrative tools
• Configuration management
• Policy management
• Condor Daemon Management
• User-priority / Group Quota Management
• User tools• Job submission
• Job control
• Prioritization
• Job Monitoring
• Grid utilization
• Usage Reporting
User Community
• Weekly user group meetings to discuss issues, planned usage, allocation trading.
• Users own the grid, IT provides care and feeding
• Promote ‘good’ behavior by highlighting ‘bad’ behavior.• Users who don’t want to share can’t take advantage of
others who do – and are prominently displayed on usage graphs as ‘exclusive’.
• Shame and ridicule can be effective tools to manage a grid.
• Avoid draconian policies by providing transparency to users and letting a cooperative community evolve.
Our Condor Environment…
• In production since 2004• Three pools (GRAPE, GA, Test)• Dedicated and non-dedicated execute nodes
• ~1000 Two-socket, dual-core HP x86 servers• ~1000 desktops, notebooks
• Linux central managers with HAD• Linux and Windows schedulers• Windows execute nodes
Conclusion
• The GRAPE project won The Hartford’s 2006 Chairman’s Award
• Condor has been a transformational technology providing users access to capabilities they wouldn’t have otherwise had – or can now live without.
• Grid computing is an integral part of our business and places The Hartford in a leadership position with respect to risk analysis in our industry.
• Condor has proven to be an invaluable asset and has time and again handled whatever challenge we’ve thrown at it.
Thank you Condor Team!