© 2018 Bloomberg Finance L.P. All rights reserved.
Monitorama
June 4-6, 2018
Allan Espinosa
Site Reliability Engineer
Autoscaling Containers…
with Math
© 2018 Bloomberg Finance L.P. All rights reserved.
@me
© 2018 Bloomberg Finance L.P. All rights reserved.
Your Distributed System
© 2018 Bloomberg Finance L.P. All rights reserved.
Your Air Conditioner
© 2018 Bloomberg Finance L.P. All rights reserved.
Autoscaling
https://docs.run.pivotal.io/appsman-services/autoscaler/using-autoscaler.html
© 2018 Bloomberg Finance L.P. All rights reserved.
Control Theory
• Influencing dynamic systems
• Corrections based on feedback loops
• Math describes effectiveness
© 2018 Bloomberg Finance L.P. All rights reserved.
Centrifugal Governor
R. Routledge, Discoveries & Inventions of the Nineteenth
Century, 13th Edition, 1901
© 2018 Bloomberg Finance L.P. All rights reserved.
Theory
© 2018 Bloomberg Finance L.P. All rights reserved.
Control Theory
• Target Output
• Input
• Actual Output
• Disturbances
© 2018 Bloomberg Finance L.P. All rights reserved.
Controller Loop Code
© 2018 Bloomberg Finance L.P. All rights reserved.
Sample Output
© 2018 Bloomberg Finance L.P. All rights reserved.
Linear-Time Invariant Systems
© 2018 Bloomberg Finance L.P. All rights reserved.
Desired Properties
• Stability
• Accuracy
• Settling time
• Overshoot
© 2018 Bloomberg Finance L.P. All rights reserved.
Stability – Unstable Systems
© 2018 Bloomberg Finance L.P. All rights reserved.
Unstable Systems
© 2018 Bloomberg Finance L.P. All rights reserved.
Unstable Systems
© 2018 Bloomberg Finance L.P. All rights reserved.
Stable Systems – Damping
© 2018 Bloomberg Finance L.P. All rights reserved.
Stable Systems
© 2018 Bloomberg Finance L.P. All rights reserved.
Accuracy
© 2018 Bloomberg Finance L.P. All rights reserved.
Settling time
© 2018 Bloomberg Finance L.P. All rights reserved.
Overshoots
© 2018 Bloomberg Finance L.P. All rights reserved.
Controllers
© 2018 Bloomberg Finance L.P. All rights reserved.
Proportional Control
© 2018 Bloomberg Finance L.P. All rights reserved.
Proportional Control
• Inherently inaccurate
• Proportional constant KP increases overshoot and settling time
© 2018 Bloomberg Finance L.P. All rights reserved.
Integral Control
• Reduces steady-state errors
• Increases settling times
© 2018 Bloomberg Finance L.P. All rights reserved.
Derivative Control
• Decreases settling time
• Sensitive to noise
© 2018 Bloomberg Finance L.P. All rights reserved.
Non-linearity
© 2018 Bloomberg Finance L.P. All rights reserved.
Other applications
• Response time vs. keep-alive time-outs
• Memory vs. worker connections
• Multiple input-multiple output
Auto-tuning, Auto-configuration
© 2018 Bloomberg Finance L.P. All rights reserved.
Summary
• Iterate on feedback
• Effectiveness of feedback
• Linear models go a long way
• Re-evaluate your models!
© 2018 Bloomberg Finance L.P. All rights reserved.
References
• P. Janert, Feedback Control for Computer Systems, Sebastapol, Ca: O’Reilly Media, 2014.
• J. Hellerstein, et. al., Feedback Control of Computing Systems, Hoboken, NJ: John Wiley & Sons, 2004.
© 2018 Bloomberg Finance L.P. All rights reserved.
Demo
© 2018 Bloomberg Finance L.P. All rights reserved.
Allan Espinosa
http://github.com/aespinosa/control-theory
Thank You!!
© 2018 Bloomberg Finance L.P. All rights reserved.
https://www.bloomberg.com/careers/technology/
Demo link: http://github.com/aespinosa/control-theory
We’re Hiring!