backy - osbconf - open source backup conference · backy vm backup beyond bacula/bareos ... bacula:...
TRANSCRIPT
![Page 1: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/1.jpg)
backy
VM backup beyond Bacula/BareosChristian Theune
@theuni
![Page 2: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/2.jpg)
Mea Culpa
![Page 3: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/3.jpg)
And I almost
missed it — again
![Page 4: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/4.jpg)
Backup!!11!!
![Page 5: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/5.jpg)
• flyingcircus.io
• DevOps as a Service
• custom, mission-critical web
applications
![Page 6: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/6.jpg)
Part I - Oh the Pain
![Page 7: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/7.jpg)
The story unfolds …
![Page 8: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/8.jpg)
![Page 9: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/9.jpg)
![Page 10: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/10.jpg)
![Page 11: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/11.jpg)
![Page 12: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/12.jpg)
Root Cause Analysis
![Page 14: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/14.jpg)
Restore script bottleneck:
global lock
![Page 15: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/15.jpg)
Undetected inconsistency in
important customer database
![Page 16: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/16.jpg)
Bacula: complexity and the
VTL
![Page 17: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/17.jpg)
Not “everything” backed up.
![Page 18: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/18.jpg)
24 hours are not a sufficient
RPO in quite a few cases
![Page 19: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/19.jpg)
Paper cuts
• Hard link farms
• Boot loaders
• The director as a “most valuable bottleneck”
![Page 20: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/20.jpg)
Recap
• Restore fiddly to script
• Undetected inconsistency that was hard to deal with
• Blind spots
• Daily Interval
• Overall complexity, performance and the VTL
• Paper cuts
![Page 21: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/21.jpg)
Part II - Make a wish
![Page 22: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/22.jpg)
Simplicity
• Restore with basic Unix tools
• No VTL
• Not mixing data of different VMs
![Page 23: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/23.jpg)
Reliability
• Verification / Scrubbing / (Repair)
• High frequency
• Integration with storage snapshots
• Not inventing new formats
![Page 24: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/24.jpg)
Operability
• Avoid bottlenecks / head-of-line blocking
• Efficient deltas for large files (ZODB)
• Parallelisation (multiple jobs and multiple servers)
• Simple scripting and environment-specific integration
• Coordination: pre/post actions on storage, hypervisor,
VM …
![Page 25: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/25.jpg)
Operability II
• Simple Nagios integration to ensure we notice RPO/
SLA failures
• RTO-compliance during mass-restore
• Self-service for customers to restore files or VMs
![Page 26: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/26.jpg)
Part III - Let’s do this!
![Page 27: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/27.jpg)
–Probably someone, maybe me
“One size fits all … not”
It’s all about size
![Page 28: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/28.jpg)
It’s all about size: backy
![Page 29: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/29.jpg)
It’s all about size: Bacula
![Page 30: Backy - OSBConf - Open Source Backup Conference · backy VM backup beyond Bacula/Bareos ... Bacula: complexity and the VTL. Not “everything” backed up. 24 hours are not a sufficient](https://reader033.vdocuments.us/reader033/viewer/2022051907/5ffacdafa2e38b518508bfcb/html5/thumbnails/30.jpg)
It’s all about size: Bareos