rush, the ruby shell and unix integration library
DESCRIPTION
Adam Wiggins\' presentation from the 2008 Silicon Valley Ruby Conference.TRANSCRIPT
Cluster management with
the Ruby shell and Unix integration library
Adam WigginsApril 18, 2008http://rush.heroku.com/
bash+ssh are the cornerstone of unix systems administration
but...
Ruby is its own universe
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
...right?
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
What are the problems here?
Quoting
Try inserting a single quote into the regexp here. How many backslashes do you need?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Security
What if this value can be entered by the user, and they type in: ";rm -rf /
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Output is all text
Why do we need the inverse grep, or for that matter the complex regexp, just to get a simple list of PIDs?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Error Trapping
How can we catch errors, like permission denied or file not found, and log them?
log_contents = `ssh #{host} "cat log/mongrel.log"`
Unit Testability
Um... yeah. Good luck with that.
`ssh #{host} "echo 'mongrel_rails start -d' | sudo -u #{user} -H bash"`
In short, Unix and Ruby do not play well together.
And that’s really a shame.
Introducing...
Bringing Ruby and Unix together.awesome x awesome!
Let’s try that again.
rush-style.
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
def start_mongrel # kill any existing mongrels box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
# don't try to restart if previous mongrel crashed log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
# do it box.bash 'mongrel_rails -d', :user => userend
Processes box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
Files log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
Bash Commands box.bash 'mongrel_rails -d', :user => user
Basic concepts of rush
Boxes box = Rush::Box.new('remote.example.com') box['/var/log/nginx/*.log'].search(/error/)
Files & Dirs dir = home['myapp/'] dir['config/environment.rb'].contents
dir['**/*.rb'].search(/^module /)
dir['README'].copy_to other_dir
Processes mongrels = box.processes.find_all_by_cmdline /mongrel_rails/ mem_hogs = mongrels.select { |p| p.mem > 50.mb } mem_hogs.each { |p| p.kill }
Permissions file.access = { :user_can => :read_and_write, :group_can => :read }
Bash Commands box.bash 'rake db:migrate'
Bash Commands box.bash 'rake db:migrate', :user => 'www', :env => { :RAILS_ENV => 'production' }
Bash Commands begin box.bash 'rake db:migrate' rescue Rush::BashFailed => e log_warning "Failed to migrate: #{e.message}" end
It’s Ruby! spec_lines = myproj['**/*_spec.rb'].line_count total_lines = myproj['**/*.rb'].line_count spec_percent = (spec_lines * 100 / total_lines).round puts "Specs are #{spec_percent}% of the total code." puts (spec_percent >= 30) ? "Nice!" : "Tsk, tsk."
It’s Ruby!
“I can not adequately convey the sheer joy it was to rewrite my Bash deployment functions in Ruby.”
- Jamie Hoover
What’s good about the traditional unix shells?
Lessons to carry forward.
Remote and local are seamless
box1['myproj/'].move_to box2
Small sharp tools, chained together
dir1['**/*.rb'].search(/^class/).copy_to dir2
Why now?
Cloud computing.Why now?
Cloud computing.
Which means automation of systems administration tasks.
Why now?
Pre-cloud computing, sysadmin tasks are occasional and manual
In cloud computing, sysadmin tasks are frequent and automatic
In cloud computing, sysadmin tasks are frequent and automatic
So we need a real programming language for them!
My need came from:
hostnames = %w(host1 host2 host3 host4)boxes = hostnames.map { |h| Rush::Box.new(h) }
arc_host = Rush::Box.new('archive')archive = arch_host['/arc/nginx_logs/']
boxes.each do |box| log = box['/var/log/nginx/access.log'] archive_name = "#{box.host}_#{log.name}" log.copy_to archive[archive_name] log.rename log.name + ".old"end
Clustering (finally)
gem install rush