transforming ruby code

Transforming Ruby Code

Ben Hughes@rubiety

http://benhugh.esTuesday, October 11, 11

http://benhugh.es

http://benhugh.es

for element in collection do_something(element)end

collection.each do |element| do_something(element)end

Tuesday, October 11, 11

Factory.define(:product) do |f| f.association :category f.sequence(:name) {|n| "Product #{n}" } f.price 19.95end

FactoryGirl.define do factory :product do category sequence(:name) {|n| "Product #{n}" } price 19.95 endend


Why?• Automated Refactoring

• Enforcing Coding Style or Best Practices

• DSL Conversion (e.g. factory_girl 1 => 2)

• Reduce Friction of changing technical “paths”


String#gsub!


gsub!( /for (\S+) in (\S+)/, '\2.each do |\1|')

for element in collection do_something(element)end



gsub!( /for (\S+) in (\S+)/, '\2.each do |\1|')

for element in find(1, 2) do_something(element)end



AST Transformations

Parse into AST

Modify AST

De-parse AST


What is an AST?2 + 3

Invoke Method (call)

2 :+ 3

2.+(3)


S-Expressions

[:lasgn, :foo, [:lit, 1]]

class Sexp < Array def kind self[0] end def body self[1..-1] endend

kind body

def s(*args) Sexp.new(*args)end


2 + 3 [:call, [:lit, 2], :+, [:arglist, [:lit, 3]]]

foo = 1 [:lasgn, :foo, [:lit, 1]]

=>

=>


if a 1else 2end

[:if, [:call, nil, :a, [:arglist] ], [:lit, 1], [:lit, 2]]

=>


class Project def name "Rails" endend

[:class, :Project, nil, [:scope, [:defn, :name, [:args], [:scope, [:block, [:str, "Rails"] ] ] ] ]]

=>


Parsing• ruby_parser: Pure Ruby (uses racc),

Parses 1.8 only

• ParseTree: Uses C Extension, same AST as ruby_parser, Parses 1.8 only

• Ripper: Internal Ruby 1.9 Parser exposes via standard library


Tree Walker Pattern

module RubyTransform class Transformer include TransformerHelpers def transform(sexp) if sexp.is_a?(Sexp) Sexp.new(*([sexp.kind] + sexp.body.map {|c| transform(c) })) else sexp end end endend

Recursion on Children AST Nodes


Eachifier (for => .each)for element in collection do_something(element)end



class Eachifier < RubyTransform::Transformer def transform(e) super(transform_fors_to_eaches(e)) end def transform_fors_to_eaches(e) if sexp?(e) && e.kind == :for transform_for_to_each(e) else e end end def transform_for_to_each(e) s(:iter, s(:call, e.body.first, :each, s(:arglist)), e.body.second, e.body.third ) endend


Clear Explicit Returns

def my_method a = 1 call_something(a) return aend

def my_method a = 1 call_something(a) aend


class ClearExplicitReturns < RubyTransform::Transformer def transform(e) super(transform_explicit_returns(e)) end def transform_explicit_returns(e) if matches_explicit_return_method?(e) transform_explicit_return_method(e) else e end end def matches_explicit_return_method?(e) e.is_a?(Sexp) && e.method? && e.block && e.block.body.last && e.block.body.last.kind == :return end def transform_explicit_return_method(e) e.clone.tap do |exp| exp.block[-1] = exp.block[-1].body[0] end endend


Block Method To Proc-ifier

collection.map {|d| d.name }

collect.map(&:name)


Tapifierdef my_method temp = "" temp << "Something" call_something(temp) tempend

def my_method "".tap do |temp| temp << "Something" call_something(temp) endend


Custom Transforms

# Reverse all string literals!RubyTransform::Transformers::Custom.new do |expression| if sexp?(expression) && expression.kind == :str s(:str, expression.body[0].reverse) else super endend


AST De-Parsing Challenges

• Lots of Lost Information!

• All Whitespace

• Comments (Depending on Parser)

• Line Numbers (Depending on Parser)

• String Literal Quote Style

• Block Style: { .. } vs do .. end


AST De-Parsers• ruby2ruby:

Ruby 1.8 only, minimal code formatting considerations

• ripper2ruby: Ruby 1.9 (operates on a ripper AST), OO s-expression abstractions

• ruby_scribe: Ruby 1.8 only, intelligent code formatting (emitter configurable)


Intelligent Code Formatting

Compositing the parsing and de-parsing operation turns it into an intelligent (and potentially configurable) code formatter.


ruby_scribe Example

require( "pp" )class Project; attr_accessor(:name) def title if active? then "Active Project: #{name}" else "Disabled Project: #{name}" end endend


[:block, [:call, nil, :require, [:arglist, [:str, "pp"]]], [:class, :Project, nil, [:scope, [:block, [:call, nil, :attr_accessor, [:arglist, [:lit, :name]]], [:defn, :title, [:args], [:scope, [:block, [:if, [:call, nil, :active?, [:arglist]], [:dstr, "Active Project: ", [:evstr, [:call, nil, :name, [:arglist]]]], [:dstr, "Disabled Project: ", [:evstr, [:call, nil, :name, [:arglist]]]] ] ]] ] ]] ]]


require "pp"

class Project attr_accessor :name def title if active? "Active Project: #{name}" else "Disabled Project: #{name}" end endend

sexp = RubyParser.new.parse(File.read(path))RubyScribe::Emitter.new.emit(sexp)


Impediments

• Standard & Solid AST Format: Ripper?

• Even Better De-Parsing

• S-expression Matchers (Tree Expressions)

• Better OO Tools for Transformations (Abstract Underlying AST Verbosity)


rspecify

class MyClass < ActiveSupport::TestCase def test_should_be_one assert_equal something, 1 end def test_should_be_one assert_not_equal something, 1 endend

describe MyClass do it "should be one" do something.should == 1 end it "should not be one" do something.should_not == 1 endend

http://github.com/rubiety/rspecify

$ rspecify cat my_class_test.rb


http://github.com/rubiety/ruby_scribe


factory_girl_upgrader

Factory.define(:product) do |f| f.association :category f.sequence(:name) {|n| "Product #{n}" } f.price 19.95end

FactoryGirl.define do factory :product do category sequence(:name) {|n| "Product #{n}" } price 19.95 endend

http://github.com/rubiety/factory_girl_upgrader

$ factory_girl_upgrader cat factories.rb




Future: AST Translation

• Parse Java AST

• Translate Java AST => Ruby AST

• Apply Idiomizing Transformations

• De-parse AST

• => Working, fairly idiomatic, JRuby!


Questions?

Ben Hughes@rubiety

http://benhugh.es


http://github.com/rubiety/ruby_transform

http://github.com/rubiety/rspecify

http://github.com/rubiety/factory_girl_upgrader


http://benhugh.es

http://benhugh.es









transforming ruby code

Documents

transformer

end gsub

collection

end

collection

kind

kind

dosomething