Send my love to Ruby II

September 19th, 2007 | 6 comments

In the last post, we saw a nice use case for the #send method. Now, we’ll see some coolness that Rubinius adds. First, some code:


class Apple
  def seed
  end
end

a = Apple.new

a.send :seed
a.__send__ :seed

Now, with the convenient and useful describe facility of the Rubinius compiler, we can see something interesting:


$ shotgun/rubinius describe send.rb
Path: send.rb
Size: 79 bytes

Sexp:
  [:block, 
  [:newline, 1, "(eval)", 
    [:class, [:colon2, :Apple], nil, 
      [:scope, 
        [:newline, 2, "(eval)", set_local_fp 1 ; local a

---- snip ----

pop
#line 8
push_literal 1
get_local_fp 1 ; local a
send send 1
pop
#line 9
get_local_fp 1 ; local a
push nil
push_literal 2
dup
is_symbol
git send_lbl1
send to_sym 0
send_lbl1:
push 0
set_args
send_off_stack
pop
push true
ret

---- snip ----

I’ve truncated that output, but feel free to run this at home. (There’s no long lasting side effects other than an itching desire to contribute to the Rubinius project.) If you correlate the assembly with the line numbers in the source, you’ll notice that the two sends are not turning out to be the same. That’s right. We have a special, and fast, __send__ operation.

The #send method is a highly useful bit of Ruby. Unfortunately, there’s some concern that using it can hurt performance. Well, with Rubinius you can have your send and use it, too.

Send my love to Ruby

September 19th, 2007 | 1 comment

Over at the Rubinius project, in between hatching plots to take over the world, we fit in some time for recreation. For example, we’ve got this masochistic interest in writing RSpec compatible specs for the Ruby core library. One of the challenges there is the large number of aliased methods that Ruby has. Using RSpec’s shared behaviors as an example, I’ve created a flavor of shared behaviors in our mini_rspec implementation. As the code below shows, this makes it straightforward to spec all these aliases.


hash_store = shared "Hash#store" do |cmd|
  describe "Hash##{cmd}" do
    it "associates the key with the value and return the value" do
      h = { :a => 1 }
      (h.send(cmd, :b, 2).should == 2
      h.should == {:b=>2, :a=>1}
    end

    it "duplicates and freezes string keys" do
      key = "foo" 
      h = {}
      h.send(cmd, key, 0)
      key << "bar" 

      h.should == { "foo" => 0 }
      h.keys[0].frozen?.should == true
    end

    it "duplicates string keys using dup semantics" do
      # dup doesn't copy singleton methods
      key = "foo" 
      def key.reverse() "bar" end
      h = {}
      h.send(cmd, key, 0)

      h.keys[0].reverse.should == "oof" 
    end  

    it "raises TypeError if called on a frozen instance" do
      should_raise(TypeError) { hash.send(cmd, 1, 2) }
    end
  end
end

describe "Hash#[]=" do
  it_behaves_like(hash_store, :[]=)
end

The very cool thing about this is how useful Ruby’s send method is. And in Rubinius, it gets even cooler, as you’ll see in part II

Rubinius sprint retrospective

September 17th, 2007 | 2 comments

Last week, Evan Phoenix, Wilson Bilkovich and I met up in Denver (ok, near Denver) for some serious Rubinius coding, aka a sprint. Charles Nutter (of JRuby fame) flew in just to hack with us for 15 hours straight and then flew back home. Very cool. When do you get collaboration like that from someone on a project that could be seen as a competitor? That’s some serious goodwill, folks, and much appreciated. Charles had his laptop out on the taxi ride from the airport digging into some issues we have with including modules. If you hang out in #rubinius, you know there’s a lot of collaboration occurring on different fronts.

This was only the second time I’ve had the pleasure to hang out with these folks in person. The sprint was intended to enable us to get some face-to-face time and work out some foundational issues, hopefully speeding other development and contribution as we head toward RubyConf07. Here’s a few things we accomplished (in no particular order):

  • Evan hooked up Syck (the YAML parser) using our very cool subtend (Ruby C API compatibility) component.
  • Wilson hammered out a bunch of StringIO specs in a couple hours.
  • Evan promptly began writing a Ruby StringIO that passed the specs.
  • Charles whipped out a bunch of def and case specs, the latter being a serious beast to compile.
  • Wilson added these wicked colorized backtraces that enable you to quickly pick out the ultimate failure line and some significant points in the backtrace.
  • Wilson implemented the rest of the missing case support in the compiler.
  • Charles added some basic ObjectSpace support.
  • I checked in a complete reorganization and a lot of fixes to our Ruby core library specs. We have around 2,800 specs, of which we’re passing over two thirds. We still have nearly 50% of the core library specs to complete.
  • Evan fixed our Thread support and added some preemption.
  • I added (with much help from Evan and some serious gdb sessions looking at x86 machine code) some coolness to our foreign function interface (FFI) to support reading and writing to C integers and doubles from Ruby. (The doubles support still needs a magic touch or two from Evan.) With that I was able to finish off our Math methods support and the rest of the Math specs.
  • Wilson hammered out a ton of the machinery necessary to support compiling eval. If you don’t realize how awesome (and painful) this is, don’t blame me.
  • I started on an implementation of File.fnmatch in preparation for Dir.glob (or dear glob is it is not-so-affectionately known to me) and cleaned up our fnmatch specs a bunch.

There’s more where that came from. For all the nitty gritty details, browse the git repository. There’s some cool stuff just over the horizon that we didn’t get to. Evan explained in more detail the architecture of our sampling profiler and our debugger support. Let’s just say you’ll be able to profile a Rails app in production with no significant performance penalty. And remote debugging is on it’s way. Seriously, let that sink in for a minute.

Interestingly, we did not consume 50 liters of coffee each and we had some incredible Japanese food one evening, thanks to a recommendation from one of Wilon’s contacts. Overall, I’m quite impressed with our progress and highly impressed with the technical prowess of these guys. The point it, Rubinius is quite accessible. It’s also some very cool and very real technology that’s going to make writing, running, living, breathing, and enjoying Ruby a lot better.

And last but not least, a big thanks to Sun for their sponsorship, and in particular Tim Bray for his advocacy. Rubinius is a small, ambitious project. But at heart, it’s very true to the value of making Ruby fun.

Dear glob

September 17th, 2007 | 0 comments

At some point, you likely needed to grab some files and do something with them. You probably had a vague recollection about Dir[] from reading the Pickaxe book. To refresh your memory, you crack it open and fire up irb. Ah, easy…


  irb(main):001:0> Dir['spec/**/*_spec.rb'].each do |file|
  irb(main):002:1* puts file
  irb(main):003:1> end

There’s no denying that Ruby makes this task dead simple. So simple, in fact, that you probably don’t think twice about how that nifty Dir[] method does its work. That is, unless you’re trying to implement it.

During the Rubinius sprint, I realized that having this in Rubinius would enable me to make our continuous integration spec runner much better. Now, I know that there’s a glob function in C that provides behavior similar to what you get in the shell using * and ? to match file names. There’s also a function fnmatch that wraps up some of that magic. No problem. We’ve got this nifty foreign-function interface (FFI) that Evan has graciously provided. Evan recommended I take that route first. Yep, took all of 10 minutes to hook everything up.

Of course, it wouldn’t be that interesting were this the end of the story. It’s not. Our fnmatch specs were mostly passing, but when I looked into the failing ones, I discovered something that I’d probably tried to shield my psyche from. Ruby implements its own fnmatch and glob functions. And when I say ‘implements’, it doesn’t really give you any idea of the pain and suffering involved. Do take a peek:

It doesn’t take but a minute to see that Ola Bini’s java code is extraordinarily more readable than the MRI source. But both are daunting to say the least. So, I’ve decided to take a different route.


  def self.fnmatch(pattern, path, flags=0)
    pattern = StringValue(pattern).dup
    path = StringValue(path).dup
    escape = (flags & FNM_NOESCAPE) == 0
    pathname = (flags & FNM_PATHNAME) != 0
    nocase = (flags & FNM_CASEFOLD) != 0
    period = (flags & FNM_DOTMATCH) == 0
    subs = { /\*{1,2}/ => '(.*)', /\?/ => '(.)', /\{/ => '\{', /\}/ => '\}' }

    return false if path[0] == ?. and pattern[0] != ?. and period
    pattern.gsub!('.', '\.')
    pattern = pattern.split(/(?<pg>\[(?:\\[\[\]]|[^\[\]]|\g<pg>)*\])/).collect do |part|
      if part[0] == ?[
        part.gsub!(/\\([*?])/, '\1')
        part.gsub(/\[!/, '[^')
      else
        subs.each { |p,s| part.gsub!(p, s) }
        if escape
          part.gsub(/\\(.)/, '\1')
        else
          part.gsub(/(\\)([^*?\[\]])/, '\1\1\2')
        end
      end
    end.join

    re = Regexp.new("^#{pattern}$", nocase ? Regexp::IGNORECASE : 0)
    m = re.match path
    if m
      return false unless m[0].size == path.size
      if pathname
        return false if m.captures.any? { |c| c.include?('/') }

        a = StringValue(pattern).dup.split '/'
        b = path.split '/'
        return false unless a.size == b.size
        return false unless a.zip(b).all? { |ary| ary[0][0] == ary[1][0] }
      end
      return true
    else
      return false
    end
  end

This code is only passing 80% of our existing specs for File.fnmatch?, so the jury is still out. And I’m sure someone can make this much better. The lesson for me is that 1) Ruby’s implementation is typically not accessible (I already knew that), and 2) writing Ruby code is a good way to handle tough problems.

But then, you already knew that. ;)

UPDATE: I’ve changed the code here to reflect our current version. It’s now passing 100% of the existing specs.