In Part 1 , we looked at the background to the gem that we are going to explore and saw how the gem works, and also how to change, rebuild and install the gem locally. Part 2 explains how the gem gets picked up and used by Ruby when you require it, and how Ruby knows what to do. In this part, we will jump across a number of files to figure out what is happening. We will not rebuild the gem or look at the Makefile, etc. in this post.
A quick note on using ‘Ruby’ and ‘Rubygems’
In this post, I use the term Ruby a lot as if Ruby does things and so on. In reality, the correct expression to use should be “Ruby Interpreter” or “Ruby runtime” or something similar. Likewise, I refer to Rubygems but I actually mean the gem called Rubygems that provides the support for gems to be installed, found and loaded. It’s a convenience that allows me to focus on other things. If you think it could be expressed better, let me know!
What happens when we require a gem?
When you require a gem, two main things happen:
The path to the gem is added to the $LOAD_PATH – this allows you to require other files from within the path of that gem
The file that you `require` gets loaded, i.e., Ruby executes the file
Let’s walk through this first. Let’s start irb and simply output the LOAD_PATH.
Most of the paths are to generic places where you would find gems or other Ruby source files to include. Now, let’s require a gem that we have on our system. Since we have installed `fast_polylines` on our system, let’s require that. This gives us the below $LOAD_PATH.
If the filename has the extension “.rb”, it is loaded as a source file; if the extension is “.so”, “.o”, or “.dll”, or the default shared library extension on the current platform, Ruby loads the shared library as a Ruby extension.
The last part applies to the native extension it loaded. When Ruby loads it as an extension , it does one more thing: it calls a function called Init_LIBRARY from that extension. In our case, the extension is called fast_polylines and so it will try to call a method called Init_fast_polylines from the compiled C source code.
If you look at the C code in the ext/fast_polylines directory, you will find this at the end of the file.
The name matches and will be called by the Ruby Interpreter when it loads the extension. This method actually defines a module called FastPolylines and creates two methods under that module, decode and encode, and makes them available to the Ruby code. Once the native extension is loaded, the FastPolylines module with the methods encode and decode are available to Ruby.
Summarising what we learnt
It might not look like a lot, but what we saw in this post so far is as below:
Once the gem is installed, we need some files to be in the correct place
The file we want to include from our code should be in lib within the gem folder. It can be in a sub-folder (e.g., lib/a but then we must do a require "a/file.rb" instead)
The compiled native extension should be in: lib/gem_name/gem_name.so
Doing a require involves Rubygems to change the load path and to load the main file
The first file that is required should load the native shared library
The native library needs to have a Init_LIBRARY function that sets up the bridge between Ruby and the C code
Ruby will call that function when the extension file is loaded
This is shown in the picture below. For simplicity, some bits (like requiring the version.rb file) have been left out. In the picture below, I use:
Script – the actual running script that uses the native gem; it might even by irb
Ruby – as described earlier, this refers to the Ruby runtime/ interpreter and includes everything other than the Script, Rubygems and the native gem
Rubygems – this refers to the Rubygems code
Gem refers to the specific native extension gem but is broken up into two parts:
Gem_Code: the main Ruby file that gets required by the Script
Gem_Extension: the shared library file that provides the native extension
The picture below shows the difference to the $LOAD_PATH between the items marked as (1) and (2) in the sequence diagram.
At the point that is marked as (3), the extension is loaded correctly and gets added to the $LOADED_FEATURES:
In reality, a lot of this is detail you will not need to worry about – Ruby and Rubygems will take care of it when the native gem is created, installed or required. However, understanding it at one level deeper might help you debug if you run into unexpected problems.
Looking ahead
With that, we come to the end of Part 2 where we looked at how the gem code and the native extension is loaded. In the next post, we will look at more specific details of our code.
I will add links and references later, possibly in the last post of the series. If you have any comments, please feel free to leave them below.
