Ruby Learning by Reversing: Native Gems, Part 1

In this series of posts, I want to look at how to create a native gem for Ruby in C. Actually, it would be more accurate to say – I want to look at how a native gem has been created in C for Ruby. So, I will look at one native gem and we will reverse back and understand what has been done and what it means.

In this post, we’ll look at the following:

Environment
Background to the Gem: fast-polylines
Installing the Gem from Rubygems
Unpacking the Gem so that we can study it
Grabbing the source from GitHub
Building the Gem
Running the Tests
Running the Performance Comparison
Making Changes
Packaging the gem and installing it locally

After that, we’ll get into topics such as:

How the interface works between Ruby and C
What the C code looks like
What the Ruby code looks like
Looking at the Makefile
Having a Makefile that works on Windows
How the specs are run and what they test
Adding Performance Comparisons

After that, we will also consolidate and review what we have learned.

Environment

So, I’m on Windows using the RubyInstaller2 with Developer Kit, installed as per the instructions on this page. Before we get started, let’s just dump out the Ruby version and environment.

$ ruby -v
ruby 3.1.1p18 (2022-02-18 revision 53f5fc4236) [x64-mingw-ucrt]

$ ridk version
---
ruby:
  path: C:/Ruby31-x64
  version: 3.1.1
  platform: x64-mingw-ucrt
ruby_installer:
  package_version: 3.1.1-1
  git_commit: d9d39f1
msys2:
  path: C:\Ruby31-x64\msys64
cc: gcc (Rev9, Built by MSYS2 project) 11.2.0
sh: GNU bash, version 5.1.8(1)-release (x86_64-pc-msys)
os: Microsoft Windows [Version 10.0.19044.2486]

So, we have everything that should allow us to build and install native gems.

Background to the Gem: fast-polylines

We will use the fast-polylines gem from Klaxit as the gem that we will look at. This is the description from the gem’s GitHub page:

Implementation of the Google polyline algorithm. About 300x faster encoding and decoding than Joshua Clayton’s gem.

The speed-up is due to a piece of native C code that runs much faster than the original Ruby code for the encoding and decoding. In this series of posts, we will see how this works as a way of learning (by reversing) how native gems can be created. To get acquainted, let’s install it and then look into it.

Installing the Gem from Rubygems

This is the easy bit. We either do a gem install fast-polylines or add it to a Gemfile and then do bundle install. For now, I will install the gem directly.

$ gem install fast-polylines
Fetching fast-polylines-2.2.2.gem
Temporarily enhancing PATH for MSYS/MINGW...
Building native extensions. This could take a while...
Successfully installed fast-polylines-2.2.2
Parsing documentation for fast-polylines-2.2.2
Installing ri documentation for fast-polylines-2.2.2
Done installing documentation for fast-polylines after 1 seconds
1 gem installed

Great! This worked as expected. There was a time when native gems for Ruby were considered a really big problem on Windows, but thanks to the RubyInstaller2 project, this has not been an issue for a very long time. In general, things work perfectly fine with the one exception that the authors did not think about Windows (since they were not working on Windows) and inadvertently overlooked something small that broke on Windows. There were a couple of minor issues with the fast-polylines gem also initially, but those issues have been fixed now.

Let’s fire up irb and see the basics of the gem working. The examples are on the GitHub page of the gem and we just run them to be sure that things look fine.

$ irb
irb(main):001:0> require 'fast_polylines'
=> true
irb(main):002:0> FastPolylines.encode([[38.5, -120.2], [40.7, -120.95], [43.252, -126.453]])
=> "_p~iF~ps|U_ulLnnqC_mqNvxq`@"
irb(main):003:0> FastPolylines.decode("_p~iF~ps|U_ulLnnqC_mqNvxq`@")
=> [[38.5, -120.2], [40.7, -120.95], [43.252, -126.453]]
irb(main):004:0>

Good! Now, let’s look at this a bit closer.

Unpacking the gem

A common approach to study a gem is to unpack it, i.e., to dump the gem source code into a folder on your computer so that you can inspect the structure, the files, and make changes to it (and yes, even do puts debugging if you really want).

What does unpack do?

If you look at the command reference for unpack this is what it says:

Unpack an installed gem to the current directory. The unpack command allows you to examine the contents of a gem or modify them to help diagnose a bug.

My working directory for gems is usually d:\projects\github\gems and under that I created a new folder called fast-polylines\unpack and went to the folder and did this:

Then, I got a copy of the gem as a single file. There are two ways to do this:

Head on over to the Rubygems page for the gem and click on the download link to download the file.
Or on the command line, do `gem fetch`

We’ll do the second one here.

$ gem fetch fast-polylines
Fetching fast-polylines-2.2.2.gem
Downloaded fast-polylines-2.2.2

There you have it. Let’s unpack it. I ran the command below in the unpack directory.

$ gem unpack fast-polylines-2.2.2.gem
Unpacked gem: 'd:/projects/github/gems/fast-polylines/unpack/fast-polylines-2.2.2'

So, the unpacked gem is in D:/projects/github/gems/fast-polylines/unpack/fast-polylines-2.2.2 – let’s see what we have there. Go into fast-polylines-2.2 and let’s see what the file tree looks like.

$ tree /F
Folder PATH listing for volume D_DRIVE
Volume serial number is C2DE-A69A
D:.
│   .rspec
│   .yardopts
│   CHANGELOG.md
│   README.md
│
├───ext
│   └───fast_polylines
│           extconf.rb
│           fast_polylines.c
│
├───lib
│   │   fast-polylines.rb
│   │   fast_polylines.rb
│   │
│   └───fast_polylines
│           version.rb
│
└───spec
        fast-polylines_spec.rb
        fast_polylines_spec.rb
        spec_helper.rb

That’s not a lot! It’s basically got just a few things:

Some files in ext (where the native code is)
Some files in lib (which is used by your code)
Some spec files to test the code
Administrative files: CHANGELOG and README, and options for RSpec and YARD

Is this really enough to modify and rebuild this gem? I think the answer is almost. When you use the gem build command, you need a gem specification – you’ll notice that we don’t have that here. So, we need to add that in if we want to rebuild the gem from this unpacked version.

Getting the Gem Specification

The answer to how to get a gem specification is in the question itself. We need to run gem specification with the correct parameters. Again, the command reference for gem specificaton and gem build provides the details to get started (but that didn’t work for me).

From the unpack directory I did this:

$ gem spec fast-polylines-2.2.2.gem --ruby > fast-polylines-2.2.2\fast-polylines.gemspec

This extracts the specification from the gem copies it into the fast-polylines-2.2.2 directory with the name fast-polylines.gemspec. You’ll note that we called gem spec with the --ruby argument which tells it to extract the specification as Ruby code (the default option is yaml) but that does not work if we want to use gem build to package the gem again.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# -*- encoding: utf-8 -*-
# stub: fast-polylines 2.2.2 ruby lib
# stub: ext/fast_polylines/extconf.rb

Gem::Specification.new do |s|
  s.name = "fast-polylines".freeze
  s.version = "2.2.2"

  s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
  s.require_paths = ["lib".freeze]
  s.authors = ["Cyrille Courti\u00E8re".freeze, "Ulysse Buonomo".freeze]
  s.date = "2022-01-12"
  s.email = ["dev@klaxit.com".freeze]
  s.extensions = ["ext/fast_polylines/extconf.rb".freeze]
  s.files = [".rspec".freeze, ".yardopts".freeze, "CHANGELOG.md".freeze, "README.md".freeze, "ext/fast_polylines/extconf.rb".freeze, "ext/fast_polylines/fast_polylines.c".freeze, "lib/fast-polylines.rb".freeze, "lib/fast_polylines.rb".freeze, "lib/fast_polylines/version.rb".freeze, "spec/fast-polylines_spec.rb".freeze, "spec/fast_polylines_spec.rb".freeze, "spec/spec_helper.rb".freeze]
  s.homepage = "https://github.com/klaxit/fast-polylines".freeze
  s.licenses = ["MIT".freeze]
  s.required_ruby_version = Gem::Requirement.new(">= 2.4.6".freeze)
  s.rubygems_version = "3.3.7".freeze
  s.summary = "Fast & easy Google polylines".freeze
  s.test_files = ["spec/fast-polylines_spec.rb".freeze, "spec/fast_polylines_spec.rb".freeze, "spec/spec_helper.rb".freeze, ".rspec".freeze]

  if s.respond_to? :specification_version then
    s.specification_version = 4
  end

  if s.respond_to? :add_runtime_dependency then
    s.add_development_dependency(%q<benchmark-ips>.freeze, ["~> 2.7"])
    s.add_development_dependency(%q<polylines>.freeze, ["~> 0.3"])
    s.add_development_dependency(%q<rspec>.freeze, ["~> 3.5"])
  else
    s.add_dependency(%q<benchmark-ips>.freeze, ["~> 2.7"])
    s.add_dependency(%q<polylines>.freeze, ["~> 0.3"])
    s.add_dependency(%q<rspec>.freeze, ["~> 3.5"])
  end
end

Ordinarily, we would be able to use the unpacked gem now in our code by adding the lib folder of the gem to the LOAD_PATH that Ruby uses. This could be easily done by using the -I flag when running Ruby or even irb. But, if we did this on your command line now (I’m doing this inside the fast-polylines-2.2.2 directory), you will be disappointed and waste a while trying to figure out which path did not work as expected.

d:\projects\github\gems\fast-polylines\unpack\fast-polylines-2.2.2>
$ irb -I./lib
irb(main):001:0> require 'fast-polylines'
<internal:C:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- fast_polylines/fast_polylines (LoadError)
        from <internal:C:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from d:/projects/github/gems/fast-polylines/unpack/fast-polylines-2.2.2/lib/fast_polylines.rb:3:in `<top (required)>'
        from d:/projects/github/gems/fast-polylines/unpack/fast-polylines-2.2.2/lib/fast-polylines.rb:3:in `require_relative'
        from d:/projects/github/gems/fast-polylines/unpack/fast-polylines-2.2.2/lib/fast-polylines.rb:3:in `<top (required)>'
        from <internal:C:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:C:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from (irb):1:in `<main>'
        from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
        from C:/Ruby31-x64/bin/irb:33:in `load'
        from C:/Ruby31-x64/bin/irb:33:in `<main>'

Why did it not work?

We added ./lib which is relative to our current path and should normally be all that we need. However, for native gems, we also need to be able to load the native extension shared libary which currently does not exist! We’ll get to that later.

Get the Source

This gem is open source, so we can just clone the repository from GitHub and start to experiment with it. Clone the repository to a folder on your computer (using whichever way you prefer to use Git).

$ git clone https://github.com/klaxit/fast-polylines.git
Cloning into 'fast-polylines'...
remote: Enumerating objects: 230, done.
remote: Counting objects: 100% (45/45), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 230 (delta 16), reused 30 (delta 11), pack-reused 185
Receiving objects: 100% (230/230), 56.58 KiB | 9.43 MiB/s, done.
Resolving deltas: 100% (92/92), done.

Let’s look at the tree for the files.

$ tree /f
Folder PATH listing for volume D_DRIVE
Volume serial number is C2DE-A69A
D:.
│   .gitignore
│   .rspec
│   .rubocop.yml
│   .yardopts
│   CHANGELOG.md
│   fast-polylines.gemspec
│   Gemfile
│   LICENSE
│   Makefile
│   README.md
│
├───.github
│   └───workflows
│           ci.yml
│
├───ext
│   └───fast_polylines
│           .gitignore
│           extconf.rb
│           fast_polylines.c
│
├───lib
│   │   fast-polylines.rb
│   │   fast_polylines.rb
│   │
│   └───fast_polylines
│           version.rb
│
├───perf
│       benchmark.rb
│
└───spec
        fast-polylines_spec.rb
        fast_polylines_spec.rb
        spec_helper.rb

If we compare this with the version from the packaged gem, we can clearly see the differences.

Since this is the development version, we notice a few more files here – github workflow, a few .gitignore files, and also an extra directory perf with a benchmark script for measuring the performance. Importantly, it includes the LICENSE, a Makefile and the fast-polylines.gemspec.

The repository comes with some good instructions in the README including:

How to contribute
How to build and test Locally
How to run the benchmark

All of these will be important for us as we proceed into the next sections.

Building the gem

Let’s build it now. Since we installed the DevKit with Ruby (which is how we were able to build the native extension when we did a gem install earlier), we should be good to go.

The instructions say we need to do things:

First, bundle install – this gets what the development of the gem requires
Then, make benchmark or make test

$ bundle install
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.3.12
Using fast-polylines 2.2.2 from source at `.`
Fetching benchmark-ips 2.10.0
Fetching diff-lcs 1.5.0
Fetching polylines 0.4.0
Fetching rspec-support 3.12.0
Installing benchmark-ips 2.10.0
Installing rspec-support 3.12.0
Installing diff-lcs 1.5.0
Installing polylines 0.4.0
Fetching rspec-core 3.12.0
Installing rspec-core 3.12.0
Fetching rspec-expectations 3.12.2
Fetching rspec-mocks 3.12.3
Installing rspec-mocks 3.12.3
Installing rspec-expectations 3.12.2
Fetching rspec 3.12.0
Installing rspec 3.12.0
Bundle complete! 4 Gemfile dependencies, 10 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

It’s possible that at this stage either make is not found or it fails with an even more exotic error like the one below.

$ make test
MAKE Version 5.43  Copyright (c) 1987, 2019 Embarcadero Technologies, Inc.
Error makefile 3: Command syntax error
Error makefile 6: Command syntax error
Error makefile 9: Command syntax error
*** 3 errors during make ***

In my case, this happens because I also use the tools from Embarcadero Technologies and their version of make is further up in the PATH resulting in the error since it cannot handle the Makefile. If you had a failure that indicated that it could not find make, don’t worry.

Remember when we installed the gem, we saw this line in the output:

Temporarily enhancing PATH for MSYS/MINGW...

This line adjusts the path so that the extension could be compiled. We can do the same using the line below (assuming you installed Ruby 3.1 into c:\Ruby31-x64)

$ c:\Ruby31-x64\ridk_use\ridk.cmd enable
$ set "RI_DEVKIT=C:\Ruby31-x64\msys64"
$ set "MSYSTEM=UCRT64"
$ set "PKG_CONFIG_PATH=/ucrt64/lib/pkgconfig:/ucrt64/share/pkgconfig"
$ set "ACLOCAL_PATH=/ucrt64/share/aclocal:/usr/share/aclocal"
$ set "MANPATH=/ucrt64/share/man"
$ set "MINGW_PACKAGE_PREFIX=mingw-w64-ucrt-x86_64"
$ set "MSYSTEM_PREFIX=/ucrt64"
$ set "MSYSTEM_CARCH=x86_64"
$ set "MSYSTEM_CHOST=x86_64-w64-mingw32"
$ set "MINGW_CHOST=x86_64-w64-mingw32"
$ set "MINGW_PREFIX=/ucrt64"
$ set "LANG=en_GB.UTF-8"
$ set "PATH=C:\Ruby31-x64\bin;C:\Ruby31-x64\msys64\ucrt64\bin;C:\Ruby31-x64\msys64\usr\bin; ... truncated ...

I have truncated the last line which shows the PATH but importantly, you can see that a number of environment variables have been SET and the PATH has been augmented to have some of the paths from the Ruby 3.1 installation at the top so that they are found first.

Now, let’s try the make test again.

$ make test
cd ext/fast_polylines && ruby extconf.rb --vendor
creating Makefile
make -C ext/fast_polylines
make[1]: Entering directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
generating fast_polylines-x64-mingw-ucrt.def
compiling fast_polylines.c
linking shared-object fast_polylines/fast_polylines.so
make[1]: Leaving directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
bundle exec rspec
.................

Finished in 0.02795 seconds (files took 0.35697 seconds to load)
17 examples, 0 failures

That looks great! Everything compiles and the tests pass.

If you search the folder for the compiled static object (fast_polylines.so), you’ll see that it currently lives in ext\fast_polylines where it was compiled.

You can now try to load this in an irb session by doing this.

$ irb -Ilib -Iext
irb(main):001:0> require 'fast-polylines'
=> true
irb(main):002:0>
irb(main):003:0> puts $LOAD_PATH
d:/projects/github/gems/fast-polylines/src/fast-polylines/lib
d:/projects/github/gems/fast-polylines/src/fast-polylines/ext
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/timeout-0.3.0/lib
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/strscan-3.0.3/lib
C:/Ruby31-x64/lib/ruby/gems/3.1.0/extensions/x64-mingw-ucrt/3.1.0/strscan-3.0.3
C:/Ruby31-x64/lib/ruby/site_ruby/3.1.0
C:/Ruby31-x64/lib/ruby/site_ruby/3.1.0/x64-ucrt
C:/Ruby31-x64/lib/ruby/site_ruby
C:/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0
C:/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0/x64-ucrt
C:/Ruby31-x64/lib/ruby/vendor_ruby
C:/Ruby31-x64/lib/ruby/3.1.0
C:/Ruby31-x64/lib/ruby/3.1.0/x64-mingw-ucrt
=> nil

We used -I and passed it two paths: lib and ext where the native extension was built. When we print the $LOAD_PATH, you can see that these are on top and we can require items from it. In this case, we pass the paths as two separate parameters -Ilib and -Iext. On Linux, you can also do -Ilib:ext and on Windows, you can do -Ilib;ext to pass multiple paths. However, on this particular gem, there is one step which runs from the Windows command prompt and one that runs using sh.exe and the two have different expectations for the path separator. For this reason, passing the paths separately is easier and guaranteed to work.

Warning: It’s possible that you don’t pass any local path to IRB and it still seems to work, or you pass the wrong path to IRB and it seems to work. This will happen if the gem is also installed on your system. In that case, the require works with the system-installed gem and you get the feeling that things are working – but they are not! You’ll spend countless hours figuring out why your changes don’t show. It may be worthwhile temporarily uninstalling the gem using gem uninstall fast-polylines so that it does not interfere with your work.

While we are at it, let’s also run the performance tests. We only need to do make benchmark in this case [note that your numbers will be different based on your computer and what else is going on when you run this].

$ make benchmark
make -C ext/fast_polylines
make[1]: Entering directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
bundle exec ruby -Ilib -Iext -r fast_polylines ./perf/benchmark.rb

                                 ENCODING

Warming up --------------------------------------
           Polylines    96.000  i/100ms
       FastPolylines    26.852k i/100ms
Calculating -------------------------------------
           Polylines      1.087k (±11.7%) i/s -      5.376k in   5.028313s
       FastPolylines    251.373k (±23.5%) i/s -      1.208M in   5.026706s

Comparison:
       FastPolylines:   251372.7 i/s
           Polylines:     1087.3 i/s - 231.19x  (± 0.00) slower


                                 DECODING

Warming up --------------------------------------
           Polylines    63.000  i/100ms
       FastPolylines    15.301k i/100ms
Calculating -------------------------------------
           Polylines    556.245  (±21.6%) i/s -      2.646k in   5.029865s
       FastPolylines    136.162k (±18.9%) i/s -    657.943k in   5.072674s

Comparison:
       FastPolylines:   136162.3 i/s
           Polylines:      556.2 i/s - 244.79x  (± 0.00) slower

You notice these lines in the output:


                                 ENCODING
           Polylines:     1087.3 i/s - 231.19x  (± 0.00) slower

                                 DECODING
           Polylines:      556.2 i/s - 244.79x  (± 0.00) slower

So, in my case (Ruby 3.1 on Windows, running on battery while doing other things), the encoding process is 231 times faster, and the decoding is 244x faster. Yay (to native extensions).

What we have seen so far

We have covered a fair bit of ground so far.

We looked a few gem commands
- We saw how to download a gem locally using gem fetch
- We then were used gem unpack to look at the files
- We grabed the gemspec using gem spec
We then grabbed the source code from GitHub
We built it, ran tests on it, used the built version from irb and ran the performance benchmark

Now, we have only one more thing to do before we move on – let’s make a small change to the code and build it; then, let’s package and install this version of the gem for our use.

Making Changes to the gem

In fact, we will make two changes – one in the Ruby side, and one on the C side.

1. Ruby: Making the version visible

If you use the console to see the version, it won’t show up.

$ make console
make -C ext/fast_polylines
make[1]: Entering directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
irb -Ilib -Iext -r fast_polylines
irb(main):001:0> FastPolylines
irb(main):002:0*
=> FastPolylines
irb(main):003:0> FastPolylines::VERSIOM
(irb):3:in `<main>': uninitialized constant FastPolylines::VERSIOM (NameError)
        from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
        from C:/Ruby31-x64/bin/irb:33:in `load'
        from C:/Ruby31-x64/bin/irb:33:in `<main>'
irb(main):004:0>
irb(main):005:0> exit

In lib\fast_polylines.rb, we’ll add a require_relative for the version.rb so that we can simply do FastPolylines::VERSION once the gem is required.

# frozen_string_literal: true

require_relative "fast_polylines/version"
require "fast_polylines/fast_polylines"

Now, we start the console again to see if it works.

$ make console
make -C ext/fast_polylines
make[1]: Entering directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
irb -Ilib -Iext -r fast_polylines
irb(main):001:0> p FastPolylines::VERSION
"2.2.2"
=> "2.2.2"
irb(main):002:0> exit

2. C: Enabling Debug Mode

The next change is a bit more difficult because we want to make a change to the C source code so that it recompiles the native extension and then we can see that the change is reflected. It’s a bit more difficult to do this because one of two things will happen when we make this change:

Either we need to understand the C extension a lot better
Or we will change something that might break a test

The C source code is in ext/fast_polylines/fast_polylines.c and the lowest hanging fruit is on Line 38: the DEBUG macro. If we set this to 1, it will output debug information. We previously saw that running the tests is a simple clean run. We would expect to have a much noisier output with debug information printed if set DEBUG to 1. Let’s uncomment this line and do a make test again.

// Uncomment this line to show debug logs.
#define DEBUG 1

Let’s make and run the tests again.

$ make test
cd ext/fast_polylines && ruby extconf.rb --vendor
creating Makefile
make -C ext/fast_polylines
make[1]: Entering directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
generating fast_polylines-x64-mingw-ucrt.def
compiling fast_polylines.c
linking shared-object fast_polylines/fast_polylines.so
make[1]: Leaving directory '/d/projects/github/gems/fast-polylines/src/fast-polylines/ext/fast_polylines'
bundle exec rspec
rb_FastPolylines__encode(..., 5)
[[38.5, -120.2], [40.7, -120.95], [43.252, -126.453]]
allocated size: 5 * 2 * 3 = 30
_polyline_encode_number("`", 3850000)
5 encoded chunks
chunks: _p~iF☺
/_polyline_encode_number
_p~iF☺
_polyline_encode_number("☺", -12020000)
5 encoded chunks
... output truncated ...

There we have it – we changed the source code and everything compiled up nicely as we expect it to. Let’s undo our change so that it’s not running in DEBUG any more, and compile it one more time.

Package and install the newly built gem

Before we wrap up this post, there is one last thing that we would like to do. After making changes to the code, we are able to now run the performance test, run the tests, and use it from a console. We also want to be able to install the locally built version into our system so that we can use it in something else that depends on it.

Before we do this, let’s change the version of the local gem so that it’s different from the one we downloaded. This is easily done in lib/fast_polylines/version.rb by changing the line there to VERSION = "2.2.2.1"

Now, we need to package the gem. The command for this is gem build and it needs you to point to the gemspec. In my src/fast-polylines directory, I did this.

$ gem build fast-polylines.gemspec
  Successfully built RubyGem
  Name: fast-polylines
  Version: 2.2.2.1
  File: fast-polylines-2.2.2.1.gem

Notice that it uses the new version (2.2.2.1) in the name of the locally built gem. If you wanted to go back the full loop, you could unpack this and get started again!

Finally, you can install the gem locally by doing gem install with the path to the local gem.

$ gem install fast-polylines-2.2.2.1.gem
Building native extensions. This could take a while...
Successfully installed fast-polylines-2.2.2.1
Parsing documentation for fast-polylines-2.2.2.1
Installing ri documentation for fast-polylines-2.2.2.1
Done installing documentation for fast-polylines after 0 seconds
1 gem installed

$ gem list fast-polylines

*** LOCAL GEMS ***

fast-polylines (2.2.2.1)

You can see it rebuilt the native gem and installed it. We can now us this from irb without doing anything about the load path, etc.

$ irb
irb(main):001:0> require 'fast-polylines'
=> true
irb(main):002:0> puts FastPolylines::VERSION
2.2.2.1
=> nil
irb(main):003:0>

Looking ahead

With that, we come to the end of this long post. We now have a working knowledge of how this gem works and does different things. We looked at a number of gem commands that allowed us to download, build and install the gem locally. We made some changes and saw them reflected. From the next post on, we will start to look at different aspects of this native gem and how it all connects up when programming a native gem.

I will add links and references later, possibly in the last post of the series. If you have any comments, please feel free to leave them below.