How to Set Up Bitbucket Pipelines on a Rails 5.2 Environment with react_on_rails and webpack
Reading time: ~ 5 minutes
I noticed a bit of a gap when it comes to documentation and setting up Bitbucket Pipelines when you have some modern webpack dependencies with yarn.
If you’ve gone through this process, you may have struggled to find helpful documentation, too. I thought it may be useful to walk through some base understanding of how Bitbucket Pipelines is set up and strategies to help debug your pipelines configuration. Read on for my experience going through the process of setting up Bitbucket Pipelines on a Rails 5.2 environment.
Setting up a Docker environment.
The first thing that is useful to know about setting up your environment for bitbucket pipelines is that you’re setting up a docker container for your Rails app to run in.
Your default docker container for your pipelines environment will be on Ubuntu 14.04, which is important to know because you will use Ubuntu command line tools to install your dependencies.
Choosing your Ruby Image
Getting started, we used a base Ruby image so we had the correct version of Ruby that matched our Ruby version for our Rails application.
In the Pipelines documentation, they say you can find your list of official ruby images. I didn’t find much luck using those listed here with the alpine/slim namespacing.
If you’re having trouble finding one that works here like I experienced, I would recommend just matching your ruby image name with your ruby version like so:
image: ruby:2.4.0
Setting up your Javascript environment
Since we have a nice integration with yarn using Rails webpacker and react_on_rails, we need to make sure that we can install yarn and have those dependencies defined your package.json when the application builds.
Unfortunately, the version of node that is installed in the default container is not high enough to support yarn. So before we can build you need to install an updated version of node.
An important thing to note: one thing we have to do in our Ubuntu environment is first download these packages via curl and then install them into our command line with apt-get.
It seemed with this example of node vs yarn, I needed to have node successfully installed before I could apt-get yarn successfully.
My pipelines.yml file looks like this so to get a version of node installed that can support yarn:
image: ruby:2.4.0
pipelines:
default:
- step:
script:
- curl -sL https://deb.nodesource.com/setup_8.x | bash -
- apt-get update && apt-get install -y nodejs
And these lines get added to install yarn into our environment:
- curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
- echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
- apt-get update
- apt-get install -y yarn
From here we need to make sure we install our packages from our package.json with:
yarn install
Setting Up Your Database Configs
In all of our Rails applications we keep our database.yml under a .gitignore so each person can modify their own local dev environments.
Our Rails environment on pipelines needs a database.yml to look at to tell it which database to use. In our case for Postgres, I created a database.ci.yml to use specifically just for bitbucket pipelines. That file looks like this:
# this file is used in the build process of bitbucket pipelines CI
default: &default
adapter: postgresql
encoding: unicode
# For details on connection pooling, see rails configuration guide
# http://guides.rubyonrails.org/configuring.html#database-pooling
pool: 5
host: localhost
development:
<<: *default
database: example_ci_development
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: example_ci_test
username: test_user
password: test_user_password
production:
<<: *default
database: example_ci_production
Now in my script I’ll want to make sure that we set our DATABASE_URL to what matches our database config as well as rename this file to database.yml so Rails knows that it’s the file to use
- export DATABASE_URL=postgresql://test_user:test_user_password@localhost/communityright_ci_test
- cp config/database.ci.yml config/database.yml
You’ll also need to make sure that you use a service that matches whatever database you are using. Here’s a guide to using services and databases from Bitbucket support.
For our app that looks like this:
pipelines:
default:
- step:
script:
- all scripts run here
services:
- postgres
Definitions define what services we call from the main pipelines task.
definitions:
services:
postgres:
image: postgres
environment:
POSTGRES_DB: example_ci_test
POSTGRES_USER: test_user
POSTGRES_PASSWORD: test_user_password
We also define ‘Redis’ in our services here since we are using Sidekiq for our ActiveJob workers in the application. You can see an example of that in the full file below.
Other .gitignored files
We also .gitiginored our config/secrets.yml - so I made sure that we made a ci specific secrets.yml and copied that over for Rails to use (it will need it to run specs)
- cp config/secrets.ci.yml config/secrets.yml
Caching Your Packages and Bundler Files
Having to do all of this installation of yarn packages and ruby gems on each pipelines build eats up a lot of time.
To cut down on this time we can cache our node_modules and vendor directories. This way these packages are available if there are no updates.
You define them first in your definitions, and make sure you are caching the correct folders.
pipelines:
default:
- step:
caches:
- bundler
- yarn
script:
- other scripts
- yarn install
- bundle install --path vendor
definitions:
caches:
bundler: ./vendor
yarn: ./node_modules
Running your feature specs with capybara-webkit
If you’re using capybara-webkit for your headless browser, you will run into issues trying to run your feature specs.
In order to get around this, you can install and configure xvfb and run your specs that way. If you are experiencing your specs timing out on your features, it’s likely you need to install xvfb for your environment to run your specs.
In the full file shown below, you can see how we are installing xvfb in our docker environment with apt-get (along with our other dependencies).
You can then run your specs with it with:
xvfb-run -a bundle exec rspec
Full file for Pipelines
There are a few other dependencies in here that I didn’t dig into in detail. You may or may not need for your Rails 5+ application. From this point, you should do a little trial and error to see which of these you need.
image: ruby:2.4.0
pipelines:
default:
- step:
caches:
- bundler
- yarn
script:
- curl -sL https://deb.nodesource.com/setup_8.x | bash -
- apt-get update && apt-get install -y nodejs apt-transport-https
- curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
- echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
- apt-get update
- apt-get install -y xvfb yarn qt5-default libqt5webkit5-dev gstreamer1.0-plugins-base gstreamer1.0-tools gstreamer1.0-x
- yarn install
- export DATABASE_URL=postgresql://test_user:test_user_password@localhost/example_ci_test
- cp config/database.ci.yml config/database.yml
- cp config/secrets.ci.yml config/secrets.yml
- bundle install --path vendor
- RAILS_ENV=development bundle exec rake db:setup
- RAILS_ENV=test bundle exec rake db:test:prepare
- xvfb-run -a bundle exec rspec
services:
- redis
- postgres
definitions:
caches:
bundler: ./vendor
yarn: ./node_modules
services:
redis:
image: redis
postgres:
image: postgres
environment:
POSTGRES_DB: example_ci_test
POSTGRES_USER: test_user
POSTGRES_PASSWORD: test_user_password
Running into issues debugging?
If things start to get really hairy, you can actually boot up a docker environment on your local machine and run through your scripts to see what’s wrong with your syntax.
You’ll first need to install Docker on your machine.
Once you get that up and running, you can use the command line tools and boot up an environment for your project using syntax like so:
docker run -it --volume=/Users/corinnekunze/dev/example-app:/example-app --workdir="/example-app" --memory=2048m ruby:2.4.0 /bin/bash
Then, you can start writing your scripts out line-by-line and see what you can get working.
Just know that if you are altering files here in your docker environment it’s going to modify your local directory, for example renaming your database.yml files.
Have you set up Bitbucket Pipelines on a Rails 5+ environment with react_on_rails and webpack? Did you run into any issues along the way? Leave a comment below with your experience if you'd like to share.