GitLab offers an excellent continuous integration (CI) service. They even provide some free worker machines shared among all users, but they're underpowered, oversubscribed, and each user is limited to 2000 minutes per month (you can check your usage in your settings).
One nice feature about GitLab CI is that it lets you Bring Your Own Compute. We can set up our own machines running the GitLab Runner service, register them on GitLab, and then our projects can start sending CI jobs to those machines.
This post tries to explain how GitLab Runner works so that when you try to install, launch, and configure it, you'll know what questions to ask.
First, we need to clear up some terminology, especially "GitLab Runner", which is overloaded:
Your GitLab instance is the GitLab service hosting your projects. It is identified by its URL. For most of us, it will be
From the perspective of a GitLab instance user, a GitLab Runner is a worker machine that executes your project's GitLab CI jobs, through work stealing. Each Runner, as it becomes available, continually sends requests to the GitLab instance, asking to be assigned jobs. Different Runners can have different capabilities (e.g. size, speed, or operating system), and they include these capabilities in their requests via tags that the GitLab Runner administrator includes in its configuration. To match your jobs to the right Runners, you have to tag your jobs as well; only Runners with all the same tags as a job will be eligible to execute that job. A Runner can be specific (available to individually chosen projects), group (available to all projects in a group), or shared (available to every project in the GitLab instance).
From the perspective of a GitLab Runner administrator, a GitLab Runner is an abstract object within a single
gitlab-runnerprocess, which can host multiple GitLab Runners. In this sense, a GitLab Runner is not a machine waiting for jobs the way a user might think of it. Instead, a GitLab Runner is like a service responsible for executing one or many jobs and reporting their progress back to the GitLab instance. It can execute the job on the same machine as its host
gitlab-runnerprocess, or on a different machine. That machine might exist before the job is submitted, or it might be launched on the fly in response to the job's submission.
From this point on, I will consistently use "GitLab Runner" to refer to the
abstract workers and "
gitlab-runner" to refer to the software or process.
When you install
know that it is generally installed as a long-running background service. On
Linux, you can check on it with
$ systemctl | grep gitlab
gitlab-runner.service loaded active running GitLab Runner
$ systemctl status gitlab-runner.service
● gitlab-runner.service - GitLab Runner
Active: active (running) since Fri 2019-03-01 11:05:36 CST; 3 weeks ago
Main PID: 929 (gitlab-runner)
Tasks: 16 (limit: 19660)
└─929 /usr/bin/gitlab-runner run
There are a few
for interacting with the service:
$ sudo gitlab-runner status
gitlab-runner: Service is running!
These commands are deprecated and will eventually be removed.
restart have corresponding
signals, but the rest do
not. It seems that GitLab expects us to use our platform's service manager
systemd) for the rest.
# Restart the service with a signal...
$ sudo killall -SIGHUP gitlab-runner
# ...or with systemd.
$ sudo systemctl restart gitlab-runner.service
GitLab Runners participate in a distributed work-stealing algorithm. They
continually poll a GitLab instance for jobs that match their tags, execute
them, and communicate the progress and results back. Even though your
gitlab-runner service is running, it is empty: it is not hosting any GitLab
Runners. It can host more than one, which is a good thing, because each GitLab
Runner is configured with many
and every combination of parameters we want to use requires a distinct GitLab
GitLab Runners are configured in the configuration file for
The file's location depends on how you run
The background service on Linux, which runs as
root, uses the configuration
Due to the idiosyncracies of GitLab Runner tokens, you cannot just copy
the configuration of a GitLab Runner. Instead, Runners must be added
gitlab-runner register, which takes a registration
token and exchanges it with the GitLab instance for a runner token that
is stored in the configuration file. It is this registration process that (1)
tells your GitLab Runner which GitLab instance to ask for jobs and (2) tells
your GitLab instance that it is allowed to assign your jobs to your GitLab
$ sudo gitlab-runner register
Running in system-mode.
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
Please enter the gitlab-ci token for this runner:
Please enter the gitlab-ci description for this runner:
Please enter the gitlab-ci tags for this runner (comma separated):
Whether to run untagged builds [true/false]:
Whether to lock the Runner to current project [true/false]:
Registering runner... succeeded runner=abcdefgh
Please enter the executor: parallels, kubernetes, ssh, virtualbox,
docker+machine, docker-ssh+machine, docker, docker-ssh, shell:
Runner registered successfully. Feel free to start it, but if it's
running already the config should be automatically reloaded!
After you register your Runners, you can edit their configurations and have
gitlab-runner service reload them.
Each Runner is configured with its own
executor that defines how it
executes jobs. There is a
shell executor that executes jobs on the same
machine as the Runner. There is an
ssh executor to execute jobs on an
existing remote machine. The
docker+machine executor uses Docker
Machine to launch cloud instances,
as needed, and execute jobs in Docker containers on those instances.
You may choose to configure a fleet of Runners offering a range of sizes, speeds, operating systems, or installed dependencies. If you do, be sure to assign tags to those Runners so that your jobs can differentiate among them.