Run drupal.org GitLab CI locally¶
You push a branch, wait for the pipeline, and a minute later it tells you what
PHPCS could have told you before you left your desk. drupal-ci-local closes
that loop: it runs the real drupal.org GitLab CI pipeline of a module on
your machine, using the same job definitions that run on drupal.org.
It is a thin wrapper around gitlab-ci-local and ships in the GitLab Drupal CI image, which the L3D development image is built from.
When to use it¶
| You want to | Use |
|---|---|
| Get quick feedback while you code | a test *module commands from Testing |
| Check your work against the real pipeline before you push | drupal-ci-local |
The ahoy commands are the fast inner loop. drupal-ci-local is the slower
pre-push check: it uses the same jobs, images and templates as drupal.org, so
you find pipeline failures before anybody else sees them.
Prerequisites¶
An image that contains the tool¶
Your L3D image must include drupal-ci-local. Check from the project:
1 | |
If the command is not found, the image is too old. Update L3D and its image.
An SSH key for git.drupal.org¶
The pipeline templates are fetched from git.drupal.org over SSH. Any
drupal.org account key works, but it must be loaded into your host's SSH agent.
L3D forwards that agent into the container. Check on the host and in the
container:
1 2 | |
Both must list your key. Without a loaded key the run stops with
Permission denied (publickey).
Docker, disk space and bandwidth¶
gitlab-ci-local starts each job in its own container through the Docker socket, which L3D provides. Expect large first runs:
- The images for ECA's full job matrix are about 1.7 GiB compressed (drupalci PHP 8.3 and 8.5 with Apache, MySQL 8, WebDriver Chromedriver, and selenium/standalone-chrome). Every PHPUnit job starts database, Chrome and Selenium services.
- The state directory (see Where state lives) was 819 MB after the trial runs on ECA.
If pulling the public drupalci images (such as webdriver-chromedriver or
mysql-8) fails with unauthorized: HTTP Basic: Access denied, stale
credentials for registry.gitlab.com in ~/.docker/config.json are the cause.
Run docker logout registry.gitlab.com or use valid credentials. Images that
are already present are not pulled again.
Usage¶
Run the command from inside the module's Git checkout. The checkout must have a
.gitlab-ci.yml at its top level. The wrapper changes to that top level itself.
In L3D, start from the module directory, for example web/modules/contrib/eca:
1 | |
See what would run¶
1 | |
For ECA this lists 14 jobs: composer (plus next minor and next major), composer-lint, phpcs, phpstan (plus next minor and next major), stylelint, eslint, cspell, and phpunit (plus next minor and next major).
No job name runs everything
l3d drupal-ci-local without job names executes the whole pipeline shown
by --list. For ECA that includes three full PHPUnit jobs, which takes a
long time. Name the jobs you want.
Run a job and what it needs¶
1 | |
--needs runs the job together with its prerequisites. For PHPCS that means the
composer job, which assembles a complete Drupal codebase around your module.
On ECA this took 3.22 minutes for composer and 47 seconds for phpcs, 5.2
minutes in total.
Run a single job and reuse earlier results¶
1 | |
A bare job name does not run prerequisites; it reuses the artifacts stored by an
earlier run. On ECA, cspell right after the run above passed in 69 seconds.
Run manual jobs¶
Manual jobs only run when you request them by name. In merge request mode (see
below) that includes composer (next minor) and composer (next major):
1 | |
Without a job name besides --manual, this runs the whole pipeline plus that
manual job. Add --needs 'composer (next minor)' to run just that job and its
prerequisites.
Choose the pipeline mode¶
The drupal.org workflow rules skip plain pushes to feature branches. The wrapper
therefore picks a pipeline source for you, controlled by DRUPAL_CI_MODE:
| Value | Effect |
|---|---|
push (default) |
CI_PIPELINE_SOURCE=push and CI_DEFAULT_BRANCH set to your current branch. Fails on a detached HEAD. |
mr |
CI_PIPELINE_SOURCE=merge_request_event and CI_MERGE_REQUEST_IID set to $DRUPAL_CI_MR_IID, default 1. |
Any other value exits with status 2.
1 | |
Merge request mode
In this mode --list shows 15 jobs: the same 14 as in push mode plus
test-only changes, a manual job that is allowed to fail and needs
composer. composer (next minor) and composer (next major) become
manual and allowed to fail (in push mode they run always), so request them
with --manual. Which jobs a full run executes in this mode has not been
checked.
Pin the templates version¶
By default the wrapper uses the default-ref tag of
gitlab_templates, which
currently equals 1.x-latest. To use another ref:
1 | |
Pass any gitlab-ci-local option¶
The wrapper adds its own arguments first and yours last, so every
gitlab-ci-local flag works: --variable, --manual, --needs, --preview,
and so on. See the
gitlab-ci-local documentation for
the full list.
Run a filtered PHPUnit job¶
The drupal.org templates read PHPUnit options from variables. This runs only
the tests matching a filter, reusing the artifacts of an earlier composer run:
1 | |
On ECA the filter took effect (OK (9 tests, 183 assertions), PHPUnit time 42
seconds). The job took 2.95 minutes, including 1.32 minutes for the first pull
of selenium/standalone-chrome.
Clean up¶
1 | |
This deletes .gitlab-ci-local in the checkout's top level and exits.
What the wrapper sets, and why¶
| Setting | Why |
|---|---|
--remote-variables pointing to project/gitlab_templates at the chosen ref, plus _GITLAB_TEMPLATES_REPO and _GITLAB_TEMPLATES_REF |
The drupal.org pipeline includes its variables from gitlab_templates. Using one ref for both keeps variables and jobs consistent. |
CI_PIPELINE_SOURCE, CI_DEFAULT_BRANCH, CI_MERGE_REQUEST_IID (per mode) |
The drupal.org workflow rules would otherwise skip your branch. gitlab-ci-local's predefined values of these are ignored with --ignore-predefined-vars. |
--no-artifacts-to-source |
Without it, the composer job copies the whole assembled Drupal codebase (vendor/, web/, recipes/, composer.lock, a modified composer.json) into your module checkout. Inside a Drupal site that pollutes the module and git status. |
--fetch-includes |
gitlab-ci-local caches included templates under .gitlab-ci-local/includes/ and reuses them. This flag re-downloads them on every run (about 3 seconds per included file over SSH), so you always test against the current gitlab_templates ref. DRUPAL_CI_FETCH_INCLUDES=0 omits it and reuses the cached copies: faster, and works offline once cached. |
.gitlab-ci-local/ in info/exclude |
Keeps the state directory out of git status without touching your .gitignore. Added once. |
A gcl-origin remote with the SSH URL |
If origin is HTTP(S) and no gcl-origin exists, the wrapper adds one. gitlab-ci-local prefers gcl-origin, and fetching includes over HTTPS is unreliable. |
In the ECA trial git status --porcelain stayed empty after the runs.
Where state lives¶
gitlab-ci-local writes to .gitlab-ci-local/ in the checkout's top level,
owned by you. Artifacts of each job stay under
.gitlab-ci-local/artifacts/<job>/; the assembled codebase from composer is
in artifacts/composer. Job workspaces, artifacts and caches move through
Docker volumes and docker cp, so the difference between L3D's /drupal mount
and the host path does not matter. After the ECA runs no volumes, networks or
containers were left behind.
Limitations¶
- FunctionalJavascript and Nightwatch tests are untested. The templates
assume services share
localhost, while gitlab-ci-local exposes them only by alias. Setting--variable SIMPLETEST_BASE_URL=http://build/webis a guess, not a confirmed fix. - amd64 only. The bundled binary is built for x86-64 Linux.
- First runs are slow because of the image downloads described above.
More information¶
- Test locally in the gitlab_templates documentation
- gitlab-ci-local (MIT license)
- Testing for the
a test *modulecommands