Pipeline definition

Where a pipeline is defined#

You can define a pipeline in three places; all three use the same schema:

• In the console, under Delivery > Pipelines.
• In a file in the repository, usually ejo.yaml. When the binding's Pipeline source is From repo — ejo.yaml is source of truth, EJO reads the file from the bound branch and updates the pipeline when the file changes.
• In the pipeline: block of the .ejo file. See The .ejo file.

The schema is strict: an unknown key (a typo such as usees:) is an error.

A first pipeline#

name: web
description: Build, scan and deploy to Kubernetes
steps:
  - name: Clone
    kind: GIT_CLONE
  - name: Build
    kind: KANIKO_BUILD
    config:
      dockerfile_path: Dockerfile
      registry_id: "{{.REGISTRY_ID}}"
    depends_on: [0]
  - name: Tests
    kind: SHELL
    image: node:20-alpine
    config:
      command: npm ci && npm test
    depends_on: [0]
  - name: Deploy
    kind: K8S_DEPLOY
    config:
      image: "{{.ImageTag}}"
      container_port: 3000
      wait_ready: true
      wait_seconds: 120
    depends_on: [1, 2]

Top-level fields#

Step fields#

Order and dependencies#

• Without depends_on, a step waits for the step before it (a sequential flow).
• depends_on: [] means the step can start right away.
• depends_on: [1, 2] makes the step wait for both steps 1 and 2; independent steps run in parallel.
• Cycles and self-dependencies are rejected.

Matrix#

- name: Test
  kind: SHELL
  image: python:3.12-slim
  matrix:
    suite: [unit, integration]
    db: [postgres, sqlite]
  config:
    command: pytest tests/{{.MatrixSuite}} --db={{.MatrixDb}}

Every combination runs as its own step. Axis values are available in the step as {{.MatrixSuite}} and {{.matrix_suite}}. A step can expand to at most 32 combinations. A step that depends on a matrix step waits for every combination.

Conditions: if and paths#

- name: Deploy to production
  kind: K8S_DEPLOY
  if: '{{eq .Branch "main"}}'
  paths: ["src/**", "Dockerfile"]

• if is a template; the step runs when the result is true, 1 or yes, and is skipped when it is false, 0, no or empty. Any other result is an error.
• paths are GitHub Actions style globs: * and ? do not cross a directory boundary, ** does. The step runs when any changed file matches. In runs where the changed files are unknown (manual, schedule, tag) the step always runs. EJO only gets the list of changed files from GitHub events; on GitLab and Azure DevOps repositories, steps with a paths filter always run.

Cache and artifacts#

- name: Build
  kind: SHELL
  image: node:20-alpine
  config:
    command: npm ci && npm run build
  cache:
    key: npm
    key_files: [package-lock.json]
    paths: [node_modules]
  artifacts:
    name: dist
    paths: [dist]
    when: on_success
    expire_in: 30d

Caches are kept on the agent's disk and also on the master (the remote cache), shared by every agent of the same repository binding. A restore tries the agent's disk first, then the master. Entries are removed when the organization's cache quota (default 5 GB) is exceeded, oldest hit first, or when they have not been hit for 14 days. Runs for a pull_request can read the branch caches but write only under their own pr-<number>- keys, and branch runs never restore those. A pull_request run never reads another pull request's keys, and a cache key or restore_keys entry cannot start with pr- (the pipeline is refused when saved; templates in keys must also parse). Manage the entries of a repository on its Caches tab.

Artifacts are deleted by an hourly sweeper once they pass their expire_in (or the retention default). Three kinds are never deleted: pinned artifacts (the pin button on the deployment's artifact list), the artifacts of the latest successful run that has artifacts, for each repository binding and pipeline, and the artifacts of the run that is currently deployed to a production environment (the newest successful run there, plus the build it promoted). The list marks the last two with a "kept" badge. After expiry the sweeper also enforces the organization's artifact quota (Settings > Organization > Storage; empty means unlimited, which is the default on self-hosted installs) by deleting the oldest unprotected artifacts first. Only artifacts of finished runs that are at least an hour old are deleted for the quota; protected, recent or unfinished artifacts can keep the total above it, and the master logs a warning when only such data remains.

Image build cache#

DOCKER_BUILD and KANIKO_BUILD can keep image layers in the registry the image is pushed to, so a build on a fresh agent or a fresh Kaniko pod reuses them.

- name: Build image
  kind: KANIKO_BUILD
  config:
    registry_id: "..."
    cache: registry          # off | registry (true also means registry)
    cache_repo: registry.example.com/team/app/cache   # optional
    cache_ttl: 168h          # optional, KANIKO_BUILD only

KANIKO_BUILD passes --cache=true --cache-repo=... --cache-ttl=... to Kaniko. DOCKER_BUILD uses docker buildx build --cache-from type=registry,ref=... --cache-to type=registry,ref=...,mode=max --load. Without a registry_id there is nowhere to keep the cache, and when docker buildx is missing or the agent's builder cannot export a registry cache (the default docker driver needs the containerd image store, or a docker-container builder), the step logs a warning and builds with the Docker daemon's own layer cache. Without a cache key DOCKER_BUILD always builds that way.

The image cache is read-only for pull_request runs: they reuse the layers of branch builds but never write any (KANIKO_BUILD adds --no-push-cache, DOCKER_BUILD leaves out --cache-to), so untrusted code cannot seed layers a branch build would trust. Saving a pipeline checks cache, cache_ttl and cache_repo.

Notes for specific registries: Amazon ECR needs the cache repository to exist (for Kaniko, create <image repository>/cache or set cache_repo), and for BuildKit EJO adds image-manifest=true,oci-mediatypes=true to the cache export on ECR hosts. Cache layers are written under the same repository namespace as the image, so any credential that can push the image can push its cache. Kaniko works with every registry that accepts a push; the cache repository grows with every distinct layer until the registry's own lifecycle policy cleans it up, so set one on registries that bill by storage.

Step kinds#

The most common config keys are listed below. The console's step form shows every field with its description.

Note: DOCKER_BUILD, PODMAN_BUILD and KANIKO_BUILD need a registry to push to. Define registries under Infrastructure > Registries and pick one on the step with registry_id. The automatic SBOM and scan also only run for images pushed to a registry.

Variables and templates#

config values may use {{.Name}} templates. The agent fills them in when the run starts.

In addition:

• Your variables: every key you define under Infrastructure > Variables is available as {{.KEY}}. See Secrets and variables.
• Outputs of earlier steps: for example {{.ImageTag}} and {{.ImageDigest}} after a build step, {{.WorkDir}} after GIT_CLONE.

Step outputs#

Each step kind produces certain outputs:

A SHELL step writes its own outputs to the $EJO_OUTPUT file as NAME=value lines and lists them under config.outputs:

- name: Find the version
  kind: SHELL
  image: alpine:3.20
  config:
    command: echo "APP_VERSION=$(cat VERSION)" >> "$EJO_OUTPUT"
    outputs: [APP_VERSION]
- name: Use it
  kind: SHELL
  image: alpine:3.20
  config:
    command: echo "Version {{.APP_VERSION}}"

Outputs cannot overwrite system variables such as WorkDir, CommitSHA or Branch.

Manual run inputs#

A pipeline can ask for values on runs started by hand:

name: web
inputs:
  - {name: target, options: [staging, production], default: staging}
  - {name: dry_run, type: bool, default: "false"}
steps:
  - name: Show
    kind: SHELL
    image: alpine:3.20
    config:
      command: echo "target=$INPUT_TARGET dry_run=$INPUT_DRY_RUN"

• Input fields: name, type (string, number, bool), required, default, options, pattern, label, help.
• Values reach every step as INPUT_<NAME> environment variables.
• Required inputs are only asked for on manual runs. Push and tag runs use the defaults; for a schedule, every required input needs a default.
• Inputs are stored in the clear, so they cannot be secret. Read secrets from variables instead.

Next steps#

• Triggers and bindings
• Secrets and variables
• Step packs
• Supply chain security