GitLab CI Cache

A slow pipeline usually repeats the same expensive setup work again and again. It downloads the same dependencies, rebuilds the same package indexes, and recreates the same local caches on every run. GitLab CI/CD caching exists to reduce that waste.

When used well, cache can cut minutes from pipelines. When used badly, it causes confusion because people expect it to behave like a guaranteed file handoff. The key is understanding what cache is actually for.

What Is Caching?

A cache in GitLab CI/CD stores files so future jobs or future pipeline runs can reuse them instead of recreating them from scratch. The goal is speed.

Common cached content includes:

• Node.js dependencies or npm cache directories
• Python virtual environments or pip caches
• Maven or Gradle dependency directories
• package manager indexes
• language toolchains downloaded during CI

Cache is best for inputs that are expensive to fetch but safe to rebuild.

Cache vs Artifacts

People often confuse cache and artifacts because both involve saved files. The difference is important.

A short rule helps:

• If a later job must use a file, prefer artifacts.
• If a later job can recreate the file but you want it to run faster, prefer cache.

Never build a pipeline that depends on cache for correctness. Cache can be missed, replaced, invalidated, or rebuilt.

Defining Cache with cache: paths:

The basic form is straightforward.

test:
  stage: test
  image: node:20
  cache:
    paths:
      - node_modules/
  script:
    - npm ci
    - npm test

This tells GitLab to store and restore the node_modules/ directory as a cache.

What should go into paths

Cache paths should be directories or files that:

• are expensive to recreate
• are safe to reuse
• do not contain unique one-time outputs that must match one exact pipeline

Good examples:

• .npm/
• .cache/pip
• .m2/repository/
• .gradle/

Potentially risky examples:

• generated release bundles
• environment-specific secrets
• compiled outputs used for deployment correctness

Those are usually better handled as artifacts.

Cache Keys with cache: key:

A cache path alone is not enough. GitLab also needs a key to know which cache bucket to use.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - node_modules/

This creates a branch-based cache. Each branch gets its own cache because $CI_COMMIT_REF_SLUG expands to a branch-safe string.

Why keys matter

Without a thoughtful key, unrelated pipelines may overwrite each other or fail to reuse useful data. A good key balances two goals:

• reuse enough to improve speed
• separate enough to avoid stale or incorrect content

Branch-based key strategy

A very common strategy is one cache per branch.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - .npm/

Benefits:

• feature branches do not trample each other
• repeated commits on the same branch stay fast

Tradeoff:

• a new branch starts cold with no cache unless you add fallbacks

File-based key strategy

Another smart approach is to base the cache on dependency lock files.

cache:
  key:
    files:
      - package-lock.json
  paths:
    - node_modules/

Now GitLab refreshes the cache when package-lock.json changes. This is often better than a pure branch key because it ties reuse to dependency state rather than branch name alone.

You can also combine ideas conceptually: branch context for isolation and file-based invalidation for freshness.

Using $CI_COMMIT_REF_SLUG

$CI_COMMIT_REF_SLUG is one of the most useful built-in variables for caching. It produces a sanitized version of the branch or tag name that is safe for use in cache keys.

cache:
  key: npm-$CI_COMMIT_REF_SLUG
  paths:
    - .npm/

This pattern is simple and readable. It is especially helpful in tutorials and smaller projects because you can immediately understand what the cache is keyed on.

Cache Policies

GitLab cache behavior can be adjusted with policy:.

Default pull-push

The default behavior is effectively pull then push.

• GitLab tries to restore an existing cache before the job runs.
• After the job finishes, GitLab uploads the updated cache.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - .npm/
  policy: pull-push

This is the general-purpose default and works well for many projects.

pull

Use pull when a job should only restore cache, not update it.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - .npm/
  policy: pull

This can speed up read-heavy jobs and prevent several jobs from all trying to upload nearly identical caches.

A common pattern is:

• one install job uses pull-push
• later test jobs use pull

push

Use push when a job should create or refresh a cache but does not need to restore one first.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - .npm/
  policy: push

This is less common, but it can be useful for dedicated warm-up jobs.

Cache when:

GitLab also lets you control when a cache is saved.

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - .npm/
  when: on_success

Typical values are:

• on_success
• on_failure
• always

on_success

This is the safest common choice. It avoids publishing a cache created by a broken setup.

on_failure

Sometimes failure-time cache updates help with debugging or with partially completed package downloads, but use this carefully because failed jobs may leave inconsistent state.

always

This saves cache regardless of job outcome. It can help in some long setup flows, but it increases the risk of uploading an incomplete or undesirable cache.

For most beginner pipelines, on_success is the right mental default.

Common Caching Example: npm and node_modules

Node.js pipelines often spend a lot of time downloading packages. Caching can help.

Example using npm cache directory

build:
  stage: build
  image: node:20
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .npm/
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run build

This pattern is often better than caching node_modules/ directly because npm can manage its own cache more predictably.

Example caching node_modules/

test:
  stage: test
  image: node:20
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
  script:
    - npm ci
    - npm test

This can work, but be aware that some teams prefer caching package manager caches rather than installed dependency directories because installed directories can be bulkier and sometimes less stable across environments.

Common Caching Example: pip and Python Environments

Python projects often cache pip downloads or even a local virtual environment.

Cache pip downloads

test-python:
  stage: test
  image: python:3.12
  cache:
    key:
      files:
        - requirements.txt
    paths:
      - .cache/pip/
  script:
    - pip install --cache-dir .cache/pip -r requirements.txt
    - pytest

This is a very common and safe optimization.

Cache a virtual environment

lint-python:
  stage: test
  image: python:3.12
  cache:
    key: venv-$CI_COMMIT_REF_SLUG
    paths:
      - .venv/
  script:
    - python -m venv .venv
    - . .venv/bin/activate
    - pip install -r requirements.txt
    - flake8 .

Caching .venv/ can be effective, but it is more environment-sensitive than caching pip downloads. Use it when you understand the tradeoffs.

Common Caching Example: Maven

Java builds often benefit greatly from caching Maven dependencies.

maven-build:
  stage: build
  image: maven:3.9-eclipse-temurin-21
  cache:
    key:
      files:
        - pom.xml
    paths:
      - .m2/repository/
  script:
    - mvn -Dmaven.repo.local=.m2/repository package

The Maven local repository can be expensive to populate, so caching it often provides major wins.

Common Caching Example: Gradle

Gradle projects have their own cache directories.

gradle-test:
  stage: test
  image: gradle:8.9-jdk21
  cache:
    key:
      files:
        - build.gradle
        - settings.gradle
    paths:
      - .gradle/
  script:
    - gradle test

Again, the principle is the same: cache dependencies and reusable tool data, not unique release outputs.

Fallback Keys

A new branch often has no cache yet. GitLab supports fallback keys so a job can try other caches if the primary one is missing.

cache:
  key: cache-$CI_COMMIT_REF_SLUG
  fallback_keys:
    - cache-$CI_DEFAULT_BRANCH
    - cache-default
  paths:
    - .npm/

This means:

1. Try the branch-specific cache
2. If not found, try the default branch cache
3. If that is not found, try a generic shared cache

Fallback keys are especially useful in feature branch workflows because the first pipeline on a new branch can still benefit from dependencies cached on main.

Cache Invalidation Strategies

The hardest part of caching is not enabling it. The hard part is knowing when old cache content should stop being reused.

Strategy 1: Lock-file based invalidation

Use package lock files in key.files. When dependencies change, the cache changes too.

This is often the best strategy for Node, Python, and Java ecosystems.

Strategy 2: Branch-based isolation

Use $CI_COMMIT_REF_SLUG so each branch has its own cache.

Good when branches diverge heavily or when you want to avoid cross-branch interference.

Strategy 3: Versioned keys

Manually bump the key when you want to invalidate everything.

cache:
  key: npm-v2-$CI_COMMIT_REF_SLUG
  paths:
    - .npm/

If the cache becomes corrupted or the structure changes, moving from v1 to v2 forces a fresh start.

Strategy 4: Fallback hierarchy

Use branch cache first, then fall back to a stable default branch cache.

This gives you both speed and reasonable freshness.

Artifacts vs Cache: Quick Comparison

If you remember only one thing, remember this table.

Practical Advice for Better Caching

Cache less, but cache smart

Caching giant directories blindly is rarely optimal. Start with the package manager cache or dependency store that creates the largest repeated cost.

Do not trust cache for correctness

A pipeline must still succeed when the cache is cold or missing. Cache is an optimization, not a dependency contract.

Avoid over-sharing stale caches

If unrelated branches or dependency versions all use the same key, weird behavior can follow. Use keys with intention.

Measure before and after

Caching should improve job duration noticeably. If it adds upload overhead without saving time, refine it.

Common Mistakes

1. Using cache to pass build outputs That is what artifacts are for.

2. Using one generic cache key forever This causes stale or conflicting content.

3. Caching too much Uploading huge caches can outweigh the benefit.

4. Saving cache on failed jobs without a reason This can publish broken cache state.

5. Assuming every runner shares cache the same way Cache behavior can depend on your runner and storage configuration, so treat it as helpful but not magical.

Final Takeaway

GitLab cache is one of the simplest ways to make pipelines faster. The winning formula is usually: cache dependency-related directories, choose a thoughtful key, use fallback keys for branch workflows, and remember that cache is about performance rather than correctness. Once you apply that mindset, your pipelines become both faster and easier to maintain.

---

Knowledge Check