Setting up Redis-backed remote caching for GitLab CI

Replacing ephemeral local runner caches with a centralized Redis backend standardizes build acceleration across distributed GitLab CI environments. This architectural shift eliminates runner-specific cache fragmentation and reduces cold-start latency for frontend and full-stack pipelines. Platform teams implementing this approach align directly with established Build Optimization & Caching Strategies frameworks. The following guide details production-grade deployment, network topology alignment, and integration with modern build systems.

How GitLab CI Redis Caching Works

GitLab Runner’s cache backend is configured in the runner daemon’s config.toml, not in .gitlab-ci.yml. When the [runners.cache] section specifies Type = "redis", the runner transparently stores and retrieves cache archives via Redis instead of the local filesystem. The .gitlab-ci.yml cache: directive controls which paths and keys to cache — the runner handles the backend transparently.

GitLab Runner supports Redis as a cache backend from version 13.0+. Redis 6.2+ is recommended for native ACL support.

Prerequisites & Redis Topology Requirements

Redis 6.2+ is required for native ACL support and optimized memory eviction. Allocate a minimum of 4 GB RAM per instance to prevent premature LRU eviction during peak pipeline concurrency. Enforce TLS 1.2+ and strict ACL rules limiting runners to +get +set +del +exists +expire commands.

Network latency between runners and the Redis cluster must remain below 50 ms RTT to avoid pipeline stalls. Store credentials securely using GitLab CI/CD masked variables (REDIS_PASSWORD, REDIS_TLS_CA). Configure the runner daemon to maintain persistent connection pools.

# /etc/gitlab-runner/config.toml — runner-level Redis cache configuration
[[runners]]
  [runners.cache]
    Type = "redis"
    Shared = true
    [runners.cache.redis]
      URL = "rediss://:${REDIS_PASSWORD}@redis.internal:6379/0"
      TLSInsecureSkipVerify = false
      PasswordFromFile = ""

Validate the runner configuration before restarting the service. Use gitlab-runner verify to confirm TLS handshake success.

Step-by-Step Configuration Implementation

With the runner daemon configured to use Redis, the .gitlab-ci.yml cache: directive works exactly as it does with the default filesystem backend — no special Redis keys are needed in the YAML.

# .gitlab-ci.yml — cache directive; backend is controlled by config.toml
build:
  cache:
    key: "${CI_PROJECT_PATH_SLUG}-${CI_COMMIT_REF_SLUG}-v2"
    paths:
      - node_modules/.vite/
      - node_modules/.cache/
    policy: pull-push
  script:
    - npm ci
    - npm run build

The runner compresses the listed paths into a cache archive and stores it in Redis under the generated key. policy: pull-push reads the archive at job start and writes back any changes on success.

Frontend Toolchain Integration & Cache Mapping

Map framework-specific cache directories to the paths defined in your pipeline configuration. Vite requires explicit cacheDir overrides (set as a top-level vite.config.ts option), while Webpack relies on cache.type: 'filesystem'. Isolate node_modules/.cache and dist directories to prevent framework state corruption across parallel jobs.

Review Optimizing Webpack and Vite for CI Environments for detailed invalidation patterns and environment variable overrides. Enforce deterministic cache routing using explicit CI variable exports.

# Force deterministic cache paths in CI
export VITE_CACHE_DIR="$CI_PROJECT_DIR/node_modules/.vite"
export WEBPACK_CACHE_DIR="$CI_PROJECT_DIR/node_modules/.cache/webpack"

# Validate directory creation before build execution
mkdir -p "$VITE_CACHE_DIR" "$WEBPACK_CACHE_DIR" || { echo "Cache path creation failed"; exit 1; }

Rollback & Parity Safeguards

Implement semantic cache versioning using v1- or v2- prefixes to enable instant invalidation during major dependency upgrades. On network degradation the runner automatically falls back to a cache miss and proceeds without the archive. Detect cache poisoning by validating artifact checksums before extraction.

# Emergency cache bypass toggle
if [ "$CI_CACHE_BYPASS" = "true" ]; then
  rm -rf node_modules/.vite node_modules/.cache dist
  echo "Cache bypassed. Performing clean build."
else
  echo "Remote cache active. Proceeding with extraction."
fi

Manual flush procedures require direct Redis CLI access or a scheduled CI job executing DEL on affected keys. Use --scan --pattern to identify keys by project prefix before deletion.

# Flush all cache keys for a project
redis-cli --scan --pattern "${CI_PROJECT_PATH_SLUG}-*" | xargs -r redis-cli DEL \
  || echo "Cleanup failed. Check Redis permissions."

Performance Trade-offs & Observability

Analyze network I/O overhead against local disk latency when scaling to multi-runner fleets. Redis typically delivers sub-10 ms read latency but introduces serialization costs for large node_modules archives. Set explicit TTLs between 24 h and 72 h based on artifact volatility; configure maxmemory-policy allkeys-lru and set expire_in in the runner’s cache configuration.

Monitor cache hit ratios using redis-cli INFO stats and correlate with GitLab pipeline duration metrics. High-throughput operations may require Redis cluster sharding to prevent connection pool exhaustion. Track eviction rates to adjust maxmemory allocations dynamically.

Common Failures & Resolution

Symptom: Pipeline stalls with connection refused or TLS handshake errors. Resolution: Inject the Redis CA bundle via the REDIS_TLS_CA CI variable. Verify ACL permissions and validate egress security groups for ports 6379/6380.

# Validate TLS connectivity before pipeline execution
openssl s_client -connect redis.internal:6379 -CAfile "$REDIS_TLS_CA" < /dev/null \
  || { echo "TLS validation failed"; exit 1; }

Symptom: High cache miss rate despite identical dependency versions. Resolution: Standardize key generation using ${CI_COMMIT_REF_SLUG} and implement deterministic hashing via cache:key:files: [package-lock.json]. Ensure all runners use the same config.toml Redis endpoint.

Symptom: Redis memory eviction causing pipeline extraction failures. Resolution: Configure maxmemory-policy allkeys-lru in Redis. Enforce expire_in: 48 hours in the runner’s config.toml cache section and schedule automated prefix cleanup jobs.

Frequently Asked Questions