Why Is Your GitLab Pipeline Still Waiting on Unrelated Jobs to Finish?

I’ve spent more time than I’d like to admit watching a GitLab pipeline spin on a single, stubborn integration test while perfectly valid deployment jobs sit idle in the next stage. It feels like waiting for a slow walker to clear the sidewalk before you can even think about crossing the street. You know the job is ready to go, the runner is available, but the system just... waits.

By default, GitLab CI/CD is a linear beast. It operates on stages. All jobs in the build stage must pass before any job in the test stage begins. All test jobs must finish before deploy kicks off. It's a safe, logical checklist, but it’s also a massive waste of time if your "Documentation" job is waiting for a "Heavy End-to-End Test" that has nothing to do with it.

The Problem with the "Wall"

Most .gitlab-ci.yml files look something like this:

stages:
  - build
  - test
  - deploy

build_app:
  stage: build
  script: npm run build

build_docs:
  stage: build
  script: npm run docs:build

test_unit:
  stage: test
  script: npm run test:unit

test_e2e:
  stage: test
  script: npm run test:e2e

deploy_preview:
  stage: deploy
  script: ./deploy.sh

In this setup, deploy_preview won't start until test_unit and test_e2e are both finished. If your E2E tests take fifteen minutes and your unit tests take thirty seconds, your preview environment is sitting in limbo for fourteen and a half minutes for no reason.

This is where the Directed Acyclic Graph (DAG) comes in, or as GitLab calls it: the needs keyword.

Breaking the Linearity

The needs keyword allows a job to ignore stage boundaries. It tells the runner: "As soon as these specific jobs are done, I’m good to go. I don't care what else is happening in the pipeline."

Let’s refactor that previous example:

stages:
  - build
  - test
  - deploy

build_app:
  stage: build
  script: npm run build

test_unit:
  stage: test
  needs: ["build_app"]
  script: npm run test:unit

deploy_preview:
  stage: deploy
  needs: ["test_unit"]
  script: ./deploy.sh

test_e2e:
  stage: test
  needs: ["build_app"]
  script: npm run test:e2e

Look at what happened to deploy_preview. It no longer waits for the entire test stage to complete. It only "needs" test_unit. If your unit tests finish quickly, your preview deployment starts immediately, even while the E2E tests are still grinding away in the background.

This isn't just about saving a few minutes; it’s about tightening the feedback loop. I want to see my preview site as fast as possible.

The Artifact Gotcha

One thing that tripped me up early on is how needs handles artifacts. Usually, a job automatically downloads all artifacts from previous stages. However, when you use needs, GitLab assumes you only want artifacts from the jobs you’ve specifically listed.

If deploy_preview needs the output of build_app, but you only listed test_unit in the needs array, your deployment will probably fail because the build folders are missing. You have to be explicit:

deploy_preview:
  stage: deploy
  needs:
    - job: build_app
      artifacts: true
    - job: test_unit
      artifacts: false # I just need the test to pass, I don't need its reports
  script: ./deploy.sh

When Should You Use This?

I tend to reach for needs when I'm dealing with monorepos or decoupled services. If you have a frontend and a backend folder in the same repo, there is zero reason for the frontend linting job to wait for the backend database migrations to finish.

By defining the relationships clearly, you create multiple parallel paths through the pipeline. It looks less like a series of hurdles and more like a river delta.

A Few Reality Checks

It’s tempting to needs-ify everything, but there are some constraints to keep in mind:

1. Complexity: If you have a pipeline with 50 jobs and they all have complex needs relationships, debugging a failure becomes a nightmare. Sometimes the "wall" of stages is actually helpful for mental mapping.
2. The "Must Exist" Rule: You cannot needs a job that hasn't been defined or is disabled via rules or except/only. If test_unit is skipped because it's a documentation-only commit, and deploy_preview needs it, the whole pipeline will error out before it even starts.
3. Cross-Project Needs: You can actually use needs to depend on jobs in *other* pipelines, which is powerful for microservices, but that's a rabbit hole for another day.

Final Thoughts

If your team is complaining that "the CI is slow," don't immediately jump to buying faster runners or more concurrency. Look at the graph. If you see big gaps where runners are idling while one long job finishes, you have a scheduling problem, not a hardware problem.

The needs keyword is probably the lowest-effort, highest-impact change you can make to a GitLab config. Stop making your jobs wait in line if they don't have to.