GitLab CI for QA

GitLab CI Overview #

GitLab CI/CD is built directly into GitLab — no plugins, no separate service, no additional setup. Every GitLab repository can use CI/CD by adding a .gitlab-ci.yml file to the repository root. GitLab detects this file automatically and runs the pipeline.

For QA engineers, GitLab CI offers several advantages: native test reporting in merge requests, built-in container registry, environment management, and review apps for testing deployments.

.gitlab-ci.yml Structure #

Basic Pipeline #

stages:
  - build
  - test
  - deploy

install:
  stage: build
  image: node:20
  script:
    - npm ci
  artifacts:
    paths:
      - node_modules/
    expire_in: 1 hour

unit-tests:
  stage: test
  image: node:20
  script:
    - npm run test:unit -- --ci --coverage
  artifacts:
    reports:
      junit: junit-results.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  coverage: '/Lines\s*:\s*(\d+\.?\d*)%/'

e2e-tests:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0-focal
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    paths:
      - playwright-report/
      - test-results/
    reports:
      junit: test-results/junit.xml
    expire_in: 7 days

Key Concepts #

Services: Test Dependencies #

GitLab CI services spin up Docker containers alongside your job. This is perfect for integration testing:

integration-tests:
  stage: test
  image: node:20
  services:
    - name: postgres:15
      alias: db
    - name: redis:7
      alias: cache
  variables:
    POSTGRES_DB: test_db
    POSTGRES_USER: test_user
    POSTGRES_PASSWORD: test_pass
    DATABASE_URL: "postgresql://test_user:test_pass@db:5432/test_db"
    REDIS_URL: "redis://cache:6379"
  script:
    - npm ci
    - npm run test:integration

The service containers are accessible by their alias names (db, cache) as hostnames within the job.

Merge Request Pipelines #

GitLab can run pipelines specifically for merge requests, showing results directly in the MR:

e2e-tests:
  stage: test
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"
  script:
    - npx playwright test
  artifacts:
    reports:
      junit: test-results/junit.xml

Test results from JUnit reports appear as a “Tests” tab in the merge request, showing pass/fail counts and individual test details.

Parallel and Matrix Jobs #

Parallel Keyword #

Split a job across multiple runners automatically:

e2e-tests:
  stage: test
  parallel: 4
  script:
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

GitLab sets CI_NODE_INDEX (1-4) and CI_NODE_TOTAL (4) automatically, enabling test sharding without manual configuration.

Matrix Strategy #

e2e-tests:
  stage: test
  parallel:
    matrix:
      - BROWSER: [chromium, firefox, webkit]
  image: mcr.microsoft.com/playwright:v1.40.0-focal
  script:
    - npx playwright test --project=$BROWSER

Caching #

Cache dependencies between pipeline runs to speed up builds:

default:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
    policy: pull-push

Environment Management #

GitLab environments let you track deployments and connect them to testing:

deploy-staging:
  stage: deploy
  script:
    - ./deploy.sh staging
  environment:
    name: staging
    url: https://staging.example.com

smoke-tests:
  stage: test
  needs: [deploy-staging]
  script:
    - npx playwright test --config=smoke.config.ts
  environment:
    name: staging
    action: verify

Exercise: Design a GitLab CI Pipeline #

Create a .gitlab-ci.yml for a web application with:

• Build stage with dependency installation
• Test stage with unit, integration (needs PostgreSQL), and E2E tests in parallel
• Deploy to staging on main branch
• Smoke tests after staging deployment
• Test reports visible in merge requests

stages:
  - build
  - test
  - deploy
  - verify

default:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/

install:
  stage: build
  image: node:20
  script:
    - npm ci
  artifacts:
    paths:
      - node_modules/
    expire_in: 1 hour

unit-tests:
  stage: test
  image: node:20
  needs: [install]
  script:
    - npm run test:unit -- --ci --coverage
  artifacts:
    reports:
      junit: junit-results.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  coverage: '/Lines\s*:\s*(\d+\.?\d*)%/'

integration-tests:
  stage: test
  image: node:20
  needs: [install]
  services:
    - name: postgres:15
      alias: db
  variables:
    POSTGRES_DB: test_db
    POSTGRES_USER: test
    POSTGRES_PASSWORD: test
    DATABASE_URL: "postgresql://test:test@db:5432/test_db"
  script:
    - npm run test:integration
  artifacts:
    reports:
      junit: integration-results.xml

e2e-tests:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0-focal
  needs: [install]
  parallel:
    matrix:
      - BROWSER: [chromium, firefox, webkit]
  script:
    - npm ci
    - npx playwright test --project=$BROWSER
  artifacts:
    when: always
    paths:
      - playwright-report/
      - test-results/
    reports:
      junit: test-results/junit.xml
    expire_in: 7 days

deploy-staging:
  stage: deploy
  image: alpine:latest
  needs: [unit-tests, integration-tests, e2e-tests]
  script:
    - ./scripts/deploy-staging.sh
  environment:
    name: staging
    url: https://staging.example.com
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

smoke-tests:
  stage: verify
  image: mcr.microsoft.com/playwright:v1.40.0-focal
  needs: [deploy-staging]
  script:
    - npm ci
    - npx playwright test --config=smoke.config.ts
  environment:
    name: staging
    action: verify
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

GitLab CI vs GitHub Actions vs Jenkins #

Best Practices for QA #

1. Always use artifacts:reports:junit to get test results in merge requests. This is one of GitLab CI’s strongest features for QA visibility.

2. Use needs instead of stage ordering when possible. The needs keyword allows jobs to start as soon as their dependencies finish, without waiting for the entire stage.

3. Set artifacts: when: always on test jobs. Without this, test reports and screenshots are lost when tests fail.

4. Use parallel for large test suites. GitLab’s built-in parallel keyword with CI_NODE_INDEX/CI_NODE_TOTAL makes sharding trivial.

5. Leverage services for integration tests. Starting PostgreSQL, Redis, or other dependencies as services is simpler and faster than installing them in the job.

6. Use rules instead of only/except. The rules keyword is more powerful and the recommended approach for controlling when jobs execute.