Olawale Lawal
Writingtransmissions / 018

CI/CD for .NET on GitHub Actions: Change Detection, Docker Caching and Deploys

7 min readDevOpsGithub Actions.Net CoreDockerMicroservice

CI/CD for .NET on GitHub Actions: Change Detection, Docker Caching and Deploys

The starter workflow GitHub offers for .NET runs dotnet build and dotnet test on every push. That’s a fine first hour. By the end of the first month you’ve added a Dockerfile per service, a container registry, a deploy step, and the pipeline takes eleven minutes to rebuild everything because someone fixed a typo in a README.

This post is the pipeline you want at the end of that month, for a solution with a couple of services sharing some libraries, deployed as containers to a host running Docker Compose. It builds only what changed, caches at every layer that can be cached, pushes images to GitHub’s own registry, and deploys with a few seconds of interruption rather than a few minutes. It also covers the things that go wrong, because most of the time spent on pipelines is spent on those.

If you want the story of how a pipeline like this evolved iteration by iteration, including the dead ends, I wrote that up separately. This is the destination, cleaned up.

The shape of it

Four jobs, in a chain:

  1. detect-changes works out which services need rebuilding from the files in the push.
  2. test restores, builds and tests the whole solution once.
  3. build-and-push builds a Docker image per affected service, in parallel, and pushes to GHCR.
  4. deploy pulls the new images on the host and restarts only those services.

Pull requests run the first two and build images without pushing them. Pushes to main run all four.

Workflow header

name: ci

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

permissions:
  contents: read

env:
  REGISTRY: ghcr.io
  IMAGE_PREFIX: ${{ github.repository_owner }}
  DOTNET_VERSION: '10.0.x'
  DOTNET_NOLOGO: true
  DOTNET_CLI_TELEMETRY_OPTOUT: true

The concurrency block is worth more than it looks. Push three commits to a PR in quick succession and only the last one runs to completion; the other two are cancelled. On main we don’t cancel, because a half-finished deploy is worse than a slow one.

permissions: contents: read at the top sets the default for every job. Jobs that need more ask for it explicitly.

Detecting what changed

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.matrix.outputs.matrix }}
      any: ${{ steps.matrix.outputs.any }}
    steps:
      - uses: actions/checkout@v4

      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            shared:
              - 'src/MyApp.Core/**'
              - 'src/MyApp.Infrastructure/**'
              - 'Directory.Build.props'
              - 'Directory.Packages.props'
              - 'Dockerfile'
            api:
              - 'src/MyApp.Api/**'
            worker:
              - 'src/MyApp.Worker/**'

      - name: Build the service matrix
        id: matrix
        shell: bash
        run: |
          services=()
          shared='${{ steps.filter.outputs.shared }}'
          [[ "$shared" == "true" || "${{ steps.filter.outputs.api }}"    == "true" ]] && services+=('{"name":"api","project":"src/MyApp.Api/MyApp.Api.csproj","image":"myapp-api"}')
          [[ "$shared" == "true" || "${{ steps.filter.outputs.worker }}" == "true" ]] && services+=('{"name":"worker","project":"src/MyApp.Worker/MyApp.Worker.csproj","image":"myapp-worker"}')

          if [ ${#services[@]} -eq 0 ]; then
            echo 'matrix=[]' >> "$GITHUB_OUTPUT"
            echo 'any=false'  >> "$GITHUB_OUTPUT"
          else
            printf -v joined '%s,' "${services[@]}"
            echo "matrix=[${joined%,}]" >> "$GITHUB_OUTPUT"
            echo 'any=true' >> "$GITHUB_OUTPUT"
          fi

paths-filter compares against the PR base on pull requests and against the previous commit on pushes, which is what you want. The shell step turns its yes/no outputs into a JSON array that the build job can feed straight into a matrix. The earlier version of this post attached a condition to each matrix entry and then never used it, so it rebuilt everything on every push. Emitting the matrix itself is the fix: if a service isn’t in the list, no job runs for it.

Anything that affects every image goes under shared: central package management files, Directory.Build.props, the Dockerfile. Forgetting one of these means a dependency bump ships to the API but not the worker.

Test once

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}
          cache: true
          cache-dependency-path: '**/packages.lock.json'

      - run: dotnet restore --locked-mode
      - run: dotnet build --no-restore -c Release
      - run: dotnet test --no-build -c Release --logger "trx;LogFileName=results.trx" --collect:"XPlat Code Coverage"

      - uses: dorny/test-reporter@v1
        if: always()
        with:
          name: tests
          path: '**/results.trx'
          reporter: dotnet-trx

NuGet caching through setup-dotnet needs lock files, so add <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile> to Directory.Build.props and commit the generated packages.lock.json files. --locked-mode then fails the build if the lock files are stale, which is the behaviour you want from CI. Without lock files the cache key has nothing stable to hash and you’ll see a restore from scratch every run.

Test results go to a check on the PR via test-reporter, so a failing test shows the test name in the GitHub UI instead of asking people to scroll through a log. The if: always() is so the report still publishes when tests fail, which is when you want it.

The Dockerfile

One Dockerfile for all services, parameterised by project. Fewer files to keep in sync, and the restore layer is shared across images.

# syntax=docker/dockerfile:1.7
ARG DOTNET_VERSION=10.0

FROM mcr.microsoft.com/dotnet/sdk:${DOTNET_VERSION} AS build
ARG PROJECT
WORKDIR /src

# Restore first, with only the files restore needs, so this layer caches
# until a dependency actually changes.
COPY Directory.Build.props Directory.Packages.props nuget.config* ./
COPY src/MyApp.Core/*.csproj            src/MyApp.Core/
COPY src/MyApp.Infrastructure/*.csproj  src/MyApp.Infrastructure/
COPY src/MyApp.Api/*.csproj             src/MyApp.Api/
COPY src/MyApp.Worker/*.csproj          src/MyApp.Worker/
COPY src/MyApp.Api/packages.lock.json    src/MyApp.Api/
COPY src/MyApp.Worker/packages.lock.json src/MyApp.Worker/

RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet restore "${PROJECT}" --locked-mode

COPY . .

RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet publish "${PROJECT}" -c Release -o /app --no-restore /p:UseAppHost=false

FROM mcr.microsoft.com/dotnet/aspnet:${DOTNET_VERSION} AS final
WORKDIR /app
COPY --from=build /app .
USER $APP_UID
ENV ASPNETCORE_HTTP_PORTS=8080
EXPOSE 8080
ENTRYPOINT ["dotnet", "MyApp.dll"]

The --mount=type=cache on the NuGet folder means packages survive between builds on the same builder even when the restore layer is invalidated. The USER $APP_UID line runs the app as the non-root user the .NET images ship with; port 8080 instead of 80 is the other half of that, since non-root can’t bind below 1024.

Set the entry DLL per service through an ARG if the names differ, or standardise on <AssemblyName> in the projects. And make sure .dockerignore excludes **/bin and **/obj; without that, COPY . . copies your local build output into the image and invalidates the layer on every run, which is a caching bug that’s easy to spend an afternoon on.

Build and push, in parallel

  build-and-push:
    needs: [detect-changes, test]
    if: needs.detect-changes.outputs.any == 'true'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    strategy:
      fail-fast: false
      matrix:
        service: ${{ fromJSON(needs.detect-changes.outputs.matrix) }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        if: github.event_name != 'pull_request'
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_PREFIX }}/${{ matrix.service.image }}
          tags: |
            type=sha,prefix=
            type=ref,event=pr
            type=raw,value=latest,enable={{is_default_branch}}

      - uses: docker/build-push-action@v6
        with:
          context: .
          build-args: |
            PROJECT=${{ matrix.service.project }}
            DOTNET_VERSION=10.0
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha,scope=${{ matrix.service.name }}
          cache-to: type=gha,scope=${{ matrix.service.name }},mode=max
          provenance: false

Points of note:

  • fail-fast: false so a broken worker build doesn’t cancel a healthy API build. You want the full picture.
  • The if guard on the job: fromJSON('[]') produces an empty matrix and the job errors out, so skip it when nothing changed.
  • Images are tagged with the short commit SHA on every build and latest only on main. The SHA tag is what you roll back to.
  • type=gha cache uses GitHub’s own cache service, scoped per service so the API and worker don’t evict each other’s layers. Registry cache (type=registry pointing at a :buildcache tag) also works and survives longer, but costs registry storage and a pull on every build. On GitHub-hosted runners, gha is the simpler default; switch when you outgrow the 10 GB cache limit.
  • provenance: false keeps the push to a single image instead of an attestation manifest list, which confuses some older tooling and shows up as an “unknown/unknown” architecture in the GHCR UI.

For solutions with more than three or four services, the matrix approach starts duplicating the shared restore work per job. At that point docker buildx bake builds all targets in one BuildKit session with shared layers, which is where the iterative pipeline post ended up.

Deploying

  deploy:
    needs: [detect-changes, build-and-push]
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Work out which services to restart
        id: svc
        run: |
          echo "list=$(echo '${{ needs.detect-changes.outputs.matrix }}' | jq -r '[.[].name] | join(" ")')" >> "$GITHUB_OUTPUT"

      - uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.DEPLOY_HOST }}
          username: ${{ secrets.DEPLOY_USER }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          fingerprint: ${{ secrets.DEPLOY_HOST_FINGERPRINT }}
          script: |
            set -euo pipefail
            cd /opt/myapp
            echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
            export IMAGE_TAG=${{ github.sha }}
            docker compose pull ${{ steps.svc.outputs.list }}
            docker compose up -d --no-deps --wait ${{ steps.svc.outputs.list }}
            docker image prune -f --filter "until=168h"

With compose.yaml on the host referencing image: ghcr.io/org/myapp-api:${IMAGE_TAG:-latest}, deploying a specific commit is setting one variable. Rolling back is running the same two commands with an older SHA. Pulling latest and hoping is how you end up unsure what’s actually running.

--wait makes compose up block until the services report healthy, so a container that starts and immediately crashes fails the deploy step instead of failing silently after the workflow has gone green. That needs a healthcheck in the Compose file, which you should have anyway.

The environment: production line attaches GitHub’s environment protection: required reviewers, a wait timer, or a branch restriction, configured in the repository settings. For a team of one it’s a confirmation click; for a team of ten it’s a change control record.

About “zero downtime”

Be honest with yourself about what compose up -d does: it stops the old container, then starts the new one. With a fast-starting .NET service and a reverse proxy in front that retries, that’s a gap of one to three seconds that most users never notice. It isn’t zero.

If a gap of seconds is unacceptable, you need two instances and a proxy that drains one while the other serves. The docker rollout Compose plugin does exactly that for Compose-managed services: scale to two, wait for the new one to be healthy, remove the old one. Pair it with Caddy as the reverse proxy and you have a genuinely interruption-free deploy for the cost of one plugin and a little extra memory during rollouts.

If you’re deploying to Kubernetes, ECS or Cloud Run instead of a Docker host, this job is the only one to replace. The platform’s own deploy action handles the rollout, and the change detection, test and image jobs carry over as they are.

Secrets and permissions

GitHub Secrets hold the deploy key, host and registry credentials. Application secrets (connection strings, API keys) should not go through the pipeline at all; the container should fetch them at startup from a secrets manager like Vault or your cloud’s equivalent. A pipeline that can read production database passwords is a pipeline you have to protect like production.

Keep the top-level permissions: contents: read and grant packages: write only to the job that pushes images. If a third-party action in the test job is compromised, it should find a token that can’t write anywhere.

Pin third-party actions to a major version at minimum (dorny/paths-filter@v3), and to a full commit SHA if your threat model includes the action’s maintainer being compromised. Dependabot will keep either up to date.

Troubleshooting

Docker layers never cache; every build restores from scratch. Either COPY . . comes before dotnet restore, .dockerignore is missing bin/ and obj/, or the cache scope is shared between services and they keep evicting each other. Check in that order.

denied: permission_denied pushing to GHCR. The job is missing packages: write, or the package already exists and is linked to a different repository (fix it in the package’s settings), or you’re on a PR from a fork, where GITHUB_TOKEN is read-only by design.

Build job fails with Matrix vector 'service' does not contain any values. Nothing changed and the if: needs.detect-changes.outputs.any == 'true' guard is missing.

paths-filter thinks everything changed on the first push to a new branch. It has no base to compare to. Set base: main explicitly in the action inputs for push events.

Restore cache never hits. No packages.lock.json files committed, or cache-dependency-path doesn’t match where they live.

SSH step fails with host key verification. Supply the host fingerprint as shown above rather than disabling verification. Get it with ssh-keyscan -t ed25519 host | ssh-keygen -lf -.

Deploy succeeds but the old version is still running. Compose didn’t pull because the tag didn’t change (you’re using latest), or --no-deps was omitted and a dependency restarted instead. Move to SHA tags.

Tests pass locally and fail in CI with culture or time-zone differences. Runners are en-US and UTC. Set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT consistently, or make the tests culture-explicit.