GitHub Actions: the tests run on a machine that is not yours, before anything ships

Works on my machine

Every test passing on your laptop proves the code works with your Python, your installed packages, your environment variables, your filesystem and the files you forgot to commit. A second machine with none of those proves the code works. Continuous integration is the habit of having that second machine run the tests on every change, automatically, before the change is merged. GitHub Actions provides the machine for free for public repositories and with a generous monthly allowance for private ones.

The workflow file

# .github/workflows/test.yml
name: test

on:
  push:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pip install -e ".[dev]"
      - run: pyright src/
      - run: pytest -q

Read it top to bottom. on: says when — every push and every pull request. runs-on: picks a fresh Ubuntu virtual machine. matrix: runs the job once per listed Python version, in parallel. The steps are what you would type: check out the code, install Python, install the package with its dev extras from module 7's pyproject.toml, run the type checker, run the tests. If any step exits non-zero, the job fails and the commit gets a red cross.

Commit the file, push, and open the Actions tab. The first run takes a couple of minutes; with cache: pip the later ones are faster.

What it catches that you did not

The clean machine has no memory of your setup, and that is its value.

A dependency you use but never declared. It was installed in your venv from some earlier experiment. On the runner, ModuleNotFoundError. The fix is in pyproject.toml, where it should have been.

A file that is not committed. A test fixture, a prompt template, a .env.example you meant to add. The runner has only what git has.

Case. macOS and Windows filesystems ignore letter case; Linux does not. import tagger.Utils finds utils.py on a Mac and fails on the runner. Paths built from Path("Data/notes.md") behave the same way. The Linux runner is the only place many developers ever discover this.

Line endings and encodings. A file saved with Windows line endings, a test that opens a file without encoding="utf-8" and relies on the Mac's default. Module 5 warned; the runner enforces.

A different Python. A syntax that arrived in 3.12 fails on 3.11 in the matrix, before a user on 3.11 finds it.

Secrets and the tests that cost money

Module 4 split tests into two layers: fast tests with a fake provider, and a few slow ones against the real service. The fast ones run on every push. The real ones need the API key, and the key must not be in the repository.

In the repository's Settings → Secrets and variables → Actions, add OPENAI_API_KEY. In the workflow:

      - run: pytest -q -m "live"
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

The secret is injected as an environment variable, and GitHub masks it in logs. The if: restricts the live tests to pushes on main, and here is the reason that matters: a pull request from a fork runs your workflow with the fork's code. If the live tests ran on pull requests with your secret available, a stranger could open a pull request whose "test" prints the key or spends the budget. GitHub withholds secrets from fork pull requests by default; the if: makes the intent explicit and keeps the paid tests off every branch push as well. Add a spend cap on the provider side regardless.

Mark the live tests with @pytest.mark.live and register the marker in pyproject.toml, so pytest -q alone runs only the free ones.

A schedule, for free

on:
  schedule:
    - cron: "0 2 * * *"      # 02:00 UTC daily
  workflow_dispatch:         # and a button to run it by hand

The same cron syntax as the previous lesson, running on GitHub's machine — a job that runs while your laptop is closed, with no server of your own. Times are UTC. Scheduled runs on free plans can be delayed under load and are paused on repositories with no activity for sixty days, so a critical job needs a heartbeat check as the previous lesson said.

Protecting the branch

In Settings → Branches, require the test job to pass before a pull request can merge. Now a red cross is not a suggestion; it is a locked door. For a project you work on alone this is discipline; for one with contributors it is the only thing standing between a well-meaning change and a broken main.

Reading a failure

Click the red cross, open the failing step, and read the log from the bottom as module 4 taught for tracebacks. The environment section at the top of the log shows the Python version and the installed packages, which is where "it passes locally" gets resolved. To reproduce locally, act (free) runs workflow files on your own machine in Docker; more often, the log alone names the cause.

The habit

Every push runs the checks; every pull request shows them; nothing merges red. It is the last piece of the course: a program that installs from one file, is tested by a machine you do not control, keeps its secrets out of the code, and runs on a schedule with a lock and a done-record. That program you can hand to someone else, or to your future self, and it will still work.

Try this now

Add the workflow to your project and push. Read the first failure — there will be one — and fix it in the project, not the workflow. Then add the live-test step with the secret and the if:, open a pull request from a branch, and confirm the live step is skipped there and runs on main.