lograder

A Python library for building programming assignment autograders. The central abstraction is a Pipeline of Step instances -- each step validates input, runs a process, and passes results forward.

Input -> Check -> Mixin -> Build -> Test

---

Installation

pip install lograder
# or for development:
uv sync --extra dev

---

Minimal example

from pathlib import Path
from lograder.pipeline.config import config
from lograder.pipeline.input.local_directory import LocalDirectory
from lograder.pipeline.check.project.simple_project import CMakeManifestCheck
from lograder.pipeline.build.cmake import CMakeBuild
from lograder.pipeline.test.output_compare import OutputCompareTest, OutputCompareCase
from lograder.pipeline.score import TestCaseScorer, AllOrNothingScorer, GradescopeConfig
from lograder.pipeline.pipeline import Pipeline

# Define test cases
cases = [
    OutputCompareCase(name="hello", args=[], expected_stdout="Hello, World!\n"),
    OutputCompareCase(name="echo",  args=["foo"], expected_stdout="foo\n"),
]

# Build the pipeline
pipeline = Pipeline()
pipeline.add(inp   := LocalDirectory())
pipeline.add(check := CMakeManifestCheck())
pipeline.add(build := CMakeBuild())
pipeline.add(tests := OutputCompareTest("hello", cases))

# Attach scorers
build.scorer = AllOrNothingScorer(10.0, label="Build")
tests.scorer = TestCaseScorer({"hello": 40.0, "echo": 50.0}, label="Correctness")

# Run
with config(root_directory=Path("/autograder/submission")):
    score = pipeline()

score.write_results_json(config=GradescopeConfig(visibility="visible"))

---

Documentation

Doc	Contents
Getting started	First autograder in 5 minutes
Concepts	Pipeline, Step, Result, Manifest, Artifact
Input	LocalDirectory, EnvironmentConfig
Check	Manifest checks, SourceCheck, constraints
Build	CMakeBuild, MakefileBuild, BashScriptBuild, PrebuiltArtifacts
Test	All 9 test step types with full examples
Scoring	Scorers, Gradescope output, results.json
Process layer	TypedExecutable, CLIArgs, registry
Output / layouts	Logging, HTML report, custom layouts
Examples	Complete worked autograders

---

Design principles

• Errors as values. Steps return Result[Ok, Err] rather than raising. A fatal Err return stops the pipeline; a yielded Err is logged and execution continues.
• Typed data flow. Each step declares its input and output types. pipeline.validate_step_types() catches mismatches before the first submission runs.
• No magic. Every process invocation goes through a TypedExecutable with validated CLIArgs. You can see exactly what command was run and what it produced.
• Library only. lograder has no CLI and no service. You write a Python script; lograder is a collection of reusable building blocks.