[zayenz]

This adds gecode/instance, a small library for reading textual benchmark instances. Each program describes its file format in a short declarative reader spec. The library reads a file with that spec, checks the values as it goes, and returns the data. Six examples now use it to read standard benchmark files, and a new gecode-instance tool prints the data in a file or converts it to MiniZinc data.

This is part of 6.5.0: feature/6.5.0-cleanups is based on this branch.

What a spec looks like

The SAT example reads DIMACS CNF with this spec:

format dimacs-cnf

comment "c"
stop "%"                    # SATLIB files end with "%" and "0"
line "p cnf {vars:0..gecode.int_max} {clauses:0..}"
repeat clauses {
  list clause: -vars..vars until 0   # the terminating 0 is not stored
}

Declarations put the name first and the type after it. A range such as -vars..vars is checked as each value is read, so this spec needs no separate checks. The TSPTW spec also uses a named type, a repeat with named records, and checks:

format tsptw

comment "#"
n: 2..1000
dist[n][n]: 0..gecode.int_max / 2 / n

type time = 0..gecode.int_max / 2
repeat n as nodes {
  each node
  ready: time
  due: time
}

check node.ready <= node.due "the time window of a node is invalid"
check nodes[0].ready == 0 "the depot (node 0) must open at time 0"

A check that uses a name read inside a repeat must hold for every record. When it fails, the error names the record and points at it in the input.

The language

• Types: int, float, num, word, and rest. Ranges are inclusive (lo..hi) or exclusive (lo..<hi), and either bound can be left out (0..). Alphabets read character grids (alphabet "123456789" blank "."). type name = ... defines a named type. The bounds can be any expression over values read earlier.
• Shapes: single values, arrays such as d[n][n] (with layout for the TSPLIB triangular formats and an optional offset), lists that end at a sentinel or at the end of the line, and repeated records.
• Text: line patterns with typed captures (line "p cnf {vars:int} {clauses:int}"), KEY : VALUE lines and sections in any order (sections until "EOF" { ... }), and comment and stop markers. In sections, each arm must match exactly one line unless it is marked optional, so a missing or repeated line is an error without a separate check.
• Checks: check, let, and if with integer arithmetic (including ^), comparisons, indexing (a[i], from 0), size, count, min, max, sum, distinct, defined, and comprehensions such as all(r in jobs: distinct(r.machine)). The constants gecode.int_max and gecode.int_min are available. Checks have a work budget proportional to the size of the input.
• Several instances per file: an instances { ... } block keeps the instance selected by number or name (-param instance=la01). A failed check, or a missing or repeated sections line, in another instance does not stop reading, and gecode-instance -list shows such failures.

The full reference is in Doxygen, under TaskInstanceSpec.

Data and errors

Records are stored column-wise as flat typed arrays, which may be ragged. The Instance::Data accessors take an IntArgs path of any depth, for example d.ints("clause", {c}). Data storage uses Gecode heap memory rather than STL containers.

Errors give the file, line, and column, the input line with a caret, the path through the spec, and the spec line:

bad.cnf:4:1: expected an int in -3..3, found '4'
     4 | 4 3 0
       | ^
  while reading: repeat [2 of 2] > clause
  spec: SAT spec:7

• The library never writes diagnostics itself. Every error in a spec or an instance is thrown as Instance::Error.
• The examples check their data in their specs, so invalid data is reported with the same location and context. The only check left in C++ is TSP's distance bound, because distances are computed there. A replacement spec given with -spec takes over responsibility for the same checks.
• gecode-instance exits with 2 for usage errors and 1 for spec or instance errors.

Performance

The SAT example replaces a hand-written DIMACS parser, so I compared the two on random 3-SAT files (Release, macOS arm64):

File	Old SAT parser	Reader spec
9 MB, 426k clauses	0.08 s	0.13 s
100 MB, 4.3M clauses	0.82 s	1.40 s
280 MB, 10.7M clauses	2.2 s	3.7 s

A whole sat run, stopped after the first search node, takes about 4.05 s on the 100 MB file, against 3.4 s on main. Peak memory there is 1.58 GB against 1.17 GB, because the parsed data stays in memory while the model is posted.

Commits

1. Add the instance module. The library, the gecode-instance tool, the tests, and CMake, Autoconf, and package wiring. The parser (spec.cpp), name resolution (resolve.cpp), expression evaluation (eval.cpp), and the reader (reader.cpp) are separate files. Without int variables, the module is left out of the build rather than turning int variables on. The ArrayTraits<ArgArray<double>> specialization moves into the kernel.

2. Let InstanceOptions read instance files. A script passes its spec to the constructor: InstanceOptions opt("SAT", instance_spec). -spec FILE reads with another spec, -param name=value sets a spec parameter (repeat the option, or separate pairs by commas, for several; gecode-instance -param works the same way), data() reads the file once, and builtin(i) tells a built-in instance number from a file name. The driver library now requires the instance library.

3. Read instance files in the examples. All but SAT keep their built-in instances.

   • SAT replaces its own parser, which assumed one clause per line.
   • TSP reads TSPLIB files and applies the TSPLIB distance rules.
   • TSPTW reads the López-Ibáñez and Blum collection.
   • Job shop reads OR-Library and JSPLIB files, including a single instance from jobshop1.txt selected by name.
   • Sudoku and advanced sudoku share a spec for the scaled sudoku corpus, with rectangular boxes and several puzzles per file.
   • Nonogram reads the .non files exported by webpbn.com.

   A new CI step runs each of these examples on a tiny file, so a mistake in an embedded spec fails CI.

4. Fix the freshness check on macOS. The generated-source check copied the source tree by piping tar into tar. On macOS, bsdtar can stop reading before the writer finishes, so the check failed in some checkouts depending on the archive size. It now copies through an archive file.

The MPG documentation is in the section "Reading instance files" of the driver chapter, on the Gecode/MPG branch feature/instance-reader.

Size

Part	Lines
Library	~6,400, including license headers and the Doxygen language reference
Tests	~1,160
Examples	+744, −314
Driver	~200
Tool	~240
Build and CI	~450, mostly Makefile.in, the smoke script, and the generated configure

Makefile.dep changes by about 13,900 lines because adding gecode/instance.hh to the existing rules rewraps them.

Testing

On the final branch:

• macOS arm64, CMake: -test Instance passes 40 of 40, also with ASan+UBSan, and the example smoke script passes. The first commit builds on its own and passes its 39 instance tests; the driver test is in the second commit. All 111 TSPLIB files of the mastqe/tsplib mirror were read; the 5 that fail use features outside the spec (FIXED_EDGES_SECTION, a free-text TYPE, more than 46340 nodes).
• Linux, clang 21, Debug, ASan+UBSan and TSan, before the optional arms and the speed-ups: -test Instance passes 40 of 40 in both builds. Also run: the smoke script, 132 runs of the examples on built-in instances and small valid and invalid files, sudokus with rectangular boxes for every sudoku model, and parallel search with -threads 4. TSan reports nothing, and ASan and UBSan report no errors. LeakSanitizer reports leaks only in code that is the same on main: a static initializer in test/int/extensional.cpp, job shop's search stop and cutoff objects, and IntSets placed in a Region in advanced sudoku.
• Real files: TSPLIB ulysses16 gives 6859 and gr17 2085; job shop ft06 gives 55 and la01 666 from jobshop1.txt, all optimal.

From the previous round, before the range types: -tag check passed 516 of 516 with CMake on macOS, and Autoconf built and passed -test Instance. TSPLIB br17 gave 39, TSPTW n20w20.001 378, and sudoku and nonogram puzzles solved correctly. Autoconf and Windows run in CI.

Known issue outside this PR

Since #233, Search::Options contains a WorkerControl whose destructor is in the search library. With Clang, every file that includes search.hh emits Search::Builder's vtable, because Builder::~Builder is defined inline. Through that vtable, ~Options() pulls the search library's ~WorkerControl() into libraries that do not link gecodesearch. On Linux Debug builds, libgecodeint, libgecodeset, and libgecodefloat are underlinked this way. This branch works around it by linking gecode-instance against gecodesearch. feature/6.5.0-cleanups fixes the cause by defining Builder::~Builder in the search library, and removes the workaround there.