From 140cfb4298bc98e4ad8be7e2d7aee92ae403f119 Mon Sep 17 00:00:00 2001 From: Esau Rodriguez Date: Thu, 22 Oct 2026 10:31:57 +0100 Subject: [PATCH 1/2] Upgrade libhegel from 0.45.0 to 0.47.1 libhegel 0.46 changed every string parameter and output of the C ABI to hegel_string_t (a {data, len} struct, by value for required strings, by nullable pointer where absent is meaningful, as a contiguous array for name lists), and renamed hegel_generate_string_result_free to hegel_owned_string_free. Both bindings marshal strings that way now: - RealLibhegel (FFM): a STRING_LAYOUT struct layout in the descriptors, str/strPtr/strArray helpers for inputs, readString/readStringPtr for outputs; hegel_context_last_error returns the struct by value, so its downcall takes an arena as allocator. - JnaLibhegel (JNA): a HegelString Structure (with ByValue) replaces the String parameters and the BufferResult struct; strArray writes the struct array and the string bytes into one Memory block so nothing inside it can be collected during the call. A null Java string crosses as {NULL, 0} by value, which the engine reads as an empty string, so a null regex pattern is now the empty pattern rather than E_INVALID_ARG; the edge tests say so, and new round-trip tests send a label, an origin, a database path and name lists through the engine and read the origin back. One behaviour change reaches the shared suite: since 0.45.2 a run that never generates a valid case ends in the engine's Unsatisfiable error even with the filter-too-much health check suppressed, instead of passing vacuously. Co-Authored-By: Claude Fable 5.1 --- .claude/CLAUDE.md | 7 +- RELEASE.md | 50 ++++ .../src/main/java/dev/hegel/JnaLibhegel.java | 233 +++++++++++------- .../test/java/dev/hegel/JnaLibhegelTest.java | 98 +++++++- .../src/main/java/dev/hegel/RealLibhegel.java | 233 ++++++++++-------- .../test/java/dev/hegel/RealLibhegelTest.java | 69 +++++- pom.xml | 2 +- .../dev/hegel/SettingsHealthDatabaseTest.java | 21 +- 8 files changed, 498 insertions(+), 215 deletions(-) create mode 100644 RELEASE.md diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 8a298db..3243be4 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -101,7 +101,12 @@ public `Generator`/`TestCase`/`Generators`/`Hegel`/`Stateful` surface stays in ` classpath); `Engine` (shared) is the process-wide lazy holder of the loaded shared object (with a test hook) — it caches only the immutable, thread-safe library, never per-test state. Both bindings keep one libhegel error context per thread (read back by `lastErrorMessage()`), pass date/time/datetime structs by - value, copy engine-allocated string/bytes buffers out and free them, and bridge the per-run + value, pass every string as the engine's `hegel_string_t` (`{const char *data; size_t len}`, + since 0.46: by value for a required string, by nullable pointer where absent is meaningful, as a + contiguous struct array for name lists; a Java `null` crosses as `{NULL, 0}` by value and as + NULL by pointer) and read strings back the same way (`readString`/`readStringPtr` in FFM, + `readString`/`readStringAt` in JNA; `hegel_context_last_error` returns the struct by value), + copy engine-allocated string/bytes buffers out and free them (`hegel_owned_string_free`), and bridge the per-run output callback (`hegel_output_callback_t`) to a `Consumer` — an FFM upcall stub whose arena lives until `runFree`, or a JNA `Callback` strongly referenced until `runFree`. In the JNA binding, C `bool` crosses as `byte` (JNA's default boolean mapping is a 32-bit int) and diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000..9967297 --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,50 @@ +RELEASE_TYPE: patch + +This patch upgrades the bundled libhegel engine from 0.45.0 to 0.47.1. The public Java API is +unchanged; the two bindings now pass every string across the C ABI as the engine's length-delimited +`hegel_string_t`, which is how the engine takes and returns strings since 0.46. + +A run that never generates a valid test case now ends in the engine's `Unsatisfiable` error (a +`HegelException` from `Hegel.test`) however the health checks are configured. Before, a test whose +`assume` or `filter` rejected every input passed vacuously once the filter-too-much health check was +suppressed. The engine also no longer cuts a run short after a handful of consecutive repeated +rejections, which could make a test over a small input space fail the filter-too-much check, or +pass without a valid input, roughly one time in twenty. + +`Generators.fromRegex` now records each drawn string as a single choice constrained by the pattern, +so a failure shrinks within the pattern instead of re-running the generator: a `\d` shrinks to `0`, +and `(?:bbbb|a)` shrinks to `a` rather than stopping at `bbbb`. Failures that earlier engine versions +saved for a regex generator no longer replay and are dropped from the database like any stale entry. +A pattern that no string over its alphabet can match (`abc` over an alphabet of `def`, `\d` over +letters, or `(?!)`) now fails up front with an `IllegalArgumentException` naming the pattern and the +alphabet, where it used to be built and then reject every draw. Regex and text generators use the +Unicode 17.0 tables (they were 15.1). + +`Failure.nondeterministic()` is now true only for a failure the run actually saw fail to reproduce. +Before, every failure of a run that had switched to nondeterministic handling carried a caveat, so a +failure that reproduced on every replay could be reported as "confirmed: failed 20 of 20 replays". + +Replaying a reproduce blob with `Settings.reproduceFailure` against a test whose draws have changed +no longer continues past the point where the test's draws leave the recording: the replay counts as +a miss, and a blob none of whose replays reproduced is reported as stale, with an output line naming +where the draws diverged. Before, a failure the changed test happened to reach on fresh draws could +be reported as the blob's failure. + +Generation and shrinking improved along the way: targeting with `TestCase.target` starts after a +short warm-up in runs of 1,000 or more test cases instead of halfway through; bounded integers whose +failing values are sparse (such as multiples of six) shrink to the minimum instead of stalling; +integer ranges wider than 128 bits draw ordinary values spread over the range's orders of magnitude +rather than almost always near the top; and span mutation no longer skews small finite spaces toward +the all-zero sequence. Seeded runs that draw such values generate different inputs from this version +on. Health check and `Unsatisfiable` messages drop their Rust syntax and name the check and setting +plainly, and verbose output announces every executed test case with `Running test case`. + +Under Antithesis, `Backend.URANDOM` (the shipped `workload` profile) now takes its entropy from the +Antithesis fuzzer directly through `libvoidstar`, so the fuzzer sees every draw as a request of its +own; elsewhere it reads `/dev/urandom` as before. + +The engine also gained a `stop_on_first_failure` setting (on by default; turned off, a run spends its +whole `testCases` budget before reporting and replays every stored failure), grammar-based string +generation from Lark grammars, and a structured run-error kind. None of these have a Java API yet; +the `HEGEL_STOP_ON_FIRST_FAILURE` environment variable and the `stop_on_first_failure` key in +`hegel.toml` already reach the engine, since it reads them itself. diff --git a/hegel-jna/src/main/java/dev/hegel/JnaLibhegel.java b/hegel-jna/src/main/java/dev/hegel/JnaLibhegel.java index e6d641e..9aa733a 100644 --- a/hegel-jna/src/main/java/dev/hegel/JnaLibhegel.java +++ b/hegel-jna/src/main/java/dev/hegel/JnaLibhegel.java @@ -5,7 +5,6 @@ import com.sun.jna.Memory; import com.sun.jna.Native; import com.sun.jna.Pointer; -import com.sun.jna.StringArray; import com.sun.jna.Structure; import com.sun.jna.ptr.ByteByReference; import com.sun.jna.ptr.DoubleByReference; @@ -44,8 +43,9 @@ final class JnaLibhegel implements Libhegel { JnaLibhegel(Path libraryPath) { try { - this.lib = Native.load( - libraryPath.toString(), HegelNative.class, Map.of(Library.OPTION_STRING_ENCODING, "UTF-8")); + // Every string crosses as an explicit hegel_string_t (UTF-8 bytes plus length), so no + // JNA String conversion is involved. + this.lib = Native.load(libraryPath.toString(), HegelNative.class); } catch (UnsatisfiedLinkError e) { throw new HegelException("Failed to open libhegel at " + libraryPath + ": " + e.getMessage()); } @@ -55,7 +55,7 @@ final class JnaLibhegel implements Libhegel { interface HegelNative extends Library { Pointer hegel_context_new(); - Pointer hegel_context_last_error(Pointer ctx); + HegelString.ByValue hegel_context_last_error(Pointer ctx); int hegel_settings_new(Pointer ctx, PointerByReference out); @@ -73,9 +73,10 @@ interface HegelNative extends Library { int hegel_settings_set_report_multiple_failures(Pointer ctx, Pointer s, byte yes); - int hegel_settings_set_database(Pointer ctx, Pointer s, String path); + // A Structure parameter crosses as a pointer to it (NULL for null): `const hegel_string_t *`. + int hegel_settings_set_database(Pointer ctx, Pointer s, HegelString path); - int hegel_settings_set_database_key(Pointer ctx, Pointer s, String key); + int hegel_settings_set_database_key(Pointer ctx, Pointer s, HegelString key); int hegel_settings_set_phases(Pointer ctx, Pointer s, int mask); @@ -97,7 +98,7 @@ int hegel_run_start( int hegel_run_start_blob( Pointer ctx, Pointer settings, - String blob, + HegelString.ByValue blob, LineCallback callback, Pointer userData, PointerByReference out); @@ -113,7 +114,7 @@ int hegel_run_start_blob( int hegel_test_case_from_blob( Pointer ctx, Pointer settings, - String blob, + HegelString.ByValue blob, LineCallback callback, Pointer userData, PointerByReference out); @@ -143,13 +144,13 @@ int hegel_generate_float( double smallestNonzeroMagnitude, DoubleByReference out); - int hegel_generate_bytes(Pointer ctx, Pointer tc, long minSize, long maxSize, BufferResult out); + int hegel_generate_bytes(Pointer ctx, Pointer tc, long minSize, long maxSize, HegelString out); - int hegel_generate_bytes_result_free(Pointer ctx, BufferResult result); + int hegel_generate_bytes_result_free(Pointer ctx, HegelString result); - int hegel_generate_string(Pointer ctx, Pointer tc, Pointer generator, BufferResult out); + int hegel_generate_string(Pointer ctx, Pointer tc, Pointer generator, HegelString out); - int hegel_generate_string_result_free(Pointer ctx, BufferResult result); + int hegel_owned_string_free(Pointer ctx, HegelString result); int hegel_generate_date(Pointer ctx, Pointer tc, HegelDate.ByValue min, HegelDate.ByValue max, HegelDate out); @@ -168,26 +169,19 @@ int hegel_string_generator_text( Pointer ctx, long minSize, long maxSize, - String codec, + HegelString codec, int minCodepoint, int maxCodepoint, Pointer categories, long categoriesLen, Pointer excludeCategories, long excludeCategoriesLen, - Pointer includeCharacters, - long includeCharactersLen, - Pointer excludeCharacters, - long excludeCharactersLen, + HegelString.ByValue includeCharacters, + HegelString.ByValue excludeCharacters, PointerByReference out); int hegel_string_generator_regex( - Pointer ctx, - Pointer pattern, - long patternLen, - byte fullmatch, - Pointer alphabet, - PointerByReference out); + Pointer ctx, HegelString.ByValue pattern, byte fullmatch, Pointer alphabet, PointerByReference out); int hegel_string_generator_email(Pointer ctx, PointerByReference out); @@ -205,7 +199,7 @@ int hegel_string_generator_regex( int hegel_collection_more(Pointer ctx, Pointer tc, long id, ByteByReference out); - int hegel_collection_reject(Pointer ctx, Pointer tc, long id, String why); + int hegel_collection_reject(Pointer ctx, Pointer tc, long id, HegelString why); int hegel_new_pool(Pointer ctx, Pointer tc, LongByReference out); @@ -213,7 +207,8 @@ int hegel_string_generator_regex( int hegel_pool_generate(Pointer ctx, Pointer tc, long poolId, byte consume, LongByReference out); - // State-machine handles cross as raw addresses (long), like every other opaque handle. + // State-machine handles cross as raw addresses (long), like every other opaque handle; the + // name lists are contiguous hegel_string_t arrays (see strArray). int hegel_new_state_machine( Pointer ctx, Pointer tc, @@ -242,9 +237,9 @@ int hegel_state_machine_should_check_invariant( int hegel_state_machine_free(Pointer ctx, long stateMachineId); - int hegel_target(Pointer ctx, Pointer tc, double value, String label); + int hegel_target(Pointer ctx, Pointer tc, double value, HegelString.ByValue label); - int hegel_mark_complete(Pointer ctx, Pointer tc, int status, String origin); + int hegel_mark_complete(Pointer ctx, Pointer tc, int status, HegelString origin); int hegel_run_result_status(Pointer ctx, Pointer result, IntByReference out); @@ -258,11 +253,13 @@ int hegel_state_machine_should_check_invariant( int hegel_failure_reproduction_blob(Pointer ctx, Pointer failure, PointerByReference out); - int hegel_failure_origin(Pointer ctx, Pointer failure, PointerByReference out); + // A present string is written into the caller's hegel_string_t; an optional one (blob, + // caveat) comes back as a `const hegel_string_t *` slot, NULL when absent. + int hegel_failure_origin(Pointer ctx, Pointer failure, HegelString out); int hegel_failure_caveat(Pointer ctx, Pointer failure, PointerByReference out); - int hegel_version(Pointer ctx, PointerByReference out); + int hegel_version(Pointer ctx, HegelString out); } /** {@code struct hegel_date_t { int32_t year; uint8_t month; uint8_t day; }} */ @@ -295,13 +292,30 @@ public static class HegelDatetime extends Structure { public static class ByValue extends HegelDatetime implements Structure.ByValue {} } - /** {@code struct hegel_generate_bytes_result_t / hegel_generate_string_result_t { T* data; size_t len; }} */ + /** + * {@code struct hegel_string_t { const char *data; size_t len; }} — the shape every string + * crosses in since libhegel 0.46 (by value, or by pointer where absent is meaningful), shared + * by {@code hegel_owned_string_t} and {@code hegel_generate_bytes_result_t}. {@code len} counts + * UTF-8 bytes; inputs need no NUL terminator and a zero length accepts a NULL {@code data}. + */ @Structure.FieldOrder({"data", "len"}) - public static class BufferResult extends Structure { + public static class HegelString extends Structure { public Pointer data; public long len; + + public HegelString() {} + + /** A view over an engine-owned struct at {@code p}, read on construction. */ + HegelString(Pointer p) { + super(p); + read(); + } + + public static class ByValue extends HegelString implements Structure.ByValue {} } + private static final int STRING_SIZE = new HegelString().size(); + interface OutputCallback extends Callback { void invoke(Pointer userData, Pointer line, long len); } @@ -359,8 +373,67 @@ private void check(String op, int code) { } } - static String readCString(Pointer ptr) { - return ptr == null ? null : ptr.getString(0, "UTF-8"); + /** + * A by-value {@code hegel_string_t} of {@code s}'s UTF-8 bytes. {@code null} crosses as {@code + * {NULL, 0}}, which the engine reads as an empty string. The struct holds its buffer, so the + * bytes stay alive as long as the struct is referenced (the call's argument list does). + */ + private static HegelString.ByValue str(String s) { + HegelString.ByValue value = new HegelString.ByValue(); + if (s != null) { + byte[] bytes = s.getBytes(StandardCharsets.UTF_8); + Memory buf = new Memory(Math.max(bytes.length, 1)); + buf.write(0, bytes, 0, bytes.length); + value.data = buf; + value.len = bytes.length; + } + return value; + } + + /** A nullable {@code const hegel_string_t*}: {@code null} maps to NULL (absent), {@code ""} to present-and-empty. */ + private static HegelString strPtr(String s) { + return s == null ? null : str(s); + } + + /** + * A NULL-distinct contiguous {@code hegel_string_t[]}: {@code null} maps to NULL, an empty list + * to a valid pointer. The structs and the bytes they point at share one allocation, so the + * block passed as the argument keeps everything alive for the call. + */ + private static Pointer strArray(List strings) { + if (strings == null) { + return null; + } + byte[][] encoded = new byte[strings.size()][]; + long total = (long) STRING_SIZE * strings.size(); + for (int i = 0; i < encoded.length; i++) { + encoded[i] = strings.get(i).getBytes(StandardCharsets.UTF_8); + total += encoded[i].length; + } + Memory block = new Memory(Math.max(total, 1)); + long offset = (long) STRING_SIZE * strings.size(); + for (int i = 0; i < encoded.length; i++) { + long entry = (long) STRING_SIZE * i; + // A zero-length string needs no bytes; NULL data is accepted with a zero length. + block.setPointer(entry, encoded[i].length == 0 ? null : block.share(offset)); + block.setLong(entry + Native.POINTER_SIZE, encoded[i].length); + block.write(offset, encoded[i], 0, encoded[i].length); + offset += encoded[i].length; + } + return block; + } + + /** + * Read a {@code hegel_string_t} the engine filled in ({@code len} bytes of UTF-8, interior NULs + * included). A NULL {@code data} pointer reads as {@code null}. + */ + static String readString(HegelString string) { + return string.data == null ? null : new String(copyBuffer(string), StandardCharsets.UTF_8); + } + + /** Read through a {@code const hegel_string_t*}: NULL (absent) reads as {@code null}. */ + static String readStringAt(Pointer string) { + return string == null ? null : readString(new HegelString(string)); } // --- settings --- @@ -416,12 +489,12 @@ public void settingsReportMultipleFailures(long s, boolean yes) { @Override public void settingsDatabase(long s, String path) { - check("hegel_settings_set_database", lib.hegel_settings_set_database(ctx(), pointer(s), path)); + check("hegel_settings_set_database", lib.hegel_settings_set_database(ctx(), pointer(s), strPtr(path))); } @Override public void settingsDatabaseKey(long s, String key) { - check("hegel_settings_set_database_key", lib.hegel_settings_set_database_key(ctx(), pointer(s), key)); + check("hegel_settings_set_database_key", lib.hegel_settings_set_database_key(ctx(), pointer(s), strPtr(key))); } @Override @@ -493,7 +566,7 @@ public long runStart(long settings, Consumer output) { public long runStartBlob(long settings, String blob, Consumer output) { LineCallback callback = output == null ? null : new LineCallback(output); PointerByReference out = new PointerByReference(); - int code = lib.hegel_run_start_blob(ctx(), pointer(settings), blob, callback, null, out); + int code = lib.hegel_run_start_blob(ctx(), pointer(settings), str(blob), callback, null, out); if (code != Abi.OK) { throw new HegelException("hegel_run_start_blob failed (rc=" + code @@ -538,7 +611,7 @@ public int testCaseFromBlob(long settings, String blob, Consumer output, // needs to live for its duration. LineCallback callback = output == null ? null : new LineCallback(output); PointerByReference outRef = new PointerByReference(); - int code = lib.hegel_test_case_from_blob(ctx(), pointer(settings), blob, callback, null, outRef); + int code = lib.hegel_test_case_from_blob(ctx(), pointer(settings), str(blob), callback, null, outRef); if (code == Abi.OK) { out[0] = address(outRef.getValue()); } @@ -627,7 +700,7 @@ public int generateFloat( @Override public int generateBytes(long tc, long minSize, long maxSize, byte[][] out) { - BufferResult result = new BufferResult(); + HegelString result = new HegelString(); int code = lib.hegel_generate_bytes(ctx(), pointer(tc), minSize, maxSize, result); if (code == Abi.OK) { out[0] = copyBuffer(result); @@ -638,17 +711,19 @@ public int generateBytes(long tc, long minSize, long maxSize, byte[][] out) { @Override public int generateString(long tc, long generator, String[] out) { - BufferResult result = new BufferResult(); + // An owned string: {data, len} plus a NUL the length excludes, released with + // hegel_owned_string_free once copied out. + HegelString result = new HegelString(); int code = lib.hegel_generate_string(ctx(), pointer(tc), pointer(generator), result); if (code == Abi.OK) { out[0] = new String(copyBuffer(result), StandardCharsets.UTF_8); - check("hegel_generate_string_result_free", lib.hegel_generate_string_result_free(ctx(), result)); + check("hegel_owned_string_free", lib.hegel_owned_string_free(ctx(), result)); } return code; } - /** Copy an engine-allocated {@code {data, len}} buffer out before it is freed. */ - private static byte[] copyBuffer(BufferResult result) { + /** Copy the {@code len} bytes an engine-filled {@code {data, len}} struct points at. */ + private static byte[] copyBuffer(HegelString result) { return result.data.getByteArray(0, (int) result.len); } @@ -769,26 +844,20 @@ public int stringGeneratorText( String includeCharacters, String excludeCharacters, long[] out) { - Pointer categoriesArr = cstrArray(categories); - Pointer excludeArr = cstrArray(excludeCategories); - byte[] include = utf8OrNull(includeCharacters); - byte[] exclude = utf8OrNull(excludeCharacters); PointerByReference outRef = new PointerByReference(); int code = lib.hegel_string_generator_text( ctx(), minSize, maxSize, - codec, + strPtr(codec), (int) minCodepoint, (int) maxCodepoint, - categoriesArr, + strArray(categories), categories == null ? 0L : categories.size(), - excludeArr, + strArray(excludeCategories), excludeCategories == null ? 0L : excludeCategories.size(), - bytesOrNull(include), - include == null ? 0L : include.length, - bytesOrNull(exclude), - exclude == null ? 0L : exclude.length, + str(includeCharacters), + str(excludeCharacters), outRef); // Read the (null-initialised) out slot unconditionally: callers check the return code // before using it. @@ -796,35 +865,12 @@ public int stringGeneratorText( return code; } - private static byte[] utf8OrNull(String s) { - return s == null ? null : s.getBytes(StandardCharsets.UTF_8); - } - - private static Pointer bytesOrNull(byte[] bytes) { - if (bytes == null) { - return null; - } - Memory buf = new Memory(Math.max(bytes.length, 1)); - buf.write(0, bytes, 0, bytes.length); - return buf; - } - - /** A NULL-distinct {@code char**}: {@code null} maps to NULL, an empty list to a valid pointer. */ - private static Pointer cstrArray(List strings) { - return strings == null ? null : new StringArray(strings.toArray(new String[0]), "UTF-8"); - } - @Override public int stringGeneratorRegex(String pattern, boolean fullmatch, long alphabet, long[] out) { - byte[] bytes = utf8OrNull(pattern); + // The pattern crosses as a length-delimited string, so it may contain NUL, which Python's + // `re` accepts. PointerByReference outRef = new PointerByReference(); - int code = lib.hegel_string_generator_regex( - ctx(), - bytesOrNull(bytes), - bytes == null ? 0L : bytes.length, - cbool(fullmatch), - pointer(alphabet), - outRef); + int code = lib.hegel_string_generator_regex(ctx(), str(pattern), cbool(fullmatch), pointer(alphabet), outRef); out[0] = address(outRef.getValue()); return code; } @@ -890,7 +936,7 @@ public int collectionMore(long tc, long id, boolean[] outMore) { @Override public int collectionReject(long tc, long id, String why) { - return lib.hegel_collection_reject(ctx(), pointer(tc), id, why); + return lib.hegel_collection_reject(ctx(), pointer(tc), id, strPtr(why)); } @Override @@ -950,11 +996,11 @@ public int newStateMachine( int code = lib.hegel_new_state_machine( ctx(), pointer(tc), - cstrArray(ruleNames), + strArray(ruleNames), groups, weights, ruleNames.size(), - cstrArray(invariantNames), + strArray(invariantNames), alwaysCheck, invariantNames.size(), minConcurrency, @@ -1013,12 +1059,12 @@ public void stateMachineFree(long stateMachineId) { @Override public int target(long tc, double value, String label) { - return lib.hegel_target(ctx(), pointer(tc), value, label); + return lib.hegel_target(ctx(), pointer(tc), value, str(label)); } @Override public int markComplete(long tc, int status, String origin) { - return lib.hegel_mark_complete(ctx(), pointer(tc), status, origin); + return lib.hegel_mark_complete(ctx(), pointer(tc), status, strPtr(origin)); } // --- results --- @@ -1034,7 +1080,7 @@ public int runResultStatus(long result) { public String runResultError(long result) { PointerByReference ref = new PointerByReference(); check("hegel_run_result_error", lib.hegel_run_result_error(ctx(), pointer(result), ref)); - return readCString(ref.getValue()); + return readStringAt(ref.getValue()); } @Override @@ -1051,7 +1097,7 @@ public String failureBlob(long result, long index) { Pointer failure = failureOut.getValue(); PointerByReference blobOut = new PointerByReference(); check("hegel_failure_reproduction_blob", lib.hegel_failure_reproduction_blob(ctx(), failure, blobOut)); - String blob = readCString(blobOut.getValue()); + String blob = readStringAt(blobOut.getValue()); check("hegel_failure_free", lib.hegel_failure_free(ctx(), failure)); return blob; } @@ -1061,9 +1107,9 @@ public String failureOrigin(long result, long index) { PointerByReference failureOut = new PointerByReference(); check("hegel_run_result_failure", lib.hegel_run_result_failure(ctx(), pointer(result), index, failureOut)); Pointer failure = failureOut.getValue(); - PointerByReference originOut = new PointerByReference(); + HegelString originOut = new HegelString(); check("hegel_failure_origin", lib.hegel_failure_origin(ctx(), failure, originOut)); - String origin = readCString(originOut.getValue()); + String origin = readString(originOut); check("hegel_failure_free", lib.hegel_failure_free(ctx(), failure)); return origin; } @@ -1075,7 +1121,7 @@ public String failureCaveat(long result, long index) { Pointer failure = failureOut.getValue(); PointerByReference caveatOut = new PointerByReference(); check("hegel_failure_caveat", lib.hegel_failure_caveat(ctx(), failure, caveatOut)); - String caveat = readCString(caveatOut.getValue()); + String caveat = readStringAt(caveatOut.getValue()); check("hegel_failure_free", lib.hegel_failure_free(ctx(), failure)); return caveat; } @@ -1084,13 +1130,14 @@ public String failureCaveat(long result, long index) { @Override public String lastErrorMessage() { - return readCString(lib.hegel_context_last_error(ctx())); + // Returned by value; the bytes borrow the context's buffer and are copied out here. + return readString(lib.hegel_context_last_error(ctx())); } @Override public String version() { - PointerByReference ref = new PointerByReference(); - check("hegel_version", lib.hegel_version(ctx(), ref)); - return readCString(ref.getValue()); + HegelString out = new HegelString(); + check("hegel_version", lib.hegel_version(ctx(), out)); + return readString(out); } } diff --git a/hegel-jna/src/test/java/dev/hegel/JnaLibhegelTest.java b/hegel-jna/src/test/java/dev/hegel/JnaLibhegelTest.java index 98ae940..d850f07 100644 --- a/hegel-jna/src/test/java/dev/hegel/JnaLibhegelTest.java +++ b/hegel-jna/src/test/java/dev/hegel/JnaLibhegelTest.java @@ -49,12 +49,19 @@ void callbackDecodesAndSwallowsExceptions() { } @Test - void readCStringHandlesNullAndValue() { - assertNull(JnaLibhegel.readCString(null)); - byte[] bytes = "hello\0".getBytes(StandardCharsets.UTF_8); - Memory value = new Memory(bytes.length); - value.write(0, bytes, 0, bytes.length); - assertEquals("hello", JnaLibhegel.readCString(value)); + void readStringHandlesNullDataAndInteriorNul() { + // A zeroed struct is {NULL, 0}: absent rather than empty. + assertNull(JnaLibhegel.readString(new JnaLibhegel.HegelString())); + // A length-delimited string keeps an interior NUL and ignores the terminator. + byte[] bytes = "a\0b\0".getBytes(StandardCharsets.UTF_8); + Memory data = new Memory(bytes.length); + data.write(0, bytes, 0, bytes.length); + Memory string = new Memory(16); + string.setPointer(0, data); + string.setLong(8, 3); + // Through a `const hegel_string_t *`, NULL is absent and a pointer is read. + assertNull(JnaLibhegel.readStringAt(null)); + assertEquals("a\0b", JnaLibhegel.readStringAt(string)); } @Test @@ -281,13 +288,86 @@ void regexGeneratorAcceptsATextAlphabet() { @Test void regexPatternCrossesAsALengthDelimitedBuffer() { - // A NUL inside the pattern must survive the call (0.45 takes pointer + byte length), and - // a null pattern is the engine's E_INVALID_ARG rather than a crash. + // A NUL inside the pattern must survive the call (the pattern crosses as a hegel_string_t: + // pointer + byte length), and a null pattern crosses as {NULL, 0}, which the engine reads + // as the empty pattern rather than crashing on it. JnaLibhegel lib = real(); long[] regex = new long[1]; assertEquals(Abi.OK, lib.stringGeneratorRegex("a\0b", true, 0, regex)); assertTrue(regex[0] != 0); lib.stringGeneratorFree(regex[0]); - assertEquals(Abi.E_INVALID_ARG, lib.stringGeneratorRegex(null, true, 0, new long[1])); + assertEquals(Abi.OK, lib.stringGeneratorRegex(null, true, 0, regex)); + assertTrue(regex[0] != 0); + lib.stringGeneratorFree(regex[0]); + } + + @Test + void stringsRoundTripThroughTheEngine() { + // Strings now cross in both directions as {data, len} structs: a by-value input (the + // target label), a nullable pointer (mark_complete's origin, the database path), a + // contiguous array (state-machine names), and the by-value/by-pointer outputs the result + // readers decode. A failure's origin coming back as the string sent proves the round trip. + JnaLibhegel lib = real(); + long s = newSettings(lib); + lib.settingsDatabase(s, ""); + lib.settingsTestCases(s, 5); + long run = lib.runStart(s, line -> {}); + long tc; + while ((tc = lib.nextTestCase(run)) != 0) { + long[] value = new long[1]; + int code = lib.generateInteger(tc, 0, 10, value); + if (code == Abi.OK) { + assertEquals(Abi.OK, lib.target(tc, value[0], "héllo wörld")); + assertEquals(Abi.OK, lib.markComplete(tc, Abi.STATUS_INTERESTING, "origin ✓")); + } else { + assertEquals(Abi.OK, lib.markComplete(tc, Abi.STATUS_OVERRUN, null)); + } + lib.testCaseFree(tc); + } + long result = lib.runResult(run); + assertEquals(Abi.RUN_STATUS_FAILED, lib.runResultStatus(result)); + assertNull(lib.runResultError(result)); + assertEquals(1, lib.runResultFailureCount(result)); + assertEquals("origin ✓", lib.failureOrigin(result, 0)); + assertNotNull(lib.failureBlob(result, 0)); + assertNull(lib.failureCaveat(result, 0)); + lib.runResultFree(result); + lib.runFree(run); + lib.settingsFree(s); + } + + @Test + void stringArraysCrossAsContiguousStructs() { + // An empty name inside the list crosses as {NULL, 0}; a valid machine comes back, proving + // the engine read every entry of the contiguous hegel_string_t array. + JnaLibhegel lib = real(); + long s = newSettings(lib); + lib.settingsDatabase(s, ""); + lib.settingsTestCases(s, 1); + long run = lib.runStart(s, line -> {}); + long tc; + while ((tc = lib.nextTestCase(run)) != 0) { + long[] id = new long[1]; + long[] concurrency = new long[1]; + assertEquals( + Abi.OK, + lib.newStateMachine( + tc, + List.of("rule", "", "other"), + new long[] {0, 0, 0}, + null, + List.of(), + new boolean[0], + 1, + 1, + 1, + id, + concurrency)); + lib.stateMachineFree(id[0]); + assertEquals(Abi.OK, lib.markComplete(tc, Abi.STATUS_VALID, null)); + lib.testCaseFree(tc); + } + lib.runFree(run); + lib.settingsFree(s); } } diff --git a/hegel/src/main/java/dev/hegel/RealLibhegel.java b/hegel/src/main/java/dev/hegel/RealLibhegel.java index 08645cd..71b0e58 100644 --- a/hegel/src/main/java/dev/hegel/RealLibhegel.java +++ b/hegel/src/main/java/dev/hegel/RealLibhegel.java @@ -72,9 +72,12 @@ final class RealLibhegel implements Libhegel { private static final StructLayout DATETIME_LAYOUT = MemoryLayout.structLayout(DATE_LAYOUT.withName("date"), TIME_LAYOUT.withName("time")); - // struct hegel_generate_bytes_result_t / hegel_generate_string_result_t { T* data; size_t len; } - private static final StructLayout BUFFER_RESULT_LAYOUT = + // struct hegel_string_t { const char *data; size_t len; } — the shape every string crosses in + // since libhegel 0.46 (by value, or by nullable pointer where absent is meaningful), shared by + // hegel_owned_string_t and hegel_generate_bytes_result_t. + private static final StructLayout STRING_LAYOUT = MemoryLayout.structLayout(ADDRESS.withName("data"), JAVA_LONG.withName("len")); + private static final long LEN_OFFSET = ADDRESS.byteSize(); private static final FunctionDescriptor OUTPUT_CALLBACK_DESC = FunctionDescriptor.ofVoid(ADDRESS, ADDRESS, JAVA_LONG); @@ -131,7 +134,7 @@ private static MethodHandle findEmitLine() { private final MethodHandle generateBytes; private final MethodHandle generateBytesResultFree; private final MethodHandle generateString; - private final MethodHandle generateStringResultFree; + private final MethodHandle ownedStringFree; private final MethodHandle generateDate; private final MethodHandle generateTime; private final MethodHandle generateDatetime; @@ -183,7 +186,9 @@ private static MethodHandle findEmitLine() { // rc(ctx, args...) descriptors; the leading JAVA_INT is the hegel_result_t return. this.contextNew = h(linker, lookup, "hegel_context_new", FunctionDescriptor.of(ADDRESS)); - this.contextLastError = h(linker, lookup, "hegel_context_last_error", FunctionDescriptor.of(ADDRESS, ADDRESS)); + // Returns a hegel_string_t by value, so the downcall takes a SegmentAllocator first. + this.contextLastError = + h(linker, lookup, "hegel_context_last_error", FunctionDescriptor.of(STRING_LAYOUT, ADDRESS)); this.settingsNew = h(linker, lookup, "hegel_settings_new", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); this.settingsFree = h(linker, lookup, "hegel_settings_free", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); this.settingsSetBackend = h( @@ -270,7 +275,7 @@ private static MethodHandle findEmitLine() { linker, lookup, "hegel_run_start_blob", - FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS, ADDRESS, ADDRESS, ADDRESS)); + FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, STRING_LAYOUT, ADDRESS, ADDRESS, ADDRESS)); this.nextTestCase = h(linker, lookup, "hegel_next_test_case", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS)); this.runResult = @@ -282,7 +287,7 @@ private static MethodHandle findEmitLine() { linker, lookup, "hegel_test_case_from_blob", - FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS, ADDRESS, ADDRESS, ADDRESS)); + FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, STRING_LAYOUT, ADDRESS, ADDRESS, ADDRESS)); this.testCaseFree = h(linker, lookup, "hegel_test_case_free", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); this.testCaseShouldCapture = h( @@ -336,8 +341,8 @@ private static MethodHandle findEmitLine() { lookup, "hegel_generate_string", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS, ADDRESS)); - this.generateStringResultFree = h( - linker, lookup, "hegel_generate_string_result_free", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); + this.ownedStringFree = + h(linker, lookup, "hegel_owned_string_free", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); this.generateDate = h( linker, lookup, @@ -367,13 +372,25 @@ private static MethodHandle findEmitLine() { lookup, "hegel_string_generator_text", FunctionDescriptor.of( - JAVA_INT, ADDRESS, JAVA_LONG, JAVA_LONG, ADDRESS, JAVA_INT, JAVA_INT, ADDRESS, JAVA_LONG, - ADDRESS, JAVA_LONG, ADDRESS, JAVA_LONG, ADDRESS, JAVA_LONG, ADDRESS)); + JAVA_INT, + ADDRESS, + JAVA_LONG, + JAVA_LONG, + ADDRESS, + JAVA_INT, + JAVA_INT, + ADDRESS, + JAVA_LONG, + ADDRESS, + JAVA_LONG, + STRING_LAYOUT, + STRING_LAYOUT, + ADDRESS)); this.stringGeneratorRegex = h( linker, lookup, "hegel_string_generator_regex", - FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, JAVA_LONG, JAVA_BOOLEAN, ADDRESS, ADDRESS)); + FunctionDescriptor.of(JAVA_INT, ADDRESS, STRING_LAYOUT, JAVA_BOOLEAN, ADDRESS, ADDRESS)); this.stringGeneratorEmail = h(linker, lookup, "hegel_string_generator_email", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); this.stringGeneratorUrl = @@ -449,7 +466,7 @@ private static MethodHandle findEmitLine() { linker, lookup, "hegel_target", - FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, JAVA_DOUBLE, ADDRESS)); + FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, JAVA_DOUBLE, STRING_LAYOUT)); this.markComplete = h( linker, lookup, @@ -527,15 +544,52 @@ private void check(String op, int code) { } } - private static MemorySegment cstr(Arena a, String s) { - return s == null ? MemorySegment.NULL : a.allocateFrom(s); + /** + * A by-value {@code hegel_string_t} of {@code s}'s UTF-8 bytes. {@code null} crosses as {@code + * {NULL, 0}}, which the engine reads as an empty string (a zero length accepts any pointer). + */ + private static MemorySegment str(Arena a, String s) { + MemorySegment seg = a.allocate(STRING_LAYOUT); // zeroed: {NULL, 0} + if (s != null) { + byte[] bytes = s.getBytes(StandardCharsets.UTF_8); + MemorySegment data = a.allocate(Math.max(bytes.length, 1)); + MemorySegment.copy(bytes, 0, data, JAVA_BYTE, 0, bytes.length); + seg.set(ADDRESS, 0, data); + seg.set(JAVA_LONG, LEN_OFFSET, bytes.length); + } + return seg; } - static String readCString(MemorySegment ptr) { - if (ptr == null || ptr.address() == 0) { - return null; + /** A nullable {@code const hegel_string_t*}: {@code null} maps to NULL (absent), {@code ""} to present-and-empty. */ + private static MemorySegment strPtr(Arena a, String s) { + return s == null ? MemorySegment.NULL : str(a, s); + } + + /** A NULL-distinct {@code hegel_string_t[]}: {@code null} maps to NULL, an empty list to a valid pointer. */ + private static MemorySegment strArray(Arena a, List strings) { + if (strings == null) { + return MemorySegment.NULL; } - return ptr.reinterpret(Long.MAX_VALUE).getString(0, StandardCharsets.UTF_8); + MemorySegment array = a.allocate(STRING_LAYOUT, Math.max(strings.size(), 1)); + for (int i = 0; i < strings.size(); i++) { + MemorySegment.copy( + str(a, strings.get(i)), 0, array, i * STRING_LAYOUT.byteSize(), STRING_LAYOUT.byteSize()); + } + return array; + } + + /** + * Read a {@code hegel_string_t} the engine filled in ({@code len} bytes of UTF-8, interior NULs + * included). A NULL {@code data} pointer reads as {@code null}. + */ + static String readString(MemorySegment string) { + return string.get(ADDRESS, 0).address() == 0 ? null : new String(copyBuffer(string), StandardCharsets.UTF_8); + } + + /** Read through a {@code const hegel_string_t*} out-slot: NULL (absent) reads as {@code null}. */ + static String readStringPtr(MemorySegment slot) { + MemorySegment string = slot.get(ADDRESS, 0); + return string.address() == 0 ? null : readString(string.reinterpret(STRING_LAYOUT.byteSize())); } /** @@ -604,13 +658,17 @@ public void settingsReportMultipleFailures(long s, boolean yes) { @Override public void settingsDatabase(long s, String path) { - // libhegel copies the string during the call, so a per-call auto arena suffices. - check("hegel_settings_set_database", rc(settingsSetDatabase, segment(s), cstr(Arena.ofAuto(), path))); + // libhegel copies the string during the call, so a per-call arena suffices. + try (Arena arena = Arena.ofConfined()) { + check("hegel_settings_set_database", rc(settingsSetDatabase, segment(s), strPtr(arena, path))); + } } @Override public void settingsDatabaseKey(long s, String key) { - check("hegel_settings_set_database_key", rc(settingsSetDatabaseKey, segment(s), cstr(Arena.ofAuto(), key))); + try (Arena arena = Arena.ofConfined()) { + check("hegel_settings_set_database_key", rc(settingsSetDatabaseKey, segment(s), strPtr(arena, key))); + } } @Override @@ -679,7 +737,7 @@ public long runStartBlob(long settings, String blob, Consumer output) { Arena arena = Arena.ofConfined(); MemorySegment callback = output == null ? MemorySegment.NULL : upcallStub(arena, output); MemorySegment out = arena.allocate(ADDRESS); - int code = rc(runStartBlob, segment(settings), cstr(arena, blob), callback, MemorySegment.NULL, out); + int code = rc(runStartBlob, segment(settings), str(arena, blob), callback, MemorySegment.NULL, out); if (code != Abi.OK) { arena.close(); throw new HegelException("hegel_run_start_blob failed (rc=" @@ -725,7 +783,7 @@ public int testCaseFromBlob(long settings, String blob, Consumer output, try (Arena arena = Arena.ofConfined()) { MemorySegment callback = output == null ? MemorySegment.NULL : upcallStub(arena, output); MemorySegment outSeg = arena.allocate(ADDRESS); - int code = rc(testCaseFromBlob, segment(settings), cstr(arena, blob), callback, MemorySegment.NULL, outSeg); + int code = rc(testCaseFromBlob, segment(settings), str(arena, blob), callback, MemorySegment.NULL, outSeg); if (code == Abi.OK) { out[0] = outSeg.get(ADDRESS, 0).address(); } @@ -816,7 +874,7 @@ public int generateFloat( @Override public int generateBytes(long tc, long minSize, long maxSize, byte[][] out) { try (Arena arena = Arena.ofConfined()) { - MemorySegment result = arena.allocate(BUFFER_RESULT_LAYOUT); + MemorySegment result = arena.allocate(STRING_LAYOUT); int code = rc(generateBytes, segment(tc), minSize, maxSize, result); if (code == Abi.OK) { out[0] = copyBuffer(result); @@ -829,20 +887,22 @@ public int generateBytes(long tc, long minSize, long maxSize, byte[][] out) { @Override public int generateString(long tc, long generator, String[] out) { try (Arena arena = Arena.ofConfined()) { - MemorySegment result = arena.allocate(BUFFER_RESULT_LAYOUT); + // An owned string: {data, len} plus a NUL the length excludes, released with + // hegel_owned_string_free once copied out. + MemorySegment result = arena.allocate(STRING_LAYOUT); int code = rc(generateString, segment(tc), segment(generator), result); if (code == Abi.OK) { out[0] = new String(copyBuffer(result), StandardCharsets.UTF_8); - check("hegel_generate_string_result_free", rc(generateStringResultFree, result)); + check("hegel_owned_string_free", rc(ownedStringFree, result)); } return code; } } - /** Copy an engine-allocated {@code {data, len}} buffer out before it is freed. */ + /** Copy the {@code len} bytes an engine-filled {@code {data, len}} struct points at. */ private static byte[] copyBuffer(MemorySegment result) { MemorySegment data = result.get(ADDRESS, 0); - long len = result.get(JAVA_LONG, ADDRESS.byteSize()); + long len = result.get(JAVA_LONG, LEN_OFFSET); return data.reinterpret(len).toArray(JAVA_BYTE); } @@ -974,26 +1034,20 @@ public int stringGeneratorText( String excludeCharacters, long[] out) { try (Arena arena = Arena.ofConfined()) { - MemorySegment categoriesSeg = cstrArray(arena, categories); - MemorySegment excludeSeg = cstrArray(arena, excludeCategories); - byte[] include = utf8OrNull(includeCharacters); - byte[] exclude = utf8OrNull(excludeCharacters); MemorySegment outSeg = arena.allocate(ADDRESS); int code = rc( stringGeneratorText, minSize, maxSize, - cstr(arena, codec), + strPtr(arena, codec), (int) minCodepoint, (int) maxCodepoint, - categoriesSeg, + strArray(arena, categories), categories == null ? 0L : (long) categories.size(), - excludeSeg, + strArray(arena, excludeCategories), excludeCategories == null ? 0L : (long) excludeCategories.size(), - bytesOrNull(arena, include), - include == null ? 0L : (long) include.length, - bytesOrNull(arena, exclude), - exclude == null ? 0L : (long) exclude.length, + str(arena, includeCharacters), + str(arena, excludeCharacters), outSeg); // Read the (zero-initialised) out slot unconditionally: callers check the return code // before using it. @@ -1002,45 +1056,13 @@ public int stringGeneratorText( } } - private static byte[] utf8OrNull(String s) { - return s == null ? null : s.getBytes(StandardCharsets.UTF_8); - } - - private static MemorySegment bytesOrNull(Arena arena, byte[] bytes) { - if (bytes == null) { - return MemorySegment.NULL; - } - MemorySegment seg = arena.allocate(Math.max(bytes.length, 1)); - MemorySegment.copy(bytes, 0, seg, JAVA_BYTE, 0, bytes.length); - return seg; - } - - /** A NULL-distinct {@code char**}: {@code null} maps to NULL, an empty list to a valid pointer. */ - private static MemorySegment cstrArray(Arena arena, List strings) { - if (strings == null) { - return MemorySegment.NULL; - } - MemorySegment array = arena.allocate(ADDRESS, Math.max(strings.size(), 1)); - for (int i = 0; i < strings.size(); i++) { - array.setAtIndex(ADDRESS, i, cstr(arena, strings.get(i))); - } - return array; - } - @Override public int stringGeneratorRegex(String pattern, boolean fullmatch, long alphabet, long[] out) { try (Arena arena = Arena.ofConfined()) { - // The pattern crosses as a length-delimited UTF-8 buffer (libhegel 0.45), so it may - // contain NUL, which Python's `re` accepts. - byte[] bytes = utf8OrNull(pattern); + // The pattern crosses as a length-delimited string, so it may contain NUL, which + // Python's `re` accepts. MemorySegment outSeg = arena.allocate(ADDRESS); - int code = rc( - stringGeneratorRegex, - bytesOrNull(arena, bytes), - bytes == null ? 0L : (long) bytes.length, - fullmatch, - segment(alphabet), - outSeg); + int code = rc(stringGeneratorRegex, str(arena, pattern), fullmatch, segment(alphabet), outSeg); out[0] = outSeg.get(ADDRESS, 0).address(); return code; } @@ -1109,7 +1131,9 @@ public int collectionMore(long tc, long id, boolean[] outMore) { @Override public int collectionReject(long tc, long id, String why) { - return rc(collectionReject, segment(tc), id, cstr(Arena.ofAuto(), why)); + try (Arena arena = Arena.ofConfined()) { + return rc(collectionReject, segment(tc), id, strPtr(arena, why)); + } } @Override @@ -1150,7 +1174,7 @@ public int newStateMachine( long[] outId, long[] outConcurrency) { try (Arena arena = Arena.ofConfined()) { - MemorySegment rules = cstrArray(arena, ruleNames); + MemorySegment rules = strArray(arena, ruleNames); MemorySegment groups = arena.allocate(JAVA_LONG, Math.max(ruleGroups.length, 1)); for (int i = 0; i < ruleGroups.length; i++) { groups.setAtIndex(JAVA_LONG, i, ruleGroups[i]); @@ -1163,7 +1187,7 @@ public int newStateMachine( weights.setAtIndex(JAVA_DOUBLE, i, ruleWeights[i]); } } - MemorySegment invariants = cstrArray(arena, invariantNames); + MemorySegment invariants = strArray(arena, invariantNames); MemorySegment alwaysCheck = arena.allocate(JAVA_BOOLEAN, Math.max(invariantAlwaysCheck.length, 1)); for (int i = 0; i < invariantAlwaysCheck.length; i++) { alwaysCheck.setAtIndex(JAVA_BOOLEAN, i, invariantAlwaysCheck[i]); @@ -1236,12 +1260,16 @@ public void stateMachineFree(long stateMachineId) { @Override public int target(long tc, double value, String label) { - return rc(target, segment(tc), value, cstr(Arena.ofAuto(), label)); + try (Arena arena = Arena.ofConfined()) { + return rc(target, segment(tc), value, str(arena, label)); + } } @Override public int markComplete(long tc, int status, String origin) { - return rc(markComplete, segment(tc), status, cstr(Arena.ofAuto(), origin)); + try (Arena arena = Arena.ofConfined()) { + return rc(markComplete, segment(tc), status, strPtr(arena, origin)); + } } // --- results --- @@ -1257,7 +1285,7 @@ public int runResultStatus(long result) { public String runResultError(long result) { MemorySegment seg = Arena.ofAuto().allocate(ADDRESS); check("hegel_run_result_error", rc(runResultError, segment(result), seg)); - return readCString(seg.get(ADDRESS, 0)); + return readStringPtr(seg); } @Override @@ -1269,42 +1297,53 @@ public long runResultFailureCount(long result) { @Override public String failureBlob(long result, long index) { - return readFailureString(result, index, failureReproductionBlob, "hegel_failure_reproduction_blob"); + return readFailureString(result, index, failureReproductionBlob, "hegel_failure_reproduction_blob", true); } @Override public String failureOrigin(long result, long index) { - return readFailureString(result, index, failureOrigin, "hegel_failure_origin"); + return readFailureString(result, index, failureOrigin, "hegel_failure_origin", false); } @Override public String failureCaveat(long result, long index) { - return readFailureString(result, index, failureCaveat, "hegel_failure_caveat"); + return readFailureString(result, index, failureCaveat, "hegel_failure_caveat", true); } - /** Fetch the {@code index}-th failure, read one of its strings with {@code reader}, free it. */ - private String readFailureString(long result, long index, MethodHandle reader, String op) { - MemorySegment failureOut = Arena.ofAuto().allocate(ADDRESS); - check("hegel_run_result_failure", rc(runResultFailure, segment(result), index, failureOut)); - MemorySegment failure = failureOut.get(ADDRESS, 0); - MemorySegment strOut = Arena.ofAuto().allocate(ADDRESS); - check(op, rc(reader, failure, strOut)); - String value = readCString(strOut.get(ADDRESS, 0)); - check("hegel_failure_free", rc(failureFree, failure)); - return value; + /** + * Fetch the {@code index}-th failure, read one of its strings with {@code reader}, free it. An + * {@code optional} string comes back through a {@code const hegel_string_t*} slot (NULL when + * absent); a present one is written into a {@code hegel_string_t} the caller provides. + */ + private String readFailureString(long result, long index, MethodHandle reader, String op, boolean optional) { + try (Arena arena = Arena.ofConfined()) { + MemorySegment failureOut = arena.allocate(ADDRESS); + check("hegel_run_result_failure", rc(runResultFailure, segment(result), index, failureOut)); + MemorySegment failure = failureOut.get(ADDRESS, 0); + MemorySegment strOut = arena.allocate(optional ? ADDRESS : STRING_LAYOUT); + check(op, rc(reader, failure, strOut)); + String value = optional ? readStringPtr(strOut) : readString(strOut); + check("hegel_failure_free", rc(failureFree, failure)); + return value; + } } // --- diagnostics --- @Override public String lastErrorMessage() { - return readCString((MemorySegment) invoke(contextLastError, ctx())); + // The string borrows the context's buffer; it is copied out before the arena closes. + try (Arena arena = Arena.ofConfined()) { + return readString((MemorySegment) invoke(contextLastError, arena, ctx())); + } } @Override public String version() { - MemorySegment seg = Arena.ofAuto().allocate(ADDRESS); - check("hegel_version", rc(version, seg)); - return readCString(seg.get(ADDRESS, 0)); + try (Arena arena = Arena.ofConfined()) { + MemorySegment seg = arena.allocate(STRING_LAYOUT); + check("hegel_version", rc(version, seg)); + return readString(seg); + } } } diff --git a/hegel/src/test/java/dev/hegel/RealLibhegelTest.java b/hegel/src/test/java/dev/hegel/RealLibhegelTest.java index 4492b00..6199b07 100644 --- a/hegel/src/test/java/dev/hegel/RealLibhegelTest.java +++ b/hegel/src/test/java/dev/hegel/RealLibhegelTest.java @@ -14,9 +14,11 @@ import java.lang.foreign.Arena; import java.lang.foreign.MemorySegment; import java.lang.foreign.SymbolLookup; +import java.lang.foreign.ValueLayout; import java.lang.invoke.MethodHandle; import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodType; +import java.nio.charset.StandardCharsets; import java.nio.file.Path; import java.util.List; import org.junit.jupiter.api.Test; @@ -47,11 +49,23 @@ void invokeRethrowsRuntimeException() throws Exception { } @Test - void readCStringHandlesNullAndValue() { - assertNull(RealLibhegel.readCString(null)); - assertNull(RealLibhegel.readCString(MemorySegment.NULL)); + void readStringHandlesNullDataAndInteriorNul() { try (Arena a = Arena.ofConfined()) { - assertEquals("hello", RealLibhegel.readCString(a.allocateFrom("hello"))); + // A zeroed struct is {NULL, 0}: absent rather than empty. + MemorySegment string = a.allocate(16); + assertNull(RealLibhegel.readString(string)); + // A length-delimited string keeps an interior NUL and ignores the terminator. + byte[] bytes = "a\0b\0".getBytes(StandardCharsets.UTF_8); + MemorySegment data = a.allocate(bytes.length); + MemorySegment.copy(bytes, 0, data, ValueLayout.JAVA_BYTE, 0, bytes.length); + string.set(ValueLayout.ADDRESS, 0, data); + string.set(ValueLayout.JAVA_LONG, 8, 3); + assertEquals("a\0b", RealLibhegel.readString(string)); + // Through a `const hegel_string_t *` slot, NULL is absent and a pointer is read. + MemorySegment slot = a.allocate(ValueLayout.ADDRESS); + assertNull(RealLibhegel.readStringPtr(slot)); + slot.set(ValueLayout.ADDRESS, 0, string); + assertEquals("a\0b", RealLibhegel.readStringPtr(slot)); } } @@ -278,13 +292,54 @@ void regexGeneratorAcceptsATextAlphabet() { @Test void regexPatternCrossesAsALengthDelimitedBuffer() { - // A NUL inside the pattern must survive the call (0.45 takes pointer + byte length), and - // a null pattern is the engine's E_INVALID_ARG rather than a crash. + // A NUL inside the pattern must survive the call (the pattern crosses as a hegel_string_t: + // pointer + byte length), and a null pattern crosses as {NULL, 0}, which the engine reads + // as the empty pattern rather than crashing on it. RealLibhegel lib = real(); long[] regex = new long[1]; assertEquals(Abi.OK, lib.stringGeneratorRegex("a\0b", true, 0, regex)); assertTrue(regex[0] != 0); lib.stringGeneratorFree(regex[0]); - assertEquals(Abi.E_INVALID_ARG, lib.stringGeneratorRegex(null, true, 0, new long[1])); + assertEquals(Abi.OK, lib.stringGeneratorRegex(null, true, 0, regex)); + assertTrue(regex[0] != 0); + lib.stringGeneratorFree(regex[0]); + } + + @Test + void stringsRoundTripThroughTheEngine() { + // Strings now cross in both directions as {data, len} structs: a by-value input (the + // target label), a nullable pointer (mark_complete's origin, the database path), a + // contiguous array (state-machine names), and the by-value/by-pointer outputs the result + // readers decode. A failure's origin coming back as the string sent proves the round trip. + RealLibhegel lib = real(); + assertNotNull(lib.version()); + String message = lib.lastErrorMessage(); + assertTrue(message == null || message.isEmpty(), String.valueOf(message)); + long s = newSettings(lib); + lib.settingsDatabase(s, ""); + lib.settingsTestCases(s, 5); + long run = lib.runStart(s, line -> {}); + long tc; + while ((tc = lib.nextTestCase(run)) != 0) { + long[] value = new long[1]; + int code = lib.generateInteger(tc, 0, 10, value); + if (code == Abi.OK) { + assertEquals(Abi.OK, lib.target(tc, value[0], "héllo wörld")); + assertEquals(Abi.OK, lib.markComplete(tc, Abi.STATUS_INTERESTING, "origin ✓")); + } else { + assertEquals(Abi.OK, lib.markComplete(tc, Abi.STATUS_OVERRUN, null)); + } + lib.testCaseFree(tc); + } + long result = lib.runResult(run); + assertEquals(Abi.RUN_STATUS_FAILED, lib.runResultStatus(result)); + assertNull(lib.runResultError(result)); + assertEquals(1, lib.runResultFailureCount(result)); + assertEquals("origin ✓", lib.failureOrigin(result, 0)); + assertNotNull(lib.failureBlob(result, 0)); + assertNull(lib.failureCaveat(result, 0)); + lib.runResultFree(result); + lib.runFree(run); + lib.settingsFree(s); } } diff --git a/pom.xml b/pom.xml index 5e6c677..c7ae7a7 100644 --- a/pom.xml +++ b/pom.xml @@ -56,7 +56,7 @@ - 0.45.0 + 0.47.1 false diff --git a/shared/src/test/java/dev/hegel/SettingsHealthDatabaseTest.java b/shared/src/test/java/dev/hegel/SettingsHealthDatabaseTest.java index e6aedef..2569703 100644 --- a/shared/src/test/java/dev/hegel/SettingsHealthDatabaseTest.java +++ b/shared/src/test/java/dev/hegel/SettingsHealthDatabaseTest.java @@ -2,6 +2,7 @@ import static dev.hegel.Generators.integers; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; @@ -50,13 +51,19 @@ void filterTooMuchHealthCheckFiresAndCanBeSuppressed() { String msg = err.getMessage().toLowerCase(); assertTrue(msg.contains("filter") || msg.contains("health"), err.getMessage()); - // Suppressing the check lets the run end without a health-check failure. - Hegel.test( - tc -> { - tc.draw(integers()); - tc.assume(false); - }, - new Settings().database(Database.disabled()).suppressHealthCheck(HealthCheck.FILTER_TOO_MUCH)); + // Suppressing the check lets the run spend its invalid budget without a health-check + // failure; since libhegel 0.45.2 a run that never generates a valid case then ends in the + // engine's Unsatisfiable error (a plain HegelException) rather than passing vacuously. + HegelException unsatisfiable = assertThrows( + HegelException.class, + () -> Hegel.test( + tc -> { + tc.draw(integers()); + tc.assume(false); + }, + new Settings().database(Database.disabled()).suppressHealthCheck(HealthCheck.FILTER_TOO_MUCH))); + assertFalse(unsatisfiable instanceof HealthCheckFailure, unsatisfiable.getMessage()); + assertTrue(unsatisfiable.getMessage().startsWith("Unsatisfiable"), unsatisfiable.getMessage()); } // With no GENERATE phase the engine produces no test cases, so the body never runs and a From 2709c301a7567f962a01a1b831d7342a0e838be9 Mon Sep 17 00:00:00 2001 From: Esau Rodriguez Date: Thu, 22 Oct 2026 15:13:33 +0100 Subject: [PATCH 2/2] Added a the missing test --- .../java/dev/hegel/generators/FloatsTest.java | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 shared/src/test/java/dev/hegel/generators/FloatsTest.java diff --git a/shared/src/test/java/dev/hegel/generators/FloatsTest.java b/shared/src/test/java/dev/hegel/generators/FloatsTest.java new file mode 100644 index 0000000..6d203ce --- /dev/null +++ b/shared/src/test/java/dev/hegel/generators/FloatsTest.java @@ -0,0 +1,51 @@ +package dev.hegel.generators; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import org.junit.jupiter.api.Test; + +/** + * Pins the subnormal default of {@link Floats#resolve} for every shape of bounds. The property + * tests that draw arbitrary bounds ({@code GenerationQualityTest}) reach these branches only when + * the engine happens to draw equal or one-sided bounds on the right side of the smallest normal, + * which depends on the random stream; this test covers them deterministically. + */ +class FloatsTest { + private static final double SMALLEST_NORMAL = Double.MIN_NORMAL; + + private static double smallestNonzero(Double min, Double max) { + return Floats.resolve("doubles", 64, min, max, null, null, null, false, false) + .smallestNonzeroMagnitude(); + } + + @Test + void subnormalDefaultFollowsTheBounds() { + // Unbounded, or bounded on one side: subnormals unless that bound excludes them all. + assertEquals(Double.MIN_VALUE, smallestNonzero(null, null)); + assertEquals(Double.MIN_VALUE, smallestNonzero(-1.0, null)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(1.0, null)); + assertEquals(Double.MIN_VALUE, smallestNonzero(null, 1.0)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(null, -1.0)); + // Distinct bounds: subnormals whenever the range reaches into (-smallestNormal, smallestNormal). + assertEquals(Double.MIN_VALUE, smallestNonzero(-1.0, 1.0)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(1.0, 2.0)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(-2.0, -1.0)); + // Equal bounds: subnormals only when the one value lies strictly inside that interval. + assertEquals(Double.MIN_VALUE, smallestNonzero(0.0, 0.0)); + assertEquals(Double.MIN_VALUE, smallestNonzero(Double.MIN_VALUE, Double.MIN_VALUE)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(1.0, 1.0)); + assertEquals(SMALLEST_NORMAL, smallestNonzero(-1.0, -1.0)); + } + + @Test + void widthPicksTheSubnormalMagnitude() { + assertEquals( + Float.MIN_VALUE, + Floats.resolve("floats", 32, null, null, null, null, null, false, false) + .smallestNonzeroMagnitude()); + assertEquals( + Float.MIN_NORMAL, + Floats.resolve("floats", 32, 1.0, 2.0, null, null, null, false, false) + .smallestNonzeroMagnitude()); + } +}