Skip to content

jit: daslang -lib emits a C-ABI library - #3927

Draft
aleksisch wants to merge 3 commits into
masterfrom
aleksisch/standalone-jit
Draft

jit: daslang -lib emits a C-ABI library#3927
aleksisch wants to merge 3 commits into
masterfrom
aleksisch/standalone-jit

Conversation

@aleksisch

Copy link
Copy Markdown
Collaborator

A host that only needs to call one daslang script had two options, and both asked
too much. The C++ API embeds the compiler. Standalone AOT emits C++ source the host
has to compile itself, and the result is a C++ class. Neither works when the host
is C, or when it wants a plain ABI boundary.

daslang -lib script.das -output build/script now writes a native library and the
C header a host calls it through. --jit-lib-static gives an archive instead. Which
functions cross is your choice: [export_c] marks them one at a time, and
-lib-export-all offers every public function of the entry module whose signature C
can spell, naming the ones it skips. Selection reads one bit, Function.flags.exports,
so the interpreter, AOT and the header emitter cannot drift apart. Scalars, string,
pointers and enums cross by value; structures and the float2..uint4 / range
families cross as const T * and return through a trailing out pointer. Every
declared structure carries a size assert and one offset assert per field, so a host
built for another target fails to compile rather than misreading memory.

A library is called by code that cannot catch anything, which shapes the entry
points. Every thunk runs its body under the context's catch boundary: a das panic
returns zero, leaves the out param untouched, and reports through
<P>_last_error(ctx). <P>_create works on any thread, because daslang's
environment is thread-local while the registration behind it happens once. A second
daslang runtime in one process declines with a null instance instead of aborting.
And <P>_destroy emits the program's [finalize] calls itself, because a jitted
SimFunction carries no shutdown flag for the runtime's own drain to find.

Where to look: modules/dasLLVM/daslib/llvm_lib.das is the entry and thunk emitter,
daslib/c_api_header.das decides what can cross and writes the header, and the four
emit_standalone_* helpers in llvm_exe.das are the exe startup both artifacts now
share. The riskiest spot is the thunk ABI: a structure result is built in an alloca
of its own alignment, never in a frame field, because an in-struct [N x i8] slot
aligns to 1 while the body stores through it at the structure's real alignment.

Validation, claims, ledger

Validation

  • untracked gate is red on 6388 files that predate this branch and belong to the
    working copy, not the change: four build directories, a soundfont, model weights and
    scratch scripts. Nothing from this branch is untracked.
  • tests-interp reports 13102/13113. Both failures are wall-clock budget overruns,
    not assertions: tests/lint/test_stale_nolint.das and
    tests/lint/test_parallel_equivalence.das against --max-file-time 60. Measured
    cleanly, twice each, they are 46.4s / 46.7s and 42.2s / 42.4s; the box was at load
    12.7 on 32 cores during the run. Neither test reaches code this branch changes.
  • sync gate could not run: it fetches over https and GitHub was rate-limiting.
    Fetched over ssh instead and rebased onto origin/master at c409fcc2a; the
    upstream diff and this one share no file.
  • Local-only, beyond what CI runs: a C host driving both artifacts through the
    generated header - scalars, struct in and out, float3 in and out, string round
    trip, bool, int64/uint64, two independent instances, an instance created on a second
    thread, a panic returning zero with its text readable and the instance still usable
    afterwards, an out param untouched after a panic, and [finalize] on destroy. The
    same host links the static archive. A C++ TU compiles the header on its
    static_assert branch, and perturbing a structure makes the layout asserts fire.
  • The alignment fix is negative-controlled: test_jit_lib_result_slot_is_aligned
    fails when the result slot moves back into the argument frame.

Claims - stated, not tested

  • Windows and macOS take untested branches: the .dll / .dylib artifact names, the
    MSVC import-library link in the ctest driver, and llvm-lib / lib as the archiver.
    Verified by reading against the neighbouring create_shared_library, which the exe
    path already exercises on those platforms. A break would be a link failure in the
    lib_capi ctest, not a wrong answer at run time.
  • --jit-lib-static reuses --jit-path-to-linker to name the archiver. Recorded in
    that flag's help text rather than given a flag of its own.

Not done

  • One daslang library per process. jit_register_Module_* carries no already-created
    guard, so a second library and an embedding host both decline. Lifting it means
    guarding the registration, which is code the exe path shares.
  • No cross-compilation: -lib with --jit-target fails early.
  • The full test_aot sweep did not run; test_aot_subset, what PR CI builds, does.
  • 81 sites across daslib/, modules/dasLLVM/daslib/ and utils/lint/ hand-roll the
    annotation-name scan that a has_annotation(fn, name) helper in ast_boost would
    cover. Two of this branch's dupe hits are instances. Left alone as a tree-wide
    refactor, surfaced here as a candidate.

🤖 Generated with Claude Code

aleksisch and others added 3 commits September 3, 2026 10:58
inject_main was one 470-line body, and the only way for a second artifact kind to
reuse the exe's startup was to copy it. Four public emitters now carry the halves,
moved verbatim: the used-function sweep, the process-global module registration,
the context creation, and the per-context startup.

Two behavior changes ride along, both needed by a caller that splits the two
halves across different functions: jit_register_fusion is get-or-add and goes to a
fusion_builder parameter (inject_main passes its own builder, so the exe emits it
where it always did), and the registration table clears inside the emitter that
consumes it rather than at inject_main's top.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A host that only calls one script wants a C header, not the compiler API. This is
the half that decides what can cross and writes the header; nothing emits code yet.

Selection reads one bit, Function.flags.exports. [export_c] sets it, and the new
export_public_functions policy sets it for every public entry-module function, so
the interpreter, AOT and this emitter cannot drift apart. Whether a signature CAN
cross is decided here instead, after infer, because argument types do not exist
when an annotation applies.

Scalars, string, pointers and enums cross by value; structures and the
float2..uint4 / range families cross as const T * and return through a trailing
out pointer. Every declared structure carries a size assert and one offset assert
per field, so a host built for another target fails to compile rather than
misreading memory. An unrepresentable [export_c] comes back as a rejection for the
caller to report; a merely-public one is skipped with a warning naming the type.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The JIT could already produce an exe and an AOT object; a host that only calls one
script had neither. -lib writes a native shared library (or a static archive under
--jit-lib-static) plus the C header from the previous commit, and links no daslang
API into the host at all.

Codegen stays in LlvmJitMode.EXE, which is what makes the exe startup shareable:
only the entry shape and the artifact differ. Each export gets an extern "C" thunk
that packs its arguments into a frame and hands the frame to a trampoline through
jit_lib_invoke_guarded, so a das panic reaches the caller as a zero return with
its text in <P>_last_error, never as an unwind through the C boundary. A structure
result is built in an alloca of its own alignment, not in a frame field: an
in-struct [N x i8] slot aligns to 1, and the body then stores through it at the
structure's real alignment.

Three facts about a library's runtime shape the entry points, and
ARCHITECTURE.md#lib-runtime-scope records them. The environment is thread-local
while the registration behind it happens once, so every later thread binds the
first one's environment. One daslang runtime fits in a process and every artifact
in it shares that one: every emitted registration goes through
jit_register_module_once, and a library that finds a populated environment is a
guest - it neither re-initializes the runtime nor drains what it did not create,
so a library loads into a daslang host and beside another library. And nothing in
the runtime finds a jitted SimFunction's shutdown functions, so <P>_destroy emits
them itself.

Nothing about the C surface is hardcoded in C++. [export_c] is a das function
macro in daslib/export_c.das; the module is !inscope and daslib/just_in_time.das
requires it, so -lib, -jit and -exe see the annotation with no require, while a
compile with no JIT asks for one. -lib carries no CodeOfPolicies field either:
llvm_jit declares jit_lib as a module option, run_jit reads it off prog._options
the way it already reads jit_split_modules, and main.cpp pushes it between compile
and simulate.

The library is tested through daslang's own dynamic binder rather than a C host
built by ctest, which is also the coexistence proof: a daslang program binds the
generated .so with [extern(cdecl, late, ...)] and drives it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@aleksisch
aleksisch force-pushed the aleksisch/standalone-jit branch from c235bd1 to bb0403c Compare September 4, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant