wasm: Add revamped tracked JavaScript values API#2359
Conversation
0145c4b to
6bbb881
Compare
[Rebased onto release-0.7 from FissionVM commit 1c9bc056 (branch jgonet/js-api-popcorn); applied unmodified.] Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com> Signed-off-by: Davide Bettio <davide@uninstall.it>
[Rebased onto release-0.7 from FissionVM commit 0c7689c2 (branch jgonet/js-api-popcorn): unrelated EM_ASM one-liner reformats and a cosmetic main.c include reorder dropped, matching the trim already made for AtomVM PR atomvm#2013.] Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>
It's a field in emscripten platform struct. It could be a callback entirely in JS but we use threads. Threads are emulated via webworkers which have their own contexts and variables aren't shared with each other. This means that we would lose uniqueness of the keys. [Rebased onto release-0.7 from FissionVM commit f8b749c9 (branch jgonet/js-api-popcorn): ES6/.mjs output is already unconditional upstream, so the original output-format change (and its message paragraph) was dropped; link flags re-expressed on upstream's JIT_LINK_FLAGS structure - cwrap, stringToNewUTF8, FS appended to EXPORTED_RUNTIME_METHODS in both JIT branches; -sEXPORTED_FUNCTIONS and -sEMULATE_FUNCTION_POINTER_CASTS added to the common link options, matching the flags Popcorn ships (FissionVM squash 75824cbe).] Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>
[Rebased onto release-0.7 from FissionVM commit a222f4c7 (branch jgonet/js-api-popcorn); applied unmodified. enif_make_resource() is deprecated on release-0.7; the call is kept 1:1 with the original and will be migrated in a follow-up commit.] Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>
[Rebased onto release-0.7 from FissionVM commit af96f671 (branch jgonet/js-api-popcorn): only the badvalue atom is added to platform_defaultatoms.def - key and value already exist as core default atoms upstream; matches Popcorn's shipped content (FissionVM squash 75824cbe) and AtomVM PR atomvm#2013.] Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>
run_script_tracked/1 never worked with the default hook: it validates the eval result with an out-of-scope variable (`keys`), so any script evaluating to an array threw a ReferenceError outside the try/catch, and the trapped caller waited forever. The check is gated on isDebug, but standard builds don't define NDEBUG, so it was always on. Signed-off-by: Davide Bettio <davide@uninstall.it>
emscripten:run_script_tracked/1 and emscripten:get_tracked/2 had no Erlang counterpart in the avm_emscripten library, so they were undocumented and invisible to dialyzer. The bare out_of_memory and badvalue answers are documented as unstable rather than spec'd, so they can become raises later. Signed-off-by: Davide Bettio <davide@uninstall.it>
The tracked NIFs had no test coverage. Cover valid and invalid inputs and value deletion on garbage collection. Web-only like the other emscripten tests: these NIFs dispatch to the browser main thread, which is blocked in main() under node. Signed-off-by: Davide Bettio <davide@uninstall.it>
The lib was extracted from eavmlib without updating the doc pipeline, so emscripten/websocket edoc was never rendered. Also fix the three links still pointing at the module's old eavmlib location. Signed-off-by: Davide Bettio <davide@uninstall.it>
The tracked values API had no prose documentation; the Module hook contract in particular was only readable from the C sources. Signed-off-by: Davide Bettio <davide@uninstall.it>
Mark the spots where a cleanup would change behavior popcorn depends on: these are deferred on purpose, not overlooked. Signed-off-by: Davide Bettio <davide@uninstall.it>
The JIT node CI leg trapped at runtime with "null function or function signature mismatch". With this flag the linker rewrites the main module's indirect calls and function table entries into uniform-signature thunks, while the JIT registers its functions in the table at runtime with their real signatures, so any indirect call from rewritten code into JIT compiled code traps. Removing the flag leaves every table entry and indirect call with its real signature, so main module code and JIT compiled code agree again. The flag was only a safety net for function pointers called with a mismatched signature, and the tracked API has none: every callback dispatched to the main thread matches the EM_FUNC_SIG it is dispatched with. Signed-off-by: Davide Bettio <davide@uninstall.it>
js_get_tracked_objects packs the fetched strings with stringToUTF8(string, ptr, size + 1), which writes size content bytes plus a NUL terminator. The NUL of every string but the last one is overwritten by the next string; the last one lands one byte past the malloc'ed buffer. Size the buffer with one extra byte for it. Signed-off-by: Davide Bettio <davide@uninstall.it>
nif_emscripten_get_tracked leaked ref_keys on every path: the key path never freed it (neither on success nor on the out of memory raise) and the value path dispatched it with a NULL satellite, so nobody freed it after do_get_tracked_objects ran. Pass it as the dispatch satellite, like run_script_tracked already does for the script. do_get_tracked_objects also leaked the buffers returned by js_get_tracked_objects when the caller had died meanwhile. Restructure both dispatched functions with an early return on that path so every exit frees what it owns. Signed-off-by: Davide Bettio <davide@uninstall.it>
do_run_script_tracked and do_get_tracked_objects built their answer on the trapped caller's heap from the main thread. The process table read lock taken there only prevents the caller from being destroyed, not from being scheduled: any signal wakes the caller up, and a scheduler thread then touches the same heap concurrently (signal processing appends heap fragments, erlang:garbage_collect/1 from another process triggers a full collection), racing with the main thread's allocations. Build the answer in a private heap instead and let mailbox_send_term_signal copy it, the same way process_info request signals are answered. The resource terms move off the deprecated enif_make_resource, which aborts on out of memory: out of memory now raises in the caller through TrapExceptionSignal instead of aborting the VM or returning the bare out_of_memory atom, and when the caller died meanwhile, destroying the private heap drops the fresh resources so their destructors delete the JS-side values instead of leaking them. Large fetched values allocate their refc buffer fallibly now, raising instead of corrupting the answer. Signed-off-by: Davide Bettio <davide@uninstall.it>
An exception thrown by an overridden tracked hook escaped the EM_JS helpers into the proxied main-thread call, so the trap answer was never sent and the trapped caller waited forever (a delete hook throw also broke the proxied call it ran in). Catch throws at the C/JS boundary and map them to the existing error answers; the delete hook is guarded in place. The helpers also used _malloc results unchecked: on allocation failure they wrote through address zero, which silently corrupts low memory on wasm. Failed allocations now take the error paths, and _free is exported for them. The default onRunTrackedJs tracked sparse array holes as key 0 (result.map skips holes and HEAPU32.set coerces them to 0), aliasing whatever real entry owns key 0; Array.from maps holes densely. A wrong-length array from an overridden onGetTrackedObjects is now a whole-call failure instead of a result list whose length silently differs from the input list. Signed-off-by: Davide Bettio <davide@uninstall.it>
Tracked object keys were juggled across five types: atomic_size_t in the counter and helper parameters, int32_t in the resource struct, uint32_t in the EM_JS helpers, a signed i32 from ccall in pre.js, and term_from_int32 (deprecated) on the way to Erlang, which silently builds wrong terms past the small integer range. Keys are uint32_t everywhere now, pre.js normalizes the ccall result to unsigned, and the key list is built with term_make_maybe_boxed_int64. The per-object fetch statuses were duplicated as JS literals and C static consts; a single enum now defines them, with the EM_JS body noted as its mirror. Also process_id naming, a (void) prototype for the exported key getter, and direct stdatomic.h/stdint.h includes in emscripten_sys.h. Signed-off-by: Davide Bettio <davide@uninstall.it>
The tracked_object() opaque alias said binary(), but handles are resource reference terms (crypto.erl uses reference() for the same kind of handle). The edoc still described the old bare out_of_memory returns, which raise now; it also missed that a stored undefined value is indistinguishable from a missing key with the default hooks, that throwing hooks are survived, and that the key related exports must not be called before main starts. Signed-off-by: Davide Bettio <davide@uninstall.it>
The suite only fetched short ASCII values through the default hooks: the multibyte UTF-8 sizing, the refc binary path taken by values of 64 bytes and more, empty strings, and the behavior with a misbehaving overridden hook (throwing or returning the wrong number of entries) were unexercised. The hook tests also pin liveness: a throwing hook used to hang the caller forever, which cypress would only surface as a timeout. The final map assertion allows sizes below the baseline: a handle dropped in earlier tests can outlive the baseline snapshot in a stale root and be collected later; garbage deletion is asserted by the absence of the garbage values instead. The failure report now escapes HTML metacharacters so diagnostics render intact. Signed-off-by: Davide Bettio <davide@uninstall.it>
Nothing consumes them through the Module object: atomvm.pre.js only uses ccall, and the emscripten-internal glue reaches its helpers directly, without the runtime method exports. Signed-off-by: Davide Bettio <davide@uninstall.it>
Signed-off-by: Davide Bettio <davide@uninstall.it>
6bbb881 to
54f63f7
Compare
|
This PR supersedes #2013. Provenance and history shape. The five original commits are Jakub's I rebased them onto Review comments from #2013. All of them are addressed:
New findings fixed on top of the review. A full pass over the whole
Compatibility with popcorn. Popcorn consumes this API in production, so Note for Software Mansion: FissionVM Testing. The cypress suite ( Thanks @jgonet for the original design and implementation, and @pguyot for |
| set(JIT_LINK_FLAGS | ||
| -sALLOW_TABLE_GROWTH | ||
| "-sEXPORTED_RUNTIME_METHODS=['ccall','addFunction','ENV']" | ||
| "-sEXPORTED_RUNTIME_METHODS=['ccall','addFunction','ENV','FS']" |
There was a problem hiding this comment.
I don't think FS is needed within this PR.
| ) | ||
| else() | ||
| set(JIT_LINK_FLAGS "-sEXPORTED_RUNTIME_METHODS=['ccall','ENV']") | ||
| set(JIT_LINK_FLAGS "-sEXPORTED_RUNTIME_METHODS=['ccall','ENV','FS']") |
There was a problem hiding this comment.
I don't think FS is needed within this PR.
| } | ||
|
|
||
| Heap heap; | ||
| size_t heap_size = LIST_SIZE(objects_n, TUPLE_SIZE(2)) + LIST_SIZE(strings_n, BINARY_HEADER_SIZE) + term_binary_data_size_in_terms(all_byte_size); |
There was a problem hiding this comment.
This allocation looks wrong and I wonder if we should use TERM_BINARY_DATA_SIZE_IN_TERMS macro. It is too complex for the CodeQL queries, but the following additional tests crash:
test_many_large_values_round_trip() ->
Strings = [lists:duplicate(50, C) || C <- lists:seq($a, $h)],
{ok, Refs} = emscripten:run_script_tracked(tracked_script(Strings)),
Expected = [{ok, list_to_binary(S)} || S <- Strings],
Expected = emscripten:get_tracked(Refs, value),
ok.
test_many_small_values_round_trip() ->
Strings = [lists:duplicate(10, C) || C <- lists:seq($a, $j)],
{ok, Refs} = emscripten:run_script_tracked(tracked_script(Strings)),
Expected = [{ok, list_to_binary(S)} || S <- Strings],
Expected = emscripten:get_tracked(Refs, value),
ok.
tracked_script(Strings) ->
Quoted = [[$', S, $'] || S <- Strings],
["[", lists:join($,, Quoted), "]"].
| Heap heap; | ||
| if (UNLIKELY(memory_init_heap(&heap, TUPLE_SIZE(2) + LIST_SIZE(keys_n, TERM_BOXED_REFERENCE_RESOURCE_SIZE)) != MEMORY_GC_OK)) { | ||
| free(keys); | ||
| send_tracked_trap_oom(global, process_id); |
There was a problem hiding this comment.
We need to clean up what js_tracked_eval inserted.
| -type html5_target() :: window | document | screen | iodata(). | ||
| -opaque listener_handle() :: binary(). | ||
| -opaque tracked_object() :: reference(). | ||
| -type get_tracked_result() :: {ok, binary()} | {error, badkey} | {error, badvalue}. |
There was a problem hiding this comment.
Not a fan of defining get_tracked_result() here, but if we do, we should export it.
There was a problem hiding this comment.
Fixed. I believe it is more readable this way.
| } | ||
| HEAPU32[size / HEAPU32.BYTES_PER_ELEMENT] = keys.length; | ||
| HEAPU32.set(keys, ptr / HEAPU32.BYTES_PER_ELEMENT); | ||
| return ptr; |
There was a problem hiding this comment.
malloc(0) could return NULL. Tests test_empty_array_yields_empty_list/0 and test_null_yields_empty_list/0 don't fail because emscripten's malloc(0) doesn't return NULL.
|
AMP, usual caveats: PR review: tracked JavaScript values APIRange reviewed: The final state of the 21 commits was reviewed as one change set; bugs fixed by later commits in the range were not reported. The review covered the JavaScript/C boundary, resource ownership, trap delivery, heap sizing, key identity, public Erlang contracts, and browser tests. Oracle was also asked to independently stress-test the final diff; its findings were verified against the cited implementation before inclusion below. Findings1. [Critical] Size each returned binary separately or
|
Fixed all the reported issues. |
do_get_tracked_objects summed the byte sizes of all fetched values and sized a single binary from the total, while the answer holds one binary per value: fetching several values in one call allocated less than the answer needs and built it past the end of the private heap. Values of REFC_BINARY_MIN bytes and more make the gap wider, as each of them takes a fixed size boxed term no matter how long it is. Sum the heap size of every fetched value instead. The strings_n out parameter of js_get_tracked_objects only fed the old expression and goes away with it. The strings buffer size also crosses to C through HEAPU32 while _malloc truncates its argument to 32 bits, so a total above 4 GB would allocate a small buffer and then write every string past its end. Fail the call instead. Signed-off-by: Davide Bettio <davide@uninstall.it>
The onRunTrackedJs hook tracks every value it evaluates before the VM
turns any key into a handle, so a failure in between left those values
in the JavaScript map with nobody left to delete them: the private heap
allocation, a resource allocation partway through the list, and the key
array allocation on the JavaScript side, which could not tell C
anything and ended up reported as an evaluation error.
js_tracked_eval reports an explicit status now (ok, error, out of
memory), and every path that receives keys without turning them into
handles deletes them through onTrackedObjectDelete, the same call the
resource destructor makes. Values tracked under keys the hook does not
return stay its own business.
An empty result no longer allocates at all: malloc(0) may return a null
pointer, which C cannot tell apart from a failed allocation, so an
empty array only yielded {ok, []} because emscripten happens to return
a minimal chunk for it.
Signed-off-by: Davide Bettio <davide@uninstall.it>
The keys an onRunTrackedJs override returns went straight into HEAPU32.set, which coerces whatever it is given: a negative key became a huge one, 2 ** 32 became 0 and a fraction was truncated, so the resulting handle addressed an unrelated entry, fetched the wrong value and deleted that entry when it was collected. Values it cannot coerce, such as a symbol, made it throw instead, out of the proxied call and past the hook try/catch, leaving the trapped caller waiting forever. Check that every key is an unsigned 32 bit integer below the reserved last value of the range, and take the evaluation error return otherwise. The hook result is copied with Array.from inside the try/catch as well, so an exotic array cannot run hook code, and throw, outside of it either. Signed-off-by: Davide Bettio <davide@uninstall.it>
mailbox_send_term_signal hands the message it builds to mailbox_post_message without checking it, and building it copies the answer into freshly allocated storage. An allocation failure there dereferenced a null pointer on the browser main thread, taking the whole VM down instead of raising out_of_memory in the caller as the other failures of these calls do. The answer of get_tracked/2 grows with the values it fetched, so the copy is not a small allocation. Build the signal here and fall back to the immediate exception signal when it cannot be allocated. Signed-off-by: Davide Bettio <davide@uninstall.it>
The key counter is 32 bit wide on wasm32 and wrapped silently, so a VM
that tracked more than 2^32 values started handing out keys that live
handles still own: storing under such a key overwrites the value of the
older handle, and collecting either handle deletes the entry both point
at.
Saturate the counter instead and reserve its last value to mean that
the key space is exhausted. The key validation rejects that value, so
an exhausted VM answers {error, badarg} rather than corrupting the
values it still tracks. The counter also drops the size_t type it had
kept from before the keys were unified on uint32_t.
Signed-off-by: Davide Bettio <davide@uninstall.it>
get_tracked/2 uses it in its exported spec, so callers writing their own specs cannot name the result type they get. Signed-off-by: Davide Bettio <davide@uninstall.it>
Nothing in the tree reaches the emscripten filesystem object through the module: the web tests pass their modules as arguments and the loader fetches them. It only serves embedders that write their modules into the in-memory filesystem before main runs, which has nothing to do with tracked values and belongs to its own change. Signed-off-by: Davide Bettio <davide@uninstall.it>
|
by now I'm just the messsenger;-) Follow-up review: tracked JavaScript values fixesFollow-up range: The follow-up diff and the resulting final PR state were reviewed. Oracle independently reviewed the same range and the five findings in Disposition of the original findings
Remaining findings1. [High] Roll back invalid keys too, or exhaustion and malformed hooks still retain values
dropKeys(keys.filter(isKey));That deletes only valid siblings. Any value stored under the invalid key remains owned by neither JavaScript nor an Erlang resource. This also breaks the new exhaustion path. The default The new invalid-key test does not catch this: Every key returned by the hook is an ownership claim for this attempted call. If validation fails before resources exist, cleanup should invoke the deletion hook for every returned key, including malformed values. Exceptions are already caught by Smallest fix: diff --git a/src/platforms/emscripten/src/lib/platform_nifs.c b/src/platforms/emscripten/src/lib/platform_nifs.c
--- a/src/platforms/emscripten/src/lib/platform_nifs.c
+++ b/src/platforms/emscripten/src/lib/platform_nifs.c
@@ -263,7 +263,7 @@ EM_JS(uint32_t *, js_tracked_eval, (const char *code, uint32_t *size, uint8_t *s
const isKey = (key) => Number.isInteger(key) && key >= 0 && key <= 0xfffffffe;
if (!keys.every(isKey)) {
console.error("onRunTrackedJs returned invalid keys", keys);
- dropKeys(keys.filter(isKey));
+ dropKeys(keys);
setOutcome(ERROR, 0);
return 0;
}Update the test hook to store values under both the valid sibling and the actual invalid key, then assert both are absent. Add an exhaustion regression by temporarily overriding only 2. [High] Copy get-hook output inside the
|
Only the valid keys were dropped when a hook returned a malformed one, so a value the hook had tracked under the malformed key stayed in the map with nothing owning it on either side. The default hook reaches this at key space exhaustion: it tracks its value before it can learn that the key it just got is the reserved one. Signed-off-by: Davide Bettio <davide@uninstall.it>
The onGetTrackedObjects result was inspected outside the try/catch that wraps the hook call, so a hook returning an array whose element getter throws, or a revoked proxy, threw out of the proxied call and left the trapped caller waiting forever. Copy the array with Array.from inside the guard, the way the tracked eval helper already does.
A key an onRunTrackedJs override returned twice got one handle each, and the first of them collected deleted the value the other still addressed, so a live handle read back badkey. Under a hook that runs cleanup on delete, that cleanup ran while a handle was still live. Take the evaluation error return instead, and document that returning a key hands it over to the VM: reuse across calls has the same effect and cannot be detected without tracking every live key.
An allocation failure in js_get_tracked_objects was reported the same way a misbehaving hook is, so a conforming hook got blamed for it with the bare badvalue atom, where the rest of the API raises out_of_memory. Report the outcome through an out parameter, as the tracked eval helper does, and share one status enum between the two.
Second follow-up review: tracked JavaScript values fixesRemote range reviewed: The contributor's remote branch was fetched and reviewed without changing the local Disposition of
|
| Finding | Status | Verification |
|---|---|---|
| 1. Invalid/exhausted keys omitted from rollback | Resolved | Validation now passes every returned key to dropKeys. Tests store under both valid and malformed keys, and the exhaustion test uses the default run hook and verifies that the sentinel entry is absent. |
| 2. Throwing getter in get-hook output traps the caller | Resolved | The hook result is copied with Array.from inside the exception guard. The new getter test expects the whole-call badvalue, restores the hook, and successfully fetches the original handle afterward. |
| 3. Duplicate/reused key ownership is not enforced or documented | Partially resolved | Duplicate keys within one result are rejected, and cross-call exclusive ownership is documented. The validation itself can still throw outside the guard, and rollback calls deletion once per occurrence rather than once per identity; see findings 1 and 2. |
4. Get-side allocation OOM is reported as badvalue |
Resolved in implementation; untested | Every explicit js_get_tracked_objects failure initializes the new call status. Allocation failures route to out_of_memory; malformed hooks retain the existing badvalue path. No allocator fault-injection test exercises the distinction. |
Findings
1. [High] Guard duplicate validation or evaluated JavaScript can trap the caller and leak its values
After onRunTrackedJs returns, duplicate validation executes outside the surrounding try/catch:
const hasDuplicates =
keys.length > 1 && new Set(keys).size !== keys.length;Set is a mutable JavaScript global and construction consumes the array iterator. Either operation can throw. This is reachable through the default hook, not only a deliberately broken override: the evaluated script runs before this line and can replace globalThis.Set, then return ordinary values:
globalThis.Set = class {
constructor() {
throw new Error("boom");
}
};
["owned"];The default hook stores "owned" and returns its key. The replacement Set constructor then throws outside the guard, so no trap answer is sent and no rollback runs. The Erlang caller waits indefinitely while the JavaScript value remains retained without a resource handle. A catchable allocation failure during Set construction has the same control flow. keys.every(isKey) is also outside the guard and resolves a mutable prototype method.
Suggested fix: perform validation inside an exception boundary, and make rollback independent of array iterators. The following combines that with the distinct-key cleanup required by finding 2:
diff --git a/src/platforms/emscripten/src/lib/platform_nifs.c b/src/platforms/emscripten/src/lib/platform_nifs.c
--- a/src/platforms/emscripten/src/lib/platform_nifs.c
+++ b/src/platforms/emscripten/src/lib/platform_nifs.c
@@ -230,14 +230,29 @@ EM_JS(uint32_t *, js_tracked_eval, (const char *code, uint32_t *size, uint8_t *s
// the hook stored these values but no resource owns them yet: drop them
// the way the resource destructor does, or they stay tracked forever
const dropKeys = (keys) => {
- for (const key of keys) {
+ for (let i = 0; i < keys.length; ++i) {
+ const key = keys[i];
+ let seen = false;
+ for (let j = 0; j < i; ++j) {
+ const previous = keys[j];
+ // SameValueZero, matching JavaScript Map key identity.
+ if (previous === key
+ || (previous !== previous && key !== key)) {
+ seen = true;
+ break;
+ }
+ }
+ if (seen) {
+ continue;
+ }
try {
Module['onTrackedObjectDelete'](key);
} catch (e) {
- console.error("onTrackedObjectDelete threw", e);
+ try {
+ console.error("onTrackedObjectDelete threw", e);
+ } catch (_) {
+ // Cleanup must not escape the dispatched callback.
+ }
}
}
};
@@ -262,11 +277,21 @@ EM_JS(uint32_t *, js_tracked_eval, (const char *code, uint32_t *size, uint8_t *s
// would coerce anything else silently (-1 to 4294967295, 2 ** 32 to 0,
// a fraction to its truncation) and alias an unrelated entry
const isKey = (key) => Number.isInteger(key) && key >= 0 && key <= 0xfffffffe;
- // a repeated key would get one handle each, and the first of them
- // collected deletes the value the others still address; only an array of
- // two or more keys can repeat one, so a single value result builds no set
- const hasDuplicates = keys.length > 1 && new Set(keys).size !== keys.length;
- if (!keys.every(isKey) || hasDuplicates) {
+ let hasInvalidKeys;
+ let hasDuplicates;
+ try {
+ hasInvalidKeys = !keys.every(isKey);
+ hasDuplicates
+ = keys.length > 1 && new Set(keys).size !== keys.length;
+ } catch (e) {
+ dropKeys(keys);
+ setOutcome(ERROR, 0);
+ return 0;
+ }
+ if (hasInvalidKeys || hasDuplicates) {
console.error("onRunTrackedJs returned invalid keys", keys);
// every returned key is an ownership claim, malformed ones included:
// the hook may have stored a value under them tooThis is the smallest control-flow correction. A stronger implementation can avoid mutable Set/Array.prototype.every entirely by validating and checking duplicates with indexed loops; that trades the temporary set allocation for quadratic duplicate checking.
Add a browser regression that replaces Set during the evaluated script, verifies that run_script_tracked/1 returns promptly, and verifies that the newly tracked value was deleted. Restore Set before making the assertion call.
2. [Medium] Roll back each distinct key once, not once per duplicate occurrence
When a hook returns [Key, Key], duplicate validation correctly rejects the result, but dropKeys(keys) invokes Module.onTrackedObjectDelete(Key) twice. The documented model transfers ownership of a key identity, not ownership of every occurrence in a malformed array. Since no resources were created, rollback should release that identity exactly once.
The default Map.delete hook is idempotent, so the current duplicate test passes. A valid custom deletion hook need not be idempotent: it may decrement a reference count, release an external object, or perform application-side cleanup. Calling it twice can therefore double-release state even if a second thrown exception is caught.
The indexed, SameValueZero-aware dropKeys diff in finding 1 fixes this without relying on Set, a mutable iterator, or native-key coercion. Update the internals wording from “every returned key is deleted” to “each distinct returned key is deleted once.”
Add a browser test that installs a counting onTrackedObjectDelete, returns [Key, Key], and asserts exactly one deletion callback before restoring the hook.
Verification performed
git diff --check 1078b72cd18ede37e46f2ae6b451c44d36958391..449ff9301c7c69ded84e00e2c8e4d29ed9eba321passes.- A clean release web build of exact remote tip
449ff9301completed with Emscripten 5.0.7 andAVM_DISABLE_JIT=ON. AtomVM.mjslinked successfully.- The
emscripten_erlang_test_modulestarget compiled all Emscripten test modules, including the revisedtest_run_script_tracked.erl. - The explicit
js_get_tracked_objectsstatus/cleanup paths were audited: malformed results setCALL_ERROR; all three allocation failures and the unrepresentable aggregate size setCALL_OOM; success setsCALL_OKafter publishing outputs. - Browser/Cypress tests and allocation fault injection were not run.
Merge recommendation
Do not merge remote tip 449ff9301 yet. Guard duplicate validation and make rollback operate once per distinct key. Both changes are localized; focused browser coverage should be added for the escaped-validation exception and deletion callback count.
|
btw I believe FS is used by popcorn - so maybe keep it around, seems unrelated to PR either way. |
I will make a PR just for it |
The key checks went through Set and Array.prototype.every, which the script that just ran can replace: it shares the realm and an undeclared assignment in sloppy mode is enough, with Set an ordinary variable name. The throw then escaped the proxied call and left the caller trapped forever, the values the hook had tracked owned by nobody. Check with operators instead, and read the keys of the fetch helper with an indexed loop rather than through the typed array iterator, for the same reason. Duplicate detection costs a pairwise scan now, over the handful of keys a result holds. Signed-off-by: Davide Bettio <davide@uninstall.it>
A hook naming one key twice got the deletion hook called twice for it, although no resource was ever created and the ownership the hook gives up is that of a key, not of an occurrence in a malformed array. The default hook deletes from a map and does not care, but a hook holding a reference count or an external object would release it twice. Signed-off-by: Davide Bettio <davide@uninstall.it>
The existing JavaScript bridge (cast, call and the promise NIFs) only
passes integers and strings, so values such as DOM nodes or callback
functions could not be handled from Erlang at all. This API lets
Erlang hold opaque handles to arbitrary JavaScript values and ties
the JavaScript value lifetime to the Erlang handle lifetime.
emscripten:run_script_tracked/1 evaluates a script on the browser's
main thread; each element of the array it evaluates to is stored in a
JavaScript-side map under a fresh integer key, and the caller gets
one opaque handle (a NIF resource wrapping the key) per element. When
the VM garbage collects a handle, the resource destructor dispatches
a call to the main thread that drops the value from the map.
emscripten:get_tracked/2 maps handles back to their integer keys, or
fetches the current values as UTF-8 binaries (values must be
JavaScript strings; anything else is typically serialized to JSON
first).
The JavaScript side is a set of hooks on the emscripten module object
that embedders may override to customize evaluation, fetching and
cleanup without patching AtomVM; the defaults are the reference
implementation of the hook contract. On the VM side the trap answers
are built in a private heap and copied into a signal, so no foreign
thread ever touches a process heap.
The API was designed and first implemented by Jakub Gonet (Software
Mansion) for Popcorn, where it powers Elixir-to-JavaScript interop in
production. This PR revamps that work for upstream: rebased onto
release-0.7, review findings fixed, hardened against throwing hooks
and allocation failures, and completed with typespecs, edoc, prose
documentation and a browser test suite.
Supersedes: #2013
Closes: #2013
These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).
SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later