Avoid throwing on blob: created via URL.createObjectURL()

[WebReflection]

The current artifact fails at instantiating blobs that contain runtime WASM artifacts.

Fixes WebAssembly/wabt#2828

[hoodmane]

Looks like it needs a test.

[WebReflection]

@hoodmane

Looks like it needs a test.

agreed, so far I've ensured no regressions but I wonder: how does one write a test? I have tested personally the change and it works as expected but tests in here are like reproducing the whole thing each time ... is there any guidance or should I just add a test to any other test that touches this change too to counter-validate it works?

thanks in advance for any possible hint!

[hoodmane]

Currently looks like MINIMUM_NODE_VERSION defaults to 18.3, we could do a conditional to check if MINIMUM_NODE_VERSION is greater than or equal to 20 and in that case drop it. Both node 18 and 20 are already end of life right?

[hoodmane]

Oh but I see this comment was already there.

[sbc100]

I think we should probably update the comment to focus on the "does not have a full fetch" part. Specifically node's fetch does not work with file:// URLs, which makes it mostly useless for loading wasm modules that exist locally.

[hoodmane]

Oh yeah that's disappointing.

[WebReflection]

it works with blobs though, which I believe are stored as temporary files because it doesn't need a server spinning to work without issues so it's extremely weird WASM files cannot be fetched ... should we escalate upstream? Right now this is blocking one of my utilities in NodeJS so we could also do a step after the other as this issue is focused to one caveat, the other one is bigger and involves upstream support/changes.

[sbc100]

I assume this only occurs for users of Module.mainScriptUrlOrBlob?

If that is true then I would take a look at the current tests we have for Module.mainScriptUrlOrBlob.. why don't they cover this case already? How could they be extended to include whatever case you ran into here?

[hoodmane]

Right @WebReflection either update test_mainScriptUrlOrBlob in test_other.py or add a new test next to it.

[WebReflection]

to clarify, this works without any issue whatsoever in NodeJS

const url = URL.createObjectURL(
  new Blob(
    [new Uint8Array([
      0, 97, 115, 109, 1, 0, 0, 0, 1, 7, 1, 96, 2, 127, 127, 1, 127, 3, 2, 1,
      0, 7, 7, 1, 3, 97, 100, 100, 0, 0, 10, 9, 1, 7, 0, 32, 0, 32, 1, 106, 11])],
    { type: 'application/wasm' }
  )
);

const buffer = await fetch(url).then(response => response.arrayBuffer());

console.log(
  await WebAssembly.instantiate(buffer).then(
    ({ instance: { exports } }) => exports.add(1, 2)
  )
);

// 3

URL.revokeObjectURL(url);

The WAT that produced that buffer, borrowed from https://developer.mozilla.org/en-US/docs/WebAssembly/Guides/Understanding_the_text_format#calling_the_function

(module
  (func $add (param $lhs i32) (param $rhs i32) (result i32)
    local.get $lhs
    local.get $rhs
    i32.add
  )
  (export "add" (func $add))

In here we should test that if that blob url is returned via { locateFile: () => url } the whole thing doesn't throw for no reason whatsoever.

If NodeJS <= 18 is an issue, they have thrown errors anyway if they try to run blob files already, and so it does any other version of NodeJS after right now ... and because it used to throw, I think it's safe to state that if the url is a blob one, the fetch is not just convinient but the only way to go.

This would require a much larger change (in terms of possible branched regressions) so I am asking: what do you expect me to do here, exactly? I can test my changes work (they do) but would that be enough or the end of this issue?

Thanks.

[WebReflection]

@sbc100

I assume this only occurs for users of Module.mainScriptUrlOrBlob?

I don't know if this helps but here how they are building the release https://github.com/WebAssembly/wabt/blob/68a7ed2769b883f6bfd83e4ea6ced6837a33dae7/Makefile#L149-L152 and the artifact just exposes a way to fetch the WASM file.

Their original source code swaps the file name by prefixing it via ../ but that's just Web based outcome, although their artifact also should work on NodeJS out of the box.

However, as soon as I bundle their own WASM via a Blob it fails because of that if condition that bails out and throw everything around.

Other runtimes I've tried don't fail, including, of course, browsers' based attempts, but all runtimes end up in there, that !NODE_THINGY breaks everything with blob URLs.

[sbc100]

Can you share an example of the failure you are trying to fix? It sounds like you are trying to run wabt under node an failing?

Can you share a full failing example?

[sbc100]

as soon as I bundle their own WASM via a Blob

Can you explain more about this? Are you doing this bundling yourself, or is it a bundler doing it? If so which bundler, and how are you running it. We do have some tests for bundlers for this kind of thing, and perhaps we missing a use case.

[WebReflection]

@sbc100 let me expand on that ... meet wat-tag, it uses libwabt behind the scene and that's produced on releases as demo with both libwabt.js and libwabt.wasm files: https://github.com/WebReflection/utils/tree/main/src/libwabt#readme

Because their artifact trusts the import.meta.url to decide how to load the WASM version via { locateFile: url => ...logic } and because Workers created via URL.createObjectURL(blob) can't carry that detail within them, I've base64-encoded after deflate that WASM file and I base64-decode via inflate that WASM so that any bundler that deals with JS will produce a single Blob that can work out of an artifact and run whatever foreign file dependency was meant.

This worked for years very well for PyScript, sql.js (WASM), and other projects that don't trust import.meta.url when things run within a URL.createObjectURL(blob) because there's no way to attach assets to that URL.

const stuff = new Worker(
  URL.createObjectURL([blobOfStuff])
);

I hope this explains the scenario, yet the first link about wat-tag I gave you, it's all about that, I want to create WASM files at runtime on either Web or NodeJS and be able to consume that just like any other Blob based stream I could encounter on the full-stack scenario, if that makes sense.

edit - to whoever is interested in doing the same, this is how I create compressed WASM artifacts that perfectly run on any JS runtime/bundler to date https://github.com/WebReflection/utils/blob/main/build/libwabt.js

[WebReflection]

@sbc100 apologizes the culprit of my previous comment was kinda this: https://github.com/WebAssembly/wabt/blob/main/docs/demo/wat2wasm/demo.js#L31-L38

that's why I am bundling the whole WASM as compressed base64 string.