api: escape error strings in batch result JSON (bulk-submit loses all results) - #144
Open
emilstahl wants to merge 1 commit into
Open
api: escape error strings in batch result JSON (bulk-submit loses all results)#144emilstahl wants to merge 1 commit into
emilstahl wants to merge 1 commit into
Conversation
BatchResultToRaw interpolated err.Error() directly into a JSON object via fmt.Sprintf. Go HTTP client errors embed quotes (url.Error format: `Get "http://host": ...`), producing invalid JSON that aborted the final marshal of the entire bulk-submit result array, losing all results. Use json.Marshal for the message instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
bulk-submiton a large URL list fails with:No output at all — the results of every URL in the batch are lost, even though all scans were submitted.
Root cause
api/batch.goBatchResultToRaw()interpolatederr.Error()directly into JSON:Go HTTP client errors embed quotes —
*url.Errorrenders asGet "http://host/": .... When a scan fails with one of these (transient network error, client timeout, connection reset), the raw message becomes:which is invalid JSON (
hofhttpappears right after the"error": "Get "key:value pair — matching the error message above). The invalidjson.RawMessagethen makes the finaljson.MarshalIndentof the whole result array incmd/scan/bulk.gofail, so the entire batch prints nothing. Small batches usually dodge it because they rarely hit a client-side network error; ~100+ URLs almost always trigger one.Fix
Marshal the error message with
json.Marshalinstead of interpolating it. Added a regression test with a*url.Error-style message.Reproduced with binary
urlscan-cli 2026.08.18on a 108-URL bulk submit; subsets of the same list passed, full list failed deterministically; fixed build returns all 108 results.