Feat: HTTP worker cancellation upon client closing connection, fix for large orchestration jobs failing to run and compression inprovements

[Zacgoose]

This pull request introduces two main improvements: (1) enhanced handling of HTTP client disconnects and request cancellation in the PowerShell dispatch pipeline, and (2) robust support for large entity splitting and reassembly in the Azure Table storage layer. These changes improve reliability, correctness, and efficiency for both HTTP endpoint execution and storage of large logical entities.

HTTP Request Cancellation and Reliability Improvements:

• HTTP endpoints now properly honor client disconnects for read (GET) requests by linking the HTTP cancellation token to the PowerShell execution pipeline, ensuring that resources are freed promptly and partial results are never cached or returned if the client aborts. Write requests (POST/PUT/DELETE/PATCH) always run to completion, even if the client disconnects, to avoid partial mutations. [1] [2] [3] [4] [5] [6] [7]

Azure Table Storage: Large Entity Splitting, Reassembly, and Cleanup:

• The storage layer now transparently splits large logical entities across multiple physical rows when necessary, and reassembles them on read. This includes logic to clean up "part" rows left behind by previous larger versions or deletes, preventing stale data from being returned on future reads. [1] [2] [3] [4] [5] [6] [7] [8] [9]
• Batch upserts and conditional replacements now handle splitting and cleanup atomically, ensuring that logical operations remain all-or-nothing even when entities are split. The batch size cap is enforced after splitting to avoid partial application. [1] [2]
• Partition and table scans now yield fully reassembled entities, except for projected queries which intentionally skip reassembly to expose all physical rows for maintenance operations. [1] [2]
• Deletes now also remove any associated "part" rows to prevent deleted entities from being accidentally reconstructed during partition scans.