chore(build): bump version to 0.20.4 + install.cmd for PowerShell-blocked Windows hosts - #231
Merged
Conversation
First of the six-repo train: THIS -> extensions 0.2.4 -> arrow 0.2.5 -> jdbc 0.2.5 -> repl 0.20.4 -> helm 0.3.5. Thirty-five changes since 0.20.3. Two themes dominate, and both are classes of SILENT wrong answers rather than crashes. Completeness — a query returned fewer rows/groups than the index holds, with HTTP 200 and no warning: #197 PIT paging sorts by `_doc`, not the injected `_shard_doc` literal (60.5 -> 8.1 ms/page on a 10M index), and the PIT gate moves 7.10 -> 7.12 #200 no total-hit accounting on the paging path #201 preferSearchAfter honored on every ES version #202 the PIT is closed exactly once per extraction #205 GROUP BY emits an explicit terms size — it used to return only ES's default top 10 groups #207 window PARTITION BY sized the same way — top 10 partitions #209 an un-LIMITed row query routes through scroll — it used to return ES's default 10 hits #224 an explicit LIMIT above index.max_result_window pages through scroll instead of failing where no LIMIT succeeded Parser correctness — the statement that ran was not the statement written: #213 the WHOLE statement must parse (`phrase`). `DELETE FROM orders WHEREE id = 1` used to run as `DELETE FROM orders` and empty the index; six more shapes measured in the commit. Plus a quote- and comment-aware multi-statement split, COPY INTO FILE_FORMAT = parsed as documented, and literals protected from the normalizer #216 the licence-activation flag is read, not returned as a constant #220 a function call closes exactly the parentheses it opened; SCRIPT AS keeps its own closing parenthesis #212 watcher/DML WHERE qualifiers resolve, and the FROM shapes DELETE and UPDATE cannot express are rejected #191 a JOIN in a watcher search input is rejected, not dropped #211 #204 #215 jest Pipeline/Template/Watcher routed through the guarded wrapper; value-discard casts and licence-path throws removed ALTER COLUMN ... SET|ADD FIELD parses and actually applies; a column is multi-field or script-defined, never both Result shape and throughput: #226 result rows no longer surface Elasticsearch hit metadata #228 every Elasticsearch response is parsed exactly once, on every client and every path #229 streamed rows are normalized in a single pass, with a zero-rebuild passthrough when nothing needs changing arrow#139 tryParseAsDateTime guarded by a cheap temporal-shape check RELEASE NOTES for 0.20.4 — five user-visible changes, none a regression: 1. Result rows no longer carry `_index`, `_score` or `_sort`. `_id` is opt-in via the new HOCON setting `elastic.include-document-id` (default false, env `ELASTIC_INCLUDE_DOCUMENT_ID`). 2. Strict whole-statement parsing. SQL that previously parsed as a PREFIX and silently discarded the rest is now a hard error ("end of input expected"). This includes bare `UNION`, which was never a synonym for `UNION ALL`, and `CREATE TABLE ... WITH (...)`, a clause the grammar does not have. 3. Generated GROUP BY / PARTITION BY JSON now carries `"size":65536`. Downstream tests pinning that JSON need their expectations updated. 4. An un-LIMITed row query now returns every row where it returned 10. The licensed `maxQueryResults` cap moved with the routing — it keys on `returnsRows`, so windowed and script-only queries are capped too rather than bypassing it. 5. The PIT gate is 7.12, not 7.10. On 7.11 `_shard_doc` fails loudly and `_doc` silently loses rows, so 7.11 now uses classic scroll. Verified: the build loads and all 20 module version keys report 0.20.4. Follow-up after publication (NOT in this PR): the user-facing version sweep across README/installers/documentation and the softclient4es.dev site, per the release-doc-sweep convention — versions are updated only once the artifacts are verified on JFrog. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nstall
The Windows client default execution policy is Restricted, and plenty of
managed hosts are set to AllSigned. On those machines a DOWNLOADED
install.ps1 cannot run at all, and the usual advice — change the machine's
execution policy — is exactly what the policy exists to prevent.
install.cmd is a wrapper and nothing else:
powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0install.ps1" %*
exit /b %ERRORLEVEL%
-ExecutionPolicy Bypass on the command line applies to THAT PROCESS only:
it changes nothing on the machine and needs no elevation. Note the piped
one-liner (`irm ... | iex`) was never affected — `iex` runs a string, not a
script file — so this is specifically for the download-then-run path, and
for double-click.
Deliberately NOT a second implementation. Every option, default, fallback
and message stays in install.ps1 and `%*` forwards the flags verbatim, so
the two entry points cannot drift. The only logic here is a guard that
names the problem when install.ps1 is not sitting next to it.
CRLF, pinned by a new .gitattributes (`*.cmd -text`, `*.bat -text`). The
documented install path is `curl -O` from raw.githubusercontent.com, which
serves the stored blob byte for byte — no checkout-time eol conversion ever
runs, so without `-text` git would normalise the CRLF out of the blob and
ship an LF-only batch file. LF-only .cmd files are a known source of
failures around labels and multi-line blocks.
Docs: README gets the cmd.exe install line; documentation/client/repl.md
gets a Windows (cmd.exe) section that says which path the execution policy
actually blocks, the two-file download, the flags-are-identical contract,
and the honest limit — `-ExecutionPolicy Bypass` is overridden when the
policy comes from GROUP POLICY (MachinePolicy / UserPolicy scopes), which
no wrapper can work around. Also documents running the generated
uninstall.ps1 the same way, since the installer still writes it as a .ps1.
Running the REPL never needed PowerShell: bin\softclient4es.bat is batch.
Verified on macOS with pwsh 7 (no Windows host here, so cmd.exe executing
the wrapper itself is NOT covered): the exact invocation form the wrapper
uses, `-NoProfile -File install.ps1 <args>`, renders -Help correctly, and
`-EsVersion 5` forwards a value-taking flag and propagates exit code 1 —
which is what `exit /b %ERRORLEVEL%` relays to the caller.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits: the version bump, and a new
install.cmdfor Windows hosts that block.ps1.1.
chore(build): bump version to 0.20.4Drops
-SNAPSHOT:ThisBuild / version := "0.20.4". One line,build.sbt.First of the six-repo train. Nothing upstream of this one, so it is the only PR in the train that is mergeable on sight:
core 0.20.4→extensions 0.2.4→arrow 0.2.5→jdbc 0.2.5→repl 0.20.4→helm 0.3.5Each downstream PR pins the released coordinate of the one before it, so each is opened only once its dependency is published on JFrog (DockerHub for helm) — no prediction window.
What 0.20.4 carries
Thirty-five changes since 0.20.3. Two themes dominate, and both are classes of silent wrong answers, not crashes.
Completeness — a query returned fewer rows/groups than the index holds, HTTP 200, no warning:
_doc, not the injected_shard_docliteral (60.5 → 8.1 ms/page on a 10M index); PIT gate 7.10 → 7.12preferSearchAfterhonored on every ES versionGROUP BYemits an explicit terms size — it used to return ES's default top 10 groupsPARTITION BYsized the same way — top 10 partitionsLIMITaboveindex.max_result_windowpages through scroll, instead of failing where no LIMIT succeededParser correctness — the statement that ran was not the statement written:
phrase).DELETE FROM orders WHEREE id = 1used to run asDELETE FROM ordersand empty the index; six more shapes are measured in the commit. Plus a quote- and comment-aware multi-statement split,COPY INTO FILE_FORMAT =parsed as documented, and literals protected from the normalizerSCRIPT ASkeeps its own closing parenthesisWHEREqualifiers resolve, and theFROMshapesDELETE/UPDATEcannot express are rejectedJOINin a watcher search input is rejected, not droppedALTER COLUMN … SET|ADD FIELDparses and actually applies; a column is multi-field or script-defined, never bothResult shape and throughput: #226 rows no longer surface ES hit metadata · #228 every ES response parsed exactly once, on every client and every path · #229 streamed rows normalized in a single pass with a zero-rebuild passthrough · arrow#139
tryParseAsDateTimeguarded by a cheap temporal-shape check._index,_scoreor_sort._idis opt-in via the new HOCON settingelastic.include-document-id(defaultfalse, envELASTIC_INCLUDE_DOCUMENT_ID).end of input expected). This includes bareUNION— never a synonym forUNION ALL, one de-duplicates — andCREATE TABLE … WITH (…), a clause the grammar does not have.GROUP BY/PARTITION BYJSON now carries"size":65536. Downstream tests pinning that JSON need their expectations updated.maxQueryResultscap moved with the routing — it keys onreturnsRows, so windowed and script-only queries are capped rather than bypassing it._shard_docfails loudly and_docsilently loses rows, so 7.11 now uses classic scroll.2.
feat(install): add install.cmd so a host that blocks .ps1 can still installThe Windows client default execution policy is
Restricted, and plenty of managed hosts areAllSigned. On those machines a downloadedinstall.ps1cannot run at all, and the usual advice — change the machine's execution policy — is exactly what the policy exists to prevent.install.cmdis a wrapper and nothing else:-ExecutionPolicy Bypasson the command line applies to that process only — it changes nothing on the machine and needs no elevation. Note the piped one-liner (irm … | iex) was never affected, sinceiexruns a string rather than a script file; this is specifically for the download-then-run path, and for double-click.Deliberately not a second implementation. Every option, default, fallback and message stays in
install.ps1, and%*forwards the flags verbatim, so the two entry points cannot drift. The only logic in the file is a guard that names the problem wheninstall.ps1is not sitting next to it.CRLF, pinned by a new
.gitattributes(*.cmd -text,*.bat -text). The documented install path iscurl -Ofrom raw.githubusercontent.com, which serves the stored blob byte for byte — no checkout-time eol conversion ever runs, so without-textgit would normalise the CRLF out of the blob and ship an LF-only batch file. Verified the pushed blob keeps its CRLF.Docs. README gets the cmd.exe install line.
documentation/client/repl.mdgets a Windows (cmd.exe) section covering which path the execution policy actually blocks, the two-file download, the flags-are-identical contract, and the honest limit:-ExecutionPolicy Bypassis overridden when the policy comes from Group Policy (MachinePolicy/UserPolicyscopes), which no wrapper can work around. It also documents running the generateduninstall.ps1the same way, since the installer still writes it as a.ps1. Running the REPL never needed PowerShell —bin\softclient4es.batis batch.Verification
Version bump:
sbt show version— the build loads and all 20 module version keys report0.20.4, no errors.Installer: exercised on macOS with pwsh 7. The exact invocation form the wrapper uses,⚠️ There is no Windows host here, so
-NoProfile -File install.ps1 <args>, renders-Helpcorrectly;-EsVersion 5forwards a value-taking flag and propagates exit code1, which is whatexit /b %ERRORLEVEL%relays to the caller.cmd.exeexecuting the wrapper itself is not covered — that is the one thing worth a manual smoke test before the release.Follow-up, explicitly not in this PR
The user-facing version sweep (README,
install.sh/install.ps1,documentation/, and the softclient4es.dev site) happens after publication, per the release-doc-sweep convention — versions are updated only once the artifacts are verified on JFrog. That sweep also owns the stale JOIN restriction note indocumentation/sql/joins.md(SELECT-alias / ordinalORDER BY+HAVINGon a JOIN, fixed by softclient4es-arrow#137).🤖 Generated with Claude Code