Repository navigation
Release 0.62.0 - #259
Merged
Merged
Release 0.62.0#259
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.62.0: what the automatic checks tell the model, plus two fixes found on 0.61.0's gate run.
pytest -q | tailno longer counts as passed. lcode then runs the checks itself.lcode bench --comparewarns when attempts are missing, and no longer treats one attempt as certain.#250, #252 and #254 change when the checks run and what the model is told, so this release goes through the gate (Releases in the contributing guide). The release candidate is this branch, built from its checkout, on the full dev slice at 32K with qwen3.6-35b, three attempts per task. It's compared with 0.60.0's full night on 4–5 November and with 0.61.0's runs on 5–6 November:
Within the noise against both. All 75 attempts ran, on the full slice with
torch-tensor-parallelismback in. The run started at 08:29 on 7 November, in the daytime, at the maintainer's request. The model ran at 72–76 tokens/s every hour.pass@1 went from 45% to 51% against 0.60.0, over the 25 tasks both ran (+5 points, noise ±10). Against 0.61.0 it went from 50% to 53%, over 24 tasks (+3 points, noise ±11).
lcode bench --compareexits 0 against 0.60.0. Against 0.61.0, it was run with that release's two runs combined. No attempt crashed inside lcode.sparql-universityattempts, and 1 passed (0/3 on 0.60.0 and 0.61.0). The note also fired in 4 attempts on other tasks.pypi-server0/3 → 3/3,rstan-to-pystan0/3 → 2/3,sparql-university0/3 → 1/3,xarray-6721anddjango-147872/3 → 3/3,django-127741/3 → 2/3,django-115552/3 → 0/3,openssl-selfsigned-cert1/3 → 0/3,pytest-100511/3 → 0/3, andbuild-cython-ext3/3 → 2/3.torch-tensor-parallelism, which now runs its code and needs most of its 15 minutes for that. 3 more arepolyglot-rust-c(The model deliberates in circles instead of acting, and lcode gives up #241).Three things on the machine differ from the earlier runs, and the comparison has to allow for them: Ollama is now 0.40.15 (
lcode updateupgraded it), swap is now 32 GB instead of 8 GB, and the Linux kernel was updated.Real tests before the night
All with qwen3.6-35b at 32K. Daytime speed limited them, so they show how lcode behaves, not scores or time.
django-12774,pytest-10051,build-cython-ext, 1 attempt eachcd tests, and a change that really breaks the project still reaches the modelpytest -q 2>&1 | tail -5sparql-university, 2 attemptspytest,ruff check,ruff format --check,basedpyrightandmkdocs build --strictpassed on each change. The same on this branch are running now.This bumps
__version__, the quickstart banner and the CHANGELOG (dated section and compare links).🤖 Generated with Claude Code