Add EmbeddingInversion (vec2text) example: weight-export scripts + RE…#26
Open
michalharakal wants to merge 6 commits into
Open
Add EmbeddingInversion (vec2text) example: weight-export scripts + RE…#26michalharakal wants to merge 6 commits into
michalharakal wants to merge 6 commits into
Conversation
…ADME Scaffolds the standalone example for embedding inversion (decode sentence embeddings back to text) built on the SKaiNET-transformers t5 / vec2text modules: - scripts/export_vec2text_weights.py — convert the jxm/gtr__nq__32 (+correct) and gtr-t5-base HuggingFace checkpoints to flat fp16 SafeTensors SKaiNET loads. - scripts/fetch-vec2text-models.sh — idempotent fetch/convert wrapper. - scripts/dump_embedder_golden.py / dump_golden_tensors.py — golden tensors for the Kotlin parity tests. - README with how-it-works, run instructions, and the planned Compose demo. Model weights (~1.1 GB) are fetched, not checked in (.gitignore). The Compose demo app depends on the t5/vec2text modules being published. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Conversion tooling doesn't belong in the Kotlin examples repo. The README now describes the one-time offline weight conversion instead (source HuggingFace checkpoints, which keys to keep per output file, tied-embedding handling, and shard merging), so weights remain reproducible without shipping Python here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the example as a Gradle composite build that includeBuilds the sibling ../../SKaiNET and ../../SKaiNET-transformers checkouts, so it compiles and runs against the just-merged t5 / vec2text source without waiting for a published skainet-transformers release. - settings.gradle.kts: includeBuild both repos. SKaiNET core modules auto-substitute (project name == artifactId); the two skainet-transformers-inference-* coords are mapped explicitly to :llm-inference:t5 / :vec2text (their publish artifactId differs from the Gradle project name). - :cli — a JVM application that loads the converted weights + T5 SentencePiece tokenizer and runs the full inversion round-trip (./gradlew :cli:run --args="..."). Verified running end-to-end via the composite. - Gradle 9.6.1 wrapper (matches the included builds); README updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e 0.36.0
- Add :app — a Compose for Desktop demo with two tabs:
* Round trip: type text -> embed -> 768-d embedding strip -> invert, showing
the per-step hypotheses with a live cosine bar as each correction completes.
* Vector arithmetic: interpolate two sentence embeddings with a slider and
invert the blend (text you never wrote, decoded from a vector).
The UI drives the correction loop step by step via the public InversionModel /
CorrectorModel APIs so each hypothesis renders as it's produced (CPU decode is
slow). Runs the load + inference off the UI thread.
- Now that SKaiNET core 0.36.0 is on Maven Central, drop the ../../SKaiNET
composite: core resolves from Central (pinned by sk.ainet:skainet-bom:0.36.0).
Only SKaiNET-transformers stays a composite (t5/vec2text not yet published).
- Declare plugin versions once at the root (apply false) to silence the
"Kotlin plugin loaded multiple times" warning across :app/:cli.
Both :app and :cli compile; the CLI round-trip is verified end-to-end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rop composite
SKaiNET-transformers 0.36.0 (incl. the t5 / vec2text modules) is now on Maven
Central, so the example no longer needs any composite build or local checkout:
- settings.gradle.kts: remove the SKaiNET-transformers includeBuild + dependency
substitutions. Plain project now.
- :cli and :app: depend on skainet-transformers-inference-{t5,vec2text}:0.36.0
directly; SKaiNET core stays aligned via sk.ainet:skainet-bom:0.36.0.
- README: drop composite instructions; document pure Maven Central consumption.
Verified: :cli and :app compile from Central with --refresh-dependencies, and
the CLI round-trip runs end-to-end against the published artifacts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… publish)
Consume skainet-transformers-inference-{t5,vec2text} 0.37.0 (adds beam search)
from a scoped mavenLocal until it lands on Central; SKaiNET core stays on
Central 0.36.0.
- CLI: VEC2TEXT_BEAM (sequence beam width) + VEC2TEXT_TOKEN_BEAMS env knobs.
- App: a "Beam width" slider; the engine runs streaming sequence-level beam
(inversion.invertBeam + corrector.correctBeam ranked by cosine each round),
emitting the best-of-beam hypothesis per step so the UI still updates live.
- README: beam usage + the publishToMavenLocal command to reproduce 0.37.0.
Verified end-to-end against the local 0.37.0 artifacts: beam ×3 reconstructs
"...ph.d. technologist ... Cornell University" at cosine 0.82 (vs ~0.77 greedy)
on the README example.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ADME
Scaffolds the standalone example for embedding inversion (decode sentence embeddings back to text) built on the SKaiNET-transformers t5 / vec2text modules:
Model weights (~1.1 GB) are fetched, not checked in (.gitignore). The Compose demo app depends on the t5/vec2text modules being published.