CAMEL-24533: camel-weaviate - store the document text in the object properties on CREATE/UPDATE by oscerd · Pull Request #25900 · apache/camel

oscerd · 2026-08-30T10:11:10Z

Issue

Problem

WeaviateEmbeddingsDataTypeTransformer builds the Weaviate write from the resolved parameters
(vectorFieldName, textFieldName, text, ...), but on CREATE and UPDATE_BY_ID it only set the body
to the embedding vector and, optionally, the id property:

message.setBody(embedding.vectorAsList(), List.class);
if (ObjectHelper.isNotEmpty(keyValue) && ObjectHelper.isNotEmpty(keyName)) {
    HashMap<String, Object> maps = new HashMap<>();
    maps.put(keyName, keyValue);
    message.setHeader(WeaviateVectorDbHeaders.PROPERTIES, maps);
}

The embedded text (text.text()) was never written into the properties, so the original document
text was silently discarded. In a RAG pipeline the vector store then cannot return the source passage
that produced a match. The Milvus sibling transformer stores the text correctly — Weaviate was an
incomplete port.

Fix

Write the text under textFieldName into the Weaviate object properties for CREATE and UPDATE_BY_ID
(via a shared setProperties helper). Also correct the default-branch error message, which said only
create and updatebyid were supported although query is handled as well.

textFieldName is a resolved header value, so this is a pure logic change — no generated metadata is
affected.

Testing

New WeaviateEmbeddingsDataTypeTransformerTest asserts that CREATE and UPDATE_BY_ID write the text
(under the default text field) and the id into the CamelWeaviateProperties map.
mvn -Psourcecheck validate green.

Claude Code on behalf of oscerd

github-actions · 2026-08-30T10:11:46Z

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

First-time contributors require MANUAL approval for the GitHub Actions to run
You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

github-actions · 2026-08-30T10:29:51Z

🧪 CI tested the following changed modules:

components/camel-ai/camel-weaviate

🔬 Scalpel shadow comparison — Scalpel: 11 tested, 27 compile-only — current: 11 all tested

Maveniverse Scalpel detected 38 affected modules (current approach: 11).

⚠️ Modules only in Scalpel (27)

apache-camel
camel-allcomponents
camel-catalog
camel-catalog-console
camel-catalog-maven
camel-catalog-suggest
camel-componentdsl
camel-endpointdsl
camel-endpointdsl-support
camel-itest
camel-jbang-core
camel-jbang-it
camel-jbang-main
camel-jbang-plugin-edit
camel-jbang-plugin-generate
camel-jbang-plugin-kubernetes
camel-jbang-plugin-test
camel-kamelet-main
camel-launcher
camel-report-maven-plugin
camel-route-parser
camel-yaml-dsl
camel-yaml-dsl-deserializers
camel-yaml-dsl-maven-plugin
coverage
docs
dummy-component

Skip-tests mode would test 11 modules (1 direct + 10 downstream), skip tests for 27 (generated code, meta-modules)

Modules Scalpel would test (11)

camel-jbang-mcp
camel-jbang-plugin-mcp
camel-jbang-plugin-route-parser
camel-jbang-plugin-tui
camel-jbang-plugin-validate
camel-langchain4j-embeddings
camel-langchain4j-embeddingstore
camel-launcher-container
camel-weaviate
camel-yaml-dsl-validator
camel-yaml-dsl-validator-maven-plugin

Modules with tests skipped (27)

apache-camel
camel-allcomponents
camel-catalog
camel-catalog-console
camel-catalog-maven
camel-catalog-suggest
camel-componentdsl
camel-endpointdsl
camel-endpointdsl-support
camel-itest
camel-jbang-core
camel-jbang-it
camel-jbang-main
camel-jbang-plugin-edit
camel-jbang-plugin-generate
camel-jbang-plugin-kubernetes
camel-jbang-plugin-test
camel-kamelet-main
camel-launcher
camel-report-maven-plugin
camel-route-parser
camel-yaml-dsl
camel-yaml-dsl-deserializers
camel-yaml-dsl-maven-plugin
coverage
docs
dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (38 modules)

Camel :: AI :: LangChain4j :: Embedding
Camel :: AI :: LangChain4j :: EmbeddingStore
Camel :: AI :: Weaviate
Camel :: All Components Sync point
Camel :: Assembly
Camel :: Catalog :: Camel Catalog
Camel :: Catalog :: Camel Report Maven Plugin
Camel :: Catalog :: Camel Route Parser
Camel :: Catalog :: Console
Camel :: Catalog :: Dummy Component
Camel :: Catalog :: Maven
Camel :: Catalog :: Suggest
Camel :: Component DSL
Camel :: Coverage
Camel :: Docs
Camel :: Endpoint DSL
Camel :: Endpoint DSL :: Support
Camel :: Integration Tests
Camel :: JBang :: Core
Camel :: JBang :: Integration tests
Camel :: JBang :: MCP
Camel :: JBang :: Main
Camel :: JBang :: Plugin :: Edit
Camel :: JBang :: Plugin :: Generate
Camel :: JBang :: Plugin :: Kubernetes
Camel :: JBang :: Plugin :: MCP
Camel :: JBang :: Plugin :: Route Parser
Camel :: JBang :: Plugin :: TUI
Camel :: JBang :: Plugin :: Testing
Camel :: JBang :: Plugin :: Validate
Camel :: Kamelet Main
Camel :: Launcher
Camel :: Launcher :: Container
Camel :: YAML DSL
Camel :: YAML DSL :: Deserializers
Camel :: YAML DSL :: Maven Plugins
Camel :: YAML DSL :: Validator
Camel :: YAML DSL :: Validator Maven Plugin

⚙️ View full build and test results

davsclaus

Reviewed against the project's Jira/commit conventions and this component's git history. Commit message, CAMEL-24533: prefix, AI attribution and sign-off all look correct, and the underlying bug (embedded text silently dropped on CREATE/UPDATE) is real and confirmed against MilvusEmbeddingsDataTypeTransformer's pattern.

However, CI is red for a reason directly caused by this diff (not flaky infra): LangChain4jEmbeddingsComponentWeaviateTargetIT.querybyid fails with Expecting actual: {"text"="hi"} to contain key: "sky". See the inline comment for the root cause and a suggested fix direction.

This review does not replace specialized tools such as CodeRabbit, Sourcery, or SonarCloud static analysis.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

oscerd · 2026-08-31T08:01:11Z

Thanks @davsclaus — fixed in the latest commit. setProperties now reads any existing PROPERTIES header and merges the text (and optional id) into a copy of it instead of building a fresh map, so caller-supplied properties survive. This restores LangChain4jEmbeddingsComponentWeaviateTargetIT.querybyid (the sky/age/dog entries set by create()/updateById() are preserved, with text added). I also extended WeaviateEmbeddingsDataTypeTransformerTest with a pre-populated-PROPERTIES case, which is exactly the scenario the original test missed.

Claude Code on behalf of oscerd

davsclaus

Review

Verified beyond the diff:

Checked out the PR branch and ran the new test class (mvn test -Dtest=WeaviateEmbeddingsDataTypeTransformerTest) — 3/3 pass.
Ran mvn install -DskipTests -Psourcecheck on camel-weaviate — clean, no formatting/import/license violations.
Compared against the Milvus sibling transformer (MilvusEmbeddingsDataTypeTransformer), which this PR says it mirrors — confirmed Milvus unconditionally writes text.text(); this PR's version is slightly more defensive (null-checks text).
Traced the PROPERTIES header downstream into WeaviateVectorDbProducer.create()/updateById() (lines 130, 156) — confirmed the map this PR builds is consumed correctly by both actions.
Checked git history: this file hasn't been touched since the original port in CAMEL-22311 (2023) — the text-drop was an incomplete port, not an intentional design decision, so this fix doesn't revert any deliberate prior behavior.
Confirmed JIRA CAMEL-24533 is correctly linked, typed Bug, and assigned to the PR author.

Notes (non-blocking)

CI is currently red on build (25, false), but that failure is unrelated to this PR — it's a pre-existing main-branch drift in generated catalog docs for unrelated AI components, being fixed separately. Not something to hold this PR on.
The second commit's approach of merging the new text/id into any existing PROPERTIES header (rather than overwriting it) is a nice defensive touch beyond a literal Milvus port — protects a caller who pre-populates custom properties before the transformer runs.
No upgrade-guide entry needed here — this restores previously-dropped data rather than changing a default or public behavior contract.

Good fix, well tested, no concerns.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

…roperties on CREATE/UPDATE WeaviateEmbeddingsDataTypeTransformer set the message body to the embedding vector and only wrote the optional id into the Weaviate object properties; the embedded text (the TextSegment body) was never written under textFieldName, so the source passage was silently lost. For a RAG pipeline this means the store cannot return the text that produced a match. The Milvus sibling stores the text correctly. Write the text under textFieldName into the properties for CREATE and UPDATE_BY_ID (shared setProperties helper). Also correct the default-branch message, which claimed only create and updatebyid were supported although query is handled too. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andrea Cosentino <ancosen@gmail.com>

…ad of replacing it Addresses review: setProperties unconditionally built a fresh PROPERTIES map, discarding properties the caller had already set on the message (which broke LangChain4jEmbeddingsComponentWeaviateTargetIT.querybyid). Read the existing PROPERTIES header and merge the text (and optional id) into a copy of it, only creating a new map when none exists. Extend the unit test to cover a pre-populated PROPERTIES header. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andrea Cosentino <ancosen@gmail.com>

gnodet

Clean bug fix — the Weaviate transformer was the only AI-component sibling (vs. Milvus, Qdrant, PgVector) that silently dropped the document text on CREATE/UPDATE. The fix correctly stores the text in the object properties, and the merge-into-existing-PROPERTIES approach with new HashMap<>(existing) is a nice defensive touch.

Non-blocking observations:

The queryEmbeddingOperation method still uses the older pattern (fresh HashMap, no shared setProperties helper). Pre-existing and out of scope for this fix, but could be aligned in a follow-up for consistency.
The null guard on text (text != null && text.text() != null) is more defensive than the Milvus sibling — good improvement.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of Guillaume Nodet

…roperties on CREATE/UPDATE WeaviateEmbeddingsDataTypeTransformer set the message body to the embedding vector and only wrote the optional id into the Weaviate object properties; the embedded text (the TextSegment body) was never written under textFieldName, so the source passage was silently lost. For a RAG pipeline this means the store cannot return the text that produced a match. The Milvus sibling transformer already stores the text correctly. Write the text under textFieldName into the properties for CREATE and UPDATE_BY_ID via a shared setProperties helper, merging into any PROPERTIES header the caller already set rather than replacing it. Also correct the default-branch error message, which claimed only create and updatebyid were supported although query is handled too. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Closes #25900 (cherry picked from commit b4c20cb)

oscerd added the bug Something isn't working label Aug 30, 2026

oscerd added this to the 4.23.0 milestone Aug 30, 2026

oscerd requested review from Croway and davsclaus August 30, 2026 10:11

github-actions Bot added components components-ai labels Aug 30, 2026

davsclaus force-pushed the fix/CAMEL-24533 branch from fe452fc to 6a00a88 Compare August 30, 2026 11:03

davsclaus requested changes Aug 30, 2026

View reviewed changes

Comment thread ...ava/org/apache/camel/component/weaviate/transform/WeaviateEmbeddingsDataTypeTransformer.java

oscerd requested a review from davsclaus August 31, 2026 08:01

davsclaus approved these changes Aug 31, 2026

View reviewed changes

davsclaus mentioned this pull request Aug 31, 2026

chore: regenerate catalog docs for structured_error_exchange_properties anchor fix #25941

Merged

2 tasks

oscerd and others added 2 commits August 31, 2026 20:17

davsclaus force-pushed the fix/CAMEL-24533 branch from f101438 to b7f8841 Compare August 31, 2026 18:18

gnodet approved these changes Aug 31, 2026

View reviewed changes

davsclaus merged commit b4c20cb into apache:main Aug 31, 2026
5 checks passed

davsclaus assigned oscerd Aug 31, 2026

This was referenced Sep 1, 2026

camel-ai fixes backport to camel-4.22.x (CAMEL-24527, CAMEL-24540, CAMEL-24533) #25996

Merged

CAMEL-24533: camel-weaviate - store the document text in the object properties on CREATE/UPDATE (backport to camel-4.18.x) #25997

Merged

Navigation Menu

Uh oh!

CAMEL-24533: camel-weaviate - store the document text in the object properties on CREATE/UPDATE - #25900

CAMEL-24533: camel-weaviate - store the document text in the object properties on CREATE/UPDATE#25900
davsclaus merged 2 commits into
apache:mainfrom
oscerd:fix/CAMEL-24533

oscerd commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026 •

edited

Loading

Uh oh!

davsclaus left a comment

Uh oh!

Uh oh!

oscerd commented Aug 31, 2026

Uh oh!

davsclaus left a comment

Uh oh!

gnodet left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

oscerd commented Aug 30, 2026

Issue

Problem

Fix

Testing

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davsclaus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

oscerd commented Aug 31, 2026

Uh oh!

davsclaus left a comment

Choose a reason for hiding this comment

Review

Notes (non-blocking)

Uh oh!

gnodet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 30, 2026 •

edited

Loading