Skip to content

Ajm/fix 2585 ignore missing lexical - #2587

Merged
amorton merged 3 commits into
mainfrom
ajm/fix-2585-ignore-missing-lexical
Sep 22, 2026
Merged

amorton merged 3 commits into
mainfrom
ajm/fix-2585-ignore-missing-lexical

Conversation

@amorton

@amorton amorton commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

What this PR does:

If the user does not explicit (by using $lexcial) request a lexical leg for findAndRerank and the collection does not have a lexical index do not return an error. We only return an error if the user actually asks for lexical and they do not have one.

Added unit tests, verified via API below:

Create collection with lexcial and rerank disabled:

{
    "createCollection": {
        "name": "rerank_no_lexical",
        "options": {
            "vector": {
                "metric": "cosine",
                "dimension": 1024,
                "service": {
                    "provider": "openai",
                    "modelName": "text-embedding-3-small"
                }
            },
            "lexical": {
                "enabled": false
            },
					"rerank": {
                "enabled": false
            }
        }
    }
}

Try findAndRerank without model override fails as expected:

{
    "findAndRerank": {
        "filter": {},
        "projection": {},
        "sort": {
            "$hybrid": "I like cheese!"
        },
        "options": {
            "limit": 20,
            "includeScores": true,
            "includeSortVector": false
        }
    }
}

{
	"errors": [
		{
			"id": "ebab6334-2f9a-4003-a2fe-c1f5f093f207",
			"family": "REQUEST",
			"scope": "SCHEMA",
			"errorCode": "RERANKING_FEATURE_NOT_ENABLED",
			"title": "Reranking feature not enabled",
			"message": "Reranking feature is not enabled for this database."
		}
	]
}

Then with a model override, the command tries to execute and sees it needs an embedding key (because open AI was used). This means it started running the command:

{
	"findAndRerank": {
		"filter": {},
		"projection": {},
		"sort": {
			"$hybrid": "I like cheese!"
		},
		"options": {
			"limit": 20,
			"includeScores": true,
			"includeSortVector": false,
			"rerank": {
				"provider": "nvidia",
				"modelName": "nvidia/llama-3.2-nv-rerankqa-1b-v2"
			}
		}
	}
}

{
	"errors": [
		{
			"id": "076fa39c-1d7b-480a-9d56-8657328799e9",
			"family": "SERVER",
			"scope": "EMBEDDING_PROVIDER",
			"errorCode": "EMBEDDING_PROVIDER_AUTHENTICATION_KEYS_NOT_PROVIDED",
			"title": "Authentication header(s) missing from embedding request",
			"message": "Missing authentication header(s) from embedding request (provider 'openai'): 'x-embedding-api-key' header is missing')."
		}
	]
}

In attached full trace running the command above that failed...

trace.json

Confirmed that the resolver only added one inner read, not 2:

"innerTaskGroup": {
	"TaskGroup": {
		"groupId": "e5d0c98a-9fa5-4254-9a40-0dbb00235a8e",
		"taskType": "IntermediateCollectionReadTask",
		"sequentialProcessing": false,
		"size": 1.0,
		"statusCount": "{READY=1}",
		"tasks": [
			{
				"IntermediateCollectionReadTask": {
					"position": 1.0,
					"status": "READY",
					"taskId": "5b846d15-162f-4f29-814e-eba48481e699",
					"schemaObject.type": "COLLECTION",
					"schemaObject.name": {
						"SchemaObjectIdentifier": {
							"tenant": {
								"Tenant": {
									"tenantId": "SINGLE-TENANT",
									"databaseType": "CASSANDRA",
									"region": "CASSANDRA_REGION"
								}
							},
							"type": "COLLECTION",
							"keyspace": "default_keyspace",
							"table": "rerank_no_lexical"
						}
					},

Which issue(s) this PR fixes:
Fixes #2585

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

if the user did not ask for lexical sort explicitly
for findAndRerank then allow collection to run without it
@amorton
amorton requested a review from a team as a code owner September 22, 2026 04:06
@github-actions

Copy link
Copy Markdown
Contributor

📈 Unit Test Coverage Delta vs Main Branch

Metric Value
Main Branch 54.12%
This PR 54.56%
Delta 🟢 +0.44%
✅ Coverage improved!

@github-actions

github-actions Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Unit Test Coverage Report

Overall Project 54.56% 🍏
Files changed 96.43% 🍏

File Coverage
FindAndRerankSortClauseDeserializer.java 99.46% 🍏
FindAndRerankOperationBuilder.java 93.55% -0.26% 🍏
CommandFeatures.java 82.68% 🍏

@erichare erichare left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One test that i believe needs to be updated, to address the DSE integration failure. Otherwise looks fantastic.

@Hazel-Datastax Hazel-Datastax left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@erichare
erichare self-requested a review September 22, 2026 19:19

@erichare erichare left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

Copy link
Copy Markdown
Contributor

📈 Integration Test Coverage Delta vs Main Branch (dse69-it)

Metric Value
Main Branch 71.99%
This PR 72.03%
Delta 🟢 +0.04%
✅ Coverage improved!

@github-actions

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (dse69-it)

Overall Project 72.03% 🍏
Files changed 91.07% 🍏

File Coverage
FindAndRerankOperationBuilder.java 88.29% -0.66% 🍏
CommandFeatures.java 73.23% 🍏
FindAndRerankSortClauseDeserializer.java 57.91% 🍏

@github-actions

Copy link
Copy Markdown
Contributor

📈 Integration Test Coverage Delta vs Main Branch (hcd-it)

Metric Value
Main Branch 73.26%
This PR 73.27%
Delta 🟢 +0.02%
✅ Coverage improved!

@github-actions

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (hcd-it)

Overall Project 73.27% 🍏
Files changed 91.07% 🍏

File Coverage
FindAndRerankOperationBuilder.java 91.84% -0.66% 🍏
CommandFeatures.java 73.23% 🍏
FindAndRerankSortClauseDeserializer.java 57.91% 🍏

@amorton
amorton merged commit f5fe333 into main Sep 22, 2026
3 checks passed
@amorton
amorton deleted the ajm/fix-2585-ignore-missing-lexical branch September 22, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suport findAndRerank on a collection with vector / vectorize but without lexical index.

3 participants