Refine vector embeddings docs - #2826
Conversation
| ``` | ||
| - Vectors stored in native `vector` type | ||
| - `vector_embedding()` function must be defined by application developers for both testing and production use. | ||
| - `vector_embedding()` function is not supported by CAP. |
There was a problem hiding this comment.
Isn't the previous sentence more helpful to developers? Or let's say more explicit about what it means for CAP developers if they want to use such a function?
There was a problem hiding this comment.
@MattSchur @vkozyura What is your reply to my comment above?
There was a problem hiding this comment.
Postgres is usually not used for testing but for production, so simple mock embeddings are not sufficient. For production, I don't think that implementing a vector_embedding function on Postgres is the best approach. I would rather go with computing the embeddings on app level or via an asynchronous vectorizer.
| ::: info Local Testing with H2 and SQLite | ||
| On H2 and SQLite the `CQL.vectorEmbedding` function is emulated using a hash-based algorithm to support local testing. For PostgreSQL, customers must define their own `vector_embedding` function for both testing and production use. | ||
| On H2 and SQLite the `CQL.vectorEmbedding` function is emulated to support local testing. | ||
| Both runtimes support a hash-based mock embedding, CAP Java additionally allows using local [ONNX](https://onnx.ai) embeddings models. |
There was a problem hiding this comment.
@vkozyura Node ships ONNX support via @cap-js/ai, correct?
| Both runtimes support a hash-based mock embedding, CAP Java additionally allows using local [ONNX](https://onnx.ai) embeddings models. | |
| Both runtimes support a hash-based mock embedding, as well as local [ONNX](https://onnx.ai) embeddings models. |
There was a problem hiding this comment.
| - **SQLite & H2:** Hash-based deterministic implementation for testing. Can be overridden by application developers to use external embedding services. | ||
| - **PostgreSQL:** No default implementation. Application developers must define their own `vector_embedding` function. | ||
| - **SAP HANA:** Uses embedding models from the [NLP](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-predictive-analysis-library/natural-language-processing-nlp) extension or an [SAP AI Core](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/what-is-sap-ai-core) remote source. | ||
| - **SQLite & H2:** Hash-based mock embedding. CAP Java additionally allows using local [ONNX](https://onnx.ai) embeddings models. |
There was a problem hiding this comment.
| - **SQLite & H2:** Hash-based mock embedding. CAP Java additionally allows using local [ONNX](https://onnx.ai) embeddings models. | |
| - **SQLite & H2:** Hash-based mock embedding or local [ONNX](https://onnx.ai) embeddings models. |
| ``` | ||
| - Vectors stored in native `vector` type | ||
| - `vector_embedding()` function must be defined by application developers for both testing and production use. | ||
| - `vector_embedding()` function is not supported by CAP. |
There was a problem hiding this comment.
Postgres is usually not used for testing but for production, so simple mock embeddings are not sufficient. For production, I don't think that implementing a vector_embedding function on Postgres is the best approach. I would rather go with computing the embeddings on app level or via an asynchronous vectorizer.
| Using vector functions in [stored calculated elements](../../cds/cdl#on-write) with [local MTXS](../../guides/multitenancy/mtxs#test-drive-locally) on SQLite isn't supported. | ||
| ::: warning Local MTXS on SQLite | ||
| Using vector functions in [stored calculated elements](../../cds/cdl#on-write) with [local MTXS](../../guides/multitenancy/mtxs#test-drive-locally) on SQLite | ||
| calls the custom functions of the CAP Node.js runtime. Using local [ONNX](https://onnx.ai) embedding models is not yet supported. |
There was a problem hiding this comment.
@vkozyura would this work with @cap-js/ai? What model would be used?
25bd72f to
a173e4c
Compare
|
Heads up @MattSchur @renejeglinsky @vkozyura — I rebased this branch and retargeted the base from Why: During conflict resolution I made one factual change to this PR's content: where it read "CAP Java additionally allows local ONNX / Node not yet supported" (info box, DB-implementation bullets), I updated it to both runtimes — Java via LangChain4j, Node.js via Everything else from this PR is kept as-is: the One open thread: @renejeglinsky asked whether the bare "not supported by CAP" for the Postgres |
|
continuing in #2837 |
4 participants
No description provided.