Lance Graph

Lance Graph is a Cypher-capable graph query engine built in Rust with Python bindings for building high-performance, scalable, and serverless multimodal knowledge graphs.

This repository contains:

• crates/lance-graph – the Cypher-capable query engine implemented in Rust
• python/ – PyO3 bindings and Python packages:
  • lance_graph – thin wrapper around the Rust query engine
  • knowledge_graph – Lance-backed knowledge graph CLI, API, and utilities

See docs/project_structure.md for the proposed workspace-based structure from issue #92.

Prerequisites

• Rust toolchain (1.82 or newer recommended)
• Python 3.11
• uv available on your PATH

Rust crate quick start

cd crates/lance-graph
cargo check
cargo test

Python package quick start

cd python
uv venv --python 3.11 .venv      # create the local virtualenv
source .venv/bin/activate         # activate the virtual environment
uv pip install 'maturin[patchelf]' # install build tool
uv pip install -e '.[tests]'     # editable install with test extras
maturin develop                   # build and install the Rust extension
pytest python/tests/ -v          # run the test suite

If another virtual environment is already active, run deactivate (or unset VIRTUAL_ENV) before the uv run command so uv binds to .venv.

Python example: Cypher query

import pyarrow as pa
from lance_graph import CypherQuery, GraphConfig

people = pa.table({
    "person_id": [1, 2, 3, 4],
    "name": ["Alice", "Bob", "Carol", "David"],
    "age": [28, 34, 29, 42],
})

config = (
    GraphConfig.builder()
    .with_node_label("Person", "person_id")
    .build()
)

query = (
    CypherQuery("MATCH (p:Person) WHERE p.age > 30 RETURN p.name AS name, p.age AS age")
    .with_config(config)
)
result = query.execute({"Person": people})
print(result.to_pydict())  # {'name': ['Bob', 'David'], 'age': [34, 42]}

Python example: Direct SQL query

For data analytics workflows where you prefer standard SQL, use SqlQuery or SqlEngine. No GraphConfig is needed:

import pyarrow as pa
from lance_graph import SqlQuery, SqlEngine

person = pa.table({
    "id": [1, 2, 3],
    "name": ["Alice", "Bob", "Carol"],
    "age": [28, 34, 29],
})

# One-off query
result = SqlQuery(
    "SELECT name, age FROM person WHERE age > 30"
).execute({"person": person})
print(result.to_pydict())  # {'name': ['Bob'], 'age': [34]}

# Multi-query with cached context
engine = SqlEngine({"person": person})
r1 = engine.execute("SELECT COUNT(*) AS cnt FROM person")
r2 = engine.execute("SELECT name FROM person ORDER BY age DESC LIMIT 2")

Python example: Unity Catalog integration

Connect to Unity Catalog (OSS) to discover and query Delta Lake or Parquet tables directly:

from lance_graph import UnityCatalog

# Connect to Unity Catalog
uc = UnityCatalog("http://localhost:8080/api/2.1/unity-catalog")

# Browse catalog metadata
catalogs = uc.list_catalogs()
schemas = uc.list_schemas("unity")
tables = uc.list_tables("unity", "default")
table = uc.get_table("unity", "default", "marksheet")
print(table.columns())  # [{"name": "id", "type_name": "INT", ...}, ...]

# Auto-register tables (Delta + Parquet) and query via SQL
engine = uc.create_sql_engine("unity", "default")
result = engine.execute("SELECT * FROM marksheet WHERE mark > 80")
print(result.to_pandas())

# For cloud storage (S3, Azure, GCS), pass storage options:
uc = UnityCatalog(
    "http://localhost:8080/api/2.1/unity-catalog",
    storage_options={
        "aws_access_key_id": "...",
        "aws_secret_access_key": "...",
        "aws_region": "us-east-1",
    }
)

Knowledge Graph CLI & API

The knowledge_graph package layers a simple Lance-backed knowledge graph service on top of the lance_graph engine. It provides:

• A CLI (knowledge_graph.main) for initializing storage, running Cypher queries, and bootstrapping data via heuristic text extraction.
• A reusable FastAPI component, plus a standalone web service (knowledge_graph.webservice) that exposes query and dataset endpoints.
• Storage helpers that persist node and relationship tables as Lance datasets.

CLI usage

uv run knowledge_graph --init                    # initialize storage and schema stub
uv run knowledge_graph --list-datasets           # list Lance datasets on disk
uv run knowledge_graph --extract-preview notes.txt
uv run knowledge_graph --extract-preview "Alice joined the graph team"
uv run knowledge_graph --extract-and-add notes.txt
uv run knowledge_graph "MATCH (n) RETURN n LIMIT 5"
uv run knowledge_graph --log-level DEBUG --extract-preview "Inline text"
uv run knowledge_graph --ask "Who is working on the Presto project?"


# Configure LLM extraction (default)
uv sync --extra llm  # install optional LLM dependencies
uv sync --extra lance-storage  # install Lance dataset support
export OPENAI_API_KEY=sk-...
uv run knowledge_graph --llm-model gpt-4o-mini --extract-preview notes.txt

# Supply additional OpenAI client options via YAML (base_url, headers, etc.)
uv run knowledge_graph --llm-config llm_config.yaml --extract-and-add notes.txt

# Fall back to the heuristic extractor when LLM access is unavailable
uv run knowledge_graph --extractor heuristic --extract-preview notes.txt

The default extractor uses OpenAI. Configure credentials via environment variables supported by the SDK (for example OPENAI_API_BASE or OPENAI_API_KEY), or place them in a YAML file passed through --llm-config. Override the model and temperature with --llm-model and --llm-temperature.

By default the CLI writes datasets under `./knowledge_graph_data`. Provide
`--root` and `--schema` to point at alternate storage locations and schema YAML.

### FastAPI service

Run the web service after installing the `knowledge_graph` package (and
dependencies such as FastAPI):

```bash
uv run --package knowledge_graph knowledge_graph-webservice

The service exposes endpoints under /graph, including /graph/health, /graph/query, /graph/datasets, and /graph/schema.

Development workflow

For linting and type checks:

# Install dev dependencies and run linters
uv pip install -e '.[dev]'
ruff format python/              # format code
ruff check python/               # lint code
pyright                          # type check

# Or run individual tests
pytest python/tests/test_graph.py::test_basic_node_selection -v

The Python README (python/README.md) contains additional details if you are working solely on the bindings.

Benchmarks

• Requirements:

  • protoc: install protobuf-compiler (Debian/Ubuntu: sudo apt-get install -y protobuf-compiler).
  • Optional: gnuplot for Criterion's gnuplot backend; otherwise the plotters backend is used.

• Run (from crates/lance-graph):

cargo bench --bench graph_execution

# Quicker local run (shorter warm-up/measurement):
cargo bench --bench graph_execution -- --warm-up-time 1 --measurement-time 2 --sample-size 10

• Reports:

  • Global index: crates/lance-graph/target/criterion/report/index.html
  • Group index: crates/lance-graph/target/criterion/cypher_execution/report/index.html

• Typical results (x86_64, quick run: warm-up 1s, measurement 2s, sample size 10):

Benchmark	Size	Median time	Approx. throughput
basic_node_filter	100	~680 µs	~147 Kelem/s
basic_node_filter	10,000	~715 µs	~13.98 Melem/s
basic_node_filter	1,000,000	~743 µs	~1.35 Gelem/s
single_hop_expand	100	~2.79 ms	~35.9 Kelem/s
single_hop_expand	10,000	~3.77 ms	~2.65 Melem/s
single_hop_expand	1,000,000	~3.70 ms	~270 Melem/s
two_hop_expand	100	~4.52 ms	~22.1 Kelem/s
two_hop_expand	10,000	~6.41 ms	~1.56 Melem/s
two_hop_expand	1,000,000	~6.16 ms	~162 Melem/s

Numbers are illustrative; your hardware, compiler, and runtime load will affect results.

External Wiki

For additional documentation, architecture, and examples, see the DeepWiki page: DeepWiki — lance-graph