From 91e5f2a65e8759be10d6bc8b3582b2eb4438b171 Mon Sep 17 00:00:00 2001 From: Erik Arvidsson Date: Tue, 20 Oct 2026 15:42:29 +0000 Subject: [PATCH 1/2] docs(zql): document JSON path filters Add a JSON Filters section under Where: the json() helper in cmp(), string vs number path segments, typed paths, type-strict comparisons, homogeneous IN lists, and how missing keys and JSON null behave. Claude-Session: https://claude.ai/code/session_013VzyesbmCjvSPN6qKkKz9X --- contents/docs/zql.mdx | 61 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/contents/docs/zql.mdx b/contents/docs/zql.mdx index 6bce3748..f63cb60b 100644 --- a/contents/docs/zql.mdx +++ b/contents/docs/zql.mdx @@ -333,6 +333,67 @@ This is particularly useful for implementing [permissions](/docs/auth#read-permi zql.issue.where(cmpLit(ctx.role, 'admin')) ``` +### JSON Filters + +You can filter on a value inside a [`json()` column](/docs/schema#custom-json-types) with the `json` helper. Pass it the column name followed by a path into the value: + +```tsx +// Find issues whose metadata.priority is 'high' +zql.issue.where(({cmp, json}) => + cmp(json('metadata', 'priority'), 'high') +) +``` + +`json` can only be used inside a callback, as the first argument to `cmp`. It works with all the comparison operators, and it can be combined with `and`, `or`, `not`, and `exists` like any other `cmp`. + +Path segments are applied left to right. A string segment reads a key of an object and a number segment reads an index of an array: + +```tsx +// metadata.address.zip +cmp(json('metadata', 'address', 'zip'), '94110') + +// The first element of metadata.tags +cmp(json('metadata', 'tags', 0), 'bug') +``` + +Array indexes must be non-negative integers. A string never indexes an array and a number never reads an object key, so `json('metadata', 'tags', '0')` doesn't match anything. + +If the column has a [custom JSON type](/docs/schema#custom-json-types), TypeScript completes the path segments and types the value you compare against. A column declared with plain `json()` accepts any path. + +#### Comparisons Are Type-Strict + +A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A value of a different type than the one you compare against never matches a positive operator and always matches a negated one: + +| Value at path | Filter | Result | +| ------------- | --------- | ------- | +| `42` | `= 42` | `true` | +| `42` | `= '42'` | `false` | +| `42` | `!= '42'` | `true` | +| `true` | `= 1` | `false` | +| `'abc'` | `> 5` | `false` | + +`LIKE` and `ILIKE` only match string values. Only strings, numbers, and booleans can be compared. You can't compare a whole object or array. + +The array passed to `IN` and `NOT IN` must contain values of a single type: + +```tsx +cmp(json('metadata', 'priority'), 'IN', [ + 'high', + 'critical' +]) +``` + +#### Missing Values + +A path that doesn't exist and a JSON `null` both read as `null`. As with columns, they never match `=`, `!=`, or the other value operators. Use `IS` and `IS NOT` to find them: + +```tsx +// Issues with no metadata.priority (missing or null) +zql.issue.where(({cmp, json}) => + cmp(json('metadata', 'priority'), 'IS', null) +) +``` + ### Relationship Filters Your filter can also test properties of relationships. Currently the only supported test is existence: From 590b2c3815666ea38e44aa8bdd37252dbe68a45c Mon Sep 17 00:00:00 2001 From: Erik Arvidsson Date: Tue, 20 Oct 2026 15:47:06 +0000 Subject: [PATCH 2/2] docs(zql): qualify JSON null rules and regenerate the search index The type-strict rule applies to non-null values (a null never matches !=), and IS / IS NOT also compare against null. Regenerate assets/search-index.json so the new JSON Filters section is searchable. Claude-Session: https://claude.ai/code/session_013VzyesbmCjvSPN6qKkKz9X --- assets/search-index.json | 64 +++++++++++++++++++++++++++++++++------- contents/docs/zql.mdx | 4 +-- 2 files changed, 55 insertions(+), 13 deletions(-) diff --git a/assets/search-index.json b/assets/search-index.json index d8d20699..e1f1a663 100644 --- a/assets/search-index.json +++ b/assets/search-index.json @@ -9654,7 +9654,7 @@ "title": "ZQL", "searchTitle": "ZQL", "url": "/docs/zql", - "content": "Inspired by SQL, ZQL is expressed in TypeScript with heavy use of the builder pattern. If you have used Drizzle or Kysely, ZQL will feel familiar. ZQL queries are composed of one or more clauses that are chained together into a query. Create a Builder To get started, use createBuilder. If you use drizzle-zero or prisma-zero, this happens automatically and an instance is stored in the zql constant exported from schema.ts: import {zql} from 'schema.ts' // zql.myTable.where(...) Otherwise, create an instance manually: // schema.ts // ... export const zql = createBuilder(schema) Select ZQL queries start by selecting a table. There is no way to select a subset of columns; ZQL queries always return the entire row, if permissions allow it. import {zql} from 'zero.ts' // Returns a query that selects all rows and columns from the // issue table. zql.issue This is a design tradeoff that allows Zero to better reuse the row locally for future queries. This also makes it easier to share types between different parts of the code. Do not directly modify JavaScript values returned from ZQL queries. ZQL caches values across queries to improve performance and reduce re-renders in frameworks like React and Solid. If you modify a value returned from ZQL, you will modify it everywhere it is used. This can lead to subtle bugs. JavaScript and TypeScript lack true immutable types so we use readonly to help enforce it. But it's easy to cast away the readonly accidentally. Ordering You can sort query results by adding an orderBy clause: zql.issue.orderBy('created', 'desc') Multiple orderBy clauses can be present, in which case the data is sorted by those clauses in order: // Order by priority descending. For any rows with same priority, // then order by created desc. zql.issue .orderBy('priority', 'desc') .orderBy('created', 'desc') All queries in ZQL have a default final order of their primary key. Assuming the issue table has a primary key on the id column, then: // Actually means: zql.issue.orderBy('id', 'asc'); zql.issue // Actually means: zql.issue.orderBy('priority', 'desc').orderBy('id', 'asc'); zql.issue.orderBy('priority', 'desc') Limit You can limit the number of rows to return with limit(): zql.issue.orderBy('created', 'desc').limit(100) Paging You can start the results at or after a particular row with start(): let start: IssueRow | undefined while (true) { let q = zql.issue .orderBy('created', 'desc') .limit(100) if (start) { q = q.start(start) } const batch = await q.run() console.log('got batch', batch) if (batch.length < 100) { break } start = batch[batch.length - 1] } By default start() is exclusive - it returns rows starting after the supplied reference row. This is what you usually want for paging. If you want inclusive results, you can do: zql.issue.start(row, {inclusive: true}) Getting a Single Result If you want exactly zero or one results, use the one() clause. This causes ZQL to return Row|undefined rather than Row[]. const result = await zql.issue .where('id', 42) .one() .run() if (!result) { console.error('not found') } one() overrides any limit() clause that is also present. Relationships You can query related rows using relationships that are defined in your Zero schema. // Get all issues and their related comments zql.issue.related('comments') Relationships are returned as hierarchical data. In the above example, each row will have a comments field, which is an array of the corresponding comments rows. You can fetch multiple relationships in a single query: zql.issue .related('comments') .related('reactions') .related('assignees') Refining Relationships By default all matching relationship rows are returned, but this can be refined. The related method accepts an optional second function which is itself a query. zql.issue.related( 'comments', // It is common to use the 'q' shorthand variable for this parameter, // but it is a _comment_ query in particular here, exactly as if you // had done zql.comment. q => q .orderBy('modified', 'desc') .limit(100) .start(lastSeenComment) ) This relationship query can have all the same clauses that top-level queries can have. Using orderBy or limit in a relationship that goes through a junction table (i.e., a many-to-many relationship) is not currently supported and will throw a runtime error. See bug 3527. You can sometimes work around this by making the junction relationship explicit, depending on your schema and usage. Nested Relationships You can nest relationships arbitrarily: // Get all issues, first 100 comments for each (ordered by modified,desc), // and for each comment all of its reactions. zql.issue.related('comments', q => q .orderBy('modified', 'desc') .limit(100) .related('reactions') ) Where You can filter a query with where(): zql.issue.where('priority', '=', 'high') The first parameter is always a column name from the table being queried. TypeScript completion will offer available options (sourced from your Zero Schema). Comparison Operators where() supports the following comparison operators: TypeScript will restrict you from using operators with types that don’t make sense – you can’t use > with boolean for example. Zero can evaluate an ILIKE filter in three places: in-memory on the client, in ZQLite against the replica, or as SQL in Postgres. ZQLite and the in-memory query engine use Unicode-aware lowercasing before matching. This means common non-ASCII case variants, such as MÜLLER and müller, match in both. If you need non-ASCII text in PG to work with ILIKE, use a Unicode-aware collation for those text columns, such as the database's default UTF-8 locale. Equals is the Default Comparison Operator Because comparing by = is so common, you can leave it out and where defaults to =. zql.issue.where('priority', 'high') Comparing to null As in SQL, ZQL’s null cannot be compared with =, !=, <, or any other normal comparison operator. Comparing any value to null with such operators is always false: These semantics feel a bit weird, but they are consistent with SQL. The reason SQL does it this way is to make join semantics work: if you’re joining employee.orgID on org.id you do not want an employee in no organization to match an org that hasn’t yet been assigned an ID. For when you purposely do want to compare to null ZQL supports IS and IS NOT operators that also work just like in SQL: // Find employees not in any org. zql.employee.where('orgID', 'IS', null) // Find employees in an org other than 42 OR employees in NO org zql.employee.where('orgID', 'IS NOT', 42) TypeScript will prevent you from comparing to null with other operators. Comparing to undefined As a convenience, you can pass undefined to where: zql.issue.where('priority', issue?.priority) This comparison is always false, so the above query always returns no results. Compound Filters The argument to where can also be a callback that returns a complex expression: // Get all issues that have priority 'critical' or else have both // priority 'medium' and not more than 100 votes. zql.issue.where(({cmp, and, or, not}) => or( cmp('priority', 'critical'), and( cmp('priority', 'medium'), not(cmp('numVotes', '>', 100)) ) ) ) cmp is short for compare and works the same as where at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below). Note that chaining where() is also a one-level and: // Find issues with priority 3 or higher, owned by aa zql.issue .where('priority', '>=', 3) .where('owner', 'aa') Comparing Literal Values The where clause always expects its first parameter to be a column name as a string. Same with the cmp helper: // \"foo\" is a column name, not a string: zql.issue.where('foo', 'bar') // \"foo\" is a column name, not a string: zql.issue.where(({cmp}) => cmp('foo', 'bar')) To compare to a literal value, use the cmpLit helper: zql.issue.where(cmpLit('foobar', 'foo' + 'bar')) This is particularly useful for implementing permissions, because the first parameter can be a field of your context: zql.issue.where(cmpLit(ctx.role, 'admin')) Relationship Filters Your filter can also test properties of relationships. Currently the only supported test is existence: // Find all orgs that have at least one employee zql.organization.whereExists('employees') The argument to whereExists is a relationship, so just like other relationships, it can be refined with a query: // Find all orgs that have at least one cool employee zql.organization.whereExists('employees', q => q.where('location', 'Hawaii') ) As with querying relationships, relationship filters can be arbitrarily nested: // Get all issues that have comments that have reactions zql.issue.whereExists('comments', q => q.whereExists('reactions') ) The exists helper is also provided which can be used with and, or, cmp, and not to build compound filters that check relationship existence: // Find issues that have at least one comment or are high priority zql.issue.where({cmp, or, exists} => or( cmp('priority', 'high'), exists('comments'), ), ) Type Helpers You can get the TypeScript type of the result of a query using the QueryResultType helper: import type {QueryResultType} from '@rocicorp/zero' const complexQuery = zql.issue.related( 'comments', q => q.related('author') ) type MyComplexResult = QueryResultType // MyComplexResult is: readonly IssueRow & { // readonly comments: readonly (CommentRow & { // readonly author: readonly AuthorRow|undefined; // })[]; // }[] You can get the type of a single row with QueryRowType: import type {QueryRowType} from '@rocicorp/zero' type MySingleRow = QueryRowType // MySingleRow is: readonly IssueRow & { // readonly comments: readonly (CommentRow & { // readonly author: readonly AuthorRow|undefined; // })[]; // } Planning Zero automatically plans queries, selecting the best indexes and join orders in most cases. Inspecting Query Plans You can inspect the plan that Zero generates for any ZQL query using the inspector. Manually Flipping Joins The process Zero uses to optimize joins is called \"join flipping\", because it involves \"flipping\" the order of joins to minimize the number of rows processed. Typically the Zero planner will pick the joins to flip automatically. But in some rare cases, you may want to manually specify the join order. This can be done by passing the flip:true option to whereExists: // Find the first 100 documents that user 42 can edit, // ordered by created desc. Because each user is an editor // of only a few documents, flip:true is much faster than // flip:false. zql.documents.whereExists('editors', e => e.where('userID', 42), {flip: true} ), .orderBy('created', 'desc') .limit(100) Or with exists: // Find issues created by user 42 or that have a comment // by user 42. Because user 42 has commented on only a // few issues, flip:true is much faster than flip:false. zql.issue.where({cmp, or, exists} => or( cmp('creatorID', 42), exists('comments', c => c.where('creatorID', 42), {flip: true}), ), ) You can manually flip just one or a few of the whereExists clauses in a query, leaving the rest to be planned automatically. Scalar Subqueries Scalar subqueries are an optimization for exists queries. Instead of doing a join at query time, Zero pre-resolves the subquery and rewrites it as a simple equality check. To use scalar subqueries, add a unique index in Postgres and declare it as unique in the Zero schema, then add {scalar: true} to your whereExists call: // Instead of joining to find issues where project.name = 'zero' // Zero resolves this server-side to: where('projectId', '123') zql.issue.whereExists( 'project', q => q.where('name', 'zero'), {scalar: true} ) Or with exists: zql.issue.where({cmp, exists} => exists('project', q => q.where('name', 'zero'), {scalar: true}, ), ) Why It Matters Joins are expensive. Sometimes they are needed, but for something like \"give me all issues where the owner's name is Alice\", you don't need a full join — you just need Alice's ID. The scalar optimization pre-fetches that ID and rewrites your query as where('ownerId', aliceId). This can improve query performance significantly. It also allows planning to work better. Since the ID is known at planning time, Zero/SQLite can choose better indexes. Trade-offs The query needs to be \"rehydrated\" (re-run) whenever the scalar subquery result changes. This is fine for relatively stable lookup data like user IDs or project IDs, but you probably wouldn't want it for rapidly-changing data. Scalar subqueries only work when the subquery is guaranteed to return at most one row. The callback must constrain every column of a primary key or a key declared with .unique(...) using equality. Compound keys require an equality constraint for every column. They also only support one-hop relationships, not junctions. Future Work Scalar subqueries are not currently integrated with Zero's planner. You need to manually choose when to use them.", + "content": "Inspired by SQL, ZQL is expressed in TypeScript with heavy use of the builder pattern. If you have used Drizzle or Kysely, ZQL will feel familiar. ZQL queries are composed of one or more clauses that are chained together into a query. Create a Builder To get started, use createBuilder. If you use drizzle-zero or prisma-zero, this happens automatically and an instance is stored in the zql constant exported from schema.ts: import {zql} from 'schema.ts' // zql.myTable.where(...) Otherwise, create an instance manually: // schema.ts // ... export const zql = createBuilder(schema) Select ZQL queries start by selecting a table. There is no way to select a subset of columns; ZQL queries always return the entire row, if permissions allow it. import {zql} from 'zero.ts' // Returns a query that selects all rows and columns from the // issue table. zql.issue This is a design tradeoff that allows Zero to better reuse the row locally for future queries. This also makes it easier to share types between different parts of the code. Do not directly modify JavaScript values returned from ZQL queries. ZQL caches values across queries to improve performance and reduce re-renders in frameworks like React and Solid. If you modify a value returned from ZQL, you will modify it everywhere it is used. This can lead to subtle bugs. JavaScript and TypeScript lack true immutable types so we use readonly to help enforce it. But it's easy to cast away the readonly accidentally. Ordering You can sort query results by adding an orderBy clause: zql.issue.orderBy('created', 'desc') Multiple orderBy clauses can be present, in which case the data is sorted by those clauses in order: // Order by priority descending. For any rows with same priority, // then order by created desc. zql.issue .orderBy('priority', 'desc') .orderBy('created', 'desc') All queries in ZQL have a default final order of their primary key. Assuming the issue table has a primary key on the id column, then: // Actually means: zql.issue.orderBy('id', 'asc'); zql.issue // Actually means: zql.issue.orderBy('priority', 'desc').orderBy('id', 'asc'); zql.issue.orderBy('priority', 'desc') Limit You can limit the number of rows to return with limit(): zql.issue.orderBy('created', 'desc').limit(100) Paging You can start the results at or after a particular row with start(): let start: IssueRow | undefined while (true) { let q = zql.issue .orderBy('created', 'desc') .limit(100) if (start) { q = q.start(start) } const batch = await q.run() console.log('got batch', batch) if (batch.length < 100) { break } start = batch[batch.length - 1] } By default start() is exclusive - it returns rows starting after the supplied reference row. This is what you usually want for paging. If you want inclusive results, you can do: zql.issue.start(row, {inclusive: true}) Getting a Single Result If you want exactly zero or one results, use the one() clause. This causes ZQL to return Row|undefined rather than Row[]. const result = await zql.issue .where('id', 42) .one() .run() if (!result) { console.error('not found') } one() overrides any limit() clause that is also present. Relationships You can query related rows using relationships that are defined in your Zero schema. // Get all issues and their related comments zql.issue.related('comments') Relationships are returned as hierarchical data. In the above example, each row will have a comments field, which is an array of the corresponding comments rows. You can fetch multiple relationships in a single query: zql.issue .related('comments') .related('reactions') .related('assignees') Refining Relationships By default all matching relationship rows are returned, but this can be refined. The related method accepts an optional second function which is itself a query. zql.issue.related( 'comments', // It is common to use the 'q' shorthand variable for this parameter, // but it is a _comment_ query in particular here, exactly as if you // had done zql.comment. q => q .orderBy('modified', 'desc') .limit(100) .start(lastSeenComment) ) This relationship query can have all the same clauses that top-level queries can have. Using orderBy or limit in a relationship that goes through a junction table (i.e., a many-to-many relationship) is not currently supported and will throw a runtime error. See bug 3527. You can sometimes work around this by making the junction relationship explicit, depending on your schema and usage. Nested Relationships You can nest relationships arbitrarily: // Get all issues, first 100 comments for each (ordered by modified,desc), // and for each comment all of its reactions. zql.issue.related('comments', q => q .orderBy('modified', 'desc') .limit(100) .related('reactions') ) Where You can filter a query with where(): zql.issue.where('priority', '=', 'high') The first parameter is always a column name from the table being queried. TypeScript completion will offer available options (sourced from your Zero Schema). Comparison Operators where() supports the following comparison operators: TypeScript will restrict you from using operators with types that don’t make sense – you can’t use > with boolean for example. Zero can evaluate an ILIKE filter in three places: in-memory on the client, in ZQLite against the replica, or as SQL in Postgres. ZQLite and the in-memory query engine use Unicode-aware lowercasing before matching. This means common non-ASCII case variants, such as MÜLLER and müller, match in both. If you need non-ASCII text in PG to work with ILIKE, use a Unicode-aware collation for those text columns, such as the database's default UTF-8 locale. Equals is the Default Comparison Operator Because comparing by = is so common, you can leave it out and where defaults to =. zql.issue.where('priority', 'high') Comparing to null As in SQL, ZQL’s null cannot be compared with =, !=, <, or any other normal comparison operator. Comparing any value to null with such operators is always false: These semantics feel a bit weird, but they are consistent with SQL. The reason SQL does it this way is to make join semantics work: if you’re joining employee.orgID on org.id you do not want an employee in no organization to match an org that hasn’t yet been assigned an ID. For when you purposely do want to compare to null ZQL supports IS and IS NOT operators that also work just like in SQL: // Find employees not in any org. zql.employee.where('orgID', 'IS', null) // Find employees in an org other than 42 OR employees in NO org zql.employee.where('orgID', 'IS NOT', 42) TypeScript will prevent you from comparing to null with other operators. Comparing to undefined As a convenience, you can pass undefined to where: zql.issue.where('priority', issue?.priority) This comparison is always false, so the above query always returns no results. Compound Filters The argument to where can also be a callback that returns a complex expression: // Get all issues that have priority 'critical' or else have both // priority 'medium' and not more than 100 votes. zql.issue.where(({cmp, and, or, not}) => or( cmp('priority', 'critical'), and( cmp('priority', 'medium'), not(cmp('numVotes', '>', 100)) ) ) ) cmp is short for compare and works the same as where at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below). Note that chaining where() is also a one-level and: // Find issues with priority 3 or higher, owned by aa zql.issue .where('priority', '>=', 3) .where('owner', 'aa') Comparing Literal Values The where clause always expects its first parameter to be a column name as a string. Same with the cmp helper: // \"foo\" is a column name, not a string: zql.issue.where('foo', 'bar') // \"foo\" is a column name, not a string: zql.issue.where(({cmp}) => cmp('foo', 'bar')) To compare to a literal value, use the cmpLit helper: zql.issue.where(cmpLit('foobar', 'foo' + 'bar')) This is particularly useful for implementing permissions, because the first parameter can be a field of your context: zql.issue.where(cmpLit(ctx.role, 'admin')) JSON Filters You can filter on a value inside a json() column with the json helper. Pass it the column name followed by a path into the value: // Find issues whose metadata.priority is 'high' zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'high') ) json can only be used inside a callback, as the first argument to cmp. It works with all the comparison operators, and it can be combined with and, or, not, and exists like any other cmp. Path segments are applied left to right. A string segment reads a key of an object and a number segment reads an index of an array: // metadata.address.zip cmp(json('metadata', 'address', 'zip'), '94110') // The first element of metadata.tags cmp(json('metadata', 'tags', 0), 'bug') Array indexes must be non-negative integers. A string never indexes an array and a number never reads an object key, so json('metadata', 'tags', '0') doesn't match anything. If the column has a custom JSON type, TypeScript completes the path segments and types the value you compare against. A column declared with plain json() accepts any path. Comparisons Are Type-Strict A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A non-null value of a different type than the one you compare against never matches a positive operator and always matches a negated one: LIKE and ILIKE only match string values. Values can be compared against strings, numbers, and booleans, and against null with IS and IS NOT (see Missing Values). You can't compare a whole object or array. The array passed to IN and NOT IN must contain values of a single type: cmp(json('metadata', 'priority'), 'IN', [ 'high', 'critical' ]) Missing Values A path that doesn't exist and a JSON null both read as null. As with columns, they never match =, !=, or the other value operators. Use IS and IS NOT to find them: // Issues with no metadata.priority (missing or null) zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'IS', null) ) Relationship Filters Your filter can also test properties of relationships. Currently the only supported test is existence: // Find all orgs that have at least one employee zql.organization.whereExists('employees') The argument to whereExists is a relationship, so just like other relationships, it can be refined with a query: // Find all orgs that have at least one cool employee zql.organization.whereExists('employees', q => q.where('location', 'Hawaii') ) As with querying relationships, relationship filters can be arbitrarily nested: // Get all issues that have comments that have reactions zql.issue.whereExists('comments', q => q.whereExists('reactions') ) The exists helper is also provided which can be used with and, or, cmp, and not to build compound filters that check relationship existence: // Find issues that have at least one comment or are high priority zql.issue.where({cmp, or, exists} => or( cmp('priority', 'high'), exists('comments'), ), ) Type Helpers You can get the TypeScript type of the result of a query using the QueryResultType helper: import type {QueryResultType} from '@rocicorp/zero' const complexQuery = zql.issue.related( 'comments', q => q.related('author') ) type MyComplexResult = QueryResultType // MyComplexResult is: readonly IssueRow & { // readonly comments: readonly (CommentRow & { // readonly author: readonly AuthorRow|undefined; // })[]; // }[] You can get the type of a single row with QueryRowType: import type {QueryRowType} from '@rocicorp/zero' type MySingleRow = QueryRowType // MySingleRow is: readonly IssueRow & { // readonly comments: readonly (CommentRow & { // readonly author: readonly AuthorRow|undefined; // })[]; // } Planning Zero automatically plans queries, selecting the best indexes and join orders in most cases. Inspecting Query Plans You can inspect the plan that Zero generates for any ZQL query using the inspector. Manually Flipping Joins The process Zero uses to optimize joins is called \"join flipping\", because it involves \"flipping\" the order of joins to minimize the number of rows processed. Typically the Zero planner will pick the joins to flip automatically. But in some rare cases, you may want to manually specify the join order. This can be done by passing the flip:true option to whereExists: // Find the first 100 documents that user 42 can edit, // ordered by created desc. Because each user is an editor // of only a few documents, flip:true is much faster than // flip:false. zql.documents.whereExists('editors', e => e.where('userID', 42), {flip: true} ), .orderBy('created', 'desc') .limit(100) Or with exists: // Find issues created by user 42 or that have a comment // by user 42. Because user 42 has commented on only a // few issues, flip:true is much faster than flip:false. zql.issue.where({cmp, or, exists} => or( cmp('creatorID', 42), exists('comments', c => c.where('creatorID', 42), {flip: true}), ), ) You can manually flip just one or a few of the whereExists clauses in a query, leaving the rest to be planned automatically. Scalar Subqueries Scalar subqueries are an optimization for exists queries. Instead of doing a join at query time, Zero pre-resolves the subquery and rewrites it as a simple equality check. To use scalar subqueries, add a unique index in Postgres and declare it as unique in the Zero schema, then add {scalar: true} to your whereExists call: // Instead of joining to find issues where project.name = 'zero' // Zero resolves this server-side to: where('projectId', '123') zql.issue.whereExists( 'project', q => q.where('name', 'zero'), {scalar: true} ) Or with exists: zql.issue.where({cmp, exists} => exists('project', q => q.where('name', 'zero'), {scalar: true}, ), ) Why It Matters Joins are expensive. Sometimes they are needed, but for something like \"give me all issues where the owner's name is Alice\", you don't need a full join — you just need Alice's ID. The scalar optimization pre-fetches that ID and rewrites your query as where('ownerId', aliceId). This can improve query performance significantly. It also allows planning to work better. Since the ID is known at planning time, Zero/SQLite can choose better indexes. Trade-offs The query needs to be \"rehydrated\" (re-run) whenever the scalar subquery result changes. This is fine for relatively stable lookup data like user IDs or project IDs, but you probably wouldn't want it for rapidly-changing data. Scalar subqueries only work when the subquery is guaranteed to return at most one row. The callback must constrain every column of a primary key or a key declared with .unique(...) using equality. Compound keys require an equality constraint for every column. They also only support one-hop relationships, not junctions. Future Work Scalar subqueries are not currently integrated with Zero's planner. You need to manually choose when to use them.", "headings": [ { "text": "Create a Builder", @@ -9720,6 +9720,18 @@ "text": "Comparing Literal Values", "id": "comparing-literal-values" }, + { + "text": "JSON Filters", + "id": "json-filters" + }, + { + "text": "Comparisons Are Type-Strict", + "id": "comparisons-are-type-strict" + }, + { + "text": "Missing Values", + "id": "missing-values" + }, { "text": "Relationship Filters", "id": "relationship-filters" @@ -9856,7 +9868,7 @@ "sectionTitle": "Where", "sectionId": "where", "url": "/docs/zql", - "content": "You can filter a query with where(): zql.issue.where('priority', '=', 'high') The first parameter is always a column name from the table being queried. TypeScript completion will offer available options (sourced from your Zero Schema). Comparison Operators where() supports the following comparison operators: TypeScript will restrict you from using operators with types that don’t make sense – you can’t use > with boolean for example. Zero can evaluate an ILIKE filter in three places: in-memory on the client, in ZQLite against the replica, or as SQL in Postgres. ZQLite and the in-memory query engine use Unicode-aware lowercasing before matching. This means common non-ASCII case variants, such as MÜLLER and müller, match in both. If you need non-ASCII text in PG to work with ILIKE, use a Unicode-aware collation for those text columns, such as the database's default UTF-8 locale. Equals is the Default Comparison Operator Because comparing by = is so common, you can leave it out and where defaults to =. zql.issue.where('priority', 'high') Comparing to null As in SQL, ZQL’s null cannot be compared with =, !=, <, or any other normal comparison operator. Comparing any value to null with such operators is always false: These semantics feel a bit weird, but they are consistent with SQL. The reason SQL does it this way is to make join semantics work: if you’re joining employee.orgID on org.id you do not want an employee in no organization to match an org that hasn’t yet been assigned an ID. For when you purposely do want to compare to null ZQL supports IS and IS NOT operators that also work just like in SQL: // Find employees not in any org. zql.employee.where('orgID', 'IS', null) // Find employees in an org other than 42 OR employees in NO org zql.employee.where('orgID', 'IS NOT', 42) TypeScript will prevent you from comparing to null with other operators. Comparing to undefined As a convenience, you can pass undefined to where: zql.issue.where('priority', issue?.priority) This comparison is always false, so the above query always returns no results. Compound Filters The argument to where can also be a callback that returns a complex expression: // Get all issues that have priority 'critical' or else have both // priority 'medium' and not more than 100 votes. zql.issue.where(({cmp, and, or, not}) => or( cmp('priority', 'critical'), and( cmp('priority', 'medium'), not(cmp('numVotes', '>', 100)) ) ) ) cmp is short for compare and works the same as where at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below). Note that chaining where() is also a one-level and: // Find issues with priority 3 or higher, owned by aa zql.issue .where('priority', '>=', 3) .where('owner', 'aa') Comparing Literal Values The where clause always expects its first parameter to be a column name as a string. Same with the cmp helper: // \"foo\" is a column name, not a string: zql.issue.where('foo', 'bar') // \"foo\" is a column name, not a string: zql.issue.where(({cmp}) => cmp('foo', 'bar')) To compare to a literal value, use the cmpLit helper: zql.issue.where(cmpLit('foobar', 'foo' + 'bar')) This is particularly useful for implementing permissions, because the first parameter can be a field of your context: zql.issue.where(cmpLit(ctx.role, 'admin')) Relationship Filters Your filter can also test properties of relationships. Currently the only supported test is existence: // Find all orgs that have at least one employee zql.organization.whereExists('employees') The argument to whereExists is a relationship, so just like other relationships, it can be refined with a query: // Find all orgs that have at least one cool employee zql.organization.whereExists('employees', q => q.where('location', 'Hawaii') ) As with querying relationships, relationship filters can be arbitrarily nested: // Get all issues that have comments that have reactions zql.issue.whereExists('comments', q => q.whereExists('reactions') ) The exists helper is also provided which can be used with and, or, cmp, and not to build compound filters that check relationship existence: // Find issues that have at least one comment or are high priority zql.issue.where({cmp, or, exists} => or( cmp('priority', 'high'), exists('comments'), ), )", + "content": "You can filter a query with where(): zql.issue.where('priority', '=', 'high') The first parameter is always a column name from the table being queried. TypeScript completion will offer available options (sourced from your Zero Schema). Comparison Operators where() supports the following comparison operators: TypeScript will restrict you from using operators with types that don’t make sense – you can’t use > with boolean for example. Zero can evaluate an ILIKE filter in three places: in-memory on the client, in ZQLite against the replica, or as SQL in Postgres. ZQLite and the in-memory query engine use Unicode-aware lowercasing before matching. This means common non-ASCII case variants, such as MÜLLER and müller, match in both. If you need non-ASCII text in PG to work with ILIKE, use a Unicode-aware collation for those text columns, such as the database's default UTF-8 locale. Equals is the Default Comparison Operator Because comparing by = is so common, you can leave it out and where defaults to =. zql.issue.where('priority', 'high') Comparing to null As in SQL, ZQL’s null cannot be compared with =, !=, <, or any other normal comparison operator. Comparing any value to null with such operators is always false: These semantics feel a bit weird, but they are consistent with SQL. The reason SQL does it this way is to make join semantics work: if you’re joining employee.orgID on org.id you do not want an employee in no organization to match an org that hasn’t yet been assigned an ID. For when you purposely do want to compare to null ZQL supports IS and IS NOT operators that also work just like in SQL: // Find employees not in any org. zql.employee.where('orgID', 'IS', null) // Find employees in an org other than 42 OR employees in NO org zql.employee.where('orgID', 'IS NOT', 42) TypeScript will prevent you from comparing to null with other operators. Comparing to undefined As a convenience, you can pass undefined to where: zql.issue.where('priority', issue?.priority) This comparison is always false, so the above query always returns no results. Compound Filters The argument to where can also be a callback that returns a complex expression: // Get all issues that have priority 'critical' or else have both // priority 'medium' and not more than 100 votes. zql.issue.where(({cmp, and, or, not}) => or( cmp('priority', 'critical'), and( cmp('priority', 'medium'), not(cmp('numVotes', '>', 100)) ) ) ) cmp is short for compare and works the same as where at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below). Note that chaining where() is also a one-level and: // Find issues with priority 3 or higher, owned by aa zql.issue .where('priority', '>=', 3) .where('owner', 'aa') Comparing Literal Values The where clause always expects its first parameter to be a column name as a string. Same with the cmp helper: // \"foo\" is a column name, not a string: zql.issue.where('foo', 'bar') // \"foo\" is a column name, not a string: zql.issue.where(({cmp}) => cmp('foo', 'bar')) To compare to a literal value, use the cmpLit helper: zql.issue.where(cmpLit('foobar', 'foo' + 'bar')) This is particularly useful for implementing permissions, because the first parameter can be a field of your context: zql.issue.where(cmpLit(ctx.role, 'admin')) JSON Filters You can filter on a value inside a json() column with the json helper. Pass it the column name followed by a path into the value: // Find issues whose metadata.priority is 'high' zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'high') ) json can only be used inside a callback, as the first argument to cmp. It works with all the comparison operators, and it can be combined with and, or, not, and exists like any other cmp. Path segments are applied left to right. A string segment reads a key of an object and a number segment reads an index of an array: // metadata.address.zip cmp(json('metadata', 'address', 'zip'), '94110') // The first element of metadata.tags cmp(json('metadata', 'tags', 0), 'bug') Array indexes must be non-negative integers. A string never indexes an array and a number never reads an object key, so json('metadata', 'tags', '0') doesn't match anything. If the column has a custom JSON type, TypeScript completes the path segments and types the value you compare against. A column declared with plain json() accepts any path. Comparisons Are Type-Strict A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A non-null value of a different type than the one you compare against never matches a positive operator and always matches a negated one: LIKE and ILIKE only match string values. Values can be compared against strings, numbers, and booleans, and against null with IS and IS NOT (see Missing Values). You can't compare a whole object or array. The array passed to IN and NOT IN must contain values of a single type: cmp(json('metadata', 'priority'), 'IN', [ 'high', 'critical' ]) Missing Values A path that doesn't exist and a JSON null both read as null. As with columns, they never match =, !=, or the other value operators. Use IS and IS NOT to find them: // Issues with no metadata.priority (missing or null) zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'IS', null) ) Relationship Filters Your filter can also test properties of relationships. Currently the only supported test is existence: // Find all orgs that have at least one employee zql.organization.whereExists('employees') The argument to whereExists is a relationship, so just like other relationships, it can be refined with a query: // Find all orgs that have at least one cool employee zql.organization.whereExists('employees', q => q.where('location', 'Hawaii') ) As with querying relationships, relationship filters can be arbitrarily nested: // Get all issues that have comments that have reactions zql.issue.whereExists('comments', q => q.whereExists('reactions') ) The exists helper is also provided which can be used with and, or, cmp, and not to build compound filters that check relationship existence: // Find issues that have at least one comment or are high priority zql.issue.where({cmp, or, exists} => or( cmp('priority', 'high'), exists('comments'), ), )", "kind": "section" }, { @@ -9920,7 +9932,37 @@ "kind": "section" }, { - "id": "729-zql#relationship-filters", + "id": "729-zql#json-filters", + "title": "ZQL", + "searchTitle": "JSON Filters", + "sectionTitle": "JSON Filters", + "sectionId": "json-filters", + "url": "/docs/zql", + "content": "You can filter on a value inside a json() column with the json helper. Pass it the column name followed by a path into the value: // Find issues whose metadata.priority is 'high' zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'high') ) json can only be used inside a callback, as the first argument to cmp. It works with all the comparison operators, and it can be combined with and, or, not, and exists like any other cmp. Path segments are applied left to right. A string segment reads a key of an object and a number segment reads an index of an array: // metadata.address.zip cmp(json('metadata', 'address', 'zip'), '94110') // The first element of metadata.tags cmp(json('metadata', 'tags', 0), 'bug') Array indexes must be non-negative integers. A string never indexes an array and a number never reads an object key, so json('metadata', 'tags', '0') doesn't match anything. If the column has a custom JSON type, TypeScript completes the path segments and types the value you compare against. A column declared with plain json() accepts any path. Comparisons Are Type-Strict A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A non-null value of a different type than the one you compare against never matches a positive operator and always matches a negated one: LIKE and ILIKE only match string values. Values can be compared against strings, numbers, and booleans, and against null with IS and IS NOT (see Missing Values). You can't compare a whole object or array. The array passed to IN and NOT IN must contain values of a single type: cmp(json('metadata', 'priority'), 'IN', [ 'high', 'critical' ]) Missing Values A path that doesn't exist and a JSON null both read as null. As with columns, they never match =, !=, or the other value operators. Use IS and IS NOT to find them: // Issues with no metadata.priority (missing or null) zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'IS', null) )", + "kind": "section" + }, + { + "id": "730-zql#comparisons-are-type-strict", + "title": "ZQL", + "searchTitle": "Comparisons Are Type-Strict", + "sectionTitle": "Comparisons Are Type-Strict", + "sectionId": "comparisons-are-type-strict", + "url": "/docs/zql", + "content": "A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A non-null value of a different type than the one you compare against never matches a positive operator and always matches a negated one: LIKE and ILIKE only match string values. Values can be compared against strings, numbers, and booleans, and against null with IS and IS NOT (see Missing Values). You can't compare a whole object or array. The array passed to IN and NOT IN must contain values of a single type: cmp(json('metadata', 'priority'), 'IN', [ 'high', 'critical' ])", + "kind": "section" + }, + { + "id": "731-zql#missing-values", + "title": "ZQL", + "searchTitle": "Missing Values", + "sectionTitle": "Missing Values", + "sectionId": "missing-values", + "url": "/docs/zql", + "content": "A path that doesn't exist and a JSON null both read as null. As with columns, they never match =, !=, or the other value operators. Use IS and IS NOT to find them: // Issues with no metadata.priority (missing or null) zql.issue.where(({cmp, json}) => cmp(json('metadata', 'priority'), 'IS', null) )", + "kind": "section" + }, + { + "id": "732-zql#relationship-filters", "title": "ZQL", "searchTitle": "Relationship Filters", "sectionTitle": "Relationship Filters", @@ -9930,7 +9972,7 @@ "kind": "section" }, { - "id": "730-zql#type-helpers", + "id": "733-zql#type-helpers", "title": "ZQL", "searchTitle": "Type Helpers", "sectionTitle": "Type Helpers", @@ -9940,7 +9982,7 @@ "kind": "section" }, { - "id": "731-zql#planning", + "id": "734-zql#planning", "title": "ZQL", "searchTitle": "Planning", "sectionTitle": "Planning", @@ -9950,7 +9992,7 @@ "kind": "section" }, { - "id": "732-zql#inspecting-query-plans", + "id": "735-zql#inspecting-query-plans", "title": "ZQL", "searchTitle": "Inspecting Query Plans", "sectionTitle": "Inspecting Query Plans", @@ -9960,7 +10002,7 @@ "kind": "section" }, { - "id": "733-zql#manually-flipping-joins", + "id": "736-zql#manually-flipping-joins", "title": "ZQL", "searchTitle": "Manually Flipping Joins", "sectionTitle": "Manually Flipping Joins", @@ -9970,7 +10012,7 @@ "kind": "section" }, { - "id": "734-zql#scalar-subqueries", + "id": "737-zql#scalar-subqueries", "title": "ZQL", "searchTitle": "Scalar Subqueries", "sectionTitle": "Scalar Subqueries", @@ -9980,7 +10022,7 @@ "kind": "section" }, { - "id": "735-zql#why-it-matters", + "id": "738-zql#why-it-matters", "title": "ZQL", "searchTitle": "Why It Matters", "sectionTitle": "Why It Matters", @@ -9990,7 +10032,7 @@ "kind": "section" }, { - "id": "736-zql#trade-offs", + "id": "739-zql#trade-offs", "title": "ZQL", "searchTitle": "Trade-offs", "sectionTitle": "Trade-offs", @@ -10000,7 +10042,7 @@ "kind": "section" }, { - "id": "737-zql#future-work", + "id": "740-zql#future-work", "title": "ZQL", "searchTitle": "Future Work", "sectionTitle": "Future Work", diff --git a/contents/docs/zql.mdx b/contents/docs/zql.mdx index f63cb60b..51910a5a 100644 --- a/contents/docs/zql.mdx +++ b/contents/docs/zql.mdx @@ -362,7 +362,7 @@ If the column has a [custom JSON type](/docs/schema#custom-json-types), TypeScri #### Comparisons Are Type-Strict -A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A value of a different type than the one you compare against never matches a positive operator and always matches a negated one: +A JSON value doesn't have a fixed type the way a column does, so comparisons follow JS strict equal (===) semantics. A non-null value of a different type than the one you compare against never matches a positive operator and always matches a negated one: | Value at path | Filter | Result | | ------------- | --------- | ------- | @@ -372,7 +372,7 @@ A JSON value doesn't have a fixed type the way a column does, so comparisons fol | `true` | `= 1` | `false` | | `'abc'` | `> 5` | `false` | -`LIKE` and `ILIKE` only match string values. Only strings, numbers, and booleans can be compared. You can't compare a whole object or array. +`LIKE` and `ILIKE` only match string values. Values can be compared against strings, numbers, and booleans, and against `null` with `IS` and `IS NOT` (see [Missing Values](#missing-values)). You can't compare a whole object or array. The array passed to `IN` and `NOT IN` must contain values of a single type: