Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
e2b5312
feat: draft Spanner Queues enqueue and ack
torreypayne Jul 23, 2026
c8eb572
feat(spanner): implement Send and Ack queue mutations and acceptance …
torreypayne Aug 4, 2026
d944cff
refactor(spanner): extract protected key_list_value helper method in …
torreypayne Aug 5, 2026
05eb9e7
test(spanner): read data back from queue after enqueue and ack mutati…
torreypayne Aug 5, 2026
ec56f1d
test(spanner): remove rescue blocks from queues acceptance tests for …
torreypayne Aug 5, 2026
3ab64e0
test(spanner): switch queues acceptance tests to Option 2 explicit Mi…
torreypayne Aug 5, 2026
a2a68d1
test(spanner): remove rescue and skip blocks from queues acceptance t…
torreypayne Aug 5, 2026
8a90dd9
chore: skip queues E2E tests when running against cloud-spanner-emulator
torreypayne Aug 14, 2026
54dea5c
fix: resolve Results#rows Enumerator test exhaustion bug in queues_test
torreypayne Aug 25, 2026
cf16b50
feat(spanner): rename deliver_at to deliver_time in Commit#enqueue to…
torreypayne Sep 1, 2026
435fd26
feat(spanner): rename keys parameter to key in Commit#enqueue and Com…
torreypayne Sep 1, 2026
37b9e98
feat(spanner): add enqueue and ack to Transaction for read-write tran…
torreypayne Sep 1, 2026
a8abc9a
feat(spanner): document and support payload types and Base64 encoding…
torreypayne Sep 1, 2026
5518ae3
refactor(spanner): require StringIO for binary queue payloads
torreypayne Oct 1, 2026
8ef7004
refactor(spanner): rename enqueue and ack to send_message and ack_mes…
torreypayne Oct 5, 2026
2fc1bc5
feat(spanner): add SendOptions and AckOptions for queue mutations
torreypayne Oct 5, 2026
0b128f2
test(spanner): run queue acceptance tests against production
torreypayne Oct 5, 2026
3017c46
test(spanner): add the TestQueue DDL fixture
torreypayne Oct 5, 2026
f2ac5bc
chore(deps): require google-cloud-spanner-v1 1.12 for queue mutations
torreypayne Oct 5, 2026
e78babd
docs(spanner): document queues in OVERVIEW.md and Client
torreypayne Oct 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions acceptance/data/fixtures.rb
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,17 @@ def items_ddl_statement
ITEMS
end

# Not part of schema_ddl_statements: queues need an Enterprise edition
# instance, and the emulator doesn't support them yet.
def queue_ddl_statement
<<~QUEUE
CREATE QUEUE IF NOT EXISTS TestQueue (
Id INT64 NOT NULL,
Payload BYTES(MAX) NOT NULL
) PRIMARY KEY (Id)
QUEUE
end

def schema_pg_ddl_statements
[
stuff_pg_ddl_statement,
Expand Down
54 changes: 54 additions & 0 deletions google-cloud-spanner/OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,60 @@ db.snapshot do |snp|
end
```

## Using queues

A queue holds messages for consumers to receive. You create queues with the
`CREATE QUEUE` statement (see the [DDL
reference](https://cloud.google.com/spanner/docs/reference/standard-sql/data-definition-language)).

Use {Google::Cloud::Spanner::Commit#send_message Commit#send_message} to send a
message. The payload is converted the same way as column values, so wrap binary
data in a `StringIO`, as you would for a `BYTES` column:

```ruby
require "google/cloud/spanner"

spanner = Google::Cloud::Spanner.new

db = spanner.client "my-instance", "my-database"

db.commit do |c|
c.send_message "OrderQueue", 101, StringIO.new("Order payload")
end
```

Receive messages with the `RECEIVE_<queue>` table-valued function, and use
{Google::Cloud::Spanner::Transaction#ack_message Transaction#ack_message} to
acknowledge each one in the same transaction as the write it caused:

```ruby
require "google/cloud/spanner"

spanner = Google::Cloud::Spanner.new

db = spanner.client "my-instance", "my-database"

results = db.execute_query "SELECT Id, Payload FROM RECEIVE_OrderQueue()"

results.rows.each do |row|
db.transaction do |tx|
tx.update "orders", [{ id: row[:Id], status: "PROCESSED" }]
tx.ack_message "OrderQueue", row[:Id]
end
end
```

Optional settings go in a {Google::Cloud::Spanner::SendOptions SendOptions} or
{Google::Cloud::Spanner::AckOptions AckOptions}, or in a hash with the same
keys:

```ruby
db.commit do |c|
c.send_message "OrderQueue", 102, StringIO.new("Order payload"),
send_options: { deliver_time: Time.now + 300 }
end
```

## Deleting databases

Use {Google::Cloud::Spanner::Admin::Database#database_admin
Expand Down
59 changes: 59 additions & 0 deletions google-cloud-spanner/acceptance/spanner/client/queues_test.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require "spanner_helper"

# An ack without ignore_not_found fails with NOT_FOUND unless the message
# exists, so a successful ack also shows that the send landed.
describe "Spanner Client Queues", :queues, :spanner do
let(:db) { spanner_client }

before do
skip "Cloud Spanner Emulator does not yet support Queues (Send & Ack)" if emulator_enabled?

# Created once per run, in the per-run test database.
$spanner_test_queue_created ||= begin
db_job = db.database.update statements: [queue_ddl_statement]
db_job.wait_until_done!
raise GRPC::BadStatus.new(db_job.error.code, db_job.error.message) if db_job.error?
true
end
end

it "sends and acks messages with Client#commit" do
commit_time = db.commit do |c|
c.send_message "TestQueue", 1, StringIO.new("payload1")
c.send_message "TestQueue", 2, StringIO.new("payload2")
end
_(commit_time).must_be_kind_of Time

commit_time = db.commit do |c|
c.ack_message "TestQueue", 1
c.ack_message "TestQueue", 2
end
_(commit_time).must_be_kind_of Time
end

it "sends and acks a message with Client#transaction" do
commit_time = db.transaction do |tx|
tx.send_message "TestQueue", 3, StringIO.new("payload3")
end
_(commit_time).must_be_kind_of Time

commit_time = db.transaction do |tx|
tx.ack_message "TestQueue", 3
end
_(commit_time).must_be_kind_of Time
end
end
2 changes: 1 addition & 1 deletion google-cloud-spanner/google-cloud-spanner.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ Gem::Specification.new do |gem|
gem.add_dependency "google-cloud-core", "~> 1.7"
gem.add_dependency "google-cloud-spanner-admin-database-v1", "~> 1.4"
gem.add_dependency "google-cloud-spanner-admin-instance-v1", "~> 1.6"
gem.add_dependency "google-cloud-spanner-v1", "~> 1.6"
gem.add_dependency "google-cloud-spanner-v1", "~> 1.12"
end
60 changes: 60 additions & 0 deletions google-cloud-spanner/lib/google/cloud/spanner/ack_options.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


module Google
module Cloud
module Spanner
##
# # AckOptions
#
# Optional settings for acknowledging a queue message with
# {Commit#ack_message} or {Transaction#ack_message}. A hash with the
# same keys can be passed instead.
#
# @example
# require "google/cloud/spanner"
#
# spanner = Google::Cloud::Spanner.new
#
# db = spanner.client "my-instance", "my-database"
#
# ack_options = Google::Cloud::Spanner::AckOptions.new ignore_not_found: true
#
# db.commit do |c|
# c.ack_message "OrderQueue", 101, ack_options: ack_options
# end
#
class AckOptions
##
# Whether the ack succeeds when the message doesn't exist.
#
# @return [Boolean]
attr_reader :ignore_not_found

##
# Creates a new AckOptions.
#
# @param [Boolean] ignore_not_found If `true`, the ack succeeds even
# if the message doesn't exist. Optional. The default is `false`,
# which fails the commit with `NOT_FOUND` when the message doesn't
# exist.
#
def initialize ignore_not_found: false
@ignore_not_found = ignore_not_found
end
end
end
end
end
20 changes: 20 additions & 0 deletions google-cloud-spanner/lib/google/cloud/spanner/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ module Spanner
#
# A client is used to read and/or modify data in a Cloud Spanner database.
#
# A client also works with queues. Send and acknowledge messages with
# {Commit#send_message} and {Commit#ack_message} inside {#commit}, or with
# {Transaction#send_message} and {Transaction#ack_message} inside
# {#transaction}. Receive messages by querying the queue's `RECEIVE_`
# function with {#execute_query}.
#
# See {Google::Cloud::Spanner::Project#client}.
#
# @example
Expand Down Expand Up @@ -506,6 +512,20 @@ def directed_read_options
# puts "User #{row[:id]} is #{row[:name]}"
# end
#
# @example Receive messages from a queue with the RECEIVE_ function:
# require "google/cloud/spanner"
#
# spanner = Google::Cloud::Spanner.new
#
# db = spanner.client "my-instance", "my-database"
#
# # RECEIVE_ needs a strong read, which is the default here.
# results = db.execute_query "SELECT Id, Payload FROM RECEIVE_OrderQueue()"
#
# results.rows.each do |row|
# puts "Received message #{row[:Id]}"
# end
#
def execute_query sql, params: nil, types: nil, single_use: nil,
query_options: nil, request_options: nil,
call_options: nil, directed_read_options: nil
Expand Down
Loading
Loading