Repository navigation
Conversation
There was a problem hiding this comment.
Code Review
This pull request increases the inbound message size limit on the Unix Domain Socket (UDS) channel to 256 MiB for both the asynchronous and synchronous accelerator clients in the Google Cloud Bigtable library. This is achieved by defining _MAX_RECEIVE_MESSAGE_LENGTH and passing it via _CHANNEL_OPTIONS to the insecure_channel constructor, preventing potential RESOURCE_EXHAUSTED errors on local reads. There are no review comments, so I have no feedback to provide.
Change-Id: I6adb2982640fe2d6736ea63ed165c9ffd6feacb7
| # the UDS leg is never the narrower of the two. | ||
| _MAX_RECEIVE_MESSAGE_LENGTH = 256 * 1024 * 1024 | ||
|
|
||
| _CHANNEL_OPTIONS = (("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),) |
There was a problem hiding this comment.
Two concerns with the one-sided cap:
1. Send-side left at the gRPC 4 MiB default — silent regression vs. the native path. The native google-cloud-bigtable Python client sets both grpc.max_send_message_length and grpc.max_receive_message_length to -1 (unlimited). This PR raises only inbound, so an ApplyBulk / MutateRow with a payload >4 MiB (allowed by the server's 100 MiB per-cell limit) will fail at the accelerator channel's outbound serialization check with no analogous server-side error. On the native path the same call goes through. Suggest raising send too:
| _CHANNEL_OPTIONS = (("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),) | |
| _CHANNEL_OPTIONS = ( | |
| ("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH), | |
| ("grpc.max_send_message_length", _MAX_RECEIVE_MESSAGE_LENGTH), | |
| ) |
2. Pick a side on the 256 MB number. Native uses -1 (unlimited). 256 MB from thin air invites "why not 128? why not 1 GiB?" and future drift when the native client bumps. Either match native (-1, both directions) and document "match native" in the comment, or if 256 MB is a deliberate memory-safety choice at the UDS hop, say that explicitly.
| self._metadata = (("x-accelerator-token", auth_secret),) | ||
| self._channel = insecure_channel(_uds_target(uds_path)) | ||
| self._channel = insecure_channel( | ||
| _uds_target(uds_path), options=_CHANNEL_OPTIONS |
There was a problem hiding this comment.
Nit: no test exercises _CHANNEL_OPTIONS. Even a one-liner asserting the option tuple is on self._channel's options would pin both the key spelling (grpc.max_receive_message_length is easy to typo) and the value. Low priority.
sushanb
left a comment
There was a problem hiding this comment.
Approving to unblock the stack, but please address the send-side cap gap above before merge — leaving outbound at the gRPC 4 MiB default is a silent regression vs. the native client, which sets both directions to -1. Simplest fix is one extra line in .
gRPC caps inbound messages at 4 MiB by default — a limit sized for a channel that crosses a network. The accelerator's UDS channel doesn't leave the machine: the daemon has already paid the cost of pulling the response off the wire, and all the cap does on this hop is turn an otherwise-fine read into
RESOURCE_EXHAUSTED. This setsgrpc.max_receive_message_lengthto 256 MB so the UDS leg is never the narrower of the two.No test yet. The Go daemon still has its own 4 MiB cap on the sending side, so nothing above 4 MiB actually crosses the socket today and a test would only be asserting on a config value. The test lands once the daemon side is raised.
Only the inbound direction is changed here;
grpc.max_send_message_lengthis untouched.