chore: update librarian to v0.21.0

[zhumin8]

Weekly update to librarian version and re-generate. This update includes a generator upgrade, see https://github.com/googleapis/librarian/releases/tag/v0.21.0.

Manually changes applied changes to postprocess config files due to generation failure after upgrade. 46dfa75

commands run:

go run github.com/googleapis/librarian/cmd/librarian@latest update version
V=$(go run github.com/googleapis/librarian/cmd/librarian@latest config get version)
go run github.com/googleapis/librarian/cmd/librarian@${V} tidy
# Build a new Docker image
go run github.com/googleapis/librarian/tool/cmd/builddockerimages@latest --language python --version=${V}
# Regenerate all libraries in Docker. Around 2 minutes.
docker run -u $(id -u):$(id -g) -v .:/repo -v ~/.cache:/.cache -w /repo docker.io/library/librarian-python:${V}  generate -v --all

For the sake of easier review, separated 3 commits:

• librarian version update f60299a
• Manual changes to config to make fix generation error: 46dfa75
• Generated code changes: b8f2240

Fixes googleapis/librarian#6450

[gemini-code-assist]

Code Review

This pull request updates dependency versions across multiple packages, bumping google-api-core to 2.24.2, protobuf to 6.33.5, proto-plus to 1.26.1, and grpc-google-iam-v1 to 0.14.2, along with updating setup.py version parsing regexes. A critical issue was identified in bigtable-integration.yaml where the paths block was accidentally deleted, breaking the YAML structure.

[zhumin8]

@ohmayr Can you please take a look at the test failures?

[zhumin8]

update:
merged main to take in this revert: #17493.
To resolve conflict and keep generated changes, run these command manually

V=$(go run github.com/googleapis/librarian/cmd/librarian@latest config get version)
docker run -u $(id -u):$(id -g) -v .:/repo -v ~/.cache:/.cache -w /repo docker.io/library/librarian-python:${V}  generate -v google-cloud-pubsub
docker run -u $(id -u):$(id -g) -v .:/repo -v ~/.cache:/.cache -w /repo docker.io/library/librarian-python:${V}  generate -v google-cloud-datastore

[zhumin8]

Turning back to draft, as I want to get #17505 in and confirm repo in healthy state first.

In the meantime, the unittest failing log: https://github.com/googleapis/google-cloud-python/actions/runs/27715880540/job/81988547476?pr=17488

Error:

ERROR: Cannot install google-api-core, google-api-core[grpc]==2.24.2, google-cloud-bigtable and google-cloud-bigtable==2.38.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    google-cloud-bigtable 2.38.0 depends on protobuf<8.0.0 and >=6.33.5
    google-api-core 2.24.2 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <7.0.0 and >=3.19.5
    google-api-core[grpc] 2.24.2 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <7.0.0 and >=3.19.5
    grpc-google-iam-v1 0.14.0 depends on protobuf!=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
    The user requested (constraint) protobuf==6.33.5

Additionally, some packages in these conflicts have no matching distributions available for your environment:
    protobuf

Looks this is introduced by the generator upgrade too:

1. protobuf Bump: The new code generator templates (in Librarian v0.21.0 ) bumped the minimum requirement of protobuf for all generated client libraries from 4.25.8 to 6.33.5 .
2. Outdated Post-processing Configs: Standard client libraries automatically resolve grpc-google-iam-v1 to 0.14.2 or later (which supports protobuf 6.x ). However, google-cloud-
   bigtable (along with google-cloud-pubsub and google-cloud-spanner ) has custom post-processing rules in .librarian/generator-input/client-post-processing/ that override the
   generator's default behavior.
3. Hardcoded Old Versions: These post-processing files hardcoded older versions of grpc-google-iam-v1 (specifically 0.14.0 for Bigtable/PubSub, and 0.12.4 for Spanner) in both
   setup.py and the testing/constraints-3.10.txt template matches.

These hardcoded overrides forced pip to install the older, incompatible versions of grpc-google-iam-v1 , causing the conflict.

[zhumin8]

Trying to resolve unit test failure above by updating these hardcoded grpc-google-iam-v1 to 0.14.2 to match ones in generator. Note that grpc-google-iam-v1 in this repo is at 0.14.4 now. But I suppose we should keep consistent with the published generator?

outdated