Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[run]
branch = True
source = ondewo/utils
omit =
ondewo/version.py

[report]
show_missing = True
exclude_lines =
pragma: no cover
@abstractmethod
raise NotImplementedError
52 changes: 52 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Runs the test suite (with 100% coverage gate) on every push and pull request.
name: tests

on:
push:
branches:
- "**"
pull_request:

concurrency:
group: tests-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
name: pytest (py${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]

steps:
- name: Checkout
uses: actions/checkout@v5

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: |
requirements.txt
requirements-dev.txt

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install .

- name: Run tests with coverage
run: python -m pytest

- name: Upload coverage report
if: always()
uses: actions/upload-artifact@v5
with:
name: coverage-${{ matrix.python-version }}
path: coverage.xml
if-no-files-found: ignore
182 changes: 182 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Repository Overview

`ondewo-client-utils-python` (distribution name `ondewo-client-utils`, importable as `ondewo.utils`) is a small
library of shared base classes and utilities for ONDEWO gRPC Python clients. Other ONDEWO client packages (nlu, s2t,
t2s, vtsi, …) depend on it. It is deliberately minimal — keep it that way.

Public building blocks (all under `ondewo/utils/`):

- `base_client.py` — `BaseClient`, the abstract base for synchronous clients (`connect` / `disconnect`, holds a
`services` container).
- `async_base_client.py` — `AsyncBaseClient`, the `async` counterpart.
- `base_services_interface.py` — `BaseServicesInterface` plus channel helpers (`get_secure_channel`,
`_get_grpc_channel`, `MAX_MESSAGE_LENGTH`) and the constant gRPC channel options.
- `async_base_services_interface.py` — `AsyncBaseServicesInterface`, the `grpc.aio` counterpart.
- `base_client_config.py` — `BaseClientConfig`, a frozen `dataclass_json` config (`host`, `port`, `grpc_cert`).
- `base_service_container.py` — `BaseServicesContainer`, the dataclass that concrete clients subclass to enumerate
their services.
- `helpers.py` — `get_struct_from_dict`, `get_attr_recursive`, `set_attr_recursive`.
- `text.py` — `TextHelper.from_camel_to_snake_case`.

The default gRPC channel options (`_DEFAULT_GRPC_OPTIONS` / `_SERVICE_CONFIG_JSON`) are assembled once at import time.
Keep them module-level constants — do not move the `json.dumps` / options-dict construction back into `__init__`, since
a client with N services would otherwise rebuild them N times per connection.

## Development

Python `>=3.9`.

- **Install:** `pip install -r requirements.txt -r requirements-dev.txt && pip install .`
- **Run tests:** `python -m pytest` — configuration lives in `setup.cfg` (`asyncio_mode = auto`, branch coverage, and a
`--cov-fail-under=100` gate). Any new runtime line needs a covering test or the suite fails.
- **Lint:** `make flake8` (or `flake8 .`); max line length is 120.
- **Types:** `make mypy` (or `pre-commit run mypy --all-files`); configuration in `mypy.ini`.
- **All hooks:** `pre-commit run --all-files`.
- **Docker parity:** `make run_tests` and `make run_code_checks` reproduce CI locally.

CI (`.github/workflows/tests.yml`) runs the test suite on every push and pull request across Python 3.9–3.12. Tests live
in `tests/`; name new files `test_*.py` (enforced by the `name-tests-test` hook).

## Working Principles

Behavioral guidelines to reduce common mistakes. They bias toward caution over speed; for trivial tasks, use judgment.

### Think before coding

Don't assume. Don't hide confusion. Surface tradeoffs.

Before implementing:

- State your assumptions explicitly. If uncertain, ask.
- If multiple interpretations exist, present them — don't pick silently.
- If a simpler approach exists, say so. Push back when warranted.
- If something is unclear, stop. Name what's confusing. Ask.

### Simplicity first

Minimum code that solves the problem. Nothing speculative.

- No features beyond what was asked.
- No abstractions for single-use code.
- No "flexibility" or "configurability" that wasn't requested.
- No error handling for impossible scenarios.
- If you write 200 lines and it could be 50, rewrite it.

Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.

### Surgical changes

Touch only what you must. Clean up only your own mess.

When editing existing code:

- Don't "improve" adjacent code, comments, or formatting.
- Don't refactor things that aren't broken.
- Match existing style, even if you'd do it differently.
- If you notice unrelated dead code, mention it — don't delete it.

When your changes create orphans:

- Remove imports/variables/functions that _your_ changes made unused.
- Don't remove pre-existing dead code unless asked.

The test: every changed line should trace directly to the user's request.

### Goal-driven execution

Define success criteria. Loop until verified.

Transform tasks into verifiable goals:

- "Add validation" → "Write tests for invalid inputs, then make them pass"
- "Fix the bug" → "Write a test that reproduces it, then make it pass"
- "Refactor X" → "Ensure tests pass before and after"

For multi-step tasks, state a brief plan:

```text
1. [Step] → verify: [check]
2. [Step] → verify: [check]
3. [Step] → verify: [check]
```

Strong success criteria let you loop independently. Weak criteria ("make it work") require constant clarification.

These guidelines are working if: fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and
clarifying questions come before implementation rather than after mistakes.

## Logging

This library uses the Python standard-library `logging` module (there is no `loguru` dependency):

```python
from logging import warning
```

- **Levels:** `debug()`, `info()`, `warning()`, `error()`, `exception()`. Choose by hotness/verbosity — `debug` for
routine method entry/exit, `info` for notable lifecycle events, `warning` / `error` / `exception` for problems.
- **Interpolate with f-strings.** Use `f"…{value}"`; only add the `f` prefix when the string actually interpolates
(`"START: …"` with no params stays a plain string).
- **`START:` / `DONE:` bracketing.** For a notable operation, wrap it with a `START:` line at entry and a `DONE:` line
at exit, both naming `ClassName: method_name` (append `: param={value}` context where useful).
- **Timing uses `perf_counter()`, rendered `:.5f`.** Measure elapsed time with `time.perf_counter()` captured as a start
value and subtracted at the `DONE:` line:

```python
from time import perf_counter

start_time: float = perf_counter()
...
log.info(f"DONE: BaseClient: connect. Elapsed time: {perf_counter() - start_time:.5f}")
```

Never measure a duration with `time.time()` — reserve `time.time()` for wall-clock timestamps. `perf_counter()` has an
undefined epoch and must not be stored or compared across processes.

## Docstrings

Google-style, triple double-quotes:

```python
"""
Short imperative summary line.

Args:
param_name (type):
Description of the parameter.

Returns:
type:
Description of the return value.

Raises:
ExceptionType:
When this exception is raised.
"""
```

## Git Commits

- **Never include Claude as author or co-author** in commit messages, PR descriptions, or any other text. Do not add
`Co-Authored-By: Claude…` trailers, "Generated with Claude Code" footers, or any similar attribution.
- The user's own git author identity (already configured in git) is the only identity that should appear on commits.
- This rule overrides the default Claude Code commit-template guidance.
- **Never prepend the JIRA ticket ID** (e.g. `[OND211-2418]`) to the commit subject yourself. The `giticket` pre-commit
hook reads the ticket from the branch name and prepends `[<ticket>]` automatically. This repo's regex is
`(?:(?:feature|bugfix|support|hotfix)/)?(OND[0-9]{3}-[0-9]{1,5})[_-][\w-]+`, so the `feature/` … prefix is optional —
a branch such as `OND211-2418-add-keycloak-for-2-fa` is matched directly and yields `[OND211-2418] `. Writing the
prefix manually produces a duplicate like `[OND211-2418] [OND211-2418] …`. Write the subject as a plain message and
let the hook add the prefix on commit.

## General Principles

- Follow existing patterns before introducing new abstractions.
- Keep changes minimal and consistent with surrounding code.
- Validate inputs early with descriptive, context-rich error messages.
- Use context managers for files, sockets, and thread pools.
- Prefer region comments for grouping methods in files that already use them.
- End edited Markdown and YAML files with a trailing newline.
40 changes: 28 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
# Makefile for ondewo-client-utils-python
#
# Documentation conventions used in this file:
# - Any target with a trailing `## <text>` comment is listed by `make help`.
# - Section banners (`####...`) are listed by `make makefile_chapters`.
#
# Common developer targets:
# make run_tests Build the pytest docker image and run the test suite
# make run_code_checks Run flake8 + mypy inside the code-checks docker image
# make flake8 / make mypy Run the linters directly in the current environment
# make setup_developer_environment_locally Install pre-commit hooks and local dependencies
# make release Run the full automated release (branch, tag, GitHub, PyPI)

PACKAGE_FOLDER := ondewo-client-utils
TESTFILE := ondewo
CODE_CHECK_IMAGE := code_check_image_${TESTFILE}
Expand Down Expand Up @@ -27,7 +40,7 @@ export

# MUST BE THE SAME AS API in Mayor and Minor Version Number
# example: API 2.9.0 --> Client 2.9.X
ONDEWO_PACKAGE_VERSION=$(shell cat ondewo/version.py | sed "s:__version__ = '::" | sed "s:'::")
ONDEWO_PACKAGE_VERSION=$(shell sed -nE "s/^__version__[^=]*= *'([^']+)'.*/\1/p" ondewo/version.py)


PYPI_USERNAME?=ENTER_HERE_YOUR_PYPI_USERNAME
Expand Down Expand Up @@ -56,7 +69,7 @@ IMAGE_UTILS_NAME=ondewo-client-utils-python:${ONDEWO_PACKAGE_VERSION}
# ONDEWO Standard Make Targets
########################################################

setup_developer_environment_locally: install_precommit_hooks install_dependencies_locally
setup_developer_environment_locally: install_precommit_hooks install_dependencies_locally ## Set up the local dev environment (pre-commit hooks + dependencies)

install_precommit_hooks: ## Installs pre-commit hooks and sets them up for the ondewo-csi-client repo
conda install -y pre-commit
Expand All @@ -76,9 +89,12 @@ flake8: ## Runs flake8
mypy: ## Run mypy static code checking
pre-commit run mypy --all-files

cythonize: ## Compile the pure-Python ondewo modules into native extensions (skips dataclasses)
python cython_compile.py build_ext --inplace

help: ## Print usage info about help targets
# (first comment after target starting with double hashes ##)
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' Makefile | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' Makefile | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

makefile_chapters: ## Shows all sections of Makefile
@echo `cat Makefile| grep "########################################################" -A 1 | grep -v "########################################################"`
Expand Down Expand Up @@ -124,11 +140,11 @@ push_to_pypi_via_docker_image: ## Push source code to pypi via docker
${IMAGE_UTILS_NAME} make push_to_pypi
rm -rf dist

show_pypi: build_package
show_pypi: build_package ## Build the package and print the contents of the resulting sdist
tar xvfz dist/ondewo-client-utils-${ONDEWO_PACKAGE_VERSION}.tar.gz
tree ondewo-client-utils-${ONDEWO_PACKAGE_VERSION}

show_pypi_via_docker_image: build_utils_docker_image ## Push source code to pypi via docker
show_pypi_via_docker_image: build_utils_docker_image ## Show the contents of the pypi package via docker
[ -d $(OUTPUT_DIR) ] || mkdir -p $(OUTPUT_DIR)
docker run --rm \
-v ${shell pwd}/dist:/home/ondewo/dist \
Expand All @@ -138,25 +154,25 @@ show_pypi_via_docker_image: build_utils_docker_image ## Push source code to pypi
rm -rf dist


push_to_pypi: build_package upload_package clear_package_data
push_to_pypi: build_package upload_package clear_package_data ## Build, upload to PyPI and clean up (run inside the release docker image)
@echo 'YAY - Pushed to pypi : )'

push_to_gh: login_to_gh build_gh_release
push_to_gh: login_to_gh build_gh_release ## Log in to GitHub and create the GitHub release (run inside the release docker image)
@echo 'Released to Github'

release_to_github_via_docker_image: ## Release to Github via docker
docker run --rm \
-e GITHUB_GH_TOKEN=${GITHUB_GH_TOKEN} \
${IMAGE_UTILS_NAME} make push_to_gh

build_package:
build_package: ## Build the sdist and wheel into dist/
python setup.py sdist bdist_wheel
chmod a+rw dist -R

upload_package:
upload_package: ## Upload the built dist/* artifacts to PyPI with twine
twine upload --verbose -r pypi dist/* -u${PYPI_USERNAME} -p${PYPI_PASSWORD}

clear_package_data:
clear_package_data: ## Remove build artifacts (build/, dist/, *.egg-info)
rm -rf build dist/* ondewo-client-utils.egg-info

ondewo_release: spc clone_devops_accounts run_release_with_devops ## Release with credentials from devops-accounts repo
Expand All @@ -169,13 +185,13 @@ clone_devops_accounts: ## Clones devops-accounts repo
DEVOPS_ACCOUNT_GIT="ondewo-devops-accounts"
DEVOPS_ACCOUNT_DIR="./${DEVOPS_ACCOUNT_GIT}"

TEST:
TEST: ## Debug: echo the resolved GitHub/PyPI credentials and current release notes
@echo ${GITHUB_GH_TOKEN}
@echo ${PYPI_USERNAME}
@echo ${PYPI_PASSWORD}
@echo ${CURRENT_RELEASE_NOTES}

run_release_with_devops:
run_release_with_devops: ## Load credentials from the devops-accounts repo and run the full release
$(eval info:= $(shell cat ${DEVOPS_ACCOUNT_DIR}/account_github.env | grep GITHUB_GH & cat ${DEVOPS_ACCOUNT_DIR}/account_pypi.env | grep PYPI_USERNAME & cat ${DEVOPS_ACCOUNT_DIR}/account_pypi.env | grep PYPI_PASSWORD))
make release $(info)

Expand Down
16 changes: 16 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,22 @@

*****************

## Release ONDEWO CLIENT UTILS PYTHON 3.2.0

### New Features

* Added a test suite with 100% code coverage
* Added a GitHub Actions workflow that runs the tests on every push and pull request

### Improvements

* Serialize the gRPC service config and default channel options once at import instead of on every service construction to reduce client construction latency
* Enabled gRPC keepalive to keep long-lived streaming RPCs warm and detect half-open connections
* Updated GitHub Actions to their Node 24 releases to remove the Node 20 deprecation warnings
* Documented all Makefile targets and added a CLAUDE.md with repository and development guidance

*****************

## Release ONDEWO CLIENT UTILS PYTHON 3.1.0

### Improvements
Expand Down
Loading
Loading