Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions .github/workflows/build-glean-sdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# SPDX-FileCopyrightText: 2026 The RISE Project
# SPDX-License-Identifier: MIT
---
# This workflow is based on upstream's own `make build-python-wheel` (a plain
# `maturin build`, see Makefile) as driven by the `pypi-linux-release`/
# `pypi-linux-additional-releases` jobs of:
# https://github.com/mozilla/glean/blob/v70.0.0/.circleci/config.yml
name: Build glean-sdk wheels (riscv64)

on:
workflow_dispatch:
inputs:
version:
description: 'Version glob to (re)build; empty builds every version of docs/packages/glean-sdk.yaml not released yet'
required: false
default: ''
pull_request:
branches: [main]
paths:
- '.github/workflows/build-glean-sdk.yml'
- 'docs/packages/glean-sdk.yaml'
- 'patches/glean-sdk/**'
push:
branches: [main]
paths:
- '.github/workflows/build-glean-sdk.yml'
- 'docs/packages/glean-sdk.yaml'
- 'patches/glean-sdk/**'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions:
contents: read # to fetch code (actions/checkout)

env:
MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64

jobs:
setup:
uses: $/.github/workflows/_setup.yml
with:
package: glean-sdk
version: ${{ inputs.version }}

build_wheels:
needs: [setup]
if: needs.setup.outputs.versions != '[]'
strategy:
fail-fast: false
matrix:
version: ${{ fromJSON(needs.setup.outputs.versions) }}
name: Build glean-sdk ${{ matrix.version }} py3-none-manylinux_riscv64
runs-on: ubuntu-24.04-riscv
timeout-minutes: 90

env:
GLEAN_SDK_VERSION: ${{ matrix.version }}

steps:
- name: Checkout glean v${{ env.GLEAN_SDK_VERSION }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: mozilla/glean
ref: v${{ env.GLEAN_SDK_VERSION }}
persist-credentials: false

- name: Checkout python-wheels
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
path: python-wheels
persist-credentials: false

- name: Patch glean source
run: git apply python-wheels/patches/glean-sdk/${{ env.GLEAN_SDK_VERSION }}/*.patch

- name: Build wheel
uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074 # v4.2.0
with:
output-dir: wheelhouse/
# bindings = "uniffi" generates ctypes bindings around a plain
# cdylib, so the wheel is py3-none (no CPython ABI); one build
# serves every interpreter.
only: cp312-manylinux_riscv64
env:
CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}
# glean-sdk ships no [tool.cibuildwheel], so the Rust toolchain
# its maturin backend needs is installed in-container.
CIBW_BEFORE_ALL_LINUX: >-
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
CIBW_ENVIRONMENT_LINUX: PATH="$PATH:$HOME/.cargo/bin"
CIBW_TEST_REQUIRES: pytest pytest-localserver semver jsonschema MarkupSafe
CIBW_TEST_SOURCES: glean-core/python/tests glean.1.schema.json
CIBW_TEST_COMMAND: >-
python -c "import glean._uniffi as m, os;
so = os.path.join(os.path.dirname(m.__file__), 'libglean_ffi.so');
assert os.path.isfile(so), so" &&
python -m pytest -v glean-core/python/tests

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: glean-sdk-${{ env.GLEAN_SDK_VERSION }}-py3-none-manylinux_riscv64
path: wheelhouse/*.whl
if-no-files-found: error

publish:
name: Publish glean-sdk ${{ matrix.version }}
needs: [setup, build_wheels]
if: needs.setup.outputs.versions != '[]'
strategy:
fail-fast: false
matrix:
version: ${{ fromJSON(needs.setup.outputs.versions) }}
permissions:
contents: write
pull-requests: write
uses: $/.github/workflows/_publish-wheel.yml
secrets:
app-private-key: ${{ secrets.RISEPROJECT_APP_PRIVATE_KEY }}
with:
artifact-pattern: glean-sdk-${{ matrix.version }}-*-manylinux_riscv64
6 changes: 6 additions & 0 deletions docs/packages/glean-sdk.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package-name: glean-sdk
source-code: https://github.com/mozilla/glean
license: MPL-2.0
versions:
- version: 70.0.0
patched: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Ludovic Henry <git@ludovic.dev>
Date: Thu, 22 Oct 2026 00:00:00 +0000
Subject: [PATCH] tests: raise the ping upload wait timeouts

The Python SDK uploads pings from a separate process: every upload batch
spawns a fresh `sys.executable _process_dispatcher_helper.py`, which has to
start an interpreter, import glean and load libglean_ffi.so before the test
uploader writes its info.txt (or the test HTTP server sees a request).

Helpers.wait_for_ping() and wait_for_requests() give that only 2 seconds.
On the riscv64 runner that cold start alone regularly takes longer, so 12
tests fail with "No ping appeared ... within 2 seconds" / "Expected 1
requests within 2 seconds. Got 0" while tests exercising the very same
upload path (test_sending_deletion_ping_if_disabled_outside_of_run,
test_ping_collection_must_happen_after_currently_scheduled_metrics_recordings,
...) pass in the same run.

Raise both defaults to 30 seconds, matching wait_for_server() in the same
conftest. Both helpers poll and return as soon as the ping/request lands, so
faster hardware pays nothing. The one negative check that passes an
explicit timeout=1 (test_uploader_capabilities_in_empty_ping) is untouched.

Upstream-Status: To upstream [fixed timeout sized for fast hardware, not riscv64-specific; not yet submitted]
---
glean-core/python/tests/conftest.py | 2 +-
glean-core/python/tests/test_glean.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/glean-core/python/tests/conftest.py b/glean-core/python/tests/conftest.py
--- a/glean-core/python/tests/conftest.py
+++ b/glean-core/python/tests/conftest.py
@@ -93,7 +93,7 @@

class Helpers:
@staticmethod
- def wait_for_ping(path, timeout=2) -> (str, str):
+ def wait_for_ping(path, timeout=30) -> (str, str):
"""
Wait for a ping to appear in `path` for at most `timeout` seconds.

diff --git a/glean-core/python/tests/test_glean.py b/glean-core/python/tests/test_glean.py
--- a/glean-core/python/tests/test_glean.py
+++ b/glean-core/python/tests/test_glean.py
@@ -39,7 +39,7 @@
ROOT = Path(__file__).parent


-def wait_for_requests(server, n=1, timeout=2):
+def wait_for_requests(server, n=1, timeout=30):
"""
Wait for `n` requests to be received by the server.

Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Ludovic Henry <git@ludovic.dev>
Date: Fri, 23 Oct 2026 00:00:00 +0000
Subject: [PATCH] tests: wait for the upload process in test_client_activity_api

test_client_activity_api submits a baseline ping, waits for the uploader
to record it, sleeps 0.1 seconds and submits the next baseline ping.

The upload process only uploads the pings it finds in pending_pings/ when
it starts, and ProcessDispatcher.dispatch() does not start a new one while
the previous one is still running. On the riscv64 runner the upload
process is still winding down (interpreter teardown) 0.1 seconds after it
uploaded the first ping, so the trigger for the "inactive" baseline ping is
dropped, the ping is never uploaded and the test fails with "No ping
appeared ... within 30 seconds" whatever the timeout.

Wait for the upload process to exit instead, like test_network.py does
with ProcessDispatcher._wait_for_last_process(), so the next trigger
always starts a fresh upload process.

Upstream-Status: To upstream [race in the test exposed by slow hardware, not riscv64-specific; not yet submitted]
---
glean-core/python/tests/test_glean.py | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/glean-core/python/tests/test_glean.py b/glean-core/python/tests/test_glean.py
--- a/glean-core/python/tests/test_glean.py
+++ b/glean-core/python/tests/test_glean.py
@@ -32,6 +32,7 @@ from glean.metrics import (
StringMetricType,
)
from glean.net import PingUploadWorker
+from glean._process_dispatcher import ProcessDispatcher
from glean.testing import _RecordingUploader
from glean._uniffi import glean_set_test_mode

@@ -850,9 +851,9 @@ def test_client_activity_api(tmpdir, monkeypatch, helpers):
# It's an empty ping.
assert "metrics" not in payload

- # The upload process is fast, but not fast enough to communicate its status.
- # We give it just a blink of an eye to wind down.
- time.sleep(0.1)
+ # The upload process only sees pings already on disk when it starts, and
+ # a new one isn't spawned while it is still running. Wait for it to exit.
+ ProcessDispatcher._wait_for_last_process()
# Making it inactive
Glean.handle_client_inactive()

@@ -861,9 +862,9 @@ def test_client_activity_api(tmpdir, monkeypatch, helpers):
assert payload["ping_info"]["reason"] == "inactive"
assert "glean.baseline.duration" in payload["metrics"]["timespan"]

- # The upload process is fast, but not fast enough to communicate its status.
- # We give it just a blink of an eye to wind down.
- time.sleep(0.1)
+ # The upload process only sees pings already on disk when it starts, and
+ # a new one isn't spawned while it is still running. Wait for it to exit.
+ ProcessDispatcher._wait_for_last_process()
# Once more active
Glean.handle_client_active()

Loading