Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 20 additions & 29 deletions .github/scripts/memory_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,32 +13,26 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Comments on a pull request with the memory that every implementation uses.
"""Comments on a pull request with the memory that the application uses.

The workflow of each implementation (.github/workflows/<implementation>.yml)
measures its application with shared/tools/measure.sh (Flutter:
flutter/tool/measure.py) and uploads the result as the artifact
memory-<implementation>, which holds one file, memory.txt, with the lines
The workflow of the implementation (.github/workflows/mygo.yml) measures the
application with shared/tools/measure.sh and uploads the result as the artifact
memory-mygo, which holds one file, memory.txt, with the lines
"memory (<point>): RSS <n> MB, PSS <n> MB in <n> processes" of the points home
and nodejs. .github/workflows/memory.yml runs this script when one of those
workflows ends. Once all of them have ended for the latest commit of the pull
request, the script writes one comment: a table like the one of docs/memory.md,
sorted by the PSS of the Home page, with the difference from the latest
measurement on the default branch next to the numbers. A later run replaces
that comment instead of adding another one.

An implementation that the pull request did not rebuild, because its workflow
only runs for changes in its own paths, shows the numbers of the default branch.
and nodejs. .github/workflows/memory.yml runs this script when that workflow
ends for the latest commit of a pull request, and the script writes one
comment: a table like the one of docs/memory.md, with the difference from the
latest measurement on the default branch next to the numbers. A later run
replaces that comment instead of adding another one.

Usage:

memory_report.py --run RUN_ID [--dry-run]
Reports on the pull request of the workflow run RUN_ID, if that was the
last of the implementation workflows to end for the commit.
Reports on the pull request of the workflow run RUN_ID.

memory_report.py --pull-request NUMBER [--dry-run]
Reports on that pull request now, with the implementations that are still
running marked as such.
Reports on that pull request now, with the implementation still running
marked as such.

--dry-run prints the comment instead of writing it. The implementations come
from --implementations or MEMORY_IMPLEMENTATIONS (names separated by spaces).
Expand Down Expand Up @@ -73,16 +67,13 @@
MAX_ARTIFACT_BYTES = 64 * 1024
MAX_MEMORY_FILE_BYTES = 4 * 1024

# The same code does not use the same memory in every CI run. Comparing every
# two of the 107 runs on 2026-10-18 and 19 that shared their source with another
# one (30 groups of 2 to 8 runs of the ten implementations; see docs/memory.md),
# the Home and the Versions PSS never differed by more than 5 MB and 5 % of the
# first number for most of the implementations, and never by more than the
# percentages below for the four others. Only a larger difference counts as a
# change.
# The same code does not use the same memory in every CI run. Runs of identical
# code (see docs/memory.md) never differed by more than 5 MB and 5 % of the
# first number, for the Home and the Versions PSS alike. Only a larger
# difference counts as a change.
MIN_CHANGE_MB = 5
DEFAULT_TOLERANCE = 0.05
TOLERANCES = {"compose": 0.08, "flet": 0.08, "gio": 0.12, "flutter": 0.15}
TOLERANCES: dict[str, float] = {}

LINE = re.compile(r"memory \((home|nodejs)\): RSS ([0-9]{1,6}) MB, PSS ([0-9]{1,6}) MB in ([0-9]{1,4}) processes")
IMPLEMENTATION = re.compile(r"[a-z0-9][a-z0-9-]*")
Expand Down Expand Up @@ -331,7 +322,7 @@ def render(rows: list[Row], *, head_sha: str) -> str:
f"[{row.name}]({row.baseline_url}) (`{row.baseline_sha[:7]}`)" for row in rows if row.baseline_url
]
details = [
"- Measured by `shared/tools/measure.sh` (Flutter: `flutter/tool/measure.py`) in the workflow of each "
"- Measured by `shared/tools/measure.sh` in the workflow of the "
"implementation, on a GitHub-hosted runner with Xvfb and no GPU. These are not the numbers of "
"`docs/memory.md`, which come from another machine: compare the rows of one comment.",
"- " + explanation(rows),
Expand Down Expand Up @@ -643,7 +634,7 @@ def main(argv: list[str] | None = None) -> int:
parser.add_argument(
"--implementations",
default=os.environ.get("MEMORY_IMPLEMENTATIONS", ""),
help="the implementations, such as 'mygo tauri', separated by spaces",
help="the implementation, such as 'mygo', separated by spaces",
)
parser.add_argument("--docs", type=Path, default=Path("docs/memory.md"), help="the page with the Technology column")
parser.add_argument("--dry-run", action="store_true", help="print the comment instead of writing it")
Expand All @@ -656,7 +647,7 @@ def main(argv: list[str] | None = None) -> int:
if not REPOSITORY.fullmatch(arguments.repository):
parser.error("--repository (or GITHUB_REPOSITORY) must be OWNER/NAME")
if not implementations or not all(IMPLEMENTATION.fullmatch(name) for name in implementations):
parser.error("--implementations (or MEMORY_IMPLEMENTATIONS) must list names such as 'mygo tauri'")
parser.error("--implementations (or MEMORY_IMPLEMENTATIONS) must list names such as 'mygo'")
technologies = {}
if arguments.docs.is_file():
technologies = read_technologies(arguments.docs.read_text(encoding="utf-8"))
Expand Down
20 changes: 6 additions & 14 deletions .github/scripts/test_memory_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,18 +266,12 @@ def test_a_small_implementation_is_judged_by_the_absolute_floor(self):
self.assertTrue(report.exceeds_noise("mygo", 42, 36))
self.assertFalse(report.exceeds_noise("mygo", 42, 37))

def test_a_large_implementation_is_judged_by_the_percentage(self):
self.assertFalse(report.exceeds_noise("tauri", 300, 315))
self.assertTrue(report.exceeds_noise("tauri", 300, 316))

def test_the_noisy_implementations_get_more_room(self):
self.assertFalse(report.exceeds_noise("flutter", 350, 350 + 52))
self.assertTrue(report.exceeds_noise("flutter", 350, 350 + 53))
self.assertFalse(report.exceeds_noise("gio", 200, 224))
self.assertTrue(report.exceeds_noise("gio", 200, 225))
def test_a_large_measurement_is_judged_by_the_percentage(self):
self.assertFalse(report.exceeds_noise("mygo", 300, 315))
self.assertTrue(report.exceeds_noise("mygo", 300, 316))

def test_no_difference_is_no_change(self):
for implementation in ("mygo", "gio", "unknown"):
for implementation in ("mygo", "unknown"):
self.assertFalse(report.exceeds_noise(implementation, 100, 100))


Expand Down Expand Up @@ -438,12 +432,10 @@ def test_links_the_runs(self):

def test_explains_the_noise_that_it_allows(self):
rows = [
Row("flutter", "Flutter", "", "measured", memory(300, 310), memory(300, 310)),
Row("gio", "Gio", "", "measured", memory(200, 210), memory(200, 210)),
Row("mygo", "mygo", "", "measured", memory(300, 310), memory(300, 310)),
]
text = self.render(rows)
self.assertIn("more than 5 MB and more than 5% of the number on main", text)
self.assertIn("12% for Gio, 15% for Flutter, which vary more", text)
self.assertIn("more than 5 MB and more than 5% of the number on main.", text)

def test_is_ascii(self):
self.render(self.rows()).encode("ascii")
Expand Down
37 changes: 18 additions & 19 deletions .github/workflows/memory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,29 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# The memory of the implementations in a pull request. The workflow of each
# implementation measures its application and uploads the result as the
# artifact memory-<implementation>. When the last of those workflows has ended
# for the latest commit of a pull request, this one writes a single comment on
# it: a table like the one of docs/memory.md with the difference from the latest
# numbers of main. Later runs replace that comment instead of adding another one
# The memory of the implementation in a pull request. The workflow of the
# mygo implementation measures the application and uploads the result as the
# artifact memory-mygo. When it has ended for the latest commit of a pull
# request, this workflow writes one comment on it: the memory with the
# difference from the latest numbers of main. Later runs replace that comment
# instead of adding another one
# (.github/scripts/memory_report.py; docs/memory.md#memory-in-pull-requests).
#
# It runs after the workflows of the pull request, in the context of the default
# It runs after the workflow of the pull request, in the context of the default
# branch, so that it can write to pull requests of forks, which a pull_request
# workflow cannot. For the same reason it never runs code of the pull request: it
# checks out main, and the artifacts it downloads are data, of which the script
# only reads numbers.
# workflow cannot. For the same reason it never runs code of the pull request:
# it checks out main, and the artifacts it downloads are data, of which the
# script only reads numbers.
#
# Run it by hand to write the report of a pull request again:
# gh workflow run memory.yml -f pull_request=<number>
name: Memory

on:
workflow_run:
# The workflows that measure an implementation, by name. Keep this in sync
# with MEMORY_IMPLEMENTATIONS below, which has their file names.
workflows: [Capacitor, Compose, Dioxus, Flet, Flutter, Gio, mygo, Tauri, Uno, Wails]
# The workflow that measures the implementation, by name. Keep this in sync
# with MEMORY_IMPLEMENTATIONS below, which has its file name.
workflows: [mygo]
types: [completed]
workflow_dispatch:
inputs:
Expand All @@ -49,16 +49,15 @@ permissions:
pull-requests: write

concurrency:
# The workflows of a commit end one after another, and each starts a report.
# They run one at a time, so that two of them cannot both write the comment,
# and the last one to start is the one that finds all of them ended.
# The run of a commit ends and then starts a report. Reports run one at a
# time, so that two of them cannot both write the comment.
group: memory-${{ github.event.workflow_run.head_sha || format('pull-request-{0}', inputs.pull_request) }}
cancel-in-progress: false

env:
# The file names of the workflows of the implementations (without .yml), which
# are also the names of their directories and of their artifacts.
MEMORY_IMPLEMENTATIONS: capacitor compose dioxus flet flutter gio mygo tauri uno wails
# The file name of the workflow of the implementation (without .yml), which
# is also the name of its directory and of its artifact.
MEMORY_IMPLEMENTATIONS: mygo

jobs:
report:
Expand Down
Loading