Skip to content

[Refactor][CI] Publish the lake-builder image with the project's Docker Hub credentials instead of a manual tag #9209

Open

Description

@DoDiODev

What and why to refactor

Five CI jobs run inside mericodev/lake-builder:latest:
test.yml (unit-test), test-e2e.yml (mysql and postgres),
golangci-lint.yml and migration-script-lint.yml.

That image is out of date and, as far as I can tell, can no longer be
refreshed through the intended path:

Fact Source
mericodev/lake-builder:latest is v0.0.15, pushed on 2023-06-30, amd64 only Docker Hub API
It was built from the Dockerfile of that time: debian:bullseye, golang:1.20.4, libgit2 1.3.2, python:3.9-slim-bullseye git show f29d31451:devops/docker/lake-builder/Dockerfile
The only publishing path is build-builder.yml, triggered by a builder-* tag. The newest such tag is builder-v0.0.7 (2022), so v0.0.8–v0.0.15 were pushed outside this workflow git ls-remote --tags origin 'builder-*'
build-builder.yml logs in with DOCKER_REGISTRY_USERNAME / DOCKER_REGISTRY_PASSWORD and pushes to the company namespace mericodev; no run of it is visible in the Actions history workflow file, Actions API

The CI copes by re-installing the toolchain on every run
(install-libgit2.sh, install-go.sh, install-mockery.sh), so the jobs pass,
but:

Simply pushing builder-v0.0.16 may not help: it depends on credentials for a
non-ASF namespace whose validity I cannot check from outside.

Describe the solution you'd like

The repository already has working Docker Hub credentials:
build.yml logs in with DOCKERHUB_USER / DOCKERHUB_TOKEN and pushes to
${{ secrets.DOCKERHUB_OWNER }}/devlake on every push to main (last
successful runs on 2026-10-11). It already publishes auxiliary, non-release
tags there as well (devlake:amd64-builder, :base, :build-cache-*).

Proposal, in two small PRs:

  1. Publish — change build-builder.yml to
    • use the same secrets as build.yml;
    • run on push to main when devops/docker/lake-builder/** (or the
      workflow itself) changes, plus workflow_dispatch — no manual tag needed;
    • push to the ASF namespace, either as a tag in the existing repository
      (e.g. apache/devlake:lake-builder-<short-sha>, following the
      amd64-builder precedent) or to a new apache/devlake-builder
      repository, if you prefer that and it can be created.
  2. Consume — once the first image exists, point the five container:
    lines to the pinned new tag instead of mericodev/lake-builder:latest.

This also completes the idea of #5041 ("push the lake builder to … apache
dockerhub"), which was closed by the stale bot without a change.

I have validated the current Dockerfile on my fork: its CI builds the builder
image from devops/docker/lake-builder/Dockerfile and runs unit, lint and e2e
(MySQL, PostgreSQL) inside it — all green
(run, with the
changes from #9208).

Questions for the maintainers:

  • Is moving the builder to the project's Docker Hub credentials acceptable,
    and which target do you prefer: a tag in apache/devlake or a separate
    apache/devlake-builder?
  • Alternatively, if the mericodev credentials are still valid and you prefer
    the existing path: could someone push a builder-v0.0.16 tag on main?

I'm happy to open the PRs for option 1 once the direction is clear.

Related issues

Additional context

The latest policy discussed in #4805 (only update latest for formal
releases) is not affected: the proposal uses explicit, pinned tags for the CI
image and does not touch the release images.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions