From f7dadad3c952f4bd6f6abf302053af0418079bd4 Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 14:48:53 +0200 Subject: [PATCH 1/6] doc: link the GitHub repository, drop the EMISO mention The agency user space needed to run capsules is the s3c-* tool set; the EMISO engine belongs to MICOFE and is not required, so it no longer appears in the capsule documentation. The introduction now points at the public GitHub repository. --- doc/source/capsules.rst | 2 +- doc/source/introduction.rst | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/source/capsules.rst b/doc/source/capsules.rst index c870c6e4c..2675591a2 100644 --- a/doc/source/capsules.rst +++ b/doc/source/capsules.rst @@ -23,7 +23,7 @@ The capsule model needs a **Linux** agency: Linux owns the devices and provides the backend drivers and the higher-level services that capsules talk to — the backend half of the frontend/backend split, the vbstore server and the capsule-management user space (``s3c-inject``, ``s3c-list``, ``s3c-save`` / -``s3c-restore``, the EMISO engine). +``s3c-restore``). The so3 **build system can fetch and build that agency itself**, the same way it fetches AVZ, U-Boot and QEMU — it need not be built out of tree. The ``linux`` diff --git a/doc/source/introduction.rst b/doc/source/introduction.rst index 39fc3a1fa..3499c5f3d 100644 --- a/doc/source/introduction.rst +++ b/doc/source/introduction.rst @@ -12,6 +12,8 @@ SO3 is the result of several years of research and development at the `HEIG-VD `__, in the field of embedded operating systems and execution environments for ARM 32/64-bit multicore systems. SO3 was publicly released in early 2020 (see also the `HEIG-VD newsletter `__). +The source code is hosted on GitHub: +`smartobjectoriented/so3 `__. `Prof. Daniel Rossier `__ initiated the development of an operating system in 2013, in the context of a Bachelor lecture focusing on the port of operating @@ -127,6 +129,7 @@ on what is selected. SO3 is therefore an excellent environment to experiment: trying out a kernel function, a processor feature or a compilation trick is quick and cheap. +.. _so3_github: https://github.com/smartobjectoriented/so3 .. _REDS: http://www.reds.ch .. _HEIG-VD: http://www.heig-vd.ch .. _heig-vd_news: https://heig-vd.ch/ From e6f43fe71ffb5bf3ea19e3f244a685ea980c2c67 Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 14:55:53 +0200 Subject: [PATCH 2/6] doc: derive the documentation version from the release tag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit conf.py carried a hand-written 6.2.0 that had drifted two patch releases behind. It now reuses so3version.sh, the same helper the boot banner uses, so the version follows the release tag with no manual bump; the helper's fallback constant covers the shallow CI checkout that publishes the pages. The build-system section no longer spells out the current recipe version either, and the release checklist gains the one version string that does need a manual bump — the SO3_KERNEL_VERSION_FALLBACK safety net, itself still on 6.2.3. --- doc/source/build_system.rst | 10 ++++++---- doc/source/conf.py | 26 +++++++++++++++++++++++--- doc/source/release_process.rst | 9 ++++++++- so3/so3/include/version.h | 2 +- 4 files changed, 38 insertions(+), 9 deletions(-) diff --git a/doc/source/build_system.rst b/doc/source/build_system.rst index f5d8019cd..15cb41c05 100644 --- a/doc/source/build_system.rst +++ b/doc/source/build_system.rst @@ -87,8 +87,10 @@ Meta-layers - base bitbake classes — notably ``patch.bbclass`` (the fetch/patch/``updiff`` machinery) and the privileged-helper plumbing. * - ``meta-so3`` - - the **SO3 kernel** recipe (``so3_6.2.0.bb``, built in tree) and the **AVZ** - hypervisor recipe (``avz_6.2.0.bb``). + - the **SO3 kernel** recipe (``so3_.bb``, built in tree) and the + **AVZ** hypervisor recipe (``avz_.bb``); both are named after the + current release and pinned by ``PREFERRED_VERSION_*`` (see + :ref:`release_process`). * - ``meta-usr`` - the **user space** (``usr-so3``, CMake + MUSL toolchain): a committed lvgl-free base, plus opt-in add-ons layered as patches via overrides — @@ -235,8 +237,8 @@ rebuilding. The SO3 kernel recipe ===================== -``so3_6.2.0.bb`` configures and builds the kernel straight from ``so3/so3``; the -mechanics below are the still-familiar Kbuild ones. +``so3_.bb`` configures and builds the kernel straight from ``so3/so3``; +the mechanics below are the still-familiar Kbuild ones. Configuration (Kconfig) ----------------------- diff --git a/doc/source/conf.py b/doc/source/conf.py index 3a42e9e7e..e16baf906 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -12,6 +12,7 @@ # serve to show the default. import os +import subprocess import sys import sphinx sys.path.insert(0, os.path.abspath('.')) @@ -66,11 +67,30 @@ # |version| and |release|, also used in various other places throughout the # built documents. # -# The short X.Y version. -version = u'6.2.0' +# It is derived from the git release tag by the very same helper the kernel boot +# banner uses (so3/so3/scripts/so3version.sh), so the documentation never has to +# be bumped by hand at release time. The helper falls back to the +# SO3_KERNEL_VERSION_FALLBACK constant of so3/so3/include/version.h when the tree +# carries no git metadata — which is the case for the shallow CI checkout that +# publishes these pages. + +def _so3_version(): + srctree = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', 'so3', 'so3')) + helper = os.path.join(srctree, 'scripts', 'so3version.sh') + + try: + out = subprocess.check_output(['sh', helper, srctree], universal_newlines=True) + except (OSError, subprocess.CalledProcessError): + return 'unknown' + + return out.strip() or 'unknown' + # The full version, including alpha/beta/rc tags. -release = u'6.2.0' +release = _so3_version() + +# The short X.Y version. +version = '.'.join(release.split('.')[:2]) # The language for content autogenerated by Sphinx. Refer to documentation diff --git a/doc/source/release_process.rst b/doc/source/release_process.rst index 3315387c2..722644d14 100644 --- a/doc/source/release_process.rst +++ b/doc/source/release_process.rst @@ -139,7 +139,14 @@ release (they drift silently otherwise): ``avz_X.Y.Z.bb`` recipes (under ``build/meta-so3/recipes-so3/``) to the new version and update the ``PREFERRED_VERSION_so3`` / ``PREFERRED_VERSION_avz`` entries in ``build/conf/local.conf`` (see the - ``v6.2.3`` bump for a template). + ``v6.2.3`` bump for a template); +* the boot-banner safety net ``SO3_KERNEL_VERSION_FALLBACK`` + (``so3/so3/include/version.h``), used only when the build tree carries no git + metadata (a bitbake ``WORKDIR`` copy, a tarball export). + +Two version strings need **no** action: the boot banner itself and the +documentation version both derive from the release tag at build time +(``so3/so3/scripts/so3version.sh``, reused by ``doc/source/conf.py``). Rules of thumb ************** diff --git a/so3/so3/include/version.h b/so3/so3/include/version.h index 9dcfa7794..3051eacd4 100644 --- a/so3/so3/include/version.h +++ b/so3/so3/include/version.h @@ -35,7 +35,7 @@ * (see scripts/so3version.sh and include/generated/autoversion.h). Keep this in * sync with the current release tag as a safety net. */ -#define SO3_KERNEL_VERSION_FALLBACK "6.2.3" +#define SO3_KERNEL_VERSION_FALLBACK "6.2.4" /* Release version resolved at build time (git tag -> base version). */ #include From bb51ea5faecb8b6f03c512724b12590a47e7ea30 Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 14:56:40 +0200 Subject: [PATCH 3/6] doc: quote kernel paths relative to the so3/so3 tree The architecture chapter drew the kernel tree as "so3/" although the sources live one level deeper, in so3/so3/, which left every subsystem path in the documentation one component short. The tree is now drawn at its real location (with the two directories it was missing, apps/ and include/) and states the convention the other chapters already follow: kernel paths are relative to so3/so3. The few paths that mixed both conventions are aligned on it: the device tree in the display chapter and the AVZ/capsule sources, whose files sit in avz/kernel/ rather than directly in avz/. --- doc/source/architecture.rst | 17 ++++++++++++----- doc/source/avz.rst | 2 +- doc/source/capsules.rst | 2 +- doc/source/display_input.rst | 2 +- 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index eb065b9ab..618b10202 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -21,9 +21,9 @@ filesystem, IPC, networking — plus a device-tree-driven device and driver mode Source tree =========== -The kernel source lives under ``so3/`` and is organised by subsystem:: +The kernel source lives under ``so3/so3/`` and is organised by subsystem:: - so3/ + so3/so3/ ├── arch/ # architecture-specific code (arm32, arm64) │ └── arm64/ # head/boot, exceptions, MMU, context switch, traps ├── kernel/ # processes, threads, scheduler, syscalls, time @@ -35,12 +35,19 @@ The kernel source lives under ``so3/`` and is organised by subsystem:: ├── dts/ # device trees (*.dts → *.dtb) ├── avz/ # the AVZ hypervisor (built with CONFIG_AVZ) ├── soo/ # the SOO framework / SO3 capsules (CONFIG_SOO) + ├── apps/ # optional kernel-space example apps (CONFIG_APP_*) + ├── include/ # kernel headers ├── configs/ # defconfig files └── lib/ # in-kernel helper libraries (libfdt, libroxml, …) -The user space lives under ``usr/`` and the surrounding tooling (bootloader, -emulator, root filesystem, deployment scripts) at the repository root — see -:ref:`build_system` and :ref:`user_space`. +.. note:: + + Kernel paths are quoted **relative to that tree** throughout this + documentation: ``arch/arm64/mmu.c`` means ``so3/so3/arch/arm64/mmu.c``. + +The user space lives under ``so3/usr/`` and the surrounding tooling (build +system, bootloader, emulator, root filesystem, deployment scripts) at the +repository root — see :ref:`build_system` and :ref:`user_space`. Exception levels ================ diff --git a/doc/source/avz.rst b/doc/source/avz.rst index 3c33fb23d..dd90ff2ca 100644 --- a/doc/source/avz.rst +++ b/doc/source/avz.rst @@ -27,7 +27,7 @@ beside it. AVZ: domains isolated by stage-2 tables, and the EL2 services beneath them. -The code lives under ``so3/avz/`` (kernel, memory, scheduler, hypercalls, grant +The code lives under ``avz/`` (kernel, memory, scheduler, hypercalls, grant tables, capsule build/inject) together with the EL2-specific parts of ``arch/arm64`` (``head.S`` MMU setup, ``exception.S`` EL2 vectors, ``context.S`` stage-2 switch, ``cache.S`` EL2 TLB ops) and the virtual GIC in diff --git a/doc/source/capsules.rst b/doc/source/capsules.rst index 2675591a2..50cbb140a 100644 --- a/doc/source/capsules.rst +++ b/doc/source/capsules.rst @@ -55,7 +55,7 @@ hypervisor support for it: * the **vbus / vbstore** clients and the event-channel / grant-table glue (``soo/kernel/``); * the hypervisor-side capsule **build / inject / snapshot** code - (``so3/avz/`` — ``capsule_build.c``, ``injector.c``). + (``avz/kernel/`` — ``capsule_build.c``, ``injector.c``). A capsule-capable guest is produced by ``virt64_capsule_defconfig`` or ``rpi4_64_capsule_defconfig`` (enabling ``CONFIG_SOO``). The agency runs diff --git a/doc/source/display_input.rst b/doc/source/display_input.rst index 3825f7477..1364115a0 100644 --- a/doc/source/display_input.rst +++ b/doc/source/display_input.rst @@ -7,7 +7,7 @@ On the QEMU ``virt`` machine SO3 drives a small set of ARM PrimeCell devices for graphics and human input. These are **not** part of the upstream ``virt`` model; they are added by the SO3 QEMU patch (``build/meta-qemu/.../files/0001-qemu-8.2.2-r0/0001-virt.c.patch``) and described -to the kernel in the device tree (``so3/dts/virt64.dts`` / ``virt32.dts``). +to the kernel in the device tree (``so3/so3/dts/virt64.dts`` / ``virt32.dts``). .. figure:: img/so3_io.png :width: 100% From b187dc72d60ffc86277233c52920a52953d275ed Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 14:58:22 +0200 Subject: [PATCH 4/6] doc: correct the hypercall, driver, ITS and application inventories MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Several lists had fallen behind the code: * the AVZ chapter presented the capsule operations as being layered on the three generic hypercalls, whereas hypercalls.c dispatches them as commands of their own, declared in soo/uapi/soo.h and compiled in with CONFIG_SOO; * the driver-model table was missing the PL031 RTC — the very device the user-space chapter credits for its file timestamps — and the mydev reference driver; * the ITS table ignored the lvperf images used by the containers and the CI, the Verdin standalone variant and the rpi4_64 capsule, and the meta-bsp entry mentioned neither bsp-linux nor bsp-capsules; * the application table advertised hello-world as a C example while it, and six undocumented test programs, are C++ built from usr/src/tests, and it listed the LVGL demos unconditionally although only the :lvgl override adds them; * the networking chapter still described the tap-and-bridge setup that st.sh replaced with user-mode slirp networking. The AVZ hypervisor also no longer hosts "a single guest": it hosts the agency plus up to five capsules. --- doc/source/avz.rst | 16 ++++++++++++---- doc/source/build_system.rst | 15 ++++++++++----- doc/source/index.rst | 5 +++-- doc/source/kernel.rst | 5 +++++ doc/source/lvgl.rst | 4 +++- doc/source/lwip.rst | 9 ++++++--- doc/source/user_guide.rst | 6 +++++- doc/source/user_space.rst | 13 ++++++++++--- 8 files changed, 54 insertions(+), 19 deletions(-) diff --git a/doc/source/avz.rst b/doc/source/avz.rst index dd90ff2ca..53d26d05f 100644 --- a/doc/source/avz.rst +++ b/doc/source/avz.rst @@ -92,15 +92,23 @@ Hypercalls Guests call into AVZ with the ``hvc`` instruction, which traps to the EL2 synchronous handler (``el12_sync_handler`` in ``arch/arm64/exception.S``) and is -dispatched by ``avz/kernel/hypercalls.c``. The generic hypercalls -(``avz/include/avz/uapi/avz.h``) are: +dispatched by ``avz/kernel/hypercalls.c``. Every hypercall is one ``cmd`` value +in that single dispatcher. Three are always compiled in +(``avz/include/avz/uapi/avz.h``): * ``AVZ_EVENT_CHANNEL_OP`` — allocate / bind / send / close event channels; * ``AVZ_CONSOLE_IO_OP`` — console output for guests; * ``AVZ_DOMAIN_CONTROL_OP`` — domain control (pause / unpause a capsule, …). -The capsule-management operations (inject, kill, read/write snapshot) used by the -SOO framework are built on top of these — see :ref:`capsules`. +The rest are the SOO commands, declared in ``soo/include/soo/uapi/soo.h`` and +compiled in only with ``CONFIG_SOO``: the **grant-table** op +(``AVZ_GRANT_TABLE_OP``), domain description (``AVZ_GET_DOM_DESC``), capsule +lifecycle (``AVZ_INJECT_CAPSULE``, ``AVZ_START_CAPSULE``, ``AVZ_KILL_S3C``, +``AVZ_GET_S3C_STATE`` / ``AVZ_SET_S3C_STATE``), the snapshot primitives +(``AVZ_S3C_READ_SNAPSHOT`` / ``AVZ_S3C_WRITE_SNAPSHOT``), the direct-communication +events (``AVZ_DC_EVENT_SET``) and the virtual-framebuffer ops +(``AVZ_FBDEV_*``). They are *not* layered on top of the three generic ones — see +:ref:`capsules`. Domain scheduling ================= diff --git a/doc/source/build_system.rst b/doc/source/build_system.rst index 15cb41c05..c3823542a 100644 --- a/doc/source/build_system.rst +++ b/doc/source/build_system.rst @@ -97,7 +97,9 @@ Meta-layers ``:lvgl`` (LVGL + ``slv`` + demos) and ``:soo`` (capsule user space). * - ``meta-bsp`` - **board support**: ``bsp-so3`` assembles the FIT image (``do_itb``) and - writes the boot media (``do_deploy_boot``). + writes the boot media (``do_deploy_boot``); ``bsp-linux`` does the same for + the Linux agency and ``bsp-capsules`` for an agency running SO3 capsules + (see :ref:`capsules`). * - ``meta-uboot`` - the **U-Boot** bootloader (fetched + patched). * - ``meta-qemu`` @@ -305,10 +307,13 @@ paths — then assembles the ``.itb`` there with ``mkimage`` (there is no commit * - ``_linux_guest.its`` - **Linux agency guest ITB**: Linux kernel + guest DTB + initrd, loaded by AVZ (``meta-bsp/.../linux/files/its/``). - * - ``virt64_capsule.its`` - - a capsule image - * - ``virt32_so3.its`` / ``rpi4_64_so3.its`` - - the 32-bit / RPi4 standalone variants + * - ``_capsule.its`` + - a capsule image (``virt64_capsule``, ``rpi4_64_capsule``) + * - ``virt32_so3.its`` / ``rpi4_64_so3.its`` / ``verdin_imx8mp_so3.its`` + - the 32-bit / RPi4 / Verdin standalone variants + * - ``_lvperf.its`` + - the LVGL-benchmark image driven by the ``docker/`` lvperf containers and + the CI (``virt64_lvperf``, ``virt32_lvperf`` — see :ref:`lvgl`) ``do_deploy_boot`` writes the resulting ``.itb`` from ``/images/`` into the FAT (boot) partition of diff --git a/doc/source/index.rst b/doc/source/index.rst index d2992b8b2..3b7211f3b 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -56,8 +56,9 @@ particularly well suited to embedded systems. From a single code base it can be built in three ways: * as a **standalone OS** running directly on the hardware (EL1 on ARM64); -* as the **AVZ hypervisor** (*Agency VirtualiZer*) running at EL2, hosting a - single guest at EL1; +* as the **AVZ hypervisor** (*Agency VirtualiZer*) running at EL2, hosting the + agency guest at EL1 — and, with the SOO framework, up to five capsules beside + it; * as an **SO3 capsule** (S3C) — a lightweight guest on top of AVZ, as part of the **SOO** framework. diff --git a/doc/source/kernel.rst b/doc/source/kernel.rst index 6f355c454..d226fcc06 100644 --- a/doc/source/kernel.rst +++ b/doc/source/kernel.rst @@ -176,6 +176,9 @@ The tree contains drivers for the main device classes: - SD/MMC controller, in-memory RAM disk * - framebuffer (``fb/``) - PL111, ramfb, virtfb (used by LVGL) + * - real-time clock (``rtc/``) + - PL031 — provides the wall-clock time behind ``gettimeofday`` and the file + timestamps (see :ref:`user_space`) * - input (``input/``) - PL050 KMI keyboard; ``so3,absmouse`` absolute pointer; PL050 relative mouse (legacy, disabled in the dts). See :ref:`display_input`. @@ -183,6 +186,8 @@ The tree contains drivers for the main device classes: - smc911x (``smsc,smc911x``), wired to lwIP — optional (``CONFIG_NET``) * - i2c, rpisense (``i2c/``, ``rpisense/``) - I²C bus and Raspberry Pi Sense HAT + * - example (``mydev.c``) + - a minimal reference driver, exercised by the ``mydev_test`` application Interrupts and time =================== diff --git a/doc/source/lvgl.rst b/doc/source/lvgl.rst index 7d4d45d8d..9a22866cd 100644 --- a/doc/source/lvgl.rst +++ b/doc/source/lvgl.rst @@ -52,7 +52,9 @@ A typical application is just: Applications ============ -The LVGL applications are defined in ``so3/usr/src/CMakeLists.txt``: +The LVGL applications are added to ``so3/usr/src/CMakeLists.txt`` by the +``:lvgl`` override — the committed, lvgl-free ``CMakeLists.txt`` does not build +them (see :ref:`build_system`): .. flat-table:: :header-rows: 1 diff --git a/doc/source/lwip.rst b/doc/source/lwip.rst index 987a0811b..5a20de7ad 100644 --- a/doc/source/lwip.rst +++ b/doc/source/lwip.rst @@ -32,6 +32,9 @@ Trying it out ============= With ``CONFIG_NET`` enabled and a supported NIC, the ``ping`` application -exercises the stack end to end. Under QEMU the launch scripts attach a tap -network device (``scripts/qemu-ifup.sh`` / ``qemu-ifdown.sh``), so the guest can -reach the host network once the tap bridge is configured. +exercises the stack end to end. Under QEMU, ``st.sh`` attaches a **user-mode +(slirp)** network device — QEMU itself plays DHCP, DNS and NAT, and forwards host +port ``2222`` to the guest's port ``22`` — so nothing has to be set up on the host +and no ``sudo`` is needed. The trade-off is that the guest is NAT'd and not +reachable from the LAN. (``scripts/qemu-ifup.sh`` / ``qemu-ifdown.sh`` are +leftovers from the earlier ``tap``-and-bridge setup and are no longer used.) diff --git a/doc/source/user_guide.rst b/doc/source/user_guide.rst index 4fa697a2d..238c7cf39 100644 --- a/doc/source/user_guide.rst +++ b/doc/source/user_guide.rst @@ -125,7 +125,11 @@ Launch scripts Both read ``IB_PLATFORM`` and the selected ITS from ``build/conf/local.conf``, attach ``filesystem/sdcard.img.`` as a virtio block device, forward the guest SSH port (host ``2222`` → guest ``22``) and expose a GDB stub on -``tcp::1234`` (see :ref:`debugging`). The exception level is chosen automatically: +``tcp::1234`` (see :ref:`debugging`). Networking is QEMU's user-mode (slirp) +stack, so no ``tap`` device and no ``sudo`` are involved — the guest is NAT'd and +not visible on the LAN. When other QEMU instances are already running, the GDB +port is shifted by their number (1235, 1236, …) and printed at startup. The +exception level is chosen automatically: * a standalone ITS → ``-M virt`` (EL1); * an ``…avz…`` ITS → ``-M virt,virtualization=on`` (EL2 for the hypervisor); diff --git a/doc/source/user_space.rst b/doc/source/user_space.rst index dc033c2c6..da26e5efe 100644 --- a/doc/source/user_space.rst +++ b/doc/source/user_space.rst @@ -17,6 +17,10 @@ facilities (for example full ``pthreads``) are intentionally kept minimal. Applications are linked **statically** against MUSL, so each executable is self-contained. +**C++** applications are supported too: the programs under +``so3/usr/src/tests/`` are C++ and exercise classes, exceptions and dynamic +allocation on top of the same toolchain. + Build system (CMake) ==================== @@ -58,12 +62,15 @@ The standard applications in ``so3/usr/src/`` include: - ICMP ping (exercises the lwIP stack) * - ``time`` - simple timing utility - * - ``hello-world`` - - minimal example * - ``thread_example`` / ``logs_example`` / ``mydev_test`` - API and subsystem demonstrations + * - ``hello-world`` / ``class-test`` / ``stdlib-test`` / ``exception-test`` / + ``allocation-test`` / ``threads-test`` / ``files-test`` + - the **C++** example and test programs of ``so3/usr/src/tests/``, built by + default (CMake option ``WITH_TESTS``, ``ON``) * - ``lvgl_widgets`` / ``lvgl_demo`` / ``lvgl_perf`` / ``lvgl_benchmark`` - - LVGL graphical demos (framebuffer builds — see :ref:`lvgl`) + - LVGL graphical demos — present only when the ``:lvgl`` override adds them + (see :ref:`lvgl`) * - ``fb_test`` - minimal framebuffer test, straight to ``/dev/fb`` (see :ref:`display_input`) * - MicroPython From 6e02c905b8c58d70b15fff7e7b918471168e03ff Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 14:59:56 +0200 Subject: [PATCH 5/6] doc: tie the coding conventions to what the CI actually enforces MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The chapter never mentioned that the layout rules are machine-checked. It now opens on the enforcement: .clang-format as the reference (tabs, 8-column indent, 128-column limit), the style.yml workflow running clang-format 19, and check-format.sh to reproduce it locally. The vague "as much as a modern screen displays" line-length rule is replaced by the 128 columns actually configured. Three Linux leftovers go away: the Syslog-ng section, which has no counterpart in SO3 — replaced by SO3's own printk/lprintk and DBG conventions — the EXPORT_SYMBOL and kmalloc/GFP_KERNEL examples, rewritten with the primitives SO3 provides, and a gratuitous jab at a vendor. --- doc/source/coding_conventions.rst | 67 ++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/doc/source/coding_conventions.rst b/doc/source/coding_conventions.rst index a974cd0f7..186c937b3 100644 --- a/doc/source/coding_conventions.rst +++ b/doc/source/coding_conventions.rst @@ -7,6 +7,26 @@ "C" coding conventions (Cconv) ############################## +How the style is enforced +************************* + +The layout rules below are not merely advisory: they are checked mechanically. +``.clang-format`` at the repository root (itself derived from the Linux kernel +one) is the reference — ``UseTab: Always``, ``IndentWidth: 8``, ``TabWidth: 8`` +and ``ColumnLimit: 128`` — and the ``style.yml`` GitHub workflow runs +**clang-format 19** over the tracked sources of ``so3/so3`` and ``so3/usr`` on +every push and pull request. + +Run the very same check locally before submitting: + +.. code-block:: bash + + check-format.sh # list the files that need reformatting + check-format.sh --fix # reformat them in place + +Everything clang-format cannot judge — naming, function length, comments, error +handling, what belongs in a macro — is the subject of the rest of this chapter. + Indentation *********** @@ -75,11 +95,11 @@ Breaking long lines and strings Coding style is all about readability and maintainability using commonly available tools. -The limit on the length of lines should correspond to what a modern screen and -editor is reasonibly able to display before the reader has to scroll horizontally -(even if it is acceptable to scroll over a few characters). +The limit on the length of lines is **128 characters** — what a modern screen and +editor is reasonably able to display without scrolling horizontally. This is the +``ColumnLimit`` of ``.clang-format``, so the check rejects longer lines. -Statements too long will be broken into sensible chunks. +Statements too long will be broken into sensible chunks. Descendants are always substantially shorter than the parent and are placed substantially to the right. The same applies to function headers with a long argument list. However, avoid to break user-visible strings such as @@ -270,8 +290,7 @@ that counts the number of active users, you should call that Encoding the type of a function into the name (so-called Hungarian notation) is brain damaged - the compiler knows the types anyway and can -check those, and it only confuses the programmer. No wonder MicroSoft -makes buggy programs. +check those, and it only confuses the programmer. LOCAL variable names should be short, and to the point. If you have some random integer loop counter, it should probably be called ``i``. @@ -324,17 +343,16 @@ generally easily keep track of about 7 different things, anything more and it gets confused. You know you're brilliant, but maybe you'd like to understand what you did 2 weeks from now. -In source files, separate functions with one blank line. If the function is -exported, the **EXPORT** macro for it should follow immediately after the -closing function brace line. E.g.: +In source files, separate functions with one blank line. A function that is not +part of a subsystem's interface should be ``static``; the ones that are belong in +the subsystem header, next to the types they operate on. E.g.: .. code-block:: c - int system_is_up(void) + static bool system_is_up(void) { return system_state == SYSTEM_RUNNING; } - EXPORT_SYMBOL(system_is_up); In function prototypes, include parameter names with their data types. Although this is not required by the C language, it is preferred @@ -372,7 +390,7 @@ The rationale for using gotos is: int result = 0; char *buffer; - buffer = kmalloc(SIZE, GFP_KERNEL); + buffer = malloc(SIZE); if (!buffer) return -ENOMEM; @@ -385,7 +403,7 @@ The rationale for using gotos is: } ... out_free_buffer: - kfree(buffer); + free(buffer); return result; } @@ -394,8 +412,8 @@ A common type of bug to be aware of is ``one err bugs`` which look like this: .. code-block:: c err: - kfree(foo->bar); - kfree(foo); + free(foo->bar); + free(foo); return ret; The bug in this code is that on some exit paths ``foo`` is NULL. Normally the @@ -405,9 +423,9 @@ fix for this is to split it up into two error labels ``err_free_bar:`` and .. code-block:: c err_free_bar: - kfree(foo->bar); + free(foo->bar); err_free_foo: - kfree(foo); + free(foo); return ret; Ideally you should simulate errors to test all exit paths. @@ -541,14 +559,17 @@ Usually, messages do not have to be terminated with a period. Coming up with good debugging messages can be quite a challenge; and once you have them, they can be a huge help for remote troubleshooting. However debug message printing is handled differently than printing other non-debug -messages. +messages. -Syslog-ng -========= +In the kernel, regular messages go through ``printk()`` (``include/printk.h``). +Its low-level counterpart ``lprintk()`` writes straight to the serial port +without going through the console layer, which is what makes it usable very +early at boot, from an interrupt handler, or when the console itself is the +suspect — the assertion and ``BUG_ON()`` paths use it for that reason. -Syslog-ng enables logging messages in various forms and configurations. -It can be used to log message on the console and/or in files typically -stored in ``/var/log`` directory. +Debug traces are kept out of a normal build: define a ``DBG()``-style macro +guarded by a local ``#define DEBUG`` in the subsystem (see +``soo/include/soo/debug.h``) rather than leaving bare ``printk()`` calls behind. Function return values and names ******************************** From 965cf4f5f82529e8d932ee84587e1998c38c1e32 Mon Sep 17 00:00:00 2001 From: Daniel Rossier Date: Thu, 30 Jul 2026 15:01:40 +0200 Subject: [PATCH 6/6] doc: bring the JTAG chapter up to date and align the heading levels The JTAG chapter still assumed a 32-bit SO3: it told the reader to select the "arm" architecture in GDB, whereas the current Raspberry Pi 4 target is the 64-bit rpi4_64. Both cases are now spelled out, and the OpenOCD user-mode limitation is scoped to the 32-bit build it actually concerns, with a pointer to the Gerrit that has since moved rather than the dead zylin.com link. The config.txt reference follows the Raspberry Pi documentation to its new home. The J-Link pinout image was pulled from segger.com at build time, which makes an offline or sandboxed build depend on the network; it becomes a plain link. Finally, this chapter and the MicroPython one used their own heading levels instead of the ones every other chapter follows, and a few typos and trailing blanks are gone. --- doc/source/coding_conventions.rst | 18 ++-- doc/source/micropython.rst | 18 ++-- doc/source/so3_jtag_rpi4.rst | 147 +++++++++++++++--------------- 3 files changed, 94 insertions(+), 89 deletions(-) diff --git a/doc/source/coding_conventions.rst b/doc/source/coding_conventions.rst index 186c937b3..8b3d35154 100644 --- a/doc/source/coding_conventions.rst +++ b/doc/source/coding_conventions.rst @@ -83,7 +83,7 @@ Don't put multiple assignments on a single line either. Coding style is super simple. Avoid tricky expressions. Outside of comments, documentation and except in some files where it is required -like Kconfig Linux kernel, spaces are never used for indentation, +like Kconfig Linux kernel, spaces are never used for indentation, and the above example is deliberately broken. Get a decent editor and don't leave whitespace at the end of lines. @@ -203,7 +203,7 @@ Also, prefer using braces when a loop contains more than a single simple stateme Spaces ====== -Use a space after (most) keywords. The notable exceptions are sizeof, typeof, alignof, +Use a space after (most) keywords. The notable exceptions are sizeof, typeof, alignof, and __attribute__, which look somewhat like functions (and are usually used with parentheses in Linux, although they are not required in the language, as in: ``sizeof info`` after ``struct fileinfo info;`` is declared). @@ -311,9 +311,9 @@ definition of a type or to define a struct and having a more readable type rather than *struct sensor*. Put a ``_t`` as suffix of a type definition. For example, ``sensor_t``. -Regarding the platform-dependent definition, it helps to define clear integer types, +Regarding the platform-dependent definition, it helps to define clear integer types, where the abstraction **helps** avoid confusion whether it is ``int`` or ``long``. -u8/u16/u32 are perfectly fine typedefs +u8/u16/u32 are perfectly fine typedefs NEVER use a typedef to hide a pointer except for the pointer to a function. For example: @@ -636,19 +636,19 @@ the next instruction in the assembly output: Conditional Compilation *********************** -Using #if or #ifdef block should always have a comment on the #else or #endif +Using #if or #ifdef block should always have a comment on the #else or #endif statement with the name of the condition, like this: .. code-block:: c #ifdef CONFIG_SOMETHING - + ... - + #else /* CONFIG_SOMETHING */ - + ... - + #endif /* !CONFIG_SOMETHING */ It will greatly help the reading of the code. diff --git a/doc/source/micropython.rst b/doc/source/micropython.rst index 365c71204..3a359ab7a 100644 --- a/doc/source/micropython.rst +++ b/doc/source/micropython.rst @@ -1,37 +1,37 @@ .. _micropython: MicroPython -=========== +########### -Micropython is an implementation of the Python 3 language designed to run on embedded platforms. -It provides a small subset of Python's standard library. +Micropython is an implementation of the Python 3 language designed to run on embedded platforms. +It provides a small subset of Python's standard library. More information is available on the `official site `__. Integration of MicroPython in SO3 ---------------------------------- +================================= -A minimal port of micropython is available in SO3. This version only supports the basic features +A minimal port of micropython is available in SO3. This version only supports the basic features of the language and some basic modules (list below) Using Micropython in the emulated environment ---------------------------------------------- +=============================================- .. note:: Micropython currently works only for the virt64 platform - + Micropython is built along with the rest of the user space (the ``usr-so3`` recipe — ``build.sh -x usr-so3``; see :ref:`user_space`). Its CMake rules drive the Makefile in ``so3/usr/src/micropython/ports/soo`` and the resulting ``firmware.elf`` is renamed ``uPython.elf`` and gathered with the other applications for the root filesystem. - + Once inside SO3, MicroPython can be launched like any other program:: so3% uPython -Launching the program will open an interactive interpreter from which code may be tested. +Launching the program will open an interactive interpreter from which code may be tested. There is currently no way to execute a python script Available `Micropython libraries `_ (modules): diff --git a/doc/source/so3_jtag_rpi4.rst b/doc/source/so3_jtag_rpi4.rst index ed423687c..c03f7e4b3 100644 --- a/doc/source/so3_jtag_rpi4.rst +++ b/doc/source/so3_jtag_rpi4.rst @@ -1,13 +1,13 @@ .. _so3_jtag_rpi4: Debugging SO3 with JTAG on RPi4 -=============================== +############################### -This pages gives instructions on how to debug SO3 or SO3 applications on +This page gives instructions on how to debug SO3 or SO3 applications on the Raspberry Pi 4 through the JTAG interface. Requirements ------------- +============ - Raspberry Pi 4 @@ -20,32 +20,32 @@ Requirements - GDB-multiarch JTAG Probe ----------- +========== To enable JTAG on the Raspberry Pi 4 add ``enable_jtag_gpio=1`` in the ``config.txt`` file. This will select Alt4 mode for GPIO pins 22-27, and set up some internal SoC connections, thus enabling the JTAG interface for the ARM CPU. It works on all models of Raspberry Pi. (See `Raspberry Pi -Documentation `__) +Documentation `__) J-Link Probe -~~~~~~~~~~~~ +------------ The `SEGGER J-Link EDU probe `__ -has been used for this project. The pinout for the 20-pin interface can be found here : -https://www.segger.com/products/debug-probes/j-link/technology/interface-description/ - -|image0| +has been used for this project. The pinout of the 20-pin interface is documented +on SEGGER's `interface description +`__ +page. Required pins are VTref, nTRST, TDI, TMS, TCK, RTCK, TDO and GND. Wiring -~~~~~~ +------ The JTAG pins should be connected to the corresponding pins on the Raspberry Pi (GPIO 22-27 in Alt4 mode). A nice representation of -Raspberry Pi header pinout can be found here : +Raspberry Pi header pinout can be found here: http://www.panu.it/raspberry/ From the ALT4 column of the table we can find which pin goes to which @@ -60,20 +60,20 @@ The extra wires at the bottom (red, green, blue) are for the UART serial port. OpenOCD -------- +======= OpenOCD is Free Open On-Chip Debugger software for In-System Programming -and Boundary-Scan Testing. Website : http://openocd.org/ +and Boundary-Scan Testing. Website: http://openocd.org/ -Prebuilt binaries are available from : https://xpack.github.io/openocd/ +Prebuilt binaries are available from: https://xpack.github.io/openocd/ -Source code is available at : +Source code is available at: https://sourceforge.net/p/openocd/code/ci/master/tree/ Building OpenOCD -~~~~~~~~~~~~~~~~ +---------------- -.. code-block:: bash +.. code-block:: bash git clone https://git.code.sf.net/p/openocd/code openocd-code cd openocd-code @@ -84,48 +84,48 @@ Building OpenOCD OpenOCD can be launched from the ``./src/`` directory e.g., -.. code-block:: bash +.. code-block:: bash ./src/openocd -v Configuration file for the Raspberry Pi 4 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +----------------------------------------- Create a configuration file (e.g., ``rpi4.cfg``) with the following -contents : +contents: .. code-block:: console adapter driver jlink - + set _CHIPNAME bcm2711 set _DAP_TAPID 0x4ba00477 - + adapter speed 1000 - + transport select jtag reset_config trst_and_srst - + telnet_port 4444 - + # create tap jtag newtap auto0 tap -irlen 4 -expected-id $_DAP_TAPID - + # create dap dap create auto0.dap -chain-position auto0.tap - + set CTIBASE {0x80420000 0x80520000 0x80620000 0x80720000} set DBGBASE {0x80410000 0x80510000 0x80610000 0x80710000} - + set _cores 4 - + set _TARGETNAME $_CHIPNAME.a72 set _CTINAME $_CHIPNAME.cti set _smp_command "" - + for {set _core 0} {$_core < $_cores} { incr _core} { cti create $_CTINAME.$_core -dap auto0.dap -ap-num 0 -ctibase [lindex $CTIBASE $_core] - + set _command "target create ${_TARGETNAME}.$_core aarch64 \ -dap auto0.dap -dbgbase [lindex $DBGBASE $_core] \ -coreid $_core -cti $_CTINAME.$_core" @@ -134,14 +134,14 @@ contents : } else { set _smp_command "target smp $_TARGETNAME.$_core" } - + eval $_command } - + eval $_smp_command targets $_TARGETNAME.0 -The configuration file was built from information found in : +The configuration file was built from information found in: https://gist.github.com/tnishinaga/46a3380e1f47f5e892bbb74e55b3cf3e and https://www.raspberrypi.org/forums/viewtopic.php?t=252551 @@ -156,9 +156,9 @@ If the ``dap`` command is not understood by OpenOCD you may be using an older version, change to a more recent version. Launching OpenOCD -~~~~~~~~~~~~~~~~~ +----------------- -OpenOCD can be launched with the following command : +OpenOCD can be launched with the following command: :: @@ -166,16 +166,16 @@ OpenOCD can be launched with the following command : |image2| -OpenOCD will listen on three ports : +OpenOCD will listen on three ports: -- *3333* : Listens for GDB connections +- *3333*: listens for GDB connections -- *4444* : Listens for telnet connections +- *4444*: listens for telnet connections -- *6666* : Listens for tcl connections +- *6666*: listens for tcl connections Connect with telnet -~~~~~~~~~~~~~~~~~~~ +------------------- Connecting to OpenOCD through telnet allows to send OpenOCD commands (see `manual `__) @@ -185,15 +185,22 @@ Connecting to OpenOCD through telnet allows to send OpenOCD commands The ``help`` command may come in handy. Connect with GDB -~~~~~~~~~~~~~~~~ +---------------- + +The CPU of the Raspberry Pi 4 is an AArch64 core and is reported as such by +OpenOCD. Debugging it from an x86 host is best done with gdb-multiarch, which can +switch architectures. + +Launch gdb-multiarch, select the architecture matching the SO3 build and connect +to OpenOCD with ``target extended-remote localhost:3333``: -The CPU from the Raspberry Pi 4 is ARM AARCH64 and will be reported as -such by OpenOCD. In order to debug ARM AARCH32 (e.g., SO3) on an X86 -host for a AARCH64 it is best to use gdb-multiarch (this allows to -switch architectures). +.. code-block:: text -Launch gdb-multiarch and set the architecture to ``arm`` (arm 32-bit), -then connect to OpenOCD with ``target extended-remote localhost:3333`` + set architecture aarch64 # rpi4_64 build (the usual case) + set architecture arm # 32-bit build (rpi4_defconfig) + +The screenshots below were taken on a 32-bit build, hence the ``arm`` +architecture; the procedure itself is identical on ``rpi4_64``. |image4| @@ -202,29 +209,28 @@ You can load the correct executable file with the ``file`` command |image5| Debugging with GDB ------------------- - -Kernel debugging works fine because the CPU is in supervisor mode, -however, debugging 32-bit user mode on AARCH64 is not supported in -OpenOCD and requires a patch. - -The patch can be found here : http://openocd.zylin.com/#/c/5826/ +================== -If you want to Debug user mode (EL0) applications (e.g., ls.elf in SO3) -you need to apply this patch and rebuild OpenOCD. +Kernel debugging works fine because the CPU is in supervisor mode. Debugging +**32-bit** user mode on an AArch64 core, on the other hand, was unsupported in +OpenOCD and needed a patch (change ``5826``, submitted on the Gerrit that has +since moved to https://review.openocd.org/): to debug user mode (EL0) of a 32-bit +build — e.g. ``ls.elf`` — that patch had to be applied and OpenOCD rebuilt. Check +whether your OpenOCD version already carries it. A 64-bit (``rpi4_64``) build is +not concerned. Debug a user app -~~~~~~~~~~~~~~~~ +---------------- -In order to break in an user app that is not currently launched in SO3 a +In order to break in a user app that is not currently launched in SO3 a hardware breakpoint is required, because since the app is not launched -there is not context for the app (MMU translation table) and setting a +there is no context for the app (MMU translation table) and setting a software breakpoint will use the current context (e.g., SO3 kernel or other app such as sh.elf) to set the software breakpoint, this will corrupt the memory of the current context. Example -^^^^^^^ +~~~~~~~ Let’s say we want to debug ls.elf @@ -251,7 +257,7 @@ another app (other context) because apps share some common addresses |image8| -One way to check is to have a look at the disassembled code : +One way to check is to have a look at the disassembled code: |image9| @@ -259,9 +265,9 @@ Here if we compare to the disassembly of ls.elf (with objdump) |image10| -We see that the instructions do not match ! We actually breaked into +We see that the instructions do not match! We actually breaked into sh.elf, if we look at the disassembly of sh.elf (with objdump) we can -see : +see: |image11| @@ -290,7 +296,7 @@ breakpoints e.g., |image14| |image15| Follow a syscall -'''''''''''''''' +^^^^^^^^^^^^^^^^ If you want to follow a syscall inside SO3 you can change the file to ``so3`` and add a hardware breakpoint inside SO3 and continue, this @@ -328,7 +334,7 @@ Here we printed to contents of the ``p_entry`` structure, filled by ``readdir()`` and we can see the entry name is ``cat.elf``. Notes -~~~~~ +----- - The reason we may break in the wrong place with hardware breakpoints is because it is a simple comparator on PC in the processor, and @@ -341,7 +347,7 @@ Notes - The reason a software breakpoint cannot be set before an app is launched is because a software breakpoint through JTAG is simply replacing the instruction where we want to break by the ARM ‘HLT’ - instruction. However is the app is not yet loaded we don’t know where + instruction. However if the app is not yet loaded we don’t know where this would be. - Once an app is loaded and is the active process the ‘HLT’ instruction @@ -353,7 +359,7 @@ Notes instruction so that the program continues execution normally. Links ------ +===== - https://sourceforge.net/projects/openocd/ @@ -364,7 +370,7 @@ Links - https://metebalci.com/blog/bare-metal-raspberry-pi-3b-jtag/ <- Very good read -- https://www.raspberrypi.org/documentation/configuration/config-txt/gpio.md +- https://www.raspberrypi.com/documentation/computers/config_txt.html#enable_jtag_gpio - http://www.panu.it/raspberry/ @@ -372,7 +378,6 @@ Links - https://stackoverflow.com/questions/53714503/openocd-error-invalid-command-name-dap-cant-connect-blue-pill-via-st-link -.. |image0| image:: https://www.segger.com/fileadmin/images/products/J-Link/Interface_Description/181129_JTAG.svg .. |image1| image:: img/rpi_jtags.jpg .. |image2| image:: img/openocd1.png .. |image3| image:: img/so3_ci_jtag.png