cockpit: use the JSON API, with per-command screens and docs; file_status / archive_status JSON objects for more commands, #9454

docs/changes.rst

@@ -170,6 +170,10 @@ New features:
 
 - create/import-tar --json: report the deduplicated size of the new archive, #10335.
   It is also included in the archive_progress JSON output.
+- extract/export-tar --list --log-json: output a file_status JSON object per listed item,
+  like create does. The text listing of export-tar has the same "+" prefix as extract's now.
+  prune/delete/undelete --list --log-json: output an archive_status JSON object per listed
+  archive, #9454.
 
 Fixes:
 
@@ -186,6 +190,13 @@ Other changes:
   botocore (S3) service models borg needs, and build cryptography against the
   bundled OpenSSL in the Linux binaries instead of bundling a second OpenSSL
   with it, #10345.
+- cockpit: process borg's --log-json output (progress, file list, log messages, prompts)
+  instead of parsing text lines, #9454. The display depends on the command: archive
+  statistics for create/import-tar/recreate/transfer (with the final statistics from
+  --json), a progress bar for extract/export-tar, the progress phases for the other
+  commands. Yes/no prompts are shown as a dialog. The cockpit exits with the exit code
+  of the borg command.
+- docs: add a usage page for the cockpit TUI.
 
 Version 2.0.0b24 (2026-09-02)
 -----------------------------

docs/installation.rst

@@ -193,7 +193,7 @@ development header files (sometimes in a separate `-dev` or `-devel` package).
   - borgstore[rest,blake3,sftp] ~= 0.6.1 (use `pip install borgbackup[sftp]`)
 * Optionally, if you wish to use rclone Backend:
   - borgstore[rest,blake3,rclone] ~= 0.6.1 (use `pip install borgbackup[rclone]`)
-* Optionally, if you wish to use the TUI (``borg --cockpit``):
+* Optionally, if you wish to use the cockpit TUI (``borg --cockpit``, see :ref:`cockpit`):
   - textual >= 6.8.0 (use `pip install borgbackup[cockpit]`)
 
 If you have troubles finding the right package names, have a look at the

docs/internals/frontends.rst

@@ -158,15 +158,47 @@ progress_percent
         Unix timestamp (float)
 
 file_status
-    This is only output by :ref:`borg_create`, :ref:`borg_import-tar` and :ref:`borg_recreate` if
-    ``--list`` is specified. The usual rules for the file listing applies, including the
-    ``--filter`` option.
+    One object per listed item, output by :ref:`borg_create`, :ref:`borg_import-tar`,
+    :ref:`borg_recreate`, :ref:`borg_extract` and :ref:`borg_export-tar` if ``--list`` is
+    specified. The usual rules for the file listing apply, including the ``--filter`` option.
 
     status
-        Single-character status as for regular list output
+        Single-character status as for regular list output: the item flags of :ref:`borg_create`,
+        or ``+`` (item extracted / exported) and ``-`` (item excluded) for :ref:`borg_extract`
+        and :ref:`borg_export-tar`.
     path
         Path of the file system object
 
+archive_status
+    One object per listed archive, output by :ref:`borg_prune` (``--list``, ``--list-kept`` and
+    ``--list-pruned``), :ref:`borg_delete` and :ref:`borg_undelete` (``--list``). With ``--json``,
+    :ref:`borg_prune` outputs the archives on *stdout* instead.
+
+    status
+        *kept* or *pruned* (:ref:`borg_prune`), *deleted* (:ref:`borg_delete`) or *undeleted*
+        (:ref:`borg_undelete`). With ``--dry-run``, this is what would be done.
+    name, archive
+        Name of the archive
+    id
+        Archive ID (hex)
+    time
+        Archive timestamp
+    message
+        The text line of the ``--list`` output, e.g. *Keeping archive (rule: daily #1): ...*
+
+    :ref:`borg_prune` additionally gives the keys of the archive objects of ``borg prune --json``:
+    the keys requested via ``--format`` and
+
+    group
+        Object mapping the ``--group-by`` keys to the values of this archive
+    kept
+        *true* if the archive is kept, *false* if it is pruned
+    keep_rule, kept_oldest, kept_archive_number
+        For a kept archive: the rule keeping it (e.g. *daily*), whether it is the oldest archive kept
+        by the rule, and its number within the rule (1 = the most recent one)
+    deleted_archive_number
+        For a pruned archive: its number among the pruned archives (1 = the first one pruned)
+
 log_message
     Any regular log output invokes this type. Regular log options and filtering applies to these as well.
 
@@ -210,6 +242,32 @@ See Prompts_ for the types used by prompts.
     {"type": "file_status", "status": "d", "path": "src"}
     {"time": 1787900398.686938, "type": "archive_progress", "finished": true}
 
+:ref:`borg_extract` file listing, with ``--exclude src/linux/baz/file3``::
+
+    {"type": "file_status", "status": "+", "path": "src"}
+    {"type": "file_status", "status": "+", "path": "src/linux"}
+    {"type": "file_status", "status": "+", "path": "src/linux/baz"}
+    {"type": "file_status", "status": "+", "path": "src/linux/baz/file2"}
+    {"type": "file_status", "status": "-", "path": "src/linux/baz/file3"}
+    {"type": "file_status", "status": "+", "path": "src/linux/file1"}
+
+:ref:`borg_prune` archive listing, with ``--list --dry-run --keep-daily=1``::
+
+    {"name": "daily", "archive": "daily", "id": "2c77c68a...", "time": "2026-09-09T02:00:00.000000+02:00",
+     "group": {"name": "daily", "host": "host"}, "kept": true, "keep_rule": "daily", "kept_oldest": false,
+     "kept_archive_number": 1, "status": "kept", "type": "archive_status",
+     "message": "Keeping archive (rule: daily #1):            daily   Wed, 2026-09-09 02:00:00 +0200 [2c77c68a...]"}
+    {"name": "daily", "archive": "daily", "id": "99a5671a...", "time": "2026-09-08T02:00:00.000000+02:00",
+     "group": {"name": "daily", "host": "host"}, "kept": false, "deleted_archive_number": 1, "status": "pruned",
+     "type": "archive_status",
+     "message": "Would prune:                                 daily   Tue, 2026-09-08 02:00:00 +0200 [99a5671a...]"}
+
+:ref:`borg_delete` archive listing, with ``--list``::
+
+    {"name": "daily", "archive": "daily", "id": "99a5671a...", "time": "2026-09-08T02:00:00.000000+02:00",
+     "status": "deleted", "type": "archive_status",
+     "message": "Deleted archive: daily   Tue, 2026-09-08 02:00:00 +0200 [99a5671a...] (1/1)"}
+
 Saving the local cache at the end of :ref:`borg_create`::
 
     {"message": "Saving files cache", "operation": 1, "msgid": "cache.close", "type": "progress_message", "finished": false, "time": 1787900398.719723}

docs/usage.rst

@@ -35,6 +35,7 @@ Usage
 
 .. toctree::
    usage/general
+   usage/cockpit
 
    usage/repo-create
    usage/repo-space

docs/usage/cockpit.rst

@@ -0,0 +1,80 @@
+.. highlight:: none
+.. _cockpit:
+
+Cockpit
+-------
+
+The cockpit is a full-screen terminal user interface showing what a borg command does
+while it runs: its progress, its statistics, the file list and the log messages, all
+updated live. To use it, put ``--cockpit`` in front of the command::
+
+    $ borg --cockpit -r /path/to/repo create --list my-files ~/Documents
+    $ borg --cockpit -r /path/to/repo extract --list my-files
+    $ borg --cockpit -r /path/to/repo check --repair
+
+The cockpit needs the ``textual`` package: ``pip install borgbackup[cockpit]`` installs
+it, the binary releases include it (see :ref:`installation`). It needs a terminal of at
+least 80x24 characters, a taller terminal gives the log more room.
+
+How it works
+~~~~~~~~~~~~
+
+The cockpit runs the borg command as a subprocess with ``--log-json`` and ``--progress``
+added, and builds its display from the JSON output borg produces for frontends, see
+:ref:`json_output`. Apart from that, the command runs exactly like it does without
+``--cockpit``, with the options you gave it.
+
+.. note::
+
+    ``--progress`` makes ``extract`` and ``export-tar`` read the archive metadata once
+    more before they start, to determine the total amount of data for the progress bar,
+    so they start a bit later than usual, see :ref:`borg_extract`.
+
+The lower part of the screen is the log: borg's messages, warnings and errors, the file
+list if you gave ``--list``, and everything else borg outputs. The panel in the upper
+right depends on the command:
+
+``create``, ``import-tar``, ``recreate``, ``transfer``
+    The statistics of the archive being created: the number of files, the original and
+    the deduplicated size, the counts of added, modified and unchanged files, the path
+    being processed and the throughput in files and bytes per second, with a history
+    graph. For ``create`` and ``import-tar``, the exact final statistics of the new
+    archive (what ``--stats`` prints) are shown in the panel and in the log when borg
+    has finished.
+
+``extract``, ``export-tar``
+    A progress bar with the percentage and the estimated remaining time, the amount of
+    data extracted so far, the throughput and the counts of the ``--list`` lines.
+
+All other commands
+    The phases of the operation borg reports progress for, e.g. "Checking index" and
+    "Checking archives" for ``check``, each with a progress bar. For ``prune``, also the
+    numbers of kept and pruned archives.
+
+Every panel also shows the elapsed time, the number of warnings and errors and, when
+borg has finished, its exit code. The cockpit stays on the screen until you press ``q``,
+so you can have a look at the log and the numbers. It then exits with the exit code of
+the borg command, see :ref:`return_codes`.
+
+Prompts and passphrases
+~~~~~~~~~~~~~~~~~~~~~~~
+
+When borg asks a yes/no question (e.g. ``check --repair`` asks whether you know what you
+are doing), the cockpit shows a dialog: answer with the YES or NO button, or type another
+answer into the input field.
+
+The cockpit can not enter a passphrase. Give it to borg via the environment, e.g. by
+setting ``BORG_PASSPHRASE`` or ``BORG_PASSCOMMAND`` (see :ref:`env_vars`). Otherwise the
+cockpit shows a hint that borg is waiting for a passphrase, and you have to quit and try
+again.
+
+Keys
+~~~~
+
+``q`` (or Ctrl-C)
+    Quit. If borg is still running, it is asked to terminate (SIGTERM) and the cockpit
+    waits until it has exited.
+
+``t``
+    Toggle the universal translator: the labels are shown in Borg speak. Resistance is
+    futile.

docs/usage_general.rst.inc

@@ -14,6 +14,8 @@
 
 .. include:: usage/general/logging.rst.inc
 
+.. _return_codes:
+
 .. include:: usage/general/return-codes.rst.inc
 
 .. include:: usage/general/config.rst.inc

src/borg/archiver/__init__.py

@@ -38,6 +38,7 @@
     from ..helpers import add_warning, BorgWarning, BackupWarning
     from ..helpers import format_file_size
     from ..helpers import remove_surrogates, text_to_json
+    from ..helpers import bin_to_hex, OutputTimestamp, BorgJsonEncoder
     from ..helpers import DatetimeWrapper, replace_placeholders
     from ..helpers.argparsing import flatten_namespace, ArgumentTypeError, ArgumentParser, SUPPRESS
     from ..helpers import is_slow_msgpack, is_supported_msgpack, sysinfo
@@ -140,6 +141,9 @@ def __init__(self, lock_wait=None, prog=None):
         self.lock_wait = lock_wait
         self.prog = prog
         self.start_backup = None
+        # for print_file_status(): the commands with a file listing set these from their options.
+        self.output_list = False
+        self.output_filter = None
 
     def print_warning(self, msg, *args, **kw):
         warning_code = kw.get("wc", EXIT_WARNING)  # note: wc=None can be used to not influence exit code
@@ -171,6 +175,27 @@ def print_file_status(self, status, path):
             else:
                 logging.getLogger("borg.output.list").info("%1s %s", status, remove_surrogates(path))
 
+    def print_archive_status(self, status, archive_info, message, data=None):
+        """
+        List an archive a command processed, like print_file_status() lists the items of a file listing.
+
+        With --log-json, an archive_status JSON object is printed: name, id and time of the archive, the
+        <status> (e.g. "kept", "pruned", "deleted"), the <message> (the text line) and the keys of <data>.
+        Without it, the text line goes to the "borg.output.list" logger. The callers check the --list options.
+        """
+        if self.log_json:
+            json_data = {
+                "name": archive_info.name,
+                "archive": archive_info.name,
+                "id": bin_to_hex(archive_info.id),
+                "time": OutputTimestamp(archive_info.ts),
+            }
+            json_data |= data or {}
+            json_data |= {"status": status, "type": "archive_status", "message": message}
+            print(json.dumps(json_data, cls=BorgJsonEncoder), file=sys.stderr)
+        else:
+            logging.getLogger("borg.output.list").info(message)
+
     def preprocess_args(self, args):
         deprecations = [
             # ('--old', '--new' or None, 'Warning: "--old" has been deprecated. Use "--new" instead.'),
@@ -281,7 +306,12 @@ def build_parser(self):
         parser.add_argument(
             "-V", "--version", action="version", version="%(prog)s " + __version__, help="show version number and exit"
         )
-        parser.add_argument("--cockpit", dest="cockpit", action="store_true", help="Start the Borg TUI")
+        parser.add_argument(
+            "--cockpit",
+            dest="cockpit",
+            action="store_true",
+            help="run the command in the cockpit TUI, a full-screen progress display",
+        )
         parser.common_options.add_common_group(parser, provide_defaults=True)
 
         common_parser = ArgumentParser(prog=self.prog)
@@ -655,10 +685,13 @@ def main():  # pragma: no cover
                 print("Please install them using: pip install 'borgbackup[cockpit]'", file=sys.stderr)
                 sys.exit(EXIT_ERROR)
 
-            app = BorgCockpitApp()
-            app.borg_args = [arg for arg in sys.argv[1:] if arg != "--cockpit"]
+            app = BorgCockpitApp(
+                borg_args=[arg for arg in sys.argv[1:] if arg != "--cockpit"], command=getattr(args, "subcommand", None)
+            )
             app.run()
-            sys.exit(EXIT_SUCCESS)  # borg subprocess RC was already shown on the TUI
+            # exit with the exit code of the borg subprocess (it was shown on the TUI); rc < 0: borg could not be run.
+            rc = app.session.rc
+            sys.exit(rc if rc is not None and rc >= 0 else EXIT_ERROR)
 
         # normal borg CLI operation
         try:

src/borg/archiver/delete_cmd.py

@@ -1,5 +1,3 @@
-import logging
-
 from ._common import with_repository, archive_match_patterns
 from ..constants import *  # NOQA
 from ..helpers import format_archive, CommandError, bin_to_hex, archivename_validator
@@ -38,7 +36,6 @@ def do_delete(self, args, repository):
             )
 
         deleted = False
-        logger_list = logging.getLogger("borg.output.list")
         for i, archive_info in enumerate(archive_infos, 1):
             name, id, hex_id = archive_info.name, archive_info.id, bin_to_hex(archive_info.id)
             # format early before deletion of the archive
@@ -54,7 +51,7 @@ def do_delete(self, args, repository):
                 deleted = True
                 if self.output_list:
                     msg = "Would delete: {} ({}/{})" if dry_run else "Deleted archive: {} ({}/{})"
-                    logger_list.info(msg.format(archive_formatted, i, count))
+                    self.print_archive_status("deleted", archive_info, msg.format(archive_formatted, i, count))
         if dry_run:
             logger.info("Finished dry-run.")
         elif deleted:

src/borg/archiver/extract_cmd.py

@@ -39,6 +39,7 @@ def do_extract(self, args, repository, manifest, archive):
 
         progress = args.progress
         output_list = args.output_list
+        self.output_list = output_list  # for print_file_status()
         dry_run = args.dry_run
         stdout = args.stdout
         sparse = args.sparse
@@ -69,8 +70,7 @@ def do_extract(self, args, repository, manifest, archive):
             is_matched = matcher.match(orig_path)
 
             if output_list:
-                log_prefix = "+" if is_matched else "-"
-                logging.getLogger("borg.output.list").info(f"{log_prefix} {remove_surrogates(item.path)}")
+                self.print_file_status("+" if is_matched else "-", item.path)
 
             if is_matched:
                 if not dry_run:

src/borg/archiver/prune_cmd.py

@@ -1,6 +1,5 @@
 from typing import Callable, NamedTuple
 from datetime import datetime, timedelta
-import logging
 import math
 from functools import partial, wraps
 import os
@@ -245,7 +244,6 @@ def do_prune(self, args, repository, manifest):
                     )
             logger.info("Keeping %d archives, pruning %d archives.", len(keep), len(archives_to_prune))
 
-        list_logger = logging.getLogger("borg.output.list")
         # set up counters for the progress display
         num_archives_deleted = 0
         pi = ProgressIndicatorPercent(total=len(archives_to_prune), msg="Pruning archives %3.0f%%", msgid="prune")
@@ -254,28 +252,30 @@ def do_prune(self, args, repository, manifest):
                 break
             # get_item_data/format_item may internally load the archive from the repository,
             # so we must call it before deleting the archive.
-            if args.json:
+            if args.json or self.log_json:
                 archive_data = formatter.get_item_data(archive_info, jsonline=True)
                 archive_data["group"] = dict(zip(group_by, group_of[archive_info]))
-            else:
+            if not args.json:
                 archive_formatted = formatter.format_item(archive_info, jsonline=False)
             if archive_info in archives_to_prune:
                 if not args.json:
                     pi.show()
                 num_archives_deleted += 1
+                status = "pruned"
                 if args.dry_run:
                     log_message = "Would prune:"
                 else:
                     log_message = f"Pruning archive ({num_archives_deleted}/{len(archives_to_prune)}):"
                     manifest.archives.delete_by_id(archive_info.id)
-                if args.json:
+                if args.json or self.log_json:
                     archive_data["kept"] = False
                     archive_data["deleted_archive_number"] = num_archives_deleted
             else:
                 result = keep[archive_info]
                 result_message = f"{result.rule.key}{'[oldest]' if result.oldest else ''} #{result.idx + 1}"
                 log_message = f"Keeping archive (rule: {result_message}):"
-                if args.json:
+                status = "kept"
+                if args.json or self.log_json:
                     archive_data["kept"] = True
                     archive_data["keep_rule"] = result.rule.key
                     archive_data["kept_oldest"] = result.oldest
@@ -293,7 +293,8 @@ def do_prune(self, args, repository, manifest):
                 or (args.list_pruned and archive_info in archives_to_prune)
                 or (args.list_kept and archive_info not in archives_to_prune)
             ):
-                list_logger.info(f"{log_message:<44} {archive_formatted}")
+                message = f"{log_message:<44} {archive_formatted}"
+                self.print_archive_status(status, archive_info, message, archive_data if self.log_json else None)
         if not args.json:
             pi.finish()
         if args.json: