refactor(ase): consume external abacuslite plugin

.github/pull_request_template.md

@@ -5,6 +5,7 @@
 - [ ] I have listed the exact verification commands run and their results.
 - [ ] I have described user-visible behavior changes, including INPUT parameter changes.
 - [ ] I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other `source/` changes.
+- [ ] If this changes `interfaces/ASE_interface/`, `source/source_relax/socket_*`, or the socket metadata schema, I linked the abacuslite counterpart and updated the compatibility contract.
 - [ ] I have requested any needed governance exception below.
 
 ### Linked Issue

.github/workflows/ase_plugin_test.yml

@@ -1,41 +1,85 @@
-name: Atomic Simulation Environment (ASE) Plugin Test
+name: Atomic Simulation Environment (ASE) Interface Test
 
 on:
   pull_request:
+  schedule:
+    - cron: '41 3 * * 1'
+  workflow_dispatch:
+    inputs:
+      plugin_ref:
+        description: Optional reviewed QuantumMisaka/abacuslite ref; empty uses published 1.1.0
+        required: false
+        default: ''
 
 defaults:
   run:
     shell: bash
 
 jobs:
   test:
-    name: abacuslite
+    name: abacuslite external
     runs-on: ubuntu-latest
+    timeout-minutes: 120
     container:
       image: ghcr.io/deepmodeling/abacus-gnu
+    env:
+      ABACUS_COMMAND: mpirun -np 2 abacus
+      ASE_ABACUS_ASSETS: ${{ github.workspace }}/tests/PP_ORB
+      ASE_ABACUS_SOCKET: '1'
 
     steps:
-      - name: Checkout code
+      - name: Checkout engine PR
         uses: actions/checkout@v7.0.1
 
+      - name: Checkout reviewed plugin candidate
+        if: github.event_name == 'workflow_dispatch' && inputs.plugin_ref != ''
+        uses: actions/checkout@v7.0.1
+        with:
+          repository: QuantumMisaka/abacuslite
+          ref: ${{ inputs.plugin_ref }}
+          path: candidate-plugin
+          persist-credentials: false
+
       - name: Set up Miniconda
         uses: conda-incubator/setup-miniconda@v4
         with:
           miniconda-version: 'latest'
           activate-environment: abacuslite
           python-version: '3.10'
 
-      - name: Install dependencies
+      - name: Install Python dependencies
         shell: bash -l {0}
         run: |
+          set -euo pipefail
           conda install -y pip
           pip install numpy scipy matplotlib
 
       - name: Install abacuslite
         shell: bash -l {0}
+        env:
+          PLUGIN_REF: ${{ inputs.plugin_ref }}
         run: |
-          cd interfaces/ASE_interface
-          pip install .
+          set -euo pipefail
+          mkdir -p "$RUNNER_TEMP/plugin-wheel"
+          if [ -n "$PLUGIN_REF" ]; then
+            python -m pip install 'build>=1.2'
+            python -m build --wheel --outdir "$RUNNER_TEMP/plugin-wheel" candidate-plugin
+            git -C candidate-plugin rev-parse HEAD
+          else
+            python -m pip download --only-binary=:all: --no-deps \
+              --dest "$RUNNER_TEMP/plugin-wheel" 'abacuslite==1.1.0'
+          fi
+          wheels=("$RUNNER_TEMP"/plugin-wheel/*.whl)
+          test "${#wheels[@]}" -eq 1
+          sha256sum "${wheels[0]}"
+          if [ -z "$PLUGIN_REF" ]; then
+            actual="$(sha256sum "${wheels[0]}")"
+            test "${actual%% *}" = c3e4504e9c9f5bceb69f70c79290e7843037ca72f625c5f66258e14d110f0ff3
+          fi
+          python -m pip install "${wheels[0]}"
+          python -m pip show abacuslite ase numpy seekpath
+          python interfaces/ASE_interface/tests/verify_external.py \
+            --checkout-root "$GITHUB_WORKSPACE"
 
       - name: Install external tools from toolchain
         run: |
@@ -45,46 +89,58 @@ jobs:
           ./scripts/stage4/install_stage4.sh
           cd ..
 
-      - name: Configure & Build ABACUS (GNU)
+      - name: Configure & build ABACUS (GNU)
         run: |
-          git config --global --add safe.directory `pwd`
+          git config --global --add safe.directory "$(pwd)"
           source toolchain/install/setup
           prepend_path CMAKE_PREFIX_PATH ${PEXSI32_ROOT}:${SUPERLU_DIST32_ROOT}:${PARMETIS32_ROOT}:${METIS32_ROOT}:${GKLIB_ROOT}
           rm -rf build
           cmake -B build -G Ninja
           cmake --build build -j2
-      
-      - name: Install and Soft Link ABACUS
+
+      - name: Install and attest the built ABACUS executable
         run: |
+          built="$GITHUB_WORKSPACE/build/abacus_basic_para"
+          test -x "$built"
           cmake --install build
-          if [ -f /usr/local/bin/abacus_2p ]; then
-            ln -sf /usr/local/bin/abacus_2p /usr/local/bin/abacus
-          fi
+          installed="$(command -v abacus)"
+          test -x "$installed"
+          built_sha="$(sha256sum "$built")"
+          installed_sha="$(sha256sum "$installed")"
+          test "${built_sha%% *}" = "${installed_sha%% *}"
+          printf 'Built ABACUS: %s %s\n' "$built" "${built_sha%% *}"
+          printf 'Installed ABACUS: %s %s\n' "$installed" "${installed_sha%% *}"
+          git rev-parse HEAD
+          abacus --version
 
-      - name: Run unit tests
+      - name: Run offline harness contracts
         shell: bash -l {0}
         run: |
-          cd interfaces/ASE_interface/abacuslite
-          chmod +x xtest.sh
-          ./xtest.sh
+          set -euo pipefail
+          cd interfaces/ASE_interface/tests
+          python -m unittest test_external_provenance test_harness_facts \
+            test_native_md_fixture test_portable_runner -v
 
-      - name: Run integration tests
+      - name: Run engine-owned integration tests
         shell: bash -l {0}
         run: |
-          cd interfaces/ASE_interface/tests
-          chmod +x xtest.sh
-          ./xtest.sh
+          set -euo pipefail
+          python interfaces/ASE_interface/tests/run_engine_compatibility.py \
+            --engine "$GITHUB_WORKSPACE/build/abacus_basic_para" \
+            --mpi-command 'mpirun -np 2' \
+            --assets "$GITHUB_WORKSPACE/tests/PP_ORB" \
+            --format latest \
+            --wheel "$RUNNER_TEMP"/plugin-wheel/*.whl \
+            --report-dir "$RUNNER_TEMP/ase-engine-tests" \
+            --engine-ref "$(git rev-parse HEAD)" \
+            --python "$(command -v python)"
 
-      # When a test fails, every integration test script (scf/relax/md/band/
-      # magnetic) preserves its whole ABACUS run directory under
-      # /tmp/abacus_ase_failure (see preserved_tmpdir in tests/common.py,
-      # overridable via ASE_ABACUS_KEEP_DIR). Upload it so remote-CI-only
-      # failures (e.g. issue #7794) can be root-caused without runner access.
       - name: Upload ASE failure artifacts
-        if: failure()
+        if: always()
         uses: actions/upload-artifact@v7
         with:
           name: ase-failure-${{ github.run_id }}-${{ github.run_attempt }}
-          path: /tmp/abacus_ase_failure/**
+          path: |
+            ${{ runner.temp }}/ase-engine-tests/**
           if-no-files-found: ignore
           retention-days: 14

docs/advanced/interface/ase.md

@@ -4,7 +4,11 @@
 
 [ASE](https://wiki.fysik.dtu.dk/ase/) (Atomic Simulation Environment) performs as a powerful Pythonic platform for atomistic simulations, in which there are plenty of functionalties supported, such as various geometry optimization algorithms for finding both the minimum energy point and the transition states, including BFGS, BFGSLineSearch, FIRE, NEB, AUTO-NEB, etc, and various molecular dynamics techniques, including thermostats (Langevin, CSVR, Nose-Hoover Chain, etc) and metadynamics (via the interface with Plumed). 
 
-Due to the growing number of softwares and machine-learning forcefields, we turn to maintain the interface with ASE by our own, while a legacy version of ASE interface can still be found at [our GitLab repository of ase-abacus](https://gitlab.com/1041176461/ase-abacus ).
+Due to the growing number of softwares and machine-learning forcefields, the
+ABACUS ASE interface is maintained as the independent
+[`abacuslite`](https://github.com/QuantumMisaka/abacuslite) package. A legacy
+version of the ASE interface can still be found at
+[our GitLab repository of ase-abacus](https://gitlab.com/1041176461/ase-abacus).
 
 ## Installation
 
@@ -15,13 +19,23 @@ conda create -n abacus python=3.10
 conda activate abacus
 ```
 
-Then, install the ASE interface by:
+Then, install the independently maintained ASE plugin:
 
 ```bash
-cd interfaces/ASE_interface
-pip install .
+python -m pip install abacuslite
 ```
 
+`abacuslite` owns the Python calculator, ABACUS input/output readers and user
+examples. The ABACUS repository keeps only the engine-side compatibility
+harness in
+[`interfaces/ASE_interface/tests`](https://github.com/deepmodeling/abacus-develop/tree/develop/interfaces/ASE_interface/tests).
+ABACUS CI installs the published plugin, or a reviewed candidate ref, and runs
+that harness against the executable built from the current source. Manual CI
+dispatch can select the candidate ref.
+
+The plugin is not vendored and is not a git submodule, so GitHub source
+archives remain complete.
+
 ## ABACUS Calculator
 
 Present calculator implementation requires a "profile" to act as an interface between the Python runtime and the file system.
@@ -99,9 +113,20 @@ with tempfile.TemporaryDirectory() as tmpdir:
 
 ## Perform Calculations
 
-In the new implementation, we limit the range of functionalties supported to mainly include the necessary ones, such as the SCF calculation, the energy and force/stress evaluation. The other features, such as starting the molecule dynamics directly in ABACUS from Python, is not supported anymore. Instead, it is encouraged to use the ASE tools to perform the molecule dynamics.
-
-Please read the examples in `interfaces/ASE_interface/examples/` for more details.
+In the new implementation, the supported surface is the one the external plugin
+exposes: SCF calculations, energy and force/stress evaluation, ASE-driven
+relaxation and molecular dynamics through the ASE optimizers and integrators,
+and ABACUS native MD, in which the engine runs its own MD integrator and the
+plugin reads the resulting trajectory. Both molecular-dynamics paths are
+validated rather than assumed: the engine-side harness runs the ASE-driven
+Langevin case (`md.py`) and the native-MD case (`native_md.py`) as required
+compatibility cases in both the develop and the legacy output-format lanes.
+Calculation types the plugin does not expose remain outside the supported
+surface, and for simulations whose integrator you control the ASE tools remain
+the recommended path.
+
+See the [abacuslite examples](https://github.com/QuantumMisaka/abacuslite/tree/main/examples)
+for complete scripts.
 
 ### Socket I/O with ASE
 
@@ -148,11 +173,10 @@ With the ABACUS toolchain workflow, build the normal ABACUS executable with LCAO
 
 For PW calculations on CUDA/ROCm with multiple MPI ranks, use a k-point layout compatible with ABACUS' GPU parallelization. In practice, make sure each k-point pool contains one MPI rank; for example, a 4-rank PW GPU socket calculation should use at least four k-points so the default GPU `kpar` adjustment can assign one rank per pool. A one-k-point PW GPU job with several MPI ranks can fail in the PW GPU transform path; reduce the rank count or use a denser k-point mesh such as a smaller `kspacing`.
 
-The ASE interface can be installed from this repository with:
+The ASE interface is installed from PyPI:
 
 ```bash
-cd interfaces/ASE_interface
-pip install .
+python -m pip install abacuslite
 ```
 
 A minimal socket calculator setup is:
@@ -193,13 +217,28 @@ Calling `atoms.get_potential_energy()` does not force a force or stress calculat
 
 The ABACUS metadata extension is required to expose force/stress presence safely. If a legacy client returns an empty extras field, the adapter accepts only an energy-only response and refuses to infer forces or stress from the fixed-wire padding. Generic i-PI/ASE clients that ignore ABACUS extras cannot distinguish mandatory padding from a computed zero; use `AbacusSocketIO` or another metadata-aware client when requesting optional properties. When launching ABACUS with a generic client, explicitly set `cal_force=1` for force-driven workflows and `cal_stress=1` for stress evaluation; an omitted switch defaults to disabled. Such clients also need their own policy for unconverged SCF results.
 
+#### Socket protocol compatibility
+
+The socket metadata extension uses the schema `abacus.socket.properties.v1`.
+The `present` list distinguishes computed properties from protocol padding, and
+`scf_converged` reports the SCF state for the returned frame. Additive metadata
+fields may be added within v1; a breaking change requires a new schema version
+and a transition period in both ABACUS and abacuslite.
+
+The ASE-side adapter contract is documented in the
+[abacuslite socket contract](https://github.com/QuantumMisaka/abacuslite/blob/main/docs/socket-contract.md).
+
+Engine and adapter changes that alter this contract should be linked and tested
+together.
+
 A socket calculator owns one ABACUS process initialized from one fixed `INPUT`/`STRU` setup. Reuse the same `AbacusSocketIO` instance only for position updates under the same electronic-structure settings and the same cell. Do not change `kpts`, `kspacing`, `nspin`, `basis_type`, `basissets`, pseudopotentials, species, atom count, cell, or other core `INPUT`/`STRU` parameters through an existing socket calculator; create a new `AbacusSocketIO` instance and a new ABACUS client process for those changes. `AbacusSocketIO` rejects cell changes before sending them to ABACUS, and the ABACUS socket driver also checks incoming POSDATA cells against the initial `STRU` cell and exits if they differ.
 
 In socket mode, ABACUS keeps one client process alive. All SCF evaluations produced by the same `AbacusSocketIO` instance are appended to the same `OUT.ABACUS/running_scf.log`, because the ABACUS calculation type remains `scf`. The authoritative per-step energy and force results are returned through the i-PI socket to ASE. Use ASE trajectory and optimizer log files, such as `BFGS(atoms, trajectory="opt.traj", logfile="opt.log")`, when each optimizer or MD step should be saved separately. Treat `running_scf.log` mainly as the ABACUS diagnostic log for the socket client, not as one independent FileIO result per structure.
 
 The i-PI protocol does not transmit element symbols. `AbacusSocketIO` therefore sorts the internal socket atoms with the same first-occurrence species grouping used when writing `STRU`, and maps returned forces back to the original ASE `Atoms` order. This avoids silent force/atom mismatches when structures are read from CIF, extxyz, POSCAR, or other formats whose atom order is not already grouped for ABACUS. Users should not manually reorder atoms for socket I/O; pass the physical ASE `Atoms` object directly to the calculator.
 
-A complete fixed-cell validation and benchmark example is available in `interfaces/ASE_interface/examples/socketio.py`.
+A complete fixed-cell validation example is available in
+[`abacuslite/examples/socketio.py`](https://github.com/QuantumMisaka/abacuslite/blob/main/examples/socketio.py).
 
 ## SPAP Analysis