diff --git a/docs/guide/execution/index.rst b/docs/guide/execution/index.rst index 61afed989..38b38e190 100644 --- a/docs/guide/execution/index.rst +++ b/docs/guide/execution/index.rst @@ -3,20 +3,32 @@ Execution ========= -With a :class:`.Transformation` defined, the next step is to execute this. -The easiest way to run it is to use the :ref:`quickrun CLI tool `. -More advanced options are available through first considering the -:ref:`theory of the execution model` -then :ref:`reading on the available Python functions`. +Once you have planned your calculations, each :class:`.Transformation` needs to be executed. +Its simulations are run and the results analysed to give a free energy estimate. +**openfe** supports two approaches, which differ in who organises the work. +Both can be used from the command line or from Python. + +:ref:`Running Transformations individually ` + You run one Transformation at a time and decide where and when each one runs, + for example by submitting one cluster job per Transformation. + On the command line this is ``openfe quickrun``; in Python, :func:`.execute_DAG`. + +:ref:`Task-based execution ` (experimental feature) + **openfe** splits a whole :class:`.AlchemicalNetwork` into tasks and keeps track of which are ready to run. + You start one or more workers, and each one picks up the next available task: + ``openfe run-task`` on the command line, or :class:`.Worker` in Python. .. toctree:: + execution_theory quickrun_execution task_based_execution - execution_theory - -.. admonition:: How-to guides - * :doc:`/cookbook/dumping_transformation` +.. admonition:: Related How-To Guides -.. TODO: add the Running calculations how-tos (quickrun, repeats in parallel, SLURM, - resuming, NVIDIA MPS, task-based execution) to this box. + * :doc:`/how_to_guides/dumping_transformation` + * :doc:`/how_to_guides/run_quickrun` + * :doc:`/how_to_guides/resume_quickrun` + * :doc:`/how_to_guides/run_on_slurm` + * :doc:`/how_to_guides/run_repeats_in_parallel` + * :doc:`/how_to_guides/run_with_mps` + * :doc:`/how_to_guides/run_task_based` diff --git a/docs/guide/execution/quickrun_execution.rst b/docs/guide/execution/quickrun_execution.rst index 25de4599d..e962a1485 100644 --- a/docs/guide/execution/quickrun_execution.rst +++ b/docs/guide/execution/quickrun_execution.rst @@ -1,6 +1,6 @@ .. _userguide_quickrun: -Execution with Quickrun +Execution with ``quickrun`` ======================= The planning and preparation of a campaign of alchemical simulations using ``openfe`` is intended to be achievable on a local workstation in a matter of minutes. @@ -10,201 +10,48 @@ These serialized JSON files are the currency of executing a campaign of simulati To read the ``Transformation`` information and execute the simulation, the command line interface provides the ``openfe quickrun`` command, the full details of which are given in :ref:`the CLI reference section`. +Overview +-------- -Basic quickrun usage --------------------- - -The ``quickrun`` command takes in the ``Transformation`` information represented as JSON, then executes a simulation according to those specifications. -For example, the following command executes a simulation defined by ``transformation.json`` and produces a results file named ``results.json``. - -:: - - > openfe quickrun transformation.json -d workdir/ -o workdir/results.json +``openfe quickrun`` executes one :class:`.Transformation`. +A Transformation contains everything needed to run a free energy calculation: +the two :class:`.ChemicalSystem`\s, the :class:`.Protocol` and, if the Protocol needs one, +an :class:`.AtomMapping` (see :ref:`alchemical_network_model`). -The ``-d`` / ``--work-dir`` flag controls where working files (checkpoints, trajectory data, etc...) are written. -If it is omitted, the current directory will be used. +When quickrun starts, it creates the ``ProtocolDAG`` of the Transformation: +the setup, simulation and analysis tasks (``ProtocolUnit``\s) of every repeat (see :ref:`userguide_execution_theory`). +quickrun then executes these tasks one at a time, and each task writes its files to its own subdirectory of the work directory. +If a task fails, quickrun retries it up to two more times; if it still fails, quickrun stops. -The ``-o`` flag controls where the results file will be written. -If it is omitted, results are written to a file named ``_results.json`` in the working directory, where ```` is a unique identifier. +When all tasks have finished, or one has failed, quickrun combines the task results into a :class:`.ProtocolResult` +and writes the free energy estimate and its uncertainty to the results file. -.. _userguide_quickrun_resume: +quickrun keeps track of which tasks have finished, so an interrupted run can be resumed with ``--resume``. +Finished tasks are skipped, and an interrupted simulation continues from its last checkpoint +(see :doc:`/how_to_guides/resume_quickrun`). -Resuming a halted job +Organizing a campaign --------------------- -When ``openfe quickrun`` starts, it saves a plan of the simulation to a cache file before execution begins: - -.. code:: bash - - /quickrun_cache/dag-cache-.json - -Where ```` is a unique identifier based on the ``-o`` file path and Transformation. -This cache is automatically removed once the job completes. - -If a job is interrupted (e.g. due to a wall-time limit, node failure, or manual cancellation), you can resume the interrupted job by passing the ``--resume`` flag: - -.. code:: bash - - > openfe quickrun transformation.json -d workdir/ -o workdir/results.json --resume - -The planned simulation cache will be used to identify where in the simulation process it left off and, if supported by the Transformation Protocol, how to resume. - -.. note:: - - The same ``-d`` / ``--work-dir`` and ``-o`` flag arguments used in the - original run must be specified so that ``quickrun`` can locate the cache file. - -If you pass ``--resume`` but no cache file is found (e.g. the job never started), the following warning is printed and a fresh execution begins. - -.. code:: bash - - openfe quickrun was run with --resume, but no cached results found at - . Starting new execution. - -If the cache file is corrupted (e.g. due to an incomplete write at the moment of interruption), ``quickrun --resume`` will raise an error with instructions to rerun the simulation: - -.. code:: bash - - Recovery failed, please remove /quickrun_cache/dag-cache-.json - before executing a new transformation simulation. - -If you do not pass the ``--resume`` flag, the code will detect the partially complete transformation and prevent you from accidentally starting a duplicate run. -The following error will be raised: - -.. code:: bash - - Transformation has been started but is incomplete. Please remove - /quickrun_cache/dag-cache-.json and rerun, or resume - execution using the ``--resume`` flag. - - -Executing within a job submission script ----------------------------------------- - -You may need to submit computational jobs to a queueing engine, such as Slurm. -The ``openfe quickrun`` command can be used within a submission script as follows: - -.. code-block:: bash - - #!/bin/bash - - #SBATCH --job-name="openfe job" - #SBATCH --mem-per-cpu=2G - - # activate an appropriate conda environment, or any "module load" commands required to - conda activate openfe_env - - openfe quickrun transformation.json -d workdir/ -o workdir/results.json - - -Parallel execution of repeats with Quickrun -=========================================== - -Serial execution of multiple repeats of a transformation can be inefficient when simulation times are long. -Higher throughput can be achieved with parallel execution by running one repeat per HPC job. -Most protocols are set up to run three repeats in serial by default, but this can be changed by either: - -1. Defining the protocol setting ``protocol_repeats`` - see the :ref:`protocol configuration guide ` for more details. -2. Using the ``openfe plan-rhfe-network`` (or ``plan-rbfe-network``) command line flag ``--n-protocol-repeats``. - -Each transformation can then be executed multiple times via the ``openfe quickrun`` command to produce a set of repeats. -However, **you must use unique results files for each repeat to ensure they don't overwrite each other**. -We recommend using folders named ``results_x`` where x is 0-2 to store the repeated calculations as our :ref:`openfe gather ` command also supports this file structure. - -Below is an example of a simple script that will create and submit a separate job script (``\*.job`` named file) for every alchemical transformation (for the simplest SLURM use case) in a network running each repeat in parallel and writing the results to a unique folder: +You decide where and when each Transformation runs, for example by +:doc:`submitting one job per Transformation to a cluster `. +You can run multiple simulations in parallel by running several quickrun calls at the same time, +for example one per Transformation, :doc:`one per repeat `, +or :doc:`several on the same GPU `. +The results files are then collected with :ref:`openfe gather `. -.. code-block:: bash +.. TODO: link the "Gather free energy results" how-to instead of the CLI reference once it exists. - for file in network_setup/transformations/*.json; do - relpath="${file:30}" # strip off "network_setup/" - dirpath=${relpath%.*} # strip off final ".json" - jobpath="network_setup/transformations/${dirpath}.job" - if [ -f "${jobpath}" ]; then - echo "${jobpath} already exists" - exit 1 - fi - for repeat in {0..2}; do - cmd="openfe quickrun ${file} -o results_${repeat}/${relpath} -d results_${repeat}/${dirpath}" - echo -e "#!/usr/bin/env bash\n${cmd}" > "${jobpath}" - sbatch "${jobpath}" - done - done +If you would rather have **openfe** schedule the tasks of a whole network for you, +see :ref:`userguide_task_based_execution`. -This should result in the following file structure after execution: +.. admonition:: How-to guides -:: - - results_parallel - ├── results_0 - │   ├── rbfe_lig_ejm_31_complex_lig_ejm_42_complex.json - │   ├── shared_HybridTopologyMultiStateAnalysisUnit-5e0825de1dd045818cdc3428205c1cf7_attempt_0 - │   │   ├── forward_reverse_convergence.png - │   │   ├── ligand_RMSD.png - │   │   ├── mbar_overlap_matrix.png - │   │   ├── replica_exchange_matrix.png - │   │   ├── replica_state_timeseries.png - │   │   └── structural_analysis.npz - │   ├── shared_HybridTopologyMultiStateSimulationUnit-144be594cf024cb19152cfe5e0b3fb7d_attempt_0 - │   │   ├── checkpoint.chk - │   │   ├── simulation.nc - │  │  └── simulation_real_time_analysis.yaml - │   └── shared_HybridTopologySetupUnit-01b5afe1972c4e2f9d0943da43b4b19c_attempt_0 - │   ├── A_db.json - │   ├── B_db.json - │   ├── hybrid_positions.npy - │   ├── hybrid_system.pdb - │   └── hybrid_system.xml.bz2 - ├── results_1 - │   ├── rbfe_lig_ejm_31_complex_lig_ejm_42_complex.json - │   ├── shared_HybridTopologyMultiStateAnalysisUnit-7986bec616a74929aee85e900535f4a2_attempt_0 - │   │   ├── forward_reverse_convergence.png - │   │   ├── ligand_RMSD.png - │   │   ├── mbar_overlap_matrix.png - │   │   ├── replica_exchange_matrix.png - │   │   ├── replica_state_timeseries.png - │   │   └── structural_analysis.npz - │   ├── shared_HybridTopologyMultiStateSimulationUnit-18eb295b7123444f9ac66ff3caffcab8_attempt_0 - │   │   ├── checkpoint.chk - │   │   ├── simulation.nc - │  │  └── simulation_real_time_analysis.yaml - │   └── shared_HybridTopologySetupUnit-3d8ccb1ef5124bd4ba20e0047aad0b5f_attempt_0 - │   ├── A_db.json - │   ├── B_db.json - │   ├── hybrid_positions.npy - │   ├── hybrid_system.pdb - │   └── hybrid_system.xml.bz2 - └── results_2 - ├── rbfe_lig_ejm_31_complex_lig_ejm_42_complex.json - ├── shared_HybridTopologyMultiStateAnalysisUnit-ac5fad8ad1fb49598f80018713dce070_attempt_0 - │   ├── forward_reverse_convergence.png - │   ├── ligand_RMSD.png - │   ├── mbar_overlap_matrix.png - │   ├── replica_exchange_matrix.png - │   ├── replica_state_timeseries.png - │   └── structural_analysis.npz - ├── shared_HybridTopologyMultiStateSimulationUnit-73abea21b423444881bd8f21415c937f_attempt_0 - │   ├── checkpoint.chk -    │   ├── simulation.nc -   │  └── simulation_real_time_analysis.yaml - └── shared_HybridTopologySetupUnit-79bc9b63321945338a3b69d9f94ee15b_attempt_0 - ├── A_db.json - ├── B_db.json - ├── hybrid_positions.npy - ├── hybrid_system.pdb - └── hybrid_system.xml.bz2 - -The results of which can be gathered from the CLI using the ``openfe gather`` command, in this case you should direct -it to the root directory which includes the repeat results and it will automatically collate the information - -:: - - > openfe gather results_parallel - -Optimizing GPU performance with NVIDIA MPS -========================================== - -You can further optimize execution of ``openfe quickrun`` using NVIDIA's Multi-Process Service (MPS). -See NVIDIA's documentation on `MPS for OpenFE free energy calculations `_ for details. + * :doc:`/how_to_guides/run_quickrun` + * :doc:`/how_to_guides/resume_quickrun` + * :doc:`/how_to_guides/run_on_slurm` + * :doc:`/how_to_guides/run_repeats_in_parallel` + * :doc:`/how_to_guides/run_with_mps` See Also -------- diff --git a/docs/guide/protocols/absolutebinding.rst b/docs/guide/protocols/absolutebinding.rst index bd43f6cac..c9986f433 100644 --- a/docs/guide/protocols/absolutebinding.rst +++ b/docs/guide/protocols/absolutebinding.rst @@ -132,7 +132,7 @@ the `Boresch restraint analysis tutorial The multistate trajectory (``simulation.nc``) and its topology (``alchemical_system.pdb``) can be analysed further using `openfe_analysis`_. -.. TODO: Link to the trajectory visualisation cookbook when published. +.. TODO: Link to the trajectory visualisation how_to_guide when published. See Also -------- @@ -154,7 +154,7 @@ See Also **How-to guides** -:ref:`How-to guides ` +:ref:`How-to guides ` **API Documentation** diff --git a/docs/guide/protocols/absolutesolvation.rst b/docs/guide/protocols/absolutesolvation.rst index db491e914..bdd10fc5e 100644 --- a/docs/guide/protocols/absolutesolvation.rst +++ b/docs/guide/protocols/absolutesolvation.rst @@ -115,7 +115,7 @@ the structural analysis is skipped. The multistate trajectories (``solvent.nc`` and ``vacuum.nc``) can be analysed further using `openfe_analysis`_. -.. TODO: Link to the trajectory visualisation cookbook when published. +.. TODO: Link to the trajectory visualisation how_to_guide when published. See Also @@ -135,7 +135,7 @@ See Also **How-to guides** -:ref:`How-to guides ` +:ref:`How-to guides ` **API Documentation** diff --git a/docs/guide/protocols/index.rst b/docs/guide/protocols/index.rst index 9d32d711a..f67ccf3c1 100644 --- a/docs/guide/protocols/index.rst +++ b/docs/guide/protocols/index.rst @@ -243,7 +243,7 @@ These limitations apply to all binding free energy protocols. See also -------- -* :doc:`Choose and configure a protocol ` +* :doc:`Choose and configure a protocol ` * :ref:`Assessing simulation health ` * :doc:`Troubleshooting simulations ` diff --git a/docs/guide/protocols/relativehybridtopology.rst b/docs/guide/protocols/relativehybridtopology.rst index b35d513cf..b8dee3515 100644 --- a/docs/guide/protocols/relativehybridtopology.rst +++ b/docs/guide/protocols/relativehybridtopology.rst @@ -201,7 +201,7 @@ evidence of ligand displacement. The multistate trajectory (``simulation.nc``) and the hybrid topology (``hybrid_system.pdb``) can be analysed further using `openfe_analysis`_. -.. TODO: Link to the trajectory visualisation cookbook when published. +.. TODO: Link to the trajectory visualisation how_to_guide when published. See Also @@ -219,7 +219,7 @@ See Also **How-to guides** -:ref:`How-to guides ` +:ref:`How-to guides ` **API Documentation** diff --git a/docs/guide/protocols/septop.rst b/docs/guide/protocols/septop.rst index 30b7fe706..de10a567b 100644 --- a/docs/guide/protocols/septop.rst +++ b/docs/guide/protocols/septop.rst @@ -144,7 +144,7 @@ the `Boresch restraint analysis tutorial The multistate trajectory (``simulation.nc``) and its topology (``alchemical_system.pdb``) can be analysed further using `openfe_analysis`_. -.. TODO: Link to the trajectory visualisation cookbook when published. +.. TODO: Link to the trajectory visualisation how_to_guide when published. See Also -------- @@ -165,7 +165,7 @@ See Also **How-to guides** -:ref:`How-to guides ` +:ref:`How-to guides ` **API Documentation** diff --git a/docs/guide/results/index.rst b/docs/guide/results/index.rst index 958ec7753..575822d8a 100644 --- a/docs/guide/results/index.rst +++ b/docs/guide/results/index.rst @@ -14,7 +14,7 @@ and entire networks of results analysed. .. admonition:: How-to guides - * :doc:`/cookbook/dumping_transformation` - * :doc:`/cookbook/jq_inspection` + * :doc:`/how_to_guides/dumping_transformation` + * :doc:`/how_to_guides/jq_inspection` .. TODO: add the gather, load results, cinnabar and openfe_analysis how-tos. diff --git a/docs/guide/results/simulation_health.rst b/docs/guide/results/simulation_health.rst index f7a662766..6ce90c987 100644 --- a/docs/guide/results/simulation_health.rst +++ b/docs/guide/results/simulation_health.rst @@ -454,7 +454,7 @@ Some problems only show up across a network. Relative free energies around a cycle should sum to zero within their uncertainties, and each edge should agree with the value implied by the rest of the network. Large deviations point to unreliable edges worth checking with the diagnostics above. These checks need a network with redundant edges -(see :doc:`Generate ligand networks `). +(see :doc:`Generate ligand networks `). To calculate cycle closure errors, see the `cinnabar cycle closure tutorial `_. diff --git a/docs/guide/setup/alchemical_network_model.rst b/docs/guide/setup/alchemical_network_model.rst index 615ecbea1..8286d49a4 100644 --- a/docs/guide/setup/alchemical_network_model.rst +++ b/docs/guide/setup/alchemical_network_model.rst @@ -43,14 +43,14 @@ is often done through a :class:`.LigandNetwork`. You can manually create a :class:`.AlchemicalNetwork` by creating a list of :class:`.Transformation` objects. For examples using the Python API, -see :ref:`cookbook on creating alchemical networks `. +see :ref:`How-to guide on creating alchemical networks `. 2. Python ``NetworkPlanner`` Convenience Classes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OpenFE also provides the convenience classes :class:`.RBFEAlchemicalNetworkPlanner` and :class:`.RHFEAlchemicalNetworkPlanner`, which use the :class:`.RelativeHybridTopologyProtocol` for creating :class:`.AlchemicalNetwork`\s. -For example usage of these convenience classes, see :ref:`Relative Alchemical Network Planners cookbook `. +For example usage of these convenience classes, see :ref:`Relative Alchemical Network Planners How-to guide `. .. note:: The Network Planners are provided for user convenience. While they cover diff --git a/docs/guide/setup/creating_ligand_networks.rst b/docs/guide/setup/creating_ligand_networks.rst index 5f6d96ff8..620a1fc98 100644 --- a/docs/guide/setup/creating_ligand_networks.rst +++ b/docs/guide/setup/creating_ligand_networks.rst @@ -44,7 +44,7 @@ Generating Ligand Networks # plan the ligand network ligand_network = network_planner(ligands=mols, mappers=[mapper], scorer=scorer) -Practical information on generating ligand networks can be found in our :ref:`cookbook for ligand network generation `. +Practical information on generating ligand networks can be found in our :ref:`How-to guide for ligand network generation `. .. note:: Like the Component objects, a ``LigandNetwork`` object is immutable once created! diff --git a/docs/guide/setup/index.rst b/docs/guide/setup/index.rst index 7741cc186..885c6d5f6 100644 --- a/docs/guide/setup/index.rst +++ b/docs/guide/setup/index.rst @@ -47,22 +47,22 @@ See :ref:`dumping transformations ` for more details. **Setup** - * :doc:`/cookbook/loading_molecules` - * :doc:`/cookbook/user_charges` - * :doc:`/cookbook/bespoke_parameters` + * :doc:`/how_to_guides/loading_molecules` + * :doc:`/how_to_guides/user_charges` + * :doc:`/how_to_guides/bespoke_parameters` **Planning: Ligand networks** - * :doc:`/cookbook/generate_ligand_network` - * :doc:`/cookbook/hand_write_ligand_network` - * :doc:`/cookbook/network_from_orion_fepp` - * :doc:`/cookbook/ligandnetwork_vis` - * :doc:`/cookbook/repairing_broken_networks` + * :doc:`/how_to_guides/generate_ligand_network` + * :doc:`/how_to_guides/hand_write_ligand_network` + * :doc:`/how_to_guides/network_from_orion_fepp` + * :doc:`/how_to_guides/ligandnetwork_vis` + * :doc:`/how_to_guides/repairing_broken_networks` **Planning: Protocols and alchemical networks** - * :doc:`/cookbook/choose_protocol` - * :doc:`/cookbook/rfe_alchemical_planners` - * :doc:`/cookbook/create_alchemical_network` + * :doc:`/how_to_guides/choose_protocol` + * :doc:`/how_to_guides/rfe_alchemical_planners` + * :doc:`/how_to_guides/create_alchemical_network` .. TODO: add the new Setup and Planning how-tos to this box as they are added. diff --git a/docs/guide/setup/input_preparation.rst b/docs/guide/setup/input_preparation.rst index 127067e35..48ab8dfba 100644 --- a/docs/guide/setup/input_preparation.rst +++ b/docs/guide/setup/input_preparation.rst @@ -82,7 +82,7 @@ The inputs are loaded as follows: * **Pre-solvated systems:** ``SolvatedPDBComponent.from_pdb_file`` or ``ProteinMembraneComponent.from_pdb_file``. - See :doc:`Loading Molecules and Chemical Systems ` for examples. + See :doc:`Loading Molecules and Chemical Systems ` for examples. .. _prep_protein: @@ -180,7 +180,7 @@ results. Partial charges are assigned during protocol setup but can also be assigned in advance for reproducibility and to avoid repeated calculations. -See :doc:`Assigning partial charges ` (Python API) or +See :doc:`Assigning partial charges ` (Python API) or :doc:`Generating Partial Charges with the OpenFE CLI `. .. rubric:: Binding poses @@ -225,7 +225,7 @@ Checklist for preparing input ``Components`` See also -------- -* :doc:`Loading Molecules and Chemical Systems ` +* :doc:`Loading Molecules and Chemical Systems ` * :doc:`Components, Chemical Systems and Thermodynamic Cycles ` * :doc:`Membrane RBFE tutorial ` * :ref:`Protocols and domain of applicability ` \ No newline at end of file diff --git a/docs/guide/troubleshooting.rst b/docs/guide/troubleshooting.rst index 03c5e9339..1c73da8e6 100644 --- a/docs/guide/troubleshooting.rst +++ b/docs/guide/troubleshooting.rst @@ -193,7 +193,7 @@ Resuming Interrupted Simulations In the case where your ``openfe quickrun`` execution has timed out, was cancelled, or was otherwise interrupted, you can attempt to resume the simulation using the ``--resume`` flag, which is available as of openfe v1.11.0. -See :ref:`Resuming Interrupted Simulations ` for more details. +See :ref:`Resuming Interrupted Simulations ` for more details. .. _troubleshooting-log-debug-information: diff --git a/docs/cookbook/bespoke_parameters.nblink b/docs/how_to_guides/bespoke_parameters.nblink similarity index 100% rename from docs/cookbook/bespoke_parameters.nblink rename to docs/how_to_guides/bespoke_parameters.nblink diff --git a/docs/cookbook/choose_protocol.nblink b/docs/how_to_guides/choose_protocol.nblink similarity index 100% rename from docs/cookbook/choose_protocol.nblink rename to docs/how_to_guides/choose_protocol.nblink diff --git a/docs/cookbook/create_alchemical_network.nblink b/docs/how_to_guides/create_alchemical_network.nblink similarity index 100% rename from docs/cookbook/create_alchemical_network.nblink rename to docs/how_to_guides/create_alchemical_network.nblink diff --git a/docs/cookbook/dumping_transformation.rst b/docs/how_to_guides/dumping_transformation.rst similarity index 94% rename from docs/cookbook/dumping_transformation.rst rename to docs/how_to_guides/dumping_transformation.rst index 6214f8a6f..fffecbb56 100644 --- a/docs/cookbook/dumping_transformation.rst +++ b/docs/how_to_guides/dumping_transformation.rst @@ -1,7 +1,7 @@ .. _dumping_transformations: -Dump a ``Transformation`` to JSON -================================= +Write a ``Transformation`` to JSON +================================== If you're trying to run a full campaign of simulations representing an alchemical network, we generally recommend saving objects using our storage diff --git a/docs/cookbook/generate_ligand_network.nblink b/docs/how_to_guides/generate_ligand_network.nblink similarity index 100% rename from docs/cookbook/generate_ligand_network.nblink rename to docs/how_to_guides/generate_ligand_network.nblink diff --git a/docs/cookbook/hand_write_ligand_network.nblink b/docs/how_to_guides/hand_write_ligand_network.nblink similarity index 100% rename from docs/cookbook/hand_write_ligand_network.nblink rename to docs/how_to_guides/hand_write_ligand_network.nblink diff --git a/docs/cookbook/index.rst b/docs/how_to_guides/index.rst similarity index 78% rename from docs/cookbook/index.rst rename to docs/how_to_guides/index.rst index a20d37fcc..970218c62 100644 --- a/docs/cookbook/index.rst +++ b/docs/how_to_guides/index.rst @@ -1,13 +1,10 @@ -.. _cookbooks: +.. _how_to_guides: How-To Guides ============= -These guides show how to carry out specific tasks with **openfe**, mostly using the Python API. +These guides show how to carry out specific tasks with **openfe**. For the concepts behind each step, see the :doc:`User Guide `. -To run a transformation with ``openfe quickrun``, see :ref:`userguide_quickrun`. - -.. TODO: remove the quickrun sentence above once moved to "Running calculations" .. module:: openfe :noindex: @@ -51,11 +48,12 @@ To run a transformation with ``openfe quickrun``, see :ref:`userguide_quickrun`. :caption: Running calculations dumping_transformation - -.. TODO: add: run a transformation with quickrun, - run repeats in parallel, run on a SLURM cluster, resume interrupted calculations and - share a GPU using NVIDIA MPS (from guide/execution/quickrun_execution), and task-based - execution. + run_quickrun + resume_quickrun + run_on_slurm + run_repeats_in_parallel + run_with_mps + run_task_based .. toctree:: :maxdepth: 1 @@ -74,4 +72,3 @@ To run a transformation with ``openfe quickrun``, see :ref:`userguide_quickrun`. :caption: Repairing and extending networks repairing_broken_networks - diff --git a/docs/cookbook/jq_inspection.rst b/docs/how_to_guides/jq_inspection.rst similarity index 100% rename from docs/cookbook/jq_inspection.rst rename to docs/how_to_guides/jq_inspection.rst diff --git a/docs/cookbook/ligandnetwork_vis.nblink b/docs/how_to_guides/ligandnetwork_vis.nblink similarity index 100% rename from docs/cookbook/ligandnetwork_vis.nblink rename to docs/how_to_guides/ligandnetwork_vis.nblink diff --git a/docs/cookbook/loading_molecules.nblink b/docs/how_to_guides/loading_molecules.nblink similarity index 100% rename from docs/cookbook/loading_molecules.nblink rename to docs/how_to_guides/loading_molecules.nblink diff --git a/docs/cookbook/network_from_orion_fepp.nblink b/docs/how_to_guides/network_from_orion_fepp.nblink similarity index 100% rename from docs/cookbook/network_from_orion_fepp.nblink rename to docs/how_to_guides/network_from_orion_fepp.nblink diff --git a/docs/cookbook/packmol_memgen_setup.nblink b/docs/how_to_guides/packmol_memgen_setup.nblink similarity index 100% rename from docs/cookbook/packmol_memgen_setup.nblink rename to docs/how_to_guides/packmol_memgen_setup.nblink diff --git a/docs/cookbook/repairing_broken_networks.nblink b/docs/how_to_guides/repairing_broken_networks.nblink similarity index 100% rename from docs/cookbook/repairing_broken_networks.nblink rename to docs/how_to_guides/repairing_broken_networks.nblink diff --git a/docs/how_to_guides/resume_quickrun.rst b/docs/how_to_guides/resume_quickrun.rst new file mode 100644 index 000000000..a18018eaa --- /dev/null +++ b/docs/how_to_guides/resume_quickrun.rst @@ -0,0 +1,108 @@ +.. _howto_recover_quickrun: + +Recover from a failed or interrupted run +======================================== + +A quickrun job can stop early in two ways: + +* **A task failed.** quickrun finished, but printed an error naming the failed task. + The results file exists, with ``estimate`` and ``uncertainty`` set to ``null``. +* **The job was interrupted.** quickrun was stopped before it finished, for example by a wall-time limit, + a node failure or a cancelled job. There is no results file, and a cache file + ``/quickrun_cache/dag-cache-.json`` is left behind. + +Rerunning a failed task +---------------- + +If a task fails, quickrun retries it up to two more times, each in its own directory +(``..._attempt_1/`` and ``..._attempt_2/``). +If it still fails, quickrun stops, writes the results file with ``estimate`` and ``uncertainty`` set to ``null``, +and exits with an error naming the failed task. + +The full traceback of each failed attempt is stored under ``unit_results`` in the results file. +To print it, use ``jq`` (see :doc:`jq_inspection`): + +.. code-block:: bash + + jq -r '.unit_results[] | select(.__qualname__ == "ProtocolUnitFailure") | .name, .traceback' \ + results.json + +Once you have fixed the cause (see :doc:`/guide/troubleshooting`), run the Transformation again with a new +work directory and results file. + +quickrun refuses to start if the results file already exists. +For example, attempting to reuse ``results.json`` produces: + +.. code-block:: text + + Error: Invalid value for '-o': Path 'results.json' is a file. + +This error occurs because the ``-o`` option does not accept an existing file. +Choose a different output filename for the new run, or remove the existing results file. + +``--resume`` only works for interrupted runs, because quickrun removes the cache once a task has failed. +By then the task has already been retried twice, so resuming would most likely fail in the same way. + +.. _howto_quickrun_resume: + +Resuming an interruped job +-------------------------- + +When ``openfe quickrun`` starts, it saves a plan of the simulation to a cache file before execution begins: + +.. code-block:: text + + /quickrun_cache/dag-cache-.json + +where ```` is a unique identifier based on the ``-o`` file path and the Transformation. +This cache is removed once quickrun finishes, including after a failed task. + +If a job is interrupted (e.g. due to a wall-time limit, node failure, or manual cancellation), +you can resume it by passing the ``--resume`` flag: + +.. code-block:: bash + + openfe quickrun transformation.json -d workdir/ -o workdir/results.json --resume + +quickrun then uses the cache to skip the tasks that had already finished. +The hybrid topology, separated topologies and absolute protocols continue an interrupted simulation from its last +checkpoint, so only the time since that checkpoint is run again. +How often checkpoints are written is set by :attr:`~.MultiStateOutputSettings.checkpoint_interval` +in the protocol's output settings (see :doc:`choose_protocol` for how to change settings). + +.. note:: + + The same ``-d`` / ``--work-dir`` and ``-o`` arguments used in the original run must be specified + so that quickrun can locate the cache file. + +If you pass ``--resume`` but no cache file is found (e.g. the job never started), the following warning is printed +and a new run begins: + +.. code-block:: text + + openfe quickrun was run with --resume, but no cached results found at + . Starting new execution. + +If the cache file is corrupted (e.g. due to an incomplete write at the moment of interruption), +``quickrun --resume`` raises the following error. +In that case, start the run again in a new work directory. + +.. code-block:: text + + Recovery failed, please remove /quickrun_cache/dag-cache-.json + before executing a new transformation simulation. + +If you do not pass the ``--resume`` flag, quickrun detects the partially complete Transformation and refuses +to start, so that you do not accidentally start a duplicate run: + +.. code-block:: text + + Transformation has been started but is incomplete. Please remove + /quickrun_cache/dag-cache-.json and rerun, or resume + execution using the ``--resume`` flag. + +See also +-------- + +* :doc:`run_quickrun` – running a Transformation with quickrun. +* :doc:`/guide/troubleshooting` – common errors and how to fix them. diff --git a/docs/cookbook/rfe_alchemical_planners.nblink b/docs/how_to_guides/rfe_alchemical_planners.nblink similarity index 100% rename from docs/cookbook/rfe_alchemical_planners.nblink rename to docs/how_to_guides/rfe_alchemical_planners.nblink diff --git a/docs/how_to_guides/run_on_slurm.rst b/docs/how_to_guides/run_on_slurm.rst new file mode 100644 index 000000000..a908344e6 --- /dev/null +++ b/docs/how_to_guides/run_on_slurm.rst @@ -0,0 +1,53 @@ +.. _howto_run_on_slurm: + +Run quickrun on a SLURM cluster +=============================== + +On a [SLURM](https://slurm.schedmd.com/) cluster, we recommend running each Transformation as its own SLURM job by using ``openfe quickrun`` in a job script. + +For example, the script below runs the Transformation JSON file given as its argument on one GPU, and follows the convention of using the input file's basename as the output JSON's basename: + +.. code-block:: bash + + #!/bin/bash + #SBATCH --job-name=openfe + #SBATCH --gres=gpu:1 + #SBATCH --mem-per-cpu=2G + #SBATCH --time=24:00:00 + + # activate an appropriate conda environment, or run any "module load" commands required + conda activate openfe_env + + name=$(basename "$1" .json) + openfe quickrun "$1" -d "workdir/${name}" -o "results/${name}.json" --resume + +Adjust the ``#SBATCH`` options to your cluster, for example to add a partition or account. + +Save the script as ``run_quickrun.sh`` and submit one job for each Transformation JSON, either manually with + +` > sbatch run_quickrun.sh ` + +or by looping over all the Transformation JSONs in a directory. +For example, to submit jobs for all Transformations written by :ref:`cli_plan-rbfe-network` to the directory `network_setup/transformations/`: + +.. code-block:: bash + + for file in network_setup/transformations/*.json; do + sbatch run_quickrun.sh "$file" + done + +Once all jobs have finished, collect the results from ``results/`` with :ref:`openfe gather `. + +.. TODO: link the "Gather free energy results" how-to instead of the CLI reference once it exists + +The ``--resume`` flag lets you resubmit a job that hit its wall-time limit: it continues where it stopped. +On the first submission, when there is nothing to resume, quickrun prints a warning and starts a new run. +See :doc:`resume_quickrun` for details. + +To run the repeats of each Transformation as separate jobs, see :doc:`run_repeats_in_parallel`. + +See also +-------- + +* :doc:`run_quickrun` – running a Transformation with quickrun. +* :doc:`run_with_mps` – running several Transformations on one GPU. diff --git a/docs/how_to_guides/run_quickrun.rst b/docs/how_to_guides/run_quickrun.rst new file mode 100644 index 000000000..24170f284 --- /dev/null +++ b/docs/how_to_guides/run_quickrun.rst @@ -0,0 +1,50 @@ +.. _howto_run_quickrun: + +Run a Transformation with quickrun +================================== + +The CLI command ``openfe quickrun`` takes a :class:`.Transformation` saved as JSON and :doc: `executes it `. +If you planned your network with either of the CLI commands :ref:`cli_plan-rbfe-network` or :ref:`cli_plan-rhfe-network`, +you already have one JSON file per Transformation. +With the Python API, you can write them yourself with :meth:`.Transformation.to_json` (see :doc:`dumping_transformation`). + +The details of how simulations are executed are determined by the protocol. +With the Relative Hybrid Topology Protocol, each file is one leg of an edge, such as the complex or the solvent leg, +so you need to run both legs of every edge. +With the Separated Topologies, Absolute Binding, and Absolute Solvation Protocols, running one Transformation executes the entire thermodynamic cycle + +.. TODO: update the Relative Hybrid Topology Protocol description once the new hybrid topology protocol is released. + +To run a Transformation: + +.. code-block:: bash + + openfe quickrun .json -d workdir/ -o workdir/_results.json + +``-d`` / ``--work-dir`` + Where quickrun writes the simulation files: system setup, trajectories, checkpoints and analysis output. + The directory is created if it doesn't exist, and defaults to the current directory. + +``-o`` + Where quickrun writes the results JSON file. Any missing parent directories are created for you. + By default, the file goes in the work directory as ``_results.json``. + +You can point several Transformations at the same work directory, since each task writes to its own subdirectory. +Each Transformation does need its own results file, and quickrun won't start if the results file already exists. + +All repeats of the Transformation (set by ``protocol_repeats``) run one after another within the same quickrun call. +To run them as separate jobs instead, see :doc:`run_repeats_in_parallel`. + +When the run finishes, quickrun prints the free energy estimate and its uncertainty and writes them to the results file. +To collect the results of many Transformations and estimate DDGs and DGs, see :ref:`openfe gather `. + +If the run fails or is interrupted, :doc:`resume_quickrun` explains what to do next. + +.. TODO: link the "Gather free energy results" how-to instead of the CLI reference once it exists + +See also +-------- + +* :ref:`userguide_quickrun` – how quickrun executes a Transformation. +* :doc:`run_on_slurm` – submitting quickrun jobs to a cluster. +* :ref:`cli_quickrun` – all quickrun options. diff --git a/docs/how_to_guides/run_repeats_in_parallel.rst b/docs/how_to_guides/run_repeats_in_parallel.rst new file mode 100644 index 000000000..f697c4e10 --- /dev/null +++ b/docs/how_to_guides/run_repeats_in_parallel.rst @@ -0,0 +1,97 @@ +.. _howto_run_repeats_in_parallel: + +Run repeats in parallel +======================= + +By default, ``openfe quickrun`` runs all repeats of a Transformation one after another (three by default for most protocols). +When simulations are long, you can instead run each repeat as a separate job. + +Set the number of repeats per Transformation to 1 +------------------------------------------------- + +Each quickrun call then runs a single repeat. Set this when you plan the Transformations: + +* with the CLI, pass ``--n-protocol-repeats 1`` to :ref:`cli_plan-rbfe-network` or :ref:`cli_plan-rhfe-network`, or +* in Python, set ``settings.protocol_repeats = 1`` before creating the Protocol (see :doc:`choose_protocol`). + +Otherwise every job by default runs 3 repeats, and you end up with three times as many repeats as intended. + +Submit one job per repeat +------------------------- + +Each Transformation can be executed multiple times with ``openfe quickrun`` to produce independent repeats. + +Each repeat must use its own results file and work directory. +We recommend using directories named ``results_0``, ``results_1``, +and ``results_2``, since :ref:`openfe gather ` supports +this directory structure. + +On a SLURM cluster, save the following job script as ``run_repeat.sh``. +It takes the Transformation JSON file and the repeat number as arguments: + +.. code-block:: bash + + #!/bin/bash + #SBATCH --job-name=openfe + #SBATCH --gres=gpu:1 + #SBATCH --mem-per-cpu=2G + #SBATCH --time=24:00:00 + + # activate an appropriate conda environment, or run any "module load" commands required + conda activate openfe_env + + name=$(basename "$1" .json) + repeat=$2 + openfe quickrun "$1" -d "results_${repeat}/${name}" -o "results_${repeat}/${name}.json" --resume + +Then submit three jobs for each Transformation: + +.. code-block:: bash + + for file in network_setup/transformations/*.json; do + for repeat in 0 1 2; do + sbatch run_repeat.sh "$file" "$repeat" + done + done + +For more on adapting the job script to your cluster, see :doc:`run_on_slurm`. + +Each job runs one repeat and writes its output to a separate directory. +Make sure the number of protocol repeats was set to 1 when planning the Transformations, as described above. + +.. warning:: + + Although quickrun does not start if the results file already + exists, two jobs started concurrently with the same output path + may both pass this check and overwrite each other's results. + + Avoid this error by making sure all output paths are unique! + +This gives one directory per repeat, each with a results file and a work directory for every Transformation: + +.. code-block:: text + + results_0/ + ├── rbfe_lig_ejm_31_complex_lig_ejm_42_complex.json + ├── rbfe_lig_ejm_31_complex_lig_ejm_42_complex/ + │ ├── shared_HybridTopologySetupUnit-_attempt_0/ + │ ├── shared_HybridTopologyMultiStateSimulationUnit-_attempt_0/ + │ └── shared_HybridTopologyMultiStateAnalysisUnit-_attempt_0/ + └── ... + results_1/ + └── ... + results_2/ + └── ... + +Gather the results +------------------ +Pass the repeat directories to :ref:`openfe gather `, for example as ``openfe gather results_*``. +It searches them recursively and combines the repeats of each Transformation. + +.. TODO: link the "Gather free energy results" how-to instead of the CLI reference once it exists. + +See also +-------- + +* :doc:`run_on_slurm` – submitting quickrun jobs to a cluster. +* :doc:`run_with_mps` – running several jobs on one GPU. diff --git a/docs/how_to_guides/run_task_based.rst b/docs/how_to_guides/run_task_based.rst new file mode 100644 index 000000000..925a77433 --- /dev/null +++ b/docs/how_to_guides/run_task_based.rst @@ -0,0 +1,55 @@ +.. _howto_run_task_based: + +Run a network with task-based execution +======================================= + +.. warning:: Task-based execution is an experimental feature and subject to change in future releases of openfe. + +With task-based execution, you run a whole :class:`.AlchemicalNetwork` instead of one Transformation at a time. +For how it works, see the :ref:`userguide_task_based_execution` user guide. + +1. Plan the network as a single AlchemicalNetwork JSON file, using ``--networks-only``: + + .. code-block:: bash + + openfe plan-rbfe-network -M ligands.sdf -p protein.pdb --networks-only -o network_setup + +2. Create the Warehouse (``warehouse_/``) and task database (``tasks_.db``) for the campaign: + + .. code-block:: bash + + openfe setup-task-campaign --alchemical-network network_setup/network_setup.json --name my_campaign + +3. Run tasks. Each invocation of ``openfe run-task`` claims and executes one available task. + + .. code-block:: bash + + openfe run-task --warehouse warehouse_my_campaign/ --task-db tasks_my_campaign.db --scratch scratch/ + + For a network with many tasks, run multiple workers in parallel. + For example, on a SLURM cluster, you can use a job array to start 15 workers, + using the ``run_tasks.sh`` worker script from the CLI tutorial linked below: + + .. code-block:: bash + + sbatch --array=1-15 run_tasks.sh + + Each worker repeatedly calls ``openfe run-task`` until its wall-time limit is reached. + All workers share the same Warehouse and task database, which coordinate task execution. + + See :ref:`task_based_execution_cli` for an example worker script. + +4. Check the progress of the campaign: + + .. code-block:: bash + + openfe status --task-db tasks_my_campaign.db --summary + +5. Once all tasks are complete, convert the results to JSON files that :ref:`openfe gather ` can read: + + .. code-block:: bash + + openfe to-legacy-json warehouse_my_campaign/ -o result_jsons + +For a full walkthrough see the :ref:`CLI ` and +:ref:`Python API ` tutorials. diff --git a/docs/how_to_guides/run_with_mps.rst b/docs/how_to_guides/run_with_mps.rst new file mode 100644 index 000000000..3aceca137 --- /dev/null +++ b/docs/how_to_guides/run_with_mps.rst @@ -0,0 +1,16 @@ +.. _howto_share_gpu_with_mps: + +Share a GPU between runs with NVIDIA MPS +======================================== + +A single ``openfe quickrun`` call often does not use the whole GPU, particularly for small systems such as solvent legs. +NVIDIA's Multi-Process Service (MPS) lets several quickrun processes run on the same GPU at the same time. +Each run is slower than on its own, but the total throughput is higher. +See NVIDIA's documentation on +`MPS for OpenFE free energy calculations `_ for details. + +See also +-------- + +* :doc:`run_on_slurm` – submitting quickrun jobs to a cluster. +* :doc:`run_repeats_in_parallel` – running each repeat as a separate job. diff --git a/docs/cookbook/user_charges.nblink b/docs/how_to_guides/user_charges.nblink similarity index 100% rename from docs/cookbook/user_charges.nblink rename to docs/how_to_guides/user_charges.nblink diff --git a/docs/index.rst b/docs/index.rst index 9366a5386..465b46839 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -39,7 +39,7 @@ The **OpenFE** ecosystem is a free and open-source framework for planning, runni installation guide/index tutorials/index - cookbook/index + how_to_guides/index reference/index CHANGELOG diff --git a/docs/tutorials/index.rst b/docs/tutorials/index.rst index 8727cb115..7ff721342 100644 --- a/docs/tutorials/index.rst +++ b/docs/tutorials/index.rst @@ -46,7 +46,7 @@ Post-Simulation Analysis Generating Partial Charges -------------------------- -.. todo: this should be in cookbook +.. todo: this should be in how_to_guides - :ref:`Generating Partial Charges CLI tutorial `: how to use the CLI to assign and store partial charges for mall molecules which can be used throughout the OpenFE ecosystem.