diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 4f6dd610..eb9a0451 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -13,8 +13,8 @@ ## Checklist - [ ] Tests added/updated (asserting against a reference value where possible) -- [ ] `black`, `isort`, `flake8`, and `mypy` pass -- [ ] `coverage run -m pytest` passes and stays above the coverage gate +- [ ] `scripts/check.sh` passes (black, isort, flake8, mypy), with the tests of what changed +- [ ] Anything that must agree with the change is updated with it (see CONTRIBUTING.md, "What must agree") - [ ] `CHANGELOG.md` updated under `[Unreleased]` - [ ] Any Monte-Carlo tests are seeded (deterministic) diff --git a/.github/workflows/actions.yml b/.github/workflows/actions.yml index 93f214af..192ca330 100644 --- a/.github/workflows/actions.yml +++ b/.github/workflows/actions.yml @@ -2,12 +2,18 @@ name: actions run-name: RePyability CI -on: [push, pull_request] +# The one automated test run: the full suite on a pull request into master +# (from dev). Nothing else runs it; release.yml publishes only what it +# passed. +on: + pull_request: + branches: [master] jobs: lint: name: lint (black, isort, flake8, mypy) runs-on: ubuntu-latest + timeout-minutes: 15 steps: - uses: actions/checkout@v5 @@ -36,6 +42,7 @@ jobs: test: name: test (Python ${{ matrix.python-version }}) runs-on: ubuntu-latest + timeout-minutes: 120 strategy: fail-fast: false matrix: @@ -53,25 +60,32 @@ jobs: python -m pip install --upgrade pip pip install -e .[dev] + - name: pytest + # Every version runs the suite on all the runner's cores; coverage is + # measured once, on the newest Python (next step). + if: matrix.python-version != '3.14' + run: python -m pytest -n auto -q --durations=30 + - name: pytest (under coverage) - # From Python 3.12 coverage can measure through sys.monitoring, - # which counts the same lines as its default tracer (sys.settrace) - # several times faster: under the tracer, Python 3.12's job took - # over an hour. Python 3.11 has no sys.monitoring, and an empty - # COVERAGE_CORE leaves it the default tracer. + # Coverage is measured on Python 3.14 alone, through sys.monitoring, + # which is several times faster than its default tracer + # (sys.settrace): under the tracer, Python 3.11's job took an hour. + # pytest-cov collects it from each of pytest-xdist's workers. + if: matrix.python-version == '3.14' env: - COVERAGE_CORE: ${{ matrix.python-version != '3.11' && 'sysmon' || '' }} - run: coverage run -m pytest + COVERAGE_CORE: sysmon + run: python -m pytest -n auto -q --durations=30 --cov --cov-report= - name: coverage report (enforces the fail_under gate) + if: matrix.python-version == '3.14' run: coverage report - name: coverage html - if: matrix.python-version == '3.11' + if: matrix.python-version == '3.14' run: coverage html - name: Upload coverage html report artifact - if: matrix.python-version == '3.11' + if: matrix.python-version == '3.14' uses: actions/upload-artifact@v4 with: name: coverage-html-report @@ -83,6 +97,7 @@ jobs: # the compiled engine runs and its results are checked against Python's. name: test (with numba, Python ${{ matrix.python-version }}) runs-on: ubuntu-latest + timeout-minutes: 120 strategy: fail-fast: false matrix: @@ -102,13 +117,16 @@ jobs: python -c "import numba; print('numba', numba.__version__)" - name: pytest - run: python -m pytest -q + run: python -m pytest -n auto -q --durations=30 test-minimum: - # CI otherwise installs the newest surpyval, so once surpyval releases, - # nothing would test the oldest one pyproject.toml still allows. - name: test (minimum surpyval) + # The other jobs install the newest of every dependency, so nothing + # else tests the oldest that pyproject.toml allows: this one installs + # each dependency at its floor (numpy, scipy, networkx, surpyval, tqdm), + # on the oldest Python, then the test tools. + name: test (minimum dependencies) runs-on: ubuntu-latest + timeout-minutes: 120 steps: - uses: actions/checkout@v5 @@ -117,29 +135,20 @@ jobs: with: python-version: '3.11' - - name: Install the package, then the oldest surpyval it allows + - name: Install the package at its dependencies' floors run: | - python -m pip install --upgrade pip - pip install -e .[dev] - minimum=$(python - <<'EOF' - import re - import tomllib - - with open("pyproject.toml", "rb") as f: - dependencies = tomllib.load(f)["project"]["dependencies"] - spec = next(d for d in dependencies if d.startswith("surpyval")) - print(re.search(r">=\s*([0-9][0-9.]*)", spec).group(1)) - EOF - ) - pip install "surpyval==$minimum" - python -c "import surpyval; print('surpyval', surpyval.__version__)" + python -m pip install --upgrade pip uv + uv pip install --system --resolution lowest-direct -e . + uv pip install --system "pytest>=7.0" "pytest-xdist>=3.5" "pytest-timeout>=2.3" + python -c "import numpy, scipy, networkx, surpyval, tqdm; print('numpy', numpy.__version__, 'scipy', scipy.__version__, 'networkx', networkx.__version__, 'surpyval', surpyval.__version__, 'tqdm', tqdm.__version__)" - name: pytest - run: python -m pytest -q + run: python -m pytest -n auto -q --durations=30 docs: name: docs (strict build) runs-on: ubuntu-latest + timeout-minutes: 30 steps: - uses: actions/checkout@v5 diff --git a/.github/workflows/close-issues.yml b/.github/workflows/close-issues.yml new file mode 100644 index 00000000..fcdfbcc4 --- /dev/null +++ b/.github/workflows/close-issues.yml @@ -0,0 +1,56 @@ +name: close issues + +# Close the issues a pull request into dev finishes, when it merges. +# GitHub itself closes an issue only when the pull request that says +# "Closes #N" merges into the default branch (master), which here happens +# at a release: this closes them as the work reaches dev, with a comment +# naming the pull request. It runs no tests (the one automated test run +# is actions.yml, on the pull request from dev into master). + +on: + pull_request: + types: [closed] + branches: [dev] + +permissions: + issues: write + pull-requests: read + +jobs: + close: + name: close the issues the pull request finishes + if: github.event.pull_request.merged == true + runs-on: ubuntu-latest + steps: + - uses: actions/github-script@v7 + with: + script: | + const pull = context.payload.pull_request; + const body = pull.body || ""; + // The keywords GitHub takes, in a body line such as "Closes #N". + const pattern = /\b(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/gi; + const numbers = new Set(); + for (const match of body.matchAll(pattern)) { + numbers.add(Number(match[1])); + } + for (const issue_number of numbers) { + const { data: issue } = await github.rest.issues.get({ + ...context.repo, + issue_number, + }); + if (issue.pull_request || issue.state === "closed") { + continue; + } + await github.rest.issues.createComment({ + ...context.repo, + issue_number, + body: `Merged into dev by #${pull.number}: it reaches users with the next release.`, + }); + await github.rest.issues.update({ + ...context.repo, + issue_number, + state: "closed", + state_reason: "completed", + }); + core.info(`Closed #${issue_number}`); + } diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index aca39c99..d5646152 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -6,7 +6,8 @@ name: release # `gh workflow run release.yml --ref master -f version=X.Y`; or the REST # API). It checks that the version typed in matches # repyability/_version.py, that CHANGELOG.md has a dated section for it, -# that the tag vX.Y does not exist yet, that CI passed on the commit and +# that the tag vX.Y does not exist yet, that the full suite passed on +# these files (on the pull request from dev into master) and # that PyPI does not have the version. It then builds, publishes to PyPI, # and last creates the tag vX.Y and a GitHub Release. The release notes # open with the CHANGELOG section's summary, followed by GitHub's @@ -87,11 +88,26 @@ jobs: echo "::error::The tag v$version already exists." exit 1 fi - passed=$(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/actions.yml/runs?head_sha=$GITHUB_SHA&status=success" --jq '.total_count') - if [ "$passed" -eq 0 ]; then - echo "::error::CI (actions.yml) has not passed on $GITHUB_SHA; wait for it, or fix it." + # The full suite runs on the pull request from dev into master + # (actions.yml), whose run is recorded against dev's commit. The + # merge commit on master holds the same files, so a pass on this + # commit, or on a parent with the same files, counts. + repo="repos/$GITHUB_REPOSITORY" + tree=$(gh api "$repo/git/commits/$GITHUB_SHA" --jq '.tree.sha') + passed="" + for sha in "$GITHUB_SHA" $(gh api "$repo/git/commits/$GITHUB_SHA" --jq '.parents[].sha'); do + [ "$(gh api "$repo/git/commits/$sha" --jq '.tree.sha')" = "$tree" ] || continue + runs=$(gh api "$repo/actions/workflows/actions.yml/runs?head_sha=$sha&status=success" --jq '.total_count') + if [ "$runs" -gt 0 ]; then + passed=$sha + break + fi + done + if [ -z "$passed" ]; then + echo "::error::The full suite (actions.yml) has not passed on these files: on $GITHUB_SHA or the dev commit it merged. Open the pull request from dev into master and wait for it." exit 1 fi + echo "The full suite passed on $passed, with the same files as $GITHUB_SHA." pypi=$(curl -s -o /dev/null -w '%{http_code}' "https://pypi.org/pypi/repyability/$version/json") if [ "$pypi" = 200 ]; then echo "::error::repyability $version is already on PyPI." diff --git a/.github/workflows/upstream.yml b/.github/workflows/upstream.yml deleted file mode 100644 index 3dcd9067..00000000 --- a/.github/workflows/upstream.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: upstream - -# Runs the tests against surpyval's development branch. RePyability requires -# surpyval with no upper bound, so a surpyval release reaches every fresh -# install at once; a surpyval change that breaks RePyability should show here -# first, before surpyval releases it. A surpyval deprecation that RePyability -# runs into fails the tests too (filterwarnings in pyproject.toml), so a -# renamed name shows here while the old one still works. -# -# Kept apart from actions.yml, whose result the release workflow checks: a -# failure here is a warning about surpyval's next release, not about the -# commit under test. - -on: - pull_request: - push: - branches: [dev, master] - schedule: - - cron: "17 5 * * *" - workflow_dispatch: - -permissions: - contents: read - -jobs: - test: - name: test (surpyval develop) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v5 - - - name: Set-up Python - uses: actions/setup-python@v6 - with: - python-version: '3.11' - - - name: Install the package, then surpyval from its develop branch - run: | - python -m pip install --upgrade pip - pip install -e .[dev] - pip uninstall -y surpyval - pip install "surpyval @ git+https://github.com/derrynknife/SurPyval.git@develop" - python -c "import surpyval; print('surpyval', surpyval.__version__, surpyval.__file__)" - - - name: pytest - run: python -m pytest -q diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ae6b7bf3..cd369daf 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -8,17 +8,10 @@ repos: # Organise imports - repo: https://github.com/PyCQA/isort - rev: "6.0.1" + rev: "9.0.1" hooks: - id: isort - # Update old python syntax - - repo: https://github.com/asottile/pyupgrade - rev: "v3.19.1" - hooks: - - id: pyupgrade - args: ["--py310-plus"] - # Format with Black - repo: https://github.com/psf/black rev: "26.3.1" @@ -27,18 +20,19 @@ repos: # Lint with flake8 (reads [tool.flake8] from pyproject via flake8-pyproject) - repo: https://github.com/PyCQA/flake8 - rev: "7.3.0" + rev: "7.4.1" hooks: - id: flake8 additional_dependencies: ["flake8-pyproject"] # Uses the locally installed mypy (the mirror uses opinionated args by - # default); tests are excluded to match the CI configuration. + # default), on the whole package as CI runs it: checking the staged files + # alone misses errors their changes cause elsewhere. - repo: local hooks: - id: mypy name: mypy - entry: mypy + entry: mypy repyability language: system - types: [python] - exclude: "^repyability/tests/" + pass_filenames: false + files: "^repyability/" diff --git a/CHANGELOG.md b/CHANGELOG.md index 21887dd2..39e74b57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,875 @@ other release, fixes included, the minor. ## [Unreleased] +## [0.14] - 2026-10-16 + +Fitted models in, every kind of model brought up to the diagram, and +maintenance questions answered in model time. A component's life may be +what surpyval fits to a repairable unit's history (a Crow-AMSAA, Duane or +HPP process, or a Kijima `GeneralizedRenewal`), and a simulation starts +such a unit from its virtual age (#269); a fitted Wiener or gamma +degradation process is a life, and replacement on condition may go by its +measured level (#271); a component may operate part of the time +(`"duty"`, #276); and a regression node's covariates are levers (#272). +The alpha-factor common-cause model joins the beta factor and MGL (#311), +and the joint importance is worked out with common-cause groups (#317). A +`FaultTree` has the lifetime analyses, the remaining importance measures, +the sensitivities, levers and uncertainty a diagram has (#314), a +`Network` its importance measures, B-lives, uncertainty, saving and routes +(#315), and a `PhasedMission` its importance measures and cut sets (#316). +Over a window, the failure frequency, MUT and MDT are worked out with +common-cause groups whose tests take time (#293), the criticality and +Fussell-Vesely importance with repair crews around a nested RBD (#312), +and the availability, events and cost of a minimally repaired component +whose repairs take time (#318); `spares_stock` takes a block-replaced +component dead on arrival (#322). The maintenance module (#319) answers, +from a part's life and prices: its failures and their cost over a window, +whether to skip the next planned replacement (#350), the long-run optimal +replacement policy over a calendar of shuts (`optimal_calendar_policy`, +whose `CalendarPolicy` a `RepairableRBD` component takes, in a maintenance +group too, #349) and when to replace a part in service next, a calendar's +cost rate as the baseline (#348). Parameter uncertainty draws a fit's +shares too (#267), conditional survival keeps its precision at old ages +(#268), a regression node's mean and draws are exact, the importance +measures with common-cause groups whose tests take time are no longer NaN +(#294), and a mixture life with imperfect repair simulates in seconds +(#295). The 0.14 persona review's fixes run through the classes: the +fault tree, network and mission take the arguments the diagram takes and +refuse what it refuses, `expected_events` counts a common-cause group's +tests that take time and a test at its period's end once, a group's +members keep their own test offsets through `with_levers`, the cost +analyses refuse a diagram with nothing priced, and messages say what to +do. `RepairableRBD`'s code is in modules by analysis, with results +unchanged; the README shows what each model can do; surpyval 0.24 is +required; and what 0.13 deprecated is gone. + +Behaviour changes: surpyval 0.24 is required; seeded uncertainty results +of limited-failure-population and zero-inflated fits change, as their +shares are drawn (#267); a regression node's mean is no longer low, and +seeded draws of a diagram with one change; seeded results with a mixture +life and imperfect repair change in their last bits (#295); a cold +standby's grid and the density's steps are sized from the lives, so +results for lives far from a unit of time move, and a LogNormal's `mu` +moves proportionally as a time does in differential importance; a +probability per demand is refused by a `RepairableRBD`'s long-run methods, +and one of 0 is a junction; a component with hidden failures repaired +minimally is refused by the exact methods, which gave the renewed unit's +values; the cost analyses refuse a diagram with nothing priced, where +`cost()` returned `None` and the others zeros; `expected_events`, +`mission_unavailability` and the measures over a window change with a +common-cause group whose tests take time (the tests are planned outages, +and their transients are integrated), and `expected_events` no longer +counts a test at its period's end twice; `FaultTree.criticality_importance` +and `differential_importance` take the success-oriented form, and +`fussell_vesely` the path-set form, as the diagram does; a simulation +option given to an exact call is refused everywhere with a `ValueError` +(a diagram raised `TypeError`), and `mean_time_to_failure_interval` and +`reliability_interval` refuse fewer than two samples; a network, a diagram +and a tree with every element a fixed probability refuse `mean()` and +`barlow_proschan_importance()` (a network refused with any fixed element, +a diagram gave `nan`); a phased mission's `components` leave the junctions +out, and its routes refuse a component with no exact reliability; a run's +`delivered_fraction` is the run's estimate (exact or conditional by +default), the simulations' own being `sample_delivered_fraction`; +`node_availability()` lists the components alone; `differential_importance` +over several windows gives one share per window; `spares_demand`, +`spares_stock` and the parameter uncertainty methods refuse a nested RBD +by default; `analysis_routes()` returns an `AnalysisRoutes` dict that +prints one method a line; `CCFGroup` refuses one name for its members and +any model naming the three it takes; a `RepairableRBD` component given a +`CalendarPolicy` worked out for another life is refused; an error in a +user's model is raised where it happens rather than taken for something +the model cannot do; private names of `repyability.rbd.repairable_rbd` are +in the modules it was split into; and what 0.13 deprecated is removed: +`optimal_inspection_intervals(offsets=)` raises `TypeError` (give +`offset_shares=`), as does calling `CapacityDistribution.mean()` (a +property), `mc_samples` or `seed` given to the exact `mean` of a +`StandbyModel`, `LoadSharingModel` or `DegradingNode` raises `ValueError`, +and `rbd._sensitivity`'s `levers` and `Lever` are gone (use `levers()`, +`Lever` and `with_levers`). Deprecated, to go in 0.15: `NodeState`'s +`phase`, renamed `since_scheduled` (#350). + +### Added + +- **The alpha-factor common-cause model (#311).** + `AlphaFactor(alphas, testing=)` takes the shares `alpha_1, ..., alpha_m` + of a group's failure events that failed exactly one, two, ..., all `m` + members, the parameters common-cause data sources publish + (NUREG/CR-5485, the NRC's CCF parameter estimates) because event counts + estimate them directly, where MGL's letters cannot be counted. It gives + each specific set of `k` members the standard basic-event probability + `Q_k`: `alpha_k / C(m-1, k-1) · Q` for staggered testing (the default) + and `k alpha_k / (alpha_t C(m-1, k-1)) · Q`, `alpha_t = Σ k alpha_k`, + for non-staggered, so it is the `MGL` model with the letters those + `Q_k` give, and goes everywhere a group's model goes: non-repairable + diagrams and fault trees, conditioned on module by module; repairable + diagrams, through the groups' chains; `basis="rate"` and + `shocks="independent"`; saved with the diagram, its testing scheme with + it. Its levers, for `levers()`, `parameter_sensitivity` and the + parameter uncertainty, are `alpha_2, ..., alpha_m` (`ccf_alpha_2`, + ...), `alpha_1` being the remainder; `testing` is not a lever. A + rounded table's alpha factors, summing to within 1% of 1, are scaled to + 1. A `CCFGroup` refuses any other model naming the three it takes, + where it said alpha-factor was not supported yet. +- **The stock of a block-replaced component dead on arrival whose repairs + or block replacements take no time (#322).** `spares_stock` refused it + (#160): a unit dead on arrival fails as it is put in, and its + replacement, put in at that instant, may be too, so replacements come + several at one instant. With repairs and block replacements in no time, + each replacement of a unit that lasts, a failure or a block + replacement, brings a geometric number more at its instant (`g` more + with probability `q^g (1 − q)`, `q` the share dead on arrival), + independent of everything else: the demand in a lead time is the + lasting units' count compounded with it (`s` or more if, of the units + put in one after another, the `s`-th has fewer than that count lasting + before it, a binomial), and a demand has a geometric number of its + instant's before it. Where the repairs or block replacements take time, + a unit dead on arrival after one that took time fails on its own, and + the recursion from a typical replacement follows it on the grid, where + it keeps the units put in at a block time after its replacement apart + from those put in just before it. An exponential life 15% dead on + arrival agrees with the closed form (Poisson failures compounded) to + 1e-7, and a Weibull one with 5 million simulated windows within their + noise, whether its repairs, its block replacements, both or neither + take time. The README's "When is a simulation needed?" row no longer + excepts it. +- **The joint importance is worked out with common-cause groups (#317).** + `joint_importance` no longer refuses a diagram with common-cause groups, + in `NonRepairableRBD`, `FaultTree` and `RepairableRBD` (in the long run, + at times and over a window): the system given each of the pair working + and failed is conditioned on their states, as the single-node measures + condition on one (#294), a member's through its group's shock outcomes + (a repairable system's, its chain's joint states), two members of one + group on their joint state, and a node outside the groups held; where a + state has no chance at a point, the system with the pair held so. The + shared causes at zero give the pairs without the groups. The routes and + the README's tables say so. Underneath, one method of the conditioning + (`_ccf_modules.Evaluation.states`, with `given` and `held`) serves one + node or a pair, on the model's outcomes and the chains' tables alike, in + place of a method a node and a method a table. + +- **A phased mission's importance measures and cut sets (#316).** + `PhasedMission` has `birnbaum_importance()`, `improvement_potential()`, + `risk_achievement_worth()`, `risk_reduction_worth()`, + `criticality_importance(kind=)` and `fussell_vesely(method=)`, each + component held across every phase, working throughout or failed from + the start, by replaying the mission's decision diagram with its + segments so held: exact, at the mission's end, a small measure keeping + its precision in a reliable mission. The criticality is the share of + the mission's failures a component accounts for (given a failure, that + it would have succeeded with the component working throughout), which + is the diagrams' form for one phase. `minimal_cut_sets()` gives the + sets of events `(component, phase)`, the component failed by the end of + that phase, that together fail the mission, the phases' own cut sets + less those another implies (a failure by one phase is one by every + later phase); the exact Fussell-Vesely importance takes the probability + that some cut set naming a component has occurred from a decision + diagram of those events over the segments, and `method="rare_event"` + sums the cut sets' probabilities instead. `analysis_routes()` reports + each of the mission's analyses, and the README's capability matrix has + the new methods. Each phase's own importance, given that the mission + reaches it, is not offered. +- **A network's importance measures, B-lives, parameter uncertainty, + saving and routes (#315).** `Network` now has, over its links and + failing nodes, the measures a `NonRepairableRBD` has beyond Birnbaum's: + `improvement_potential`, `risk_achievement_worth`, + `risk_reduction_worth`, `criticality_importance`, `fussell_vesely` + (`fv_type`, `method`), `structural_importance`, + `differential_importance`, `joint_importance` and + `barlow_proschan_importance`, with the same definitions + (`rbd/_measures.py`, the formulas over the system's values with an + element held, which the diagram's methods share for the densities, + the Barlow-Proschan shares and the inversion of the reliability), each + from the network's decision diagram replayed with an element held, the + exact Fussell-Vesely from a decision diagram of each element's cut + sets; `bx_life` and `time_to_reliability`; `sf_uncertainty` and + `mean_uncertainty`, whose draws are a diagram's with the same seed and + sources; `to_dict`, `to_json`, `from_dict` and `from_json`, the links + and nodes saved as a diagram's nodes are; and `analysis_routes()`, + which refuses every analysis but the simulation for a network too + large for its diagram, with the message the methods raise. On the + bridge, each measure is the diagram's of the same system to 1e-12. +- **A fault tree's lifetime, density and hazard, the remaining importance + measures, its parameter sensitivities and levers, and the uncertainty + from fitted event models (#314).** `FaultTree` now has what + `NonRepairableRBD` had and it lacked, on the same structures: the mean + time to the top event (`mean`, the area under the exact probability that + it has not occurred, to about 1e-10), the B-lives (`bx_life`, + `time_to_reliability`, by root-finding), the density, hazard and + cumulative hazard (`df`, `hf`, `Hf`), the structural importance (from + the gates alone), the improvement potential, the Barlow-Proschan + importance (each event's Birnbaum importance times its model's density, + integrated), `parameter_sensitivity` (the Birnbaum importance times each + parameter's derivative; a common-cause group's parameters and its model's + by differences), `levers()` and `with_levers` (an event given as a + probability has the lever `p`, and stays a probability), and + `sf_uncertainty` and `mean_uncertainty`, drawn as the diagram of the + tree draws them (`to_rbd()`, converted once). Each honours the tree's + common-cause groups as `sf` does, conditioned module by module, and + gives the values of `to_rbd()`'s diagram. The lifetime analyses call the + functions `NonRepairableRBD`'s now are, with the tree's own evaluation + (`cumulative_hazard`, `density_by_differences`, `invert_reliability`, + `rate_parts`, `barlow_proschan`, `ccf_group_sensitivity` in + `rbd/non_repairable_rbd.py`), rather than convert the tree at each call. + `FaultTree.analysis_routes()` says how each of the tree's analyses is + computed, as a diagram's does, and the README's capability matrix shows + the methods. + +- **The long-run optimal replacement policy over a calendar of shuts + (#319).** `NonRepairable.optimal_calendar_policy(calendar)` finds, for a + part whose planned replacement costs differently at different times + (shuts on periods or dates, each `Opportunity` with its own price; a + dearer day, or none; a failure), the threshold age at each time and the + long-run cost rate (`CalendarPolicy`): what it does and costs by kind + of replacement (`replacement_rates`, `cost_rates`, failures included), + each shut's threshold and long-run chance of a replacement + (`schedule()`), and when a unit in service will next be replaced + (`next_replacement(age, now)`, a `PlannedReplacement`). Dynamic programming on a grid (exact for the grid over a + periodic calendar's cycle), integration between opportunities in + continuous time, or, for a cycle too long to sweep, a sampled cost rate + with its standard error. `calendar_policy_cost(calendar, rule)` prices + a given rule. A bearing (Weibull, 1000 days, shape 2.5) with an annual + shut at 1, quarterly at 2, weekly at 3, any day at 4 and a failure at + 20 costs 0.006943 a day, against 0.007000 replacing at every annual + shut and 0.01385 by age replacement on any day. In a `RepairableRBD`, + a component's `"preventive": {"policy": "calendar", "calendar": + policy}` replaces each unit at the first time its age reaches the + policy's threshold, charged the calendar's price there: simulated (in + the Python loop), refused by the exact and numerical analyses, and not + saved. In a maintenance group it is a member like any other (#349): + its planned replacements are stops of the group, and it takes an + `"opportunity"` age (any age: the policy has no interval) from which a + stop renews it early, charged the calendar's price for a planned + replacement on the day (where the calendar allows none, its + `replace_cost`), its next replacement then counted from the new unit; + the policy is not re-optimised for the group's stops. Decisions to + take on all of it: `repyability/maintenance/CALENDAR_DECISIONS.md`. +- **When to replace a part in service next (#319).** + `NonRepairable.optimal_next_replacement(age, now_cost=, + scheduled_outages=)` compares replacing it now, at each scheduled + outage (given as time from now and price), at a planned replacement at + its best time, and running it to failure. Each choice's cost is its + expected cost until the part is replaced, less the time it serves + valued at the baseline, the long-run cost a unit of time of its + `optimal_replacement_policy`, so with no outages replacing now is the + best from the optimal replacement age. Each choice comes with its + break-even age, the youngest at which replacing now costs no more + (`NextReplacement`, `ReplacementChoice`). A bearing 200 days old + (Weibull, 365 days, shape 2; $150 planned, $400 at a failure, outages + in 150 and 330 days at $100) is best kept to the outage in 150 days; + replacing now is the best from 297.6 days old. Where cheap shuts keep + coming, `policy=` makes the `cost_rate` of a `CalendarPolicy` the + baseline (#348; a `MaintenancePolicy` is taken too, its cost rate + alone refused): with a shut every 180 days at $100, a day of the bearing's + service is worth 0.990 rather than 1.117, and replacing now beats the + outage in 150 days from 256.1 days old rather than 288.4, and every + later choice from 263.9. +- **A part's failures over a window, and its costs given by name (#319).** + `NonRepairable` takes `planned_cost=` and `unplanned_cost=` (each alone + if need be; both as `set_costs_planned_and_unplanned` sets them), and + its `failures(t, age=)` gives, over a window `t` from the current + unit's age, the chance it fails, the expected number of failures (each + a replacement, new units' too, each taking the part's + `time_to_replace`; exact, with no simulation) and their expected cost + (`FailureForecast`). A bearing with a Weibull life (scale 365 days, + shape 2), 224 days old, fails in the next 82 days with a chance of + 27.8%, with 0.283 replacements expected. Right after a failure, a new + unit just in, `skip_next_replacement(every, resets=, time_to_next=, + discount_rate=)` says whether to skip the next planned replacement of + a plan in force (#350): the plan resets at each replacement (age) or + keeps to fixed times (block, the next `time_to_next` away), and the + result (`SkipValue`) is the present-value cost of doing the + replacement against that of skipping it, their difference (the NPV of + skipping) and the best, exactly, at a continuous discount rate per + unit of the life's time (0 by default). The bearing on a plan every + 180 days, at 8% a year, is best left: skipping saves $26.56 on a plan + that resets, and $123.08 on fixed times with the next replacement 30 + days away. Built one use case at a time + (`repyability/maintenance/README.md`); its calendar side (tasks on + dates, operating schedules, meters read by date) went to Reliafy, as + RePyability is about timelines and Reliafy about calendars. +- **The failure frequency, MUT and MDT with common-cause groups whose + tests, or another block's replacements, take time (#293).** They were + refused there (#220): a test or a block replacement that takes time is + a planned outage, at its time on the calendar, which takes the system + down if it was up just before and is not just after. With groups, + their members' joint states just before and just after each such time + now come from their chains (a member's test starting is one of their + jumps), the other nodes' as without groups. The common SIL case, a + 1oo2 with a beta factor and a proof test that takes the channel off + line, has its exact MUT, MDT and MTBF, and Barlow-Proschan importance + its shares; with no shared cause they are the independent members'. +- **Fitted repairable-unit models as components (#269).** A spec's + `"reliability"` may be what surpyval fits to a repairable unit's failure + history. A Poisson process (`CrowAMSAA`, `Duane`, `HPP`) is minimal + repair of the life whose cumulative hazard is its cumulative intensity + (a Weibull, or an exponential life), so its expected failures with + repairs in no time are exactly the fitted ones; a `GeneralizedRenewal` + (Kijima I or II) is its life distribution with the `"repair"` of its + Kijima model and restoration factor. The spec is saved as that life and + repair. surpyval's ARA, ARI and G1 renewal models are refused, their + repairs not being Kijima's (SurPyval#833). +- **A regression node's covariates are levers (#272).** A + `RegressionNode` at fixed covariates (a load, a temperature) lists each + covariate in `NonRepairableRBD.levers()` as `"covariate."` (the + model's feature name, or its place), `parameter_sensitivity` reports the + system's derivative in it, and `with_levers` builds the diagram with the + component run at another value. A covariate is unbounded: running + outside the fitted conditions is extrapolation, as an accelerated life + test's use level is, and is not refused. +- **A simulation starts an imperfectly repaired unit from its virtual age + (#269).** `NodeState(age=a, virtual_age=v)`: its virtual age `v` at its + last repair and its operating time `a` since; its life left is drawn + from virtual age `v + a`, and its next repair takes the virtual age on + by the whole time since its last. A fitted `GeneralizedRenewal`'s + `unit_states()` gives both. The exact methods still refuse such a state, + as do the simulations for a component also replaced after some + failures, maintained or tested, which would need its history since it + was renewed. Seeded runs from other states are unchanged. +- **Fitted degradation processes as lives (#271).** A surpyval + `WienerProcess` or `GammaProcess` fit (with its failure `threshold`) is + a node's life in `NonRepairableRBD` and a component's in + `RepairableRBD`: the first-passage time from its starting level, which + every exact method and both simulation engines take, and saving keeps. + One fitted with stress covariates is refused: fit it at the stress the + component runs at. +- **Replacement on condition by the measured level (#271).** + `"preventive": {"policy": "condition", "level": x, ...}`, in place of + `"threshold"`, for a component whose life is a degradation process: + each inspection measures its level and replaces it at or past `x`. The + level an inspection finds is drawn given that the unit has not failed + since the last (a gamma increment truncated at the threshold, or a + Wiener path killed there); surpyval does not give it yet (SurPyval#836). + Simulated only, on the Python engine: the exact methods and a start + state refuse such a component. Seeded runs without it are unchanged. +- **A component that operates part of the time (`"duty"`, #276).** A + `RepairableRBD` spec's `"duty"`, the fraction of the time the component + operates, puts a life fitted in operating time on the diagram's clock: + `R(d t)`, the same surpyval distribution with its scale moved, which + every method takes. Repairs, maintenance and tests stay on the clock; + levers, draws and saving keep the life as given. The rescaling rule of + each distribution is RePyability's until surpyval has one (SurPyval#845). +- **Criticality and Fussell-Vesely over time with repair crews around a + nested RBD (#312).** With limited repair crews, the two measures at + times or over a window are averages over the crews' Markov chain's + states, and refused a nested RBD, which the Birnbaum importance, the + improvement potential, the risk worths and `availability_rate` took + in. A nested RBD has crews of its own, so it is independent of the + chain: the chain's vectors are now taken for each pattern of the + nested RBDs up and down and weighted by their own availabilities at + each time, as `point_availability` is; over a window the ratio of the + integrals, by quadrature. A nested two-unit group with its own crew in + series with a pair sharing a crew: the pair's criticality is 0.317 five + time units in, for 0.513 in the long run. Checked against the closed + form from the two halves, and against seeded histories where the + structure couples them. +- **Minimal repair whose repairs take time, over a window (#318).** A + component repaired minimally (Kijima, `q = 1`) with repairs that take + time, a fitted Crow-AMSAA, Duane or HPP process with a repair time + among them, has its availability over time, its expected events and + cost over a window, and the importance and sensitivity over time + numerical, where every analysis but the simulation refused it. It fails + at its life's hazard at its operating age, the time it has been up, + which its repairs leave as it is, so its failures by operating age `a` + are Poisson with mean `H(a)` and the repairs so far a compound Poisson + sum: its availability and expected failures are sums, over the repairs + so far, of that Poisson probability against the distribution of their + total, on a grid of its operating age (`MinimalRepairCurve`, by + `minimal_repair_lattice`; the error falls as the square of the step, + about 1e-7). Checked against the two-state chain of an exponential + unit, the renewal equation (an exponential life is renewed by any + repair) and the simulations. The long-run values stay refused: a + minimally repaired unit's rate of failures need not settle. A run's + means are the numerical values by default, as for any system the + methods over time work out. `analysis_routes()` says so, and the + README's tables follow. + +### Changed + +- **The README shows what each model can do.** Its "What each model can + do" section has two kinds of table, made from the code (`python -m + repyability.tests.capabilities` writes them, and a test checks they are + current): which analyses each model type has, by method, and how each + family of analyses is worked out (exact, numerical, simulated or + refused) for each kind of node or component, from `analysis_routes()` + on the test catalogue's diagrams. +- **One automated test run.** The full suite runs on the pull request + from `dev` into `master`, on every core (`pytest -n auto`), with + coverage measured on Python 3.14 alone; nothing runs on pushes or on + pull requests into `dev` (CONTRIBUTING.md says what to run locally). + The `release` workflow publishes only what that run passed. +- **`NodeState.phase` is `since_scheduled` (#350).** The time since a + component's last block replacement or test placed it on its calendar as + its "phase", a word a phased mission uses for its own phases: the field + is `since_scheduled`, and so is the keyword + (`NodeState(age=300.0, since_scheduled=100.0)`) and what its `repr` + shows. `phase=` still works and warns (a `FutureWarning`), as does + reading `.phase`; giving both is refused. Both go in 0.15. +- **A roadmap and user stories.** `ROADMAP.md` holds the larger ideas and + their designs (first among them a net present value simulator for + maintenance plans), and `stories/` the questions users and agents should + be able to answer, with how we will know they are answered; the README + links both. Statuses are as of this release. +- **The README and the documentation mention Reliafy**, the hosted app built + on RePyability and SurPyval: the README's new "Related" section, the docs + home page, and one line at the end of the tutorial and of the repairable + systems and common-cause guides, linking the matching Reliafy guide. +- **Parameter uncertainty draws a fit's shares too (#267).** `"fit"` drew + only a distribution's own parameters (from surpyval's `hess_inv`) and + kept a limited failure population's share that ever fails (`lfp_p`) and + a zero-inflated fit's share dead on arrival (`f0`) at their fitted + values. Long after the units that fail have failed, the reliability is + that share alone, so its spread came out as nothing: with 42% of units + ever failing, known to 0.029, every draw had 42%. It now draws every + parameter the fit estimated from surpyval's `covariance()`, the shares on + the logit scale, in `sf_uncertainty` and the other uncertainty methods, + vega and both classes' `uncertainty_importance`, and seeded uncertainty + results of such fits change. An offset is still held where it was + fitted, as surpyval's covariance leaves it out (SurPyval#830); `"fit"` + now warns that it does. +- **`lfp_p` and `f0` are parameters (#267).** Where a node's model has a + limited failure population or zero inflation, `parameter_sensitivity`, + `levers()` and `with_levers` take the shares as parameters, after the + distribution's own (the order of surpyval's `covariance()`), with the + range (0, 1), and an uncertainty `{parameter: distribution}` may name + them. +- **A `RegressionNode`'s mean and draws are exact.** Its `mean()` + integrated the survival curve tabulated on 4,096 points and stopped + where the survival fell to 1e-4, so it came out low (8e-5 for a + LogNormal AFT model; the heavier the tail, the lower), and its draws + were read off that grid, so none fell past its end. The mean is now + integrated as an RBD's MTTF is, to a relative 1e-10, on pieces split at + the model's quantiles or the schedule's change points, and `inf` where + some lifetimes never end (a `ValueError` before). A draw is the model's + quantile `qf(u, Z)` at fixed covariates (surpyval 0.24), and along a + schedule the time at which the cumulative hazard reaches `-log(1 - u)`, + by bisection. Seeded draws of a diagram with a regression node change. + surpyval's own `mean(Z)` is not used: on an additive-hazards model with + a Normal, Logistic or Gumbel baseline it integrates a survival function + above 1 before 0 (SurPyval#828), a model `RegressionNode` refuses. +- **surpyval 0.24 or later is required** (0.23 was). It gives a mixture + life a survival function that keeps its precision in the far tail, and + draws a life given an age (imperfect repair) reading a limited failure + population's share by its own name, `lfp_p`, which a mixture lacks. The + mixture's new quantile function loses the long lives' precision and is + slow (SurPyval#821), so the simulations keep drawing a mixture's lives + with RePyability's own; seeded results are unchanged. +- **The documentation is easier to find one's way in.** The API reference + is a page per section (the largest 0.9 MB, where the one page was + 5.7 MB with its source inlined), each class's page lists what it + inherits, and docstrings no longer show bare issue numbers. Maintaining + a system (preventive maintenance in a diagram, replacement on + condition, opportunistic maintenance, hidden failures and their + intervals) has its own guide page, out of Costs. Concepts is the + reference for how RePyability computes each value, and links the Learn + lesson that teaches an idea where it restated it. Three statements were + wrong: Concepts had `mean_time_to_failure()` average simulated + lifetimes (it integrates exactly by default), and the Learn lesson on + dependence had load sharing of identical non-exponential units + simulated (it is numerical) and the standby arrangements without a + reliability simulated with `seed=0` (since 0.12 they refuse, and the + system's simulations draw them). +- **`RepairableRBD`'s code is in modules by analysis.** + `repyability/rbd/repairable_rbd.py` (some 20,500 lines) holds the class + alone (some 7,100): its constructor, its public methods, each calling a + function of the same name in a module of its kind (`_runs`, + `_event_loop`, `_curves`, `_windows`, `_long_run`, ...), and the hooks + `RBD` calls. Results are unchanged. Code that imported private names + from `repyability.rbd.repairable_rbd` (`_PythonRunner`, `_aged_life`, + the worker functions, ...) finds them in those modules; the criticality + indices of a run's counts are still importable from there. A + simulation's state is one object (`_events._Run`), which a run's end + forgets whole (it had left the common-cause groups' state behind), and + `system_state`, `component_status`, `t_simulation` and + `last_change_planned`, which `initialize_event_queue` and `next_event` + document as kept on the diagram, are read-only properties of the run in + progress. +- **A fault in a model is raised, not taken for something it cannot do.** + Where RePyability probes a node's model (its quantiles to split a grid + at, its mean, whether its rate is constant, whether a group's members + save alike), it caught any error as "the model cannot do this" and went + on without it. It now catches only what a model that cannot take the + values raises (an arithmetic, type, value, lookup, attribute, + not-implemented or runtime error; a save's type, value, attribute or + not-implemented error), so a fault in a user's model, a `NameError` or + an `AssertionError` say, is raised where it happens. `analysis_routes()` + likewise reports only refusals, and raises any other error a check + meets. + +### Deprecated + +- **`NodeState.phase`, renamed `since_scheduled` (#350).** Giving `phase=` + and reading `.phase` still work and warn (a `FutureWarning`); 0.15 + refuses them (`deprecation.NEXT_REMOVAL`). + +### Removed + +- **What 0.13 deprecated.** `optimal_inspection_intervals` no longer takes + `offsets=`, renamed `offset_shares=` as its values are shares of the + interval where `with_intervals`' and the plan's `offsets` are times + (#222): passing it raises `TypeError`. `CapacityDistribution.mean` is a + property alone: calling it, `mean()`, raises `TypeError` (#235). The + exact `mean` of a `StandbyModel`, `LoadSharingModel` or `DegradingNode` + refuses `mc_samples` and `seed`, which it ignored, with the `ValueError` + every exact call gives a simulation option (#233). `rbd._sensitivity`'s + `levers` and `Lever`, the names callers used before the levers were + public, are gone: use `levers()`, `Lever` and `with_levers` (#244). + `test_removed_in_0_14.py` keeps them gone. + +### Fixed + +- **A fixed maintenance duration is fixed whatever numpy's rounding.** A + duration known exactly (an `ExactEventTime`) is followed as a fixed + time, where a spread one is convolved; it was told from its variance, + a difference of two numbers about the duration squared, against a + tolerance a few units of rounding wide: on numpy 2.0 the variance came + out at 1e-15, and the exact and numerical analyses of a component + maintained for a fixed time refused ("its later age replacements ... + are too many ... to follow exactly"). The tolerance is a millionth of + the duration. +- **A shut at an offset off the grid keeps its period (#319).** The + grid's step for `NonRepairable.optimal_calendar_policy` was sized to + put every period, offset and date on it, and fell back to the + life-sized step when that made it far finer: a shut every 180 days at + an offset of 0.1234 days (three hours into the day, what a date turned + into model time gives) then had a step that did not divide 180, so the + calendar lost its period, `"auto"` sampled it (57 s for 0.5 s, the + cost rate 0.04% off, no `period`) and its schedule drifted (0.0, + 180.17, 360.03). The step is now a whole fraction of the periods (and + a plan's repeat), the offsets and dates on it too only where that + keeps it near the life's; an offset or date off the grid is laid at + the grid time nearest it, within half a step, with the warning that + said so already. The policy is the aligned calendar's: the same step, + cost rate, thresholds and chances, exact, `period` 180, in the same + time. +- **The maintenance planner, from the 0.14 persona review.** + `optimal_calendar_policy` warns when `"auto"` falls to sampling (the + cycle has more than 20,000 steps; each window is an exact solve of a + second or more, so `mc_samples` sets the cost), and its docs say that + a sampled policy's rates by kind and the schedule's chances are + estimates from a unit run in, with no standard error (only + `cost_rate_se` has one); it refuses a calendar that allows no planned + replacement at all (no opportunities, and none on any day), as + `optimal_replacement_policy` refuses with no costs, where it returned + running to failure; a `CalendarPolicy` carries the saved form of the + life it was worked out for (`life`), and a `RepairableRBD` component + given a policy worked out for another life is refused, naming both + (it was accepted, and simulated at a cost rate 19% off); and + `next_replacement` replaces a unit past the oldest age the thresholds + were worked out to at the next opportunity (or on any day now), where + it answered `inf`. `optimal_next_replacement` takes a planned + replacement as one later than now: where no later time beats + replacing at once, the planned choice is replacing now, at `now_cost` + (it was the planned price at time 0, which `now_cost` contradicted), + so `now` is True exactly when the best choice is at time 0. + `skip_next_replacement` on a plan that resets warns of a discount + rate that discounts the costs away, as the plan on fixed times and + `RepairableRBD.expected_cost` do, and gives `replace` and `skip` one + meaning, the present value of the whole future under each choice: + with no discounting both are endless (`inf`), and only `npv` is + finite (they held the first cycles' comparison terms, 0 and -26.33). + `MaintenancePolicy` and `FailureLimitPolicy` are result mappings like + every other result, with `to_dict()` for JSON. +- **A junction is no component of a phased mission (from the 0.14 persona + review).** A `PerfectReliability` node of a phase's diagram (a vote + point, or one `FaultTree.to_rbd()` draws) was a component of the + mission that could fail: it had a Birnbaum importance (the mission's + reliability), a risk achievement worth and a place in every measure's + dict, where the diagram's measures leave it out. The mission's + `components` now leave the junctions out, as the diagram folds them out + of its structure, so the measures have the real components alone, and + the mission of one phase gives the diagram's values at the phase's end. + `PhasedMission.analysis_routes()` also said every analysis was exact + for a mission with a component that has no exact or numerical + reliability (a `StandbyModel` only simulations take), while `sf()` + raised: the exact analyses are now routed as refused with that + component's message, and as numerical when a component's reliability + is, as the diagrams' routes have it. +- **An irrelevant node behind a junction is found (from the 0.14 persona + review).** `find_irrelevant_components()` and + `structure_check["irrelevant_nodes"]` missed a node in parallel with a + path through a `PerfectReliability` junction alone (`s-a-j-t` with + `a-x-t`): `x` is in no minimal path set and has a Birnbaum importance + of 0, but the structure was read before the junction was folded in. + They now read the structure with the junctions folded in, as the path + and cut sets do; a junction itself is no component and is never listed. +- **A network refuses surpyval's distribution itself (from the 0.14 + persona review).** `Network` took `surv.Weibull` (the class, not a + model of it) for a link or node, and failed later with surpyval's own + error; it now refuses it where it is given, as the diagrams and trees + do, saying to fit it or give its parameters. +- **A fixed probability among lives, in a network (from the 0.14 persona + review).** `Network.mean()`, `barlow_proschan_importance()` and + `mean_uncertainty()` refused a network with any fixed-probability + element, even a perfect link given as `0.0`, where a diagram and a + tree give the finite answer: a fixed element fails at the start or + never, so it has no density (no share of the failures over time) and + the mean is `inf` where it keeps the terminals joined. They now refuse + only when every element is fixed, and `mean(method="simulate")` is + taken likewise. With every element fixed, a diagram's and a tree's + `barlow_proschan_importance()` gave `{node: nan}` silently where the + network refused: all three now refuse it, with the mean's reason (no + lifetimes), and `analysis_routes()` says so. +- **The classes' methods take the same arguments (from the 0.14 persona + review).** `FaultTree.criticality_importance(t, kind=)` and + `differential_importance(t, kind=)` take the success-oriented form, + and `fussell_vesely(t, fv_type=, method=)` the path-set form, as + `NonRepairableRBD` and `Network` do, agreeing with `to_rbd()` with and + without common-cause groups. `FaultTree.mean(method="simulate", + mc_samples=, seed=)` and `FaultTree.random(size, seed=)` estimate and + draw the times to the top event through the tree's diagram (the draws + are `to_rbd()`'s), where a tree had no simulation at all. `Network` + has `minimal_cut_sets()` and `minimal_path_sets()`, lists smallest + first as every other class gives them (`cut_sets()` and `path_sets()` + stay, as sets), and the README's table shows them. A simulation + option given to an exact call (`mean(mc_samples=100)`, + `sf(50.0, seed=1)`) is refused the same way everywhere, a + `ValueError` saying it applies only to `method="simulate"`: a diagram + raised `TypeError`, a network and a mission `ValueError` for + `mc_samples` and nothing for `seed`. `mc_samples` and `size` are + checked in `NonRepairableRBD.mean(method="simulate")` and `random()` + (0 gave `nan` with numpy's warnings, 1.5 a numpy error), and an + interval (`mean_time_to_failure_interval`, + `PhasedMission.reliability_interval`) refuses fewer than two + samples, where it returned a `nan` interval silently; both say + `method="simulated"` in their `ConfidenceInterval`. A common-cause + group's own parameters can be given to the uncertainty under its + members' tuple by their lever names (`{("a", "b"): {"ccf_beta": + dist}}`, as `levers()` keys them), which was refused as not a + parameter of the members' model ("Did you mean 'beta'?"), besides + under the `CCFGroup` as before. +- **Messages and printing (from the 0.14 persona review).** A misspelt + keyword (`rbd.sf(t=50)`) is Python's `TypeError` for it, where the + time check said "x is required", and that message names the kind of + diagram ("this network", "this fault tree"). `RBD.from_dict` of a + document with no `"type"` says so. `bx_life("10")` is refused as not + a number, and a B-life the reliability never falls to (a fixed + survivor in parallel keeps it at 0.9) says the target is never + reached and what the reliability never falls below, where it asked + for an `upper_bound`. A number landing in `working_nodes` or + `broken_nodes` (`mission_availability(0, 100)`) is refused naming the + argument, where it was "'int' object is not iterable", through one + check every method that holds nodes shares. `analysis_routes()` + returns an `AnalysisRoutes` dict that prints one method a line, + `name: route -- reason`, with `to_dict()` for JSON, in every class. + `NonRepairable` prints its life model, its replacement and its costs. + The measures at `Q = 0` (`t = 0`) give their documented `nan` without + numpy's warnings; a network's measure of an irrelevant link prints + `0.0`, not `-0.0`. `with_levers` names the kind of diagram in its + message for an unknown key (a diagram no longer mentions the repair + crews). `path_set_probabilities` refuses a missing node with a + `ValueError` naming it, and a time in place of the probabilities, and + its values are in `minimal_path_sets()`'s order, as documented. + `FaultTree.from_rbd` warns that a diagram's capacities are left out. + `FaultTree.sf` and `Network.sf` (and their measures) take a 2-D array + of times, shaped as given, as a diagram's do. `levers()` and the + Greeks guide say that nested diagrams' and composite models' units are + not reached. The tutorial's controller criticality reads 0.916 (it + said 0.922), the costs lesson's share of years above the mean 0.48, + and the quickstart uses `minimal_path_sets()` and + `minimal_cut_sets()`. +- **`expected_events` counts a common-cause group's tests that take + time (from the 0.14 persona review).** A group whose members' tests + take time (a `duration`) had no planned outages over a window, and its + down time was the instant tests' (a 1oo2 pair tested together for 8 + hours a year with a 0.1 beta factor: 0 outages and 84 h of down time + over ten years, where the same pair without the group counts 9 and + 81 h, and `mission_unavailability` gave 156 h): the groups' chains + were followed between the pieces of the integrals without their tests + among the pieces' edges, and a test's jump was no event. The members' + tests that take time are now the system's planned outages at exact + times, from the chains just before and at each test, as a node's + scheduled outages are, and the chains' bends are the integrals' + pieces' edges: `system_planned_outages`, `system_downtime` (now `t - + mission_availability(t) · t` again) and the members' down times agree + with the pair without the group when nothing is shared, and with the + simulation. +- **`mission_unavailability` with a common-cause group whose tests or + repairs take an exponential time.** The transient after a test, at + the test's or repair's rate, fell inside pieces of the integral far + longer than it and was missed: the same pair's tests of 8 hours on + average gave the instant tests' value (0.00096 for 0.00138 over ten + years). The group's chain now bends, for the integrals, at geometric + offsets after each test at the transients' scales, and the integral + of the unavailability refines to those scales, as it does to a unit's + closed form. `mission_availability` and `expected_events` take the + same pieces. +- **A common-cause group's members keep their own test offsets through + `with_levers`, and `levers()` lists them.** A group's levers were the + first member's spec, rebuilt for every member: moving the group's test + interval (or any lever) lost a staggered member's `offset`, so the + derivative `parameter_sensitivity` reported for the interval was the + pair tested together's (-9.8e-5 for -5.8e-5), and with tests that can + miss a failure nonsense (-10.4). The levers the members hold at one + value are the group's, moved in each member's own spec; a member's + offset, its place on the calendar, and any lever the members hold at + different values, are the member's own, under its name, in the range + the constructor checks (`[0, interval)` for an offset). A group's test + at the end of its calendar's period could also be worked out a + rounding past it (3 × 99.9 for 299.7) and skipped by the chain, which + then lost that member's full test (the unavailability doubled at + intervals near 100): it is now the period's end. +- **`expected_events` counted a test at the end of its period twice + (from the 0.14 persona review).** The counts over a window are + extended, once every component has settled, a period at a time: a + test a rounding short of the period's end (248.642 + 6 × 731.3, for + a period ending at 248.642 + 5 × 731.3 + 731.3) fell in the period + and was counted again in the extension, a quarter more planned + outages for a staggered pair tested every 731.3 hours. The period + now ends at such an event. +- **From the 0.14 persona review, smaller.** `BetaFactor` and `MGL` + check their parameters as numbers, as `AlphaFactor` does (`True` read + as 1, and `"0.1"` raised a bare `TypeError`); `CCFGroup` refuses one + name for its members (`"v1"` read as the members `v` and `1`); + `differential_importance` over several windows gives one share per + window, as the other measures do, where it gave the first window's; + the importance measures at a test's instant are documented as of the + state just after it (a pair tested together cannot then be down, and + its ratio measures are 0/0, NaN), as `point_availability` is; a + shared cause's system failure in `simulate_timelines` is documented + as the member's whose fall took the system down, and the timelines + guide no longer says groups are refused; the README's capability row + for a beta-factor group with revealed failures shows the group's own + routes, exact, from a diagram of exponential members and repairs (its + catalogue diagram had a block-replaced node); and a model's constant + failure rate is kept while the model lives, where every diagram built + around the same model (hundreds, in `optimal_inspection_intervals`) + worked it out tens of times over: a staggered search over four + intervals for a 2oo3 MGL group takes 2.6 s for 13 s, and without the + group 0.6 s for 6.8 s. +- **A component with hidden failures repaired minimally was computed as + if every repair renewed it.** A spec with `"repair": {"q": 1}` and an + `"inspection"` went to the tested unit's curve before the imperfect + repair was looked at, so `point_availability`, `mission_availability`, + `expected_events` and `expected_cost` returned the renewed (`q = 0`) + unit's values (0.45 at 40 for a Weibull life (10, 1.5) tested every 10, + against 0.29 simulated), while `analysis_routes()` said they refuse, + and `availability()` took that value as its exact mean. They now refuse + as the routes say (its failures are found only by its tests), and a + run's means fall back to the simulations' own. +- **The cost analyses refuse a diagram with nothing priced.** With no + cost at all, not even an acquisition cost, `cost()` returned `None` + (and `.mean` of it failed), `expected_cost` zeros and + `expected_cost_rate` and `total_cost` `0.0`, while `analysis_routes()` + said `cost` was simulated. They now refuse with `routes.Refused` + ("Nothing is priced: give the components a repair_cost, ..."), as + every analysis refuses a diagram with nothing it works on, and the + routes say so; `compare(quantity="cost")` refuses the same way. With + an acquisition cost alone they give the one-off purchase and no + running cost, exactly, as before, and `availability()`'s `cost` stays + `None` with nothing at all priced. +- **From the 0.14 persona review of repairable systems.** A nested + `RepairableRBD` that carries costs is warned of when the outer diagram + is built, naming the node, as the outer one does not count them; in + `expected_events` a nested node's `node_preventive` and + `node_corrective` are its system's planned outages and failures, as + the simulation counts them for the node (its preventive count was 0); + `spares_demand` and `spares_stock` refuse by default with a nested RBD, + naming the components counted, where they left its components out + silently. The parameter uncertainty methods say, by default and given a + nested node, that a nested diagram's fits are not drawn here (its own + methods take them), the default refuses with `routes.Refused` when no + component's model is a fit, and `analysis_routes()` quotes that + refusal in the methods' reason (they stay simulated, given the uncertain + nodes). `parameter_sensitivity` over time leaves out the `repair.q` + lever of a minimally repaired component (`q = 1`), where it reported + `nan`, as the values over time take `q = 1` only. One `RepairableRBD` + object given for two nodes is refused as the outer diagram is built, + naming the nodes, where a simulation failed late. A run's + `delivered_fraction` (the production availability) is taken as its + `mean_availability` is: exact where `mission_capacity` works it out, + given the modules' histories where a conditional run applies, and the + simulations' own otherwise (`sample_delivered_fraction`), with the + interval's `method` filled in every case. `node_availability()` lists + the components alone, as `node_sf` and the importance measures do, + without the input and output nodes. `NodeState` and the guide say a + state's `age` is on the diagram's clock (a duty component's operating + age is `duty * age`). `spares_demand` convolves on a grid once per + replacement likely in the horizon, with each distribution's transform + taken once (about half the time, to the bit), and its docstring says + how the cost grows; a diagram rebuilt with a lever moved reuses the + renewal cycles and tested units of the components the lever leaves, + for `parameter_sensitivity`. +- **Spares from new of a block-replaced component dead on arrival, put in + at a block time in no time (#322).** The unit a block replacement in no + time puts in may be dead on arrival, and the one put in at its failure, + in no time, is at the block time too, after its replacement: it is not + replaced there again. On the grid rounded up, `spares_demand` took it + for one put in just before the block time and replaced there, unless + dead on arrival too, and so counted replacements that never come: a + Weibull life (100, shape 2) 10% dead on arrival, block replaced every + 60, was 0.035 out by 130 (the chance of exactly three replacements + 0.331, for 0.366 simulated). The count of a typical replacement's + followers with repairs or block replacements that take time now keeps + such a unit apart too (a block replacement or a repair in under a step). +- **A mixture life with imperfect repair simulates in a few seconds + (#295).** A component whose life is a `MixtureModel` and whose + repairs are imperfect (Kijima) drew each life left given its virtual + age through surpyval's `conditional_gaps`: the mixture's quantile + below a cumulative hazard of 20, and a root search of the mixture's + `H` above it, which a Kijima unit's growing virtual age reaches ever + more often (a 15,000 h run of the issue's diagram took about 53 s, + and 20,000 h did not finish). It is now drawn by Newton's steps on the + mixture's cumulative hazard, worked out from its components, in about + 4 evaluations a draw (that run: about 2.6 s). The draws are the same + roots to the last bits of the virtual age plus the life left, so + seeded results with such a component change only in their last bits + (about 1e-15 on the catalogue's diagram). +- **Importance measures with common-cause groups whose tests take time + (#294).** While a member's test kept it off line it could not be up, + and the system given it up was 0/0, so Birnbaum's measure, the + improvement potential, the risk reduction worth, the criticality and + the differential importance were NaN for every node, with no warning. + Where a member's state has no chance, the system given it is now the + system with the member held in that state, as for a node outside the + groups: with no shared cause the measures are the independent members' + own. +- **A component that never fails is a junction again.** A `RepairableRBD` + component whose life is a probability of failing of 0 + (`FixedEventProbability` at 0, as a vote point is often drawn) was + refused by the exact methods since a probability per demand was (and + before that was read as a mean life of 0, an availability of 0). It is + now a junction, as a life of `PerfectReliability` is: it never fails, + every analysis leaves it out, and a cost or schedule given it is + refused. +- **Conditional survival keeps its precision at old ages (#268).** The + chance of surviving a further `x` given survival to an age `X` was the + ratio `R(X + x) / R(X)`, which came out 0 once both were too small for a + float: for a Weibull(100, 3) unit 1000 hours old, the next 10 hours' + survival is 6.9e-14, where it gave 0. It is now worked out from the + model's cumulative hazard, `exp(-(H(X + x) - H(X)))`, wherever the model + has one (surpyval's models, and a diagram's own `Hf`), in + `NonRepairableRBD.cs` and a node's age in `sf_given_state`. A diagram's + own `cs` is still 0 once its reliability at `X` is below the smallest + float, as its `Hf` is then infinite. +- **A probability per demand has no long run in a `RepairableRBD`.** A + `FixedEventProbability` life (a unit that fails at once with that + probability, or never) was read as a mean life by the long-run methods: + `p = 0.1` gave an availability of 0.09 where the simulations give 0.999. + The long-run values, frequencies, cost rate and importance now refuse + it, as the values over time did, and the routes say so. +- **`RegressionNode` takes surpyval's accelerated-life models.** An + Arrhenius, Eyring or power-law life was refused, its covariate count + taken from its life model's parameters (two, for one stress); it is now + the number of stresses the model was fitted with. +- **Results no longer depend on the unit of time.** A cold-standby + group's grid started at time 1 whatever its lives' scale, and the + density and hazard (and the reliability rate's parts) took steps of + `1e-6` of time below time 1: in a diagram whose lives were about 1e-5 + of the unit long, a cold standby's probability of failing was off by + up to 27% (and its density by 5%). Both + are relative now: the grid starts at the typical life, a step is `1e-6` + of the time (of the shortest typical life at time 0). A regression + node's check for a proper survival curve looks just after 0, not at + `1e-9`. +- **Proportional differential importance over parameters does not depend + on the unit.** A LogNormal's `mu` (the log of a time) was moved by a + share of itself, which changes with the unit (in hours its share of one + diagram was 0.89, in years 19); a proportional change of it is now a + proportional change of the time (`d mu = epsilon`), as for a Cox-Lewis + process's `alpha`. A regression node's covariate, whose zero is its + unit's, is refused (`change="uniform"` takes it). + ## [0.13] - 2026-10-09 Simpler, faster and harder to misuse. Each computation now has one diff --git a/CLAUDE.md b/CLAUDE.md index fbc4489c..1b60227f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,6 +13,23 @@ open source) handles plotting/dashboards/reporting. RePyability is the computational reliability engine; keep it free of plotting dependencies. +- **RePyability is about timelines; Reliafy is about calendars.** + RePyability works in model time (the models' own unit, from 0 or from + a state: ages, phases and meter readings as numbers) and answers in + plain data. Dates, calendars, operating schedules, meters read by date, + financial years, and the CMMS (work orders, the asset register, the + history store, the forecast by date) and RCM's records and workflow + belong to Reliafy, which turns dates into model time, calls + RePyability, and turns the answers back into dates. A calendar of + opportunities in model time (shuts on periods or at given times, each + with a price: `maintenance.Calendar`) is RePyability's; laying it on + real dates is Reliafy's. RePyability keeps + the quantitative questions: RCM's per failure mode (run to failure, + scheduled replacement, failure finding, on-condition, the options at a + planned replacement), a failure's consequences at system level, and + the probabilities a forecast is made of (expected failures and costs + over a window from a state, and their distribution). + - **Model behaviour belongs in surpyval too.** When a surpyval model misbehaves (its draws, means, densities or save format), raise an issue in derrynknife/surpyval rather than only working around it here. Keep any @@ -21,12 +38,16 @@ - **surpyval releases reach RePyability users at once.** surpyval is required with no upper bound, so its next release is what a fresh install - gets. `.github/workflows/upstream.yml` runs the tests against surpyval's - `develop`; when it fails, fix RePyability (working with both the released - surpyval and `develop`) and release that before surpyval releases. CI's - `test (minimum surpyval)` job tests the oldest surpyval `pyproject.toml` - allows; raise that minimum, rather than keep code for older versions, - once RePyability needs what a newer surpyval does. + gets. No automated run tests surpyval's `develop` (the maintainer wants + one automated test run, below): before surpyval releases, run + RePyability's tests against its `develop` by hand (`pip install + "surpyval @ git+https://github.com/derrynknife/SurPyval@develop"`), and + when they fail, fix RePyability (working with both the released surpyval + and `develop`) and release that first. CI's `test (minimum dependencies)` + job tests the oldest surpyval `pyproject.toml` allows (and every other + dependency at its floor); raise that minimum, + rather than keep code for older versions, once RePyability needs what a + newer surpyval does. ## Code @@ -40,10 +61,40 @@ into both. If code needs a "keep in sync with X" note, simplify it instead. +- **`RepairableRBD`'s code lives in modules by analysis.** The class + (`repairable_rbd.py`) keeps its constructor, its public methods (their + signatures, decorators and docstrings, each calling a function of the + same name), the hooks `RBD` calls, and the methods a test overrides on + one diagram (`_crews_couple`, `_conditional_modules`). The work is done + by functions taking the diagram, in `repyability/rbd/`: `_event_loop` + (the Python loop and its streams), `_runs` (availability and cost runs, + the twin, conditional runs, chunks and shards), `_curves` (each node's + curve, the start states, `_settling`), `_windows` (values over time), + `_long_run`, `_costs`, `_repairable_capacity`, `_repairable_importance`, + `_ccf_groups`, `_crews`, `_intervals`, `_repairable_allocation`, + `_spares`, `_repairable_uncertainty`, `_sensitivity`, + `_repairable_routes`, `_requirements` (the `_require_*` checks) and + `_common` (small helpers they share). A new analysis goes into the + module of its kind, and its public method onto the class. Modules low in + the imports (`_compiled`, `_tally`, `_timeline_runs`, ...) import these + inside the functions that use them, as they import the modules above + them. + +- **Maintenance (#319) is built one use case at a time.** Its calendar + side (tasks on dates, operating schedules, meters read by date) moved + to Reliafy (Reliafy/reliafy#389), by the scope note above; what stays + works in model time (`NonRepairable.failures`, the choice at a planned + replacement). `repyability/maintenance/README.md` records each use case + (who asks, what they know and want back, how it is solved, the + hand-worked answer with today's methods, the checks, what was learned) + before it is built. Add a use case there before the code that answers + it, and add nothing no use case needs; the maintainer chooses which is + next. + ## Simulation engines and seeded results - **A `RepairableRBD` simulation has two engines that must agree to the last - bit**: the Python event loop (`RepairableRBD._replicate`) and the compiled + bit**: the Python event loop (`_event_loop._replicate`) and the compiled one (`repyability/rbd/_kernel.py`, numba, the optional `fast` extra). A change to the loop's events, arithmetic or order goes into both; `test_simulation_engines.py` checks them against each other (run by CI's @@ -61,10 +112,10 @@ function this size, and passing the run's tuples to a function per simulation cost several times the loop (measured in #206). - **Simulations take turns across threads (#216).** The event loop keeps - a run's state on the diagram (`_RUN_STATE`) and draws that cannot be + a run's state on the diagram (`_run_state`, an `_events._Run`) and draws that cannot be streamed come from numpy's global RNG, so a run holds `repyability.utils.wrappers.SIMULATIONS`, a process-wide `RLock`: - `RepairableRBD._run` (but a sharded run's parent, whose shards take it + `_event_loop._run` (but a sharded run's parent, whose shards take it where they run, so a `shard_map` on threads cannot wait on it) and `numpy_seed`. New code that runs the loop or seeds the global RNG goes through one of these; threads that work for a run (numba's) must @@ -79,7 +130,7 @@ too, `importlib.util.find_spec("numba")` returning None and `import numba` raising `ModuleNotFoundError` (which `importorskip` needs). - **`simulate_timelines`' histories are the event loop's, on every - engine.** Both loops record them as they run (`_replicate` with + engine.** Both loops record them as they run (`_event_loop._replicate` with `_Context.history`; `_kernel._simulate` when given room to record, the system's own level only): each top-level component's changes, and each of the system's with the component that @@ -87,7 +138,7 @@ loops record goes into both: `test_timelines.py` checks every engine's histories against each other and against `availability`. - **Capacity states are worked out in batches** - (`_CapacityRecorder.evaluate`): a simulation's in Python, a batch's + (`_tally._CapacityRecorder.evaluate`): a simulation's in Python, a batch's compiled, so a state's capacity must not depend on what is worked out with it. It does not while every node works at one level (each probability is then 0 or 1, and every sum exact); with several levels @@ -99,15 +150,16 @@ `_System`, `_Store` and `_structure` and on `_streams`' blocks. Keep those compatible, or raise `engines.API` (with a CHANGELOG entry) when an engine would have to change with them. -- **A conditional run (#189, `repyability/rbd/_conditional.py`) simulates +- **A conditional run (#189, `repyability/rbd/_conditional.py`, and + `RepairableRBD._conditional_modules` and `_runs._modules_rbd`) simulates the modules alone** (`_conditional_modules`: the nodes - `_node_over_time` refuses, with their maintenance groups), as a diagram - of their own (`_modules_rbd`) whose streams are named as in the system, + `_curves._node_over_time` refuses, with their maintenance groups), as a diagram + of their own (`_runs._modules_rbd`) whose streams are named as in the system, so its simulations are a plain run's; its tally keeps each simulation's cost beside its histories, on both engines. Given each joint state of - the modules, the rest comes from `_window` and `point_availability` + the modules, the rest comes from `_windows._window` and `point_availability` with the modules held. A change to what makes a node need simulating - goes into `_node_over_time`, and anything that ties a module to the + goes into `_curves._node_over_time`, and anything that ties a module to the other nodes (crews, a group's `system_down`) must refuse in `_conditional_modules`. Changes at one instant are ordered as the loop orders them (`_conditional.paths`: failures, the stops they open, then @@ -117,18 +169,18 @@ history, and the estimates against plain runs. - **A run's means are exact or conditional by default (#187, #189).** `availability()` and `cost()` take the exact methods' expected values - where they work them out (`_exact_means`, as the controls of the system + where they work them out (`_runs._exact_means`, as the controls of the system itself), and otherwise, where a conditional run applies, each simulation's expected values given its modules' histories - (`_conditioned_run`, the modules simulated again from the run's + (`_runs._conditioned_run`, the modules simulated again from the run's entropy); `availability_from_chunks` gives merged chunks the same - (`_default_means`). The simulations are a plain run's either way. A test + (`_runs._default_means`). The simulations are a plain run's either way. A test that checks the simulation against the exact methods must run plainly (`control_variate=False`), or it compares the exact values with themselves. - **A run's changes are put in time order by numpy alone (#201, #208)** - in `repairable_rbd.py`: `_net_by_time` nets the +1 and -1 changes of - state with one sort of integers, each time's bits above the change's + in `repyability/rbd/_time_order.py`: `_net_by_time` nets the +1 and -1 + changes of state with one sort of integers, each time's bits above the change's sign (a time is never negative); `_by_time` sorts the capacity's (time, change) pairs, and the changes in how many systems can carry an unlimited amount, as complex numbers; `_capacity_totals` merges the two @@ -153,14 +205,14 @@ `MAX_WIDTH`, a chunk's rows) only affects speed: `test_streams.py` checks it changes nothing. - **The repair crews' Markov chain (`repyability/rbd/_crew_chain.py`) - copies the simulation's queue (`_Crews`)**: which waiting job a free crew + copies the simulation's queue (`_events._Crews`)**: which waiting job a free crew takes, and how instant jobs pass through. A change to one goes into the other; `test_crew_chain.py` checks the chain's exact values against the simulation. Likewise a standby group's chain (`_standby_chain.py`) copies - `_StandbyGroup`'s rules (switching, spares, repairs), checked by + `_events._StandbyGroup`'s rules (switching, spares, repairs), checked by `test_repairable_standby.py`. - **A common-cause group's chains (`repyability/rbd/_ccf_chain.py`) copy - the simulation's causes** (`_Cause`, `_strike`, #158): each cause, a + the simulation's causes** (`_events._Cause`, `_event_loop._strike`, #158): each cause, a member's own or a shared one, strikes at its share of the failure rate and fails the members it names that are up, at once; with tests that can miss, one coin for all the failures it makes. A change to one goes into @@ -171,15 +223,22 @@ against the chain of every copy. - **A common-cause group's chain with tests and repairs that take time (`_ccf_chain._Timed`, #220) copies the simulation's inspections** too - (`_inspected_follow_up`, `_inspected_next`, `_strike`): a working member + (`_event_loop._inspected_follow_up`, `_event_loop._inspected_next`, `_event_loop._strike`): a working member is off line for its test, unaged and not struck; a failure found is repaired once the test is over; a test in a member's own test or repair is not done. A change to one goes into the other: `test_ccf_timed.py` checks the chain against the simulation, and against the members' own - model (`_hidden_tests`) where no cause is shared. + model (`_hidden_tests`) where no cause is shared. Over a window, a + member's test that takes time is a planned outage of the system at an + exact time (`OverTime.jump_times`, which `_windows._atom_groups` counts + from the chain just before and at it, as a node's scheduled outages), + and the chain's bends (`OverTime._bends`: the tests, the fixed ends, + geometric offsets at an exponential length's scale) are the integrals' + pieces' edges, with the integral of the unavailability refined to those + scales (`_ccf_groups._timing_rates`). - **A tested unit's numerical model (`repyability/rbd/_hidden_tests.py`, - #159) copies the simulation's inspections** (`_inspected_follow_up`, - `_inspected_next`): a test takes a working unit off line without ageing + #159) copies the simulation's inspections** (`_event_loop._inspected_follow_up`, + `_event_loop._inspected_next`): a test takes a working unit off line without ageing it, a failure is repaired once its test is over, the tests in a repair are not done, and a failure a test misses waits for the next full test. A change to one goes into the other; `test_hidden_failures_timed.py` @@ -188,7 +247,9 @@ ## How each analysis is computed - **A `RepairableRBD`'s junctions are folded out of its structure.** A node - given `PerfectReliability` is no component: `RBD._decomposition()` folds + given `PerfectReliability`, or a spec whose life never fails + (`_model_utils.always_works`: that, or a probability of failing of 0), is + no component: `RBD._decomposition()` folds it in as always working (`modular.fold`), so whatever evaluates the structure (the curves, the long-run values, both simulation engines, the timelines, the path and cut sets) never sees it. Evaluate the structure @@ -197,19 +258,22 @@ `_capacity_arrays` passes them as working. `test_junctions.py` checks every public method against the same system drawn without a junction. -- **Identical components share one curve** (`_availability_curves`, - `_curve_twin`): a plain unit's curve follows from its life and repair +- **Identical components share one curve** (`_curves._availability_curves`, + `_curves._curve_twin`): a plain unit's curve follows from its life and repair models and its state at 0 alone. Anything new that makes a unit's curve depend on more (a schedule, a group, a coupling) must leave it out in - `_plain_unit`, or two units would be given one curve that is only one's: + `_curves._plain_unit`, or two units would be given one curve that is only one's: `test_identical_components_share_one_curve` checks the shared curves against a curve each. - **`analysis_routes()` (both RBD classes) must agree with the methods.** It says, without running anything, whether each public analysis is exact, numerical, simulated or refused. Refusals go through checks the report - calls too (`_require_*` helpers, `_inspected_rate`, ...), so its reasons - are the methods' own messages. When a method is added, or gains a refusal + calls too (`_requirements._require_*` helpers, `_requirements._inspected_rate`, ...), so its reasons + are the methods' own messages. A refusal is a `NotImplementedError` (what + an analysis cannot work out) or a `routes.Refused` (a `ValueError`: the + diagram has nothing the analysis works on); `routes.refusal` raises any + other error, as a fault. When a method is added, or gains a refusal or changes how it computes, update `analysis_routes`: `test_analysis_routes.py` checks that it covers every public method, that each method does what it says on diagrams of every kind, and that the @@ -220,11 +284,24 @@ exact-against-simulated checks (`test_catalogue.py`) share. A new node class, spec key, policy or diagram option needs a diagram there: `test_the_catalogue_has_every_kind` names what no diagram uses. + `FaultTree.analysis_routes()` (#314), `Network.analysis_routes()` + (#315) and `PhasedMission.analysis_routes()` (#316) are held to the + same: their calls in `FAULT_TREE_CALLS` (on the trees of + `catalogue.fault_tree_kinds()`), `NETWORK_CALLS` and `PHASED_CALLS`. - **The README's "When is a simulation needed?" table follows the routes.** It says, by what is asked, the components and the maintenance, what is simulated and whether it must be (or could be exact, with the issue). `test_the_readme_says_what_is_simulated` checks each row against `analysis_routes()`: when an analysis is made exact, update its row. +- **The README's capability matrices are generated** by + `repyability/tests/capabilities.py` (`python -m + repyability.tests.capabilities` writes them between its markers): which + analyses each model type has, and, from `analysis_routes()` on the + catalogue's diagrams, how each family is worked out for each kind of + node. `test_readme_capabilities.py` checks they are current and that + every public method is in a row (or in `NOT_ANALYSES`): a new method + goes into a row of `CAPABILITIES`, and a new catalogue kind worth a + row of its own into `NODES`. - **A diagram too meshed to work out keeps only its graph (#172).** When a core's decision diagram passes `bdd.STEP_LIMIT`, `RBD._decomposition()` is a `modular.GraphStructure`, which works out a state and a lifetime @@ -254,7 +331,7 @@ members up or down, an owner's laid out as a last axis of the arrays; `test_ccf_repairable_modules.py` checks every long-run value, measure and the groups' system over time against the times split by every - combination (`_with_ccf_groups`, which the capacity distribution and the + combination (`_ccf_groups._with_ccf_groups`, which the capacity distribution and the allocations still take, behind `_ccf_chain.check_split`). - **The integrals over a window are summed on coarse pieces** (`repyability/rbd/_quadrature.py`, #164): the curves' breaks, cut to a @@ -268,10 +345,10 @@ knot. - **Unavailability over time is worked out as itself (#237)**, not as one less the availability, which rounds to 0 below about 1e-16: - `RepairableRBD._system_at(..., down=True)` takes each component's + `_windows._system_at(..., down=True)` takes each component's probability of being down from its closed form where it has one - (`_closed_form_down`: an exponential life and repair, nothing else - about it), from its curve otherwise (`_curves_down_at`), and from the + (`_curves._closed_form_down`: an exponential life and repair, nothing else + about it), from its curve otherwise (`_curves._curves_down_at`), and from the crews' and common-cause groups' chains (`GroupsCurve.down_at`); the mission's integral is refined to its own size (`_integrated(..., relative=)`). A change to how `point_availability` works out a kind of @@ -288,6 +365,17 @@ halvings (right, but slow). `test_evaluation_costs.py` checks the points an MTTF takes and its accuracy. +- **A measure that is a function of the system's values lives once, in + `repyability/rbd/_measures.py`** (#314, #315): the ratios (improvement + potential, the risk worths, criticality, Fussell-Vesely from numerators), + an element's density, the finite-difference steps, the cumulative + hazard, the rate parts and Barlow-Proschan shares, a life's end and the + inversion of a reliability for a B-life. A diagram, a fault tree and a + network differ in how they work out the system with an element held + (a decomposition, the top event conditioned, a decision diagram + replayed), not in the measures: a new measure goes there, and each + class calls it with its own values. + ## API conventions - **One name for the number of simulations**: `mc_samples`, and `max_samples` @@ -301,9 +389,14 @@ `checks.seed` where it is taken. - **Inputs are checked where they are given (#233)**: numbers through `checks.is_number`/`number_or_nan` (not `bool`, not text), times through - `checks.real_array`, and models through `checks.no_distribution`, which - refuses surpyval's distribution itself (`surv.Weibull`) for a model of - it. A new argument that takes a number, a time or a model uses them. + `checks.real_array`, a choice of option through `checks.one_of`, and + models through `checks.no_distribution`, which refuses surpyval's + distribution itself (`surv.Weibull`) for a model of it. A new argument + that takes a number, a time, an option or a model uses them. +- **A model is probed with `_model_utils.MODEL_ERRORS`** (and saved with + `SAVE_ERRORS`): code that asks a model what it can do (a quantile, a + mean, a saved form) catches what a model that cannot do it raises, never + `Exception`, so a fault in a model or in RePyability is raised. - **Warnings point outside the package (#232)**: `warnings.warn(..., stacklevel=outside_level())` (`repyability/utils/wrappers.py`), never a counted level, which goes stale as calls are wrapped. A message never @@ -341,22 +434,37 @@ deprecated went in 0.13, and `test_removed_in_0_13.py` keeps it gone: calling `SparesDemand.mean()`/`std()` (#184) and those models' `mc_samples`, `lower` and `seed` raise `TypeError`, and `StandbyModel`'s - options after them are given by name. What 0.13 deprecates goes in 0.14 - (`NEXT_REMOVAL`): `optimal_inspection_intervals(offsets=)`, renamed - `offset_shares=` (#222, through `deprecation.renamed`); calling - `CapacityDistribution.mean()`, now a property (#235, through - `deprecation.called`); and `mc_samples` and `seed` given to the exact - `mean` of a `StandbyModel`, `LoadSharingModel` or `DegradingNode`, - which ignores them (#233, through `deprecation.ignored` in - `standby_node.drawn_mean`), all of which - `test_what_0_13_deprecates_goes_in_0_14` holds to: it fails once the - version reaches `NEXT_REMOVAL`. + options after them are given by name. What 0.13 deprecated went in + 0.14, and `test_removed_in_0_14.py` keeps it gone: + `optimal_inspection_intervals(offsets=)` (#222) raises `TypeError`, for + `offset_shares=`; `CapacityDistribution.mean` is a plain number or + array, not callable (#235); the exact `mean` of a `StandbyModel`, + `LoadSharingModel` or `DegradingNode` refuses `mc_samples` and `seed` + (#233, `standby_node.drawn_mean`); and `rbd._sensitivity`'s `levers` + and `Lever` are gone (#244). What 0.14 deprecates goes in 0.15 + (`NEXT_REMOVAL`): `NodeState`'s `phase`, renamed `since_scheduled` + (#350, through `deprecation.renamed` for the argument and + `renamed_attribute` for reading it); a new deprecation warns through + those, `called` or `ignored`. `test_what_0_14_deprecates_goes_in_0_15` + fails once the version reaches it. +- **No result depends on the unit of time (or of a covariate).** A step, + grid or probe is sized from the models (a typical life, + `failure_time_scale`), never in absolute time, and a proportional + change of a lever is of its scale (`_sensitivity._proportional_scale`: + 1 for a log-scale parameter, none for a covariate). `test_units.py` + checks results scale with the unit, from 1e-6 to 1e6. A model's unit + is surpyval's to carry (SurPyval#845); until it does, every node is + taken to be in one unit, unchecked. - **Exact by default, simulation on request.** Where an analysis can be computed exactly or numerically, that is the default, and the Monte-Carlo estimate is a `method="simulate"` away (as for `NonRepairableRBD.mean`). ## Performance +- **A model's constant failure rate is kept while the model lives** + (`_common._constant_rate`, a `WeakKeyDictionary` by model): the checks + and the chains ask for it tens of times per diagram, and a search over + intervals builds hundreds of diagrams around the same models. - **In a loop, a dot product of long vectors goes through `repyability.utils.vectors.dot`, not `@`.** OpenBLAS threads one of more than about 10,000 terms, at milliseconds a call; a loop of thousands of @@ -365,11 +473,14 @@ ## Testing -- **Test what a change touches; the full suite only when asked.** Work - going into dev is checked by the tests of what it changes (the files - that exercise the code touched, and new tests for it). The full suite - runs only when the maintainer asks for it, and before merging to main - (master). +- **One automated test run: the full suite on the pull request from dev + into master.** `actions.yml` runs on nothing else: not on pushes, not on + pull requests into dev, not on a schedule, and there is no other test + workflow. Work going into dev is checked locally by the tests of what it + changes (the files that exercise the code touched, and new tests for + it), and is merged without waiting for CI. Do not start, re-run or add + other automated test runs; run the full suite locally only when the + maintainer asks. ## Releasing @@ -387,9 +498,13 @@ major.minor. `[Unreleased]` section into `## [X.Y] - YYYY-MM-DD`, opening with a summary paragraph and the behaviour changes (they open the release notes), and update the version in `docs/guide/saving.md`. PR to dev, then dev to - master, listing "Closes #N" for each finished issue: commit messages' - "(#N)" close nothing. -2. When CI has passed on master's merge commit, run the workflow with + master. A pull request into dev lists "Closes #N" for each issue it + finishes, and `.github/workflows/close-issues.yml` closes them when it + merges, with a comment naming it (GitHub itself closes an issue only on a + merge into master): commit messages' "(#N)" close nothing, and the + dev-to-master pull request need not list them again. +2. When the full suite has passed on the pull request from dev into + master, merge it (a merge commit) and run the workflow with `actions_run_trigger`: `run_workflow`, workflow `release.yml`, ref `master`, inputs `{"version": "X.Y", "dry_run": "true"}`. If that passes, run it again with `"dry_run": "false"`. Then check the run, the @@ -397,20 +512,43 @@ major.minor. ## surpyval workarounds to remove -surpyval 0.23 (the minimum) pickles its fits (#573), names the +surpyval 0.24 (the minimum) gives a mixture a precise tail (`sf`, #671) and +has `conditional_gaps` read a limited failure population's `lfp_p`, not a +mixture's `p` (#626); 0.23 pickles its fits (#573), names the limited-failure proportion `lfp_p` (#608) and takes `success_run`'s -`alpha_ci` (#580), which RePyability now uses directly. What remains: - -- **A mixture's quantile, `p` and tail** (surpyval #651, #626, #671). - surpyval's `MixtureModel` has no `qf` (#651), keeps its EM - responsibilities as `p` (#626), which surpyval's `conditional_gaps` - takes for a limited failure population's, and has an `sf` of `1 - ff` - (#671), imprecise in the tail. `_sampling.mixture_quantile` inverts its - distribution (with its components' `sf` summed above the median), which +`alpha_ci` (#580). RePyability uses these directly. What remains: + +- **A mixture's quantile** (surpyval #821). surpyval 0.24's `MixtureModel` + has a `qf` (#651), but it inverts `F` in the upper tail, losing the long + lives' precision (about 1e-4), and takes some 200 times as long as + RePyability's. `_sampling.mixture_quantile` inverts the distribution + (with its components' `sf` summed above the median), which `stream_sampler` gives a `RepairableRBD`'s streams (a change to it - changes seeded results), and `MixtureLife` hands `conditional_gaps` its - `Hf` and that `qf` (`_aged_life`). Take the mixture's own `qf`, and drop - `MixtureLife`, once the minimum surpyval has them. + changes seeded results). Take the mixture's own `qf` once the minimum + surpyval's is as precise and fast. +- **A mixture's life left at a virtual age** (surpyval #857, #295: + `conditional_gaps` inverts `H` past its quantile's limit in 12–15 + evaluations where Newton's steps take 3–4, and `MixtureModel.Hf` costs + about 100 µs a call). `MixtureLife.aged` + draws it for `_aged_life` by Newton's steps on the mixture's `H`, + worked out from its components, from a table of `H` (a change to it + changes seeded results in the last bits). Take `conditional_gaps` + again, and drop `MixtureLife`, once the minimum surpyval has both. +- **A degradation process's level given survival** (surpyval #836). + Replacement on condition by level (#271) needs the level at the next + inspection of a unit that has not failed by then, which surpyval's + `WienerProcessModel` and `GammaProcessModel` do not give. + `_degradation.level_after` works it out from the fitted parameters (the + method of images for a Wiener path; a gamma increment truncated at the + threshold), and `_condition_next` draws it from the `LEVEL` stream (a + change to it changes seeded results). Take surpyval's once the minimum + surpyval has it. + +- **Rescaling a model's time for a duty cycle** (surpyval #845). + A spec's `"duty"` puts a life in operating time on the calendar, + `R(d t)`, by `_duty._SCALED`, one rule per surpyval distribution (a new + one needs its rule there, which `test_units` checks against `R(d t)`). + Take surpyval's rescaling once the minimum surpyval has it. List each new workaround here with its surpyval issue and where it lives, so it can go once the minimum surpyval in `pyproject.toml` includes the diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index af29dd26..49fa6fdd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,60 +1,111 @@ # Contributing to RePyability -Thanks for your interest in improving RePyability! This guide covers setting up -a development environment and the checks your change needs to pass. +Thanks for your interest in improving RePyability! This guide covers setting +up, the checks a change needs, and the rules the code keeps to: several +parts of the package must agree with each other, and the tests that hold +them together say which. ## Development setup Use a virtual environment, then install the package with its dev tooling -(pinned in `pyproject.toml`): +(pinned in `pyproject.toml`) and, to run the compiled simulation engine, the +`fast` extra (numba): ```bash python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate -pip install -e .[dev] +pip install -e .[dev] # add ,fast for numba, ,docs for mkdocs +pre-commit install # run the formatters and linters on commit ``` -This installs RePyability in editable mode plus `black`, `isort`, `flake8`, -`mypy`, `pytest`, `coverage`, and `pre-commit`. +RePyability needs Python 3.11 or later and surpyval 0.24 or later. -## Pre-commit hooks +## Branches and the one automated test run -Install the git hooks so the formatters and linters run automatically on -commit: +Work branches off **`dev`** and goes back into `dev` by pull request. +Releases go from `dev` into `master`. -```bash -pre-commit install -pre-commit run --all-files # optional: run against the whole tree once -``` - -## Running the checks locally - -CI runs the same checks; running them before pushing avoids round-trips: +There is **one automated test run**: the full suite on the pull request from +`dev` into `master` (`.github/workflows/actions.yml`). Nothing runs on pushes +or on pull requests into `dev`, so check your change locally before it goes +in: ```bash -black --check repyability # formatting (drop --check to auto-format) -isort --check-only repyability # import ordering (drop --check-only to fix) -flake8 repyability # linting -mypy repyability # static type checks -coverage run -m pytest # tests -coverage report # enforces the coverage fail_under gate +scripts/check.sh # black, isort, flake8, mypy, as CI runs them +scripts/check.sh repyability/tests/test_units.py # the same, then those tests on every core ``` +Run the tests of what you changed: the files that exercise the code you +touched, and new tests for it. The whole suite has over 6,000 tests and takes +a long time even on every core (`python -m pytest -n auto`); it runs on the +release pull request. + ## Tests -- Every behavioural change should come with a test. Prefer asserting against a - **closed-form or textbook reference value** (as the existing suite does) - rather than only self-consistency. -- Monte-Carlo tests must be deterministic — pass a `seed=` to the simulation - entry points. -- Keep the library coverage above the `fail_under` gate in `pyproject.toml`. +- Every behavioural change comes with a test. Assert against a + **closed-form or textbook reference value** where there is one, not only + self-consistency. +- Monte-Carlo tests are deterministic: pass a `seed=`. +- Keep the library's coverage above the `fail_under` gate in + `pyproject.toml`. +- A test that asks for the compiled engine (`engine="numba"`) skips without + numba (`pytest.importorskip("numba")`). With numba installed, also check + such tests with it hidden, as CI's plain jobs have none. +- The documentation's code is run too: every ```` ```python ```` block in + `docs/` and the README runs in order, and a value quoted after `# ->` is + checked against what the line returns (`test_docs_examples.py`). + +## What must agree + +Several computations exist twice, by design, and a test compares them. A +change to one goes into the other: + +- **The two simulation engines**: the Python event loop + (`RepairableRBD._replicate`) and the compiled one (`rbd/_kernel.py`) agree + to the last bit (`test_simulation_engines.py`, run with numba). +- **The Markov chains that copy the simulation's rules**: repair crews + (`_crew_chain.py`), standby groups (`_standby_chain.py`), common-cause + groups (`_ccf_chain.py`) and tested units (`_hidden_tests.py`), each + checked against the simulation by its own test. +- **`analysis_routes()`**, which says without running anything whether each + public method is exact, numerical, simulated or refused, agrees with the + methods on every diagram of `tests/catalogue.py` + (`test_analysis_routes.py`). A new method, refusal, node class, spec key + or option needs its route, and a catalogue diagram that uses it + (`test_the_catalogue_has_every_kind` names what no diagram uses). The + README's "When is a simulation needed?" table follows the routes too. +- **Seeded results** are defined by the random streams (`rbd/_streams.py`). + Changing a stream's name or definition changes them: re-record + `tests/seeded_event_loop.json`, update the docs' quoted numbers, and say + so in the CHANGELOG. + +The maintainer's notes in `CLAUDE.md` list these and the other conventions +(input checks, warnings, result objects, levers) in full. + +## surpyval + +RePyability consumes fitted surpyval models; fitting stays in surpyval. When +a surpyval model misbehaves, raise an issue in +[derrynknife/SurPyval](https://github.com/derrynknife/SurPyval), keep any +workaround here small, and list it with its issue in `CLAUDE.md`, so it can +go once the minimum surpyval version has the fix. ## Pull requests -1. Branch off `master`. -2. Keep changes focused; update `CHANGELOG.md` under `[Unreleased]`. -3. Make sure all checks above pass. -4. Open a PR describing the change and its motivation. +1. Branch off `dev`. +2. Keep changes focused, and update `CHANGELOG.md` under `[Unreleased]`. +3. Run `scripts/check.sh` with the tests of what you changed. +4. Open a pull request into `dev` describing the change and its motivation, + with a "Closes #N" line for each issue it finishes: they are closed when + it merges (`.github/workflows/close-issues.yml`), with a comment naming + the pull request. + +## Deprecations + +A deprecation warns (a `FutureWarning`, through +`repyability/utils/deprecation.py`) for one minor release, and the next +removes it. `test_what_0_14_deprecates_goes_in_0_15` and its kind fail once +the version reaches the removal, as a reminder. ## Releases (maintainers) @@ -66,16 +117,13 @@ release: 1. On a branch, bump `repyability/_version.py` and roll the `CHANGELOG.md` `[Unreleased]` section into a dated `## [X.Y] - YYYY-MM-DD` section. Its opening paragraphs become the summary at the top of the release notes. - Update any version shown in the docs, then merge to `dev`, and `dev` to - `master`. -2. Once CI has passed on master, run the `release` workflow on master with - the version: Actions → release → Run workflow, or + Update the version in `docs/guide/saving.md`. Merge to `dev`, then open + the pull request from `dev` into `master`, which runs the full suite. +2. Once it has passed, merge it and run the `release` workflow on master + with the version: Actions → release → Run workflow, or `gh workflow run release.yml --ref master -f version=X.Y`. Add `-f dry_run=true` to check and build without publishing. The workflow checks the version, the CHANGELOG section, that the tag is new, that CI passed and that PyPI does not have the version. Then it builds, publishes to PyPI via trusted publishing, and creates the tag `vX.Y` and a GitHub Release. - -Publishing a GitHub Release by hand for a new tag also works: the workflow -then builds that tag and publishes it to PyPI. diff --git a/README.md b/README.md index 34c5531f..52718a87 100755 --- a/README.md +++ b/README.md @@ -3,9 +3,8 @@ [![actions](https://github.com/derrynknife/RePyability/actions/workflows/actions.yml/badge.svg)](https://github.com/derrynknife/RePyability/actions/workflows/actions.yml) [![docs](https://github.com/derrynknife/RePyability/actions/workflows/docs.yml/badge.svg)](https://derrynknife.github.io/RePyability/) -Reliability Engineering Tools - -This is a series of tools created to make an open source set of methods to be used by reliability engineers to make it more accessible for students right through to practicing professionals. +Reliability engineering tools, open source, for students through to +practising professionals. RePyability builds and analyses systems as reliability block diagrams (RBDs), taking already-fitted lifetime models (from @@ -14,7 +13,9 @@ taking already-fitted lifetime models (from - **Fault trees**: static fault trees (OR, AND and VOTE gates, repeated events) evaluated exactly, with common-cause groups, cut sets, importance - measures and conversion to and from block diagrams. + measures, the mean time to the top event, B-lives and hazard, parameter + sensitivities, uncertainty from fitted event models, and conversion to + and from block diagrams. - **Reliability**: exact system reliability, hazard and conditional survival; the exact MTTF (or simulated, with confidence intervals); B*X* life; uncertainty intervals on the reliability, MTTF, B*X* life and time to a @@ -112,6 +113,175 @@ that teaches system reliability from a single part's lifetime to designing and maintaining whole systems, working every idea out by hand and then with RePyability, with exercises and worked answers. +## Install + +RePyability needs Python 3.11 or later. Install it from +[PyPI](https://pypi.org/project/repyability/): + +```bash +pip install repyability +``` + +Repairable systems simulate about ten times as fast with the optional +compiled engine, which needs [numba](https://numba.pydata.org) (available for +64-bit Linux, macOS on Apple silicon and Windows): + +```bash +pip install "repyability[fast]" +``` + +## What each model can do + +The analyses each model type has, and how each family of them is worked +out for each kind of node or component: the gaps are where a cell says +— or ✗. The tables are made from the code (`python -m +repyability.tests.capabilities` writes them, and a test checks they are +current). + + + +Which analyses each model type has (the method; — where it has none): + +| | `NonRepairableRBD` | `RepairableRBD` | `FaultTree` | `Network` | `PhasedMission` | +|---|---|---|---|---|---| +| **Reliability and lifetime** | | | | | | +| Reliability R(t) | `sf` | — | `sf` | `sf` | `sf` | +| Unreliability F(t) | `ff` | — | `ff` | `ff` | `ff` | +| Density, hazard, cumulative hazard | `df` | — | `df` | — | — | +| Mean time to (first) failure | `mean` | — | `mean` | `mean` | — | +| Each node's own R(t), F(t) and MTTF | `node_sf` | — | — | — | — | +| B-life, time to a reliability | `bx_life` | — | `bx_life` | `bx_life` | — | +| Given the system's age | `cs` | — | — | — | — | +| Given the components' ages or states | `sf_given_state` | — | — | — | — | +| Mean residual and remaining life | `mean_residual_life` | — | — | — | — | +| Random lifetimes | `random` | — | `random` | `random` | — | +| Top event probability | — | — | `top_event_probability` | — | — | +| Each phase's failure probability | — | — | — | — | `phase_failure_probabilities` | +| **Availability and outages** | | | | | | +| Long-run availability | — | `mean_availability` | — | — | — | +| Each node's availability | — | `node_availability` | — | — | — | +| Availability over time | — | `point_availability` | — | — | — | +| Over a mission | — | `mission_availability` | — | — | — | +| Unavailability (precise when small) | — | `mean_unavailability` | — | — | — | +| Failure frequency, MTBF | — | `system_failure_frequency` | — | — | — | +| Mean up and down times | — | `mean_up_time` | — | — | — | +| Expected failures and events in a window | — | `expected_events` | — | — | — | +| Simulated availability (distribution) | — | `availability` | — | — | — | +| Simulated histories | — | `simulate_timelines` | — | — | — | +| Compare two designs | `compare` | `compare` | — | — | — | +| **Costs and spares** | | | | | | +| Long-run cost rate | — | `expected_cost_rate` | — | — | — | +| Expected cost in a window | — | `expected_cost` | — | — | — | +| Simulated cost (distribution) | — | `cost` | — | — | — | +| Total cost of ownership | — | `total_cost` | — | — | — | +| Spares demand and stock | — | `spares_demand` | — | — | — | +| **Structure** | | | | | | +| Minimal path and cut sets | `minimal_cut_sets` | `minimal_cut_sets` | `minimal_cut_sets` | `minimal_cut_sets` | `minimal_cut_sets` | +| Each path set's probability | `path_set_probabilities` | `path_set_probabilities` | — | — | — | +| Cut sets ranked by probability | — | — | `ranked_cut_sets` | — | — | +| Irrelevant components | `find_irrelevant_components` | `find_irrelevant_components` | — | — | — | +| Convert to and from an RBD | — | — | `to_rbd` | — | — | +| Capacity distribution | `capacity_distribution` | `capacity_distribution` | — | — | — | +| Capacity over time | — | `point_capacity` | — | — | — | +| How each analysis is computed | `analysis_routes` | `analysis_routes` | `analysis_routes` | `analysis_routes` | `analysis_routes` | +| Save and load (dict, JSON) | `to_dict` | `to_dict` | `to_dict` | `to_dict` | — | +| **Importance measures** | | | | | | +| Birnbaum | `birnbaum_importance` | `birnbaum_importance` | `birnbaum_importance` | `birnbaum_importance` | `birnbaum_importance` | +| Structural | `structural_importance` | `structural_importance` | `structural_importance` | `structural_importance` | — | +| Improvement potential | `improvement_potential` | `improvement_potential` | `improvement_potential` | `improvement_potential` | `improvement_potential` | +| Risk achievement worth | `risk_achievement_worth` | `risk_achievement_worth` | `risk_achievement_worth` | `risk_achievement_worth` | `risk_achievement_worth` | +| Risk reduction worth | `risk_reduction_worth` | `risk_reduction_worth` | `risk_reduction_worth` | `risk_reduction_worth` | `risk_reduction_worth` | +| Criticality | `criticality_importance` | `criticality_importance` | `criticality_importance` | `criticality_importance` | `criticality_importance` | +| Fussell-Vesely | `fussell_vesely` | `fussell_vesely` | `fussell_vesely` | `fussell_vesely` | `fussell_vesely` | +| Differential (DIM) | `differential_importance` | `differential_importance` | `differential_importance` | `differential_importance` | — | +| Joint (second order) | `joint_importance` | `joint_importance` | `joint_importance` | `joint_importance` | — | +| Barlow-Proschan | `barlow_proschan_importance` | `barlow_proschan_importance` | `barlow_proschan_importance` | `barlow_proschan_importance` | — | +| Rate of change over time | `reliability_rate` | `availability_rate` | — | — | — | +| Uncertainty importance | `uncertainty_importance` | `uncertainty_importance` | — | — | — | +| All measures given a state | `importances_given_state` | — | — | — | — | +| Parameter sensitivity, levers | `parameter_sensitivity` | `parameter_sensitivity` | `parameter_sensitivity` | — | — | +| **Uncertainty** | | | | | | +| Of the reliability or availability | `sf_uncertainty` | `mean_availability_uncertainty` | `sf_uncertainty` | `sf_uncertainty` | `reliability_interval` | +| Of the MTTF | `mean_uncertainty` | — | `mean_uncertainty` | `mean_uncertainty` | — | +| Of the B-life | `bx_life_uncertainty` | — | — | — | — | +| Of the cost rate | — | `expected_cost_rate_uncertainty` | — | — | — | +| **Design and maintenance** | | | | | | +| Reliability allocation | `simple_allocation` | `simple_allocation` | — | — | — | +| Availability, MTTF and MTTR allocation | — | `availability_allocation` | — | — | — | +| Redundancy allocation | `allocate_redundancy` | `allocate_redundancy` | — | — | — | +| Reliability-redundancy allocation | `allocate_reliability_redundancy` | — | — | — | — | +| Cost-reliability front | `redundancy_front` | — | — | — | — | +| Optimal replacement intervals | — | `optimal_replacement_intervals` | — | — | — | +| Optimal proof-test intervals | — | `optimal_inspection_intervals` | — | — | — | + +How each family of analyses is worked out on a diagram with each kind of node or component: **E** exact, **N** numerical (to a stated accuracy), **S** simulated, **✗** refused as not worked out (naming the methods, or counting them, where only some are), – where the diagram has nothing the analysis works on. Made from `analysis_routes()` on the test catalogue's diagrams. + +
NonRepairableRBD, by node + +| | Reliability | MTTF | B-life | Given a state | Importance | Sensitivity | Uncertainty | Allocation | Capacity | Simulation | +|---|---|---|---|---|---|---|---|---|---|---| +| A surpyval model | E | N | N | E N | E N | N | S | E N | – | S | +| A fitted model (with its covariance) | E | N | N | E N | E N | N | S | E N | – | S | +| A fit with a cure share and zero-inflation | E | N | N | E N | E N | N | S | E N | – | S | +| A probability per demand | E | – | – | E | E | N | S | E N | – | S | +| A mixture (`MixtureModel`) | E | N | N | E N | E N | N | S | E N | – | S | +| A regression node (covariates) | E | N | N | E N | E N | N | S | E N | – | S | +| A degrading node | N | N | N | N | N | N | S | N | N | S | +| A fitted degradation process | E | N | N | E N | E N | N | S | E N | – | S | +| Cold standby (convolved) | N | N | N | N | N | N | S | N | – | S | +| Standby with no reliability of its own | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | N; ✗ `allocate_redundancy` | – | S | +| Load sharing | E | N | N | E N | E N | N | S | E N | – | S | +| A repeated node | E | N | N | E N | E N | N | S | E N | – | S | +| A repeated standby node | N | N | N | N | N | N | S | N | – | S | +| A nested RBD | E | N | N | E N | E N | N | S | E N | – | S | +| A junction | E | N | N | E N | E N | N | S | E N | – | S | +| A beta-factor common-cause group | E | ✗ | N | ✗ | E N | N | S; ✗ `mean_uncertainty` | E N | – | S | +| An MGL common-cause group | E | ✗ | N | ✗ | E N | N | S; ✗ `mean_uncertainty` | E N | – | S | +| An alpha-factor common-cause group | E | ✗ | N | ✗ | E N | N | S; ✗ `mean_uncertainty` | E N | – | S | +| Capacities | E | N | N | E N | E N | N | S | E N | E | S | +| A structure too meshed to work out | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | S | + +
+ +
RepairableRBD, by component + +| | Long run | Frequency | Over time | Costs | Importance | Sensitivity | Uncertainty | Allocation | Intervals | Spares | Capacity | Simulation | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| Any life and repair | E | E | N | E N | E N | N | S | E | – | N | – | S | +| A fitted life | E | E | N | – | E N | N | S | E | – | N | – | S | +| A fitted Crow-AMSAA process | ✗ | ✗ | N | N; ✗ `expected_cost_rate` | N; ✗ 9 of 10 | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| A probability per demand | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ `mean_availability_uncertainty` | ✗ | – | N | ✗ `capacity_distribution` | S | +| A mixture life | E | E | N | E N | E N | N | S | E | – | N | – | S | +| A fitted degradation process life | E | E | N | – | E N | N | S | E | – | N | – | S | +| Operated part of the time (duty) | N | N | N | – | N | N | S | N | – | N | – | S | +| A standby or load-sharing life, simulated | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ `mean_availability_uncertainty` | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| k-out-of-n | E | E | N | – | E N | N | S | E | – | N | – | S | +| A nested RBD | E | E | N | E N | E N | N | S | E | – | – | – | S | +| A junction | E | E | N | E N | E N | N | S | E | – | N | – | S | +| Capacities | E | E | N | – | E N | N | S | E | – | N | E N | S | +| Age replacement | N | N | N | N | N | N | S | N | N | N | – | S | +| Block replacement | N | N | N | – | N | N | S | N | – | N | – | S | +| Opportunistic maintenance | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ `optimal_replacement_intervals` | ✗ | ✗ `capacity_distribution` | S | +| Replacement on condition | N | N | N | N | N | N | S | N | – | ✗ | – | S | +| Replacement on condition by level | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| Hidden failures, tested | N | N | N | – | N | N | S | N | – | N | – | S | +| Tests that take time | N | N | N | – | N | N | S | N | – | N | – | S | +| Tests that can miss a failure | N | N | N | – | N | N | S | N | – | N | – | S | +| Imperfect repair (Kijima) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| Minimal repair | ✗ | ✗ | N | N; ✗ `expected_cost_rate` | N; ✗ 9 of 10 | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| Minimal repair, repairs that take time | ✗ | ✗ | N | N; ✗ `expected_cost_rate` | N; ✗ 9 of 10 | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| Limited repair crews | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | ✗ | ✗ `capacity_distribution` | S | +| A standby group | E | E | N | E N | E N | N | S | E | – | ✗ | – | S | +| A beta-factor group, revealed failures | E | E | N | – | E N | N | S | E | – | N | – | S | +| A beta-factor group, tested | E | E | N | E N | E N | N | S | E | – | N | – | S | +| An alpha-factor group, revealed failures | E | E | N | – | E N | N | S | E | – | N | – | S | +| A group whose tests take time | N | N | N | N | N | N | S | N | – | N | – | S | +| A structure too meshed to work out | ✗ | ✗ | ✗ | E; ✗ `expected_cost` | ✗ | ✗ | S; ✗ `mean_availability_uncertainty` | ✗ | – | N | – | S | + +
+ + + ## When is a simulation needed? RePyability computes exactly by default: by closed forms, or by @@ -130,7 +300,8 @@ only when *k*-out-of-*n*, bridges and other meshed diagrams, components in several places, nested diagrams): -- **components:** surpyval lifetime distributions and fixed +- **components:** surpyval lifetime distributions, degradation processes + (Wiener and gamma, as the time to their threshold) and fixed probabilities (a diagram refuses a non-parametric fit, such as Kaplan–Meier: fit a parametric distribution in surpyval); repeated nodes; cold standby with one or @@ -176,11 +347,12 @@ places, nested diagrams): | **Architecture and maintenance** | | | | Phased missions and networks too large for their decision diagrams | Refused, pointing to `method="simulate"` | Only in practice: the diagrams grow with the phases' and the network's width rather than their paths, so meshed missions and networks are exact (a network grid of 100 nodes in a second and a half); one of 121 nodes passes the limit, `repyability.network.MAX_STATES`, which can be raised. | | Block diagrams too meshed for their decision diagrams | Simulated (lifetimes, availability, cost and timelines, in Python); the exact and numerical analyses refused | Only in practice: the diagram grows with how wide the mesh is rather than with its paths, so most meshes are exact (a 10 × 10 grid in 0.04 seconds, a random mesh of 60 nodes and 345 links in 2); one of 70 nodes and 485 links passes the limit, `repyability.rbd.bdd.STEP_LIMIT`, which can be raised. | -| Common-cause groups in a repairable diagram | For exponential lives, tested or repaired: the long run, importance and the allocations exact (a beta-factor member's copies join its group, and the availability allocations keep the members' availability), and numerical where the members' tests and repairs take a fixed or an exponential time (#220); the values over time from new numerical (the groups' Markov chains), and the simulations draw the shared causes, in Python. Members of other lives refused, as are tests and repairs of other lengths, copies of members whose tests or repairs take time, the failure frequency where their tests do, and members held or started from a current state | Other lives: a shared cause has no one rate for members of different ages, so they need a model first. Tests and repairs of other lengths, no: the chain could follow them on a grid, as a single component's model does. From a current state, no: the chains and the simulations could start from the members' states. | +| Common-cause groups in a repairable diagram | For exponential lives, tested or repaired: the long run, importance and the allocations exact (a beta-factor member's copies join its group, and the availability allocations keep the members' availability), and numerical where the members' tests and repairs take a fixed or an exponential time (#220); the values over time from new numerical (the groups' Markov chains), and the simulations draw the shared causes, in Python. Members of other lives refused, as are tests and repairs of other lengths, copies of members whose tests or repairs take time, and members held or started from a current state | Other lives: a shared cause has no one rate for members of different ages, so they need a model first. Tests and repairs of other lengths, no: the chain could follow them on a grid, as a single component's model does. From a current state, no: the chains and the simulations could start from the members' states. | | Shared repair crews | For exponential lives and repairs, the long run and importance exact, and the values over time numerical (the same Markov chain, followed by uniformization), but the allocations; other lives simulated. The maintenance and test intervals are chosen as if every repair started at once on request (`assume_unlimited_crews=True`), for a plan to simulate with the crews | Other lives: yes, in general. | | Standby groups (a duty unit and its spares, repaired) | For exponential units, the long run and importance exact, and the values over time numerical (the units' Markov chain, followed by uniformization); other units simulated | Other units: yes, in general. | | Opportunistic maintenance (renewals at a group's stops) | Simulated | Yes: each member's renewals depend on the others' ages. | | Imperfect repair (Kijima), with or without replacement at the *N*-th failure | Simulated; but minimal repair (`q = 1`) in no time is numerical over a window from new (it fails `H(t)` times by `t`), its long run refused | Yes, in general: a repair does not renew the unit. | +| Replacement on condition by a measured degradation level | Simulated (in Python), from new; the exact and numerical analyses refused | No: between inspections the level is a Markov process, so the long run and the values over time could follow its distribution on a grid of levels, as replacement on condition by age follows ages. | | Spares with repair crews, standby groups, opportunistic maintenance or imperfect repair | Refused, pointing to `spares_demand(method="simulate")`. Otherwise numerical, demand and stock: block-replaced components' whether their repairs and block replacements take time or not (from a typical replacement in the long run), but for one dead on arrival while they may take none, and tested components', whatever their tests and repairs take and whether their tests miss failures | Yes, in general: a crew's queue, a group's switching or stops and an imperfect repair make the replacements depend on more than each unit's own lives. A unit dead on arrival whose renewals may take no time: no, the replacements at one instant could be counted. | Where a system's expected values over a window are exact, @@ -199,21 +371,6 @@ anything; a refusal's message names the simulation to run instead. The [guide](https://derrynknife.github.io/RePyability/guide/saving/#what-is-exact-and-what-is-simulated) lists every method's route. -## Install -RePyability can be installed via pip using the PyPI [repository](https://pypi.org/project/repyability/) - -```bash -pip install repyability -``` - -Repairable systems simulate about ten times as fast with the optional -compiled engine, which needs [numba](https://numba.pydata.org) (available for -64-bit Linux, macOS on Apple silicon and Windows): - -```bash -pip install "repyability[fast]" -``` - ## Documentation The full documentation — tutorial, user guide, concepts and API reference — is hosted at @@ -230,28 +387,20 @@ mkdocs serve # then open http://127.0.0.1:8000 The source lives in `docs/` and `mkdocs.yml`; start with `docs/index.md`. -## Testing -Run the testing suite by simply executing: -```bash -pytest -``` -or use coverage to get a coverage report: -```bash -coverage run -m pytest # Run pytest under coverage's watch -coverage report # Print coverage report -coverage html # Make a html coverage report (really useful), open htmlcov/index.html -``` +## Roadmap + +Where RePyability is going, beyond the issue tracker: [ROADMAP.md](ROADMAP.md) +holds the larger ideas and their designs (first among them a net present +value simulator for maintenance plans, kept current as the plant changes), +and [`stories/`](stories/README.md) the questions users and agents should be +able to answer, with how we will know they are answered. + +## Related + +[Reliafy](https://reliafy.com) is a hosted, browser-based app built on RePyability and SurPyval, for building reliability block diagrams and analysing their reliability and availability without writing code. + +## Contributing -## Pre-commit -### TL;DR -- Pip install `pre-commit` (it's in `requirements_dev.txt` anyways) -- Run `pre-commit install` which sets up the git hook scripts -- If you'd like, run `pre-commit run --all-files` to run the hooks on all files -- When you go to commit, it will only proceed after all the hooks succeed - -### Why? -To ensure the good code quality and consistency it is recommended that when contributing to this -repository to use the provided `.pre-commit-config.yaml` configuration for the Python package -`pre-commit` (https://pre-commit.com). Upon making a commit, it checks that imports -and requirements are sorted, syntax is up-to-date, code is formatted, linted, and statically type-checked, -all with the same tools and configurations as one another. +See [CONTRIBUTING.md](CONTRIBUTING.md): setting up, the checks a change +needs (`scripts/check.sh`), and the parts of the package that must agree +with each other. diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 00000000..0d79655e --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,365 @@ +# RePyability roadmap + +The bigger ideas for RePyability: features that need design, research or +several weeks of work. They live here, not in the issue tracker, so the +issues stay a list of things someone could start this week. The outcomes +they serve, and how we will know they are met, are the user stories in +[`stories/`](stories/README.md). + +Statuses are as of 0.13 and what is merged on `dev` for 0.14 (16 October +2026). + +## How this file works + +- **An idea starts here**, as a section with the problem, why it matters, + its rough size, what it depends on, and a status: + - *idea*: worth doing, not designed; + - *designed*: the approach is settled (the design is summarised here); + - *scheduled*: broken into issues that are being worked on. +- **When an idea is scheduled**, its design is split into a few actionable + issues, linked from its section here. +- **When it is done**, its section moves to "Done" with a line and the + release; the changelog is the full record. +- **Ideas we have decided against** go under "Not planned", with the + reason, so they are not raised again. + +## Scope: timelines, not calendars + +RePyability works in model time: the models' own unit, from 0 or from a +state, with ages, phases and meter readings as numbers, and it answers in +plain data. Dates, calendars, operating schedules, meters read by date and +financial years belong to the application that uses it +([Reliafy](https://reliafy.com), for one), which turns dates into model +time, calls RePyability, and turns the answers back into dates. Every +design below is in model time: an outage is a time on the model's +timeline, a price curve a function of model time, and a horizon a length +of it. + +--- + +## The NPV simulator: what is a maintenance plan worth? + +*Status: designed. Size: large (phased, several weeks). Stories: MX-01 to +MX-17 in [`stories/maintenance-economics.md`](stories/maintenance-economics.md).* + +**Problem.** Maintenance plans, spares, crews and redundancy are decided on +money, at a cost of capital, over an asset's life. RePyability prices a +plan per hour in the long run; 0.12 added a `discount_rate` to +`total_cost` and `allocate_redundancy` that discounts that long-run rate. +It does not value a plan from where the plant is, over a horizon, with +what the plant earns while it runs. The question users and agents ask is +"which plan creates the most value, at the least cost, and how sure are +we?", and today they answer it by hand. + +**Why it can be exact.** By linearity of expectation the expected present +value of a cost charged at events is the cost times the discounted expected +number of events, and of a cost or value accruing per hour the discounted +integral of the (un)availability: + +```text +E[PV(events)] = c * integral_0^H exp(-delta t) dM(t) +PV(value) = integral_0^H exp(-delta t) * v(t) * A(t) dt +``` + +`M(t)` and `A(t)` are what `expected_events(t)` and `point_availability(t)` +already give exactly, from new or from a state, with maintenance, +inspections and (for exponential units) crews. So the expected NPV is exact +wherever those are, and simulation is needed only for what is not linear +in the events: percentiles, the chance of missing a hurdle, availability +contracts with penalties. The equivalent annual cost `EAC = delta * PV` +reduces to `expected_cost_rate` as `delta -> 0`, a regression check for +every plan. A prototype on `expected_cost(t)` (two Weibull pumps feeding a +compressor, 10 years at 8%) moved the best pump PM interval from 10,000 h +undiscounted to 14,000 h: deferring spend is worth something. + +**What exists for one part (0.14, #319).** The maintenance module answers +a single part's questions exactly, in model time: its failures and their +cost over a window from its age (`NonRepairable.failures`), the NPV of +skipping its next planned replacement (`ReplacementPlan.skip_next`, the +first discounted answer), the best replacement policy when a shut is the +cheapest time to replace (`replacement_policies`), and when to replace a +part in service next, given scheduled outages as times from now +(`NonRepairable.optimal_next_replacement`). The NPV simulator takes these +questions to a whole plant. + +**Design.** Three layers, all plain data so they save and load, all in +model time: + +- **Plan**: the actions, mixed freely. Periodic maintenance (as today); + outages at given times on the model's timeline, with a duration and a + scope (renew some components, restore others to a virtual age, take the + system down); recurring campaigns with opportunistic work; upgrades + that swap a component's life model at a capital cost; crews and + call-out contracts; retirement with a salvage value from each + component's age. Renewals at given times generalise block replacement + from multiples of an interval to any list of times. +- **Cash flows**: value (a rate, capacity times a price curve, any `v(t)`), + event costs (repair, replace, preventive, inspection, set-up), rate costs + (downtime, crews, holding spares), capital (acquisition, upgrades, + salvage), each with its own escalation; real or nominal rates. +- **Valuation**: the rate (per unit of model time), the horizon, and the + method: exact where the curves are (fixed-time events discounted at their + exact times, not grid midpoints), simulated where needed (each event + discounted at its time, plans compared with common random numbers). The + result: NPV, EAC, the undiscounted total, a cash-flow table by period of + model time, category and component, how it was computed, and the units. + Which period is a financial year, and which time is which date, is the + application's. + +**Phases** (each becomes issues when scheduled; the stories' building +blocks B1 to B7): + +1. Expected NPV and EAC on the exact curves, with a constant value rate and + the cash-flow table (B1). Validated against Fox's (1966) closed form + for discounted age replacement and the `delta -> 0` limit. +2. Value and prices over model time (a price curve `v(t)`), and escalation + (B2). +3. Plans on the timeline: outages at given times, campaigns, upgrades, + salvage (B3). +4. The simulated NPV distribution and non-linear payoffs (B4). +5. Crews as cash flows and the call-out option, valued in the crews' + Markov chain with "surge crew active" states (B5). +6. `objective="npv"` / `"eac"` in the optimisers, with their constraints, + and rho, the discount-rate sensitivity (B6); parameter uncertainty in + the NPV (B7, on 0.12's #200). + +**For agents** (B10). Every input and result is JSON with units; every NPV +analysis is in `analysis_routes()`; refusals say what to do; a valuation is +reproducible from its saved inputs. The 0.12 persona check's integrator +findings (#223, #231, #235) are the starting list. + +**Depends on.** Nothing new in SurPyval for phases 1 to 6. + +--- + +## Live NPV: keep the plan's value current as the plant changes + +*Status: designed (architecture below; details settle in phase 1 of the +NPV simulator). Size: large. Depends on: the NPV simulator. Stories: MX-18 +to MX-25.* + +**Problem.** A plan's value changes every time the plant does: a vibration +reading rises, a pump is replaced, a seal fails, new failure data narrow a +fit, the forward curve moves. The aim is that RePyability holds the plant +as it is now and re-values (and, when asked, re-optimises) the plan as each +event arrives, so that an application can keep every maintenance plan +optimal by NPV through all the churn of real maintenance work, and an agent +can say at any moment what the plan is worth, what changed it, and what to +do next. + +**What exists.** Every exact analysis of a `RepairableRBD` starts from the +components' current states (`state=`, `NodeState`: age, up or down, how +long down, whether in maintenance, the phase of its schedule, or +stationary), and `simulate_timelines` does too (#163). The simulations +also start an imperfectly repaired unit from its virtual age +(`NodeState(virtual_age=)`, #269), which a fitted `GeneralizedRenewal`'s +`unit_states()` gives; a fitted repairable-unit model is a component +(#269), and a fitted gamma or Wiener degradation process a life, with +replacement on condition by its measured level (#271). A +`NonRepairableRBD` gives the reliability, remaining life and importance +given the states. What is missing is the state a sensor reading gives (a +component's current level), the events that move the state, and a +valuation that follows it. + +**Design.** + +- **A live plant object** (working name `Plant`): the `RepairableRBD`, the + `Plan`, the `Valuation`, each component's current state, and the log of + events, as of a time. It is event-sourced: the state is the fold of the + events, so events can arrive late or out of order (a work order closed + some time after the job) and the result is the same as in order, and any + past valuation can be rebuilt. Every event's time is a model time. +- **Events**: + - `reading(node, level, at)`: a condition measurement. The component's + life becomes its remaining-life distribution from that level, from a + SurPyval degradation model (gamma or Wiener process, or a path model); + - `replaced(node, at)`: age 0; `repaired(node, at, virtual_age=)`: an + imperfect repair (Kijima, with SurPyval's fitted restoration factor); + - `failed(node, at)`: down, queued for a crew if crews are limited, + spares drawn from stock; `inspected(node, at, found=)`: a test of a + hidden failure; + - `refit(component_type, model)`: a new life model (with its covariance) + for every component of that type, each keeping its own age or + condition; + - `prices(curve)`, `plan(changes)`, `covariates(node, values, at)` (a + load or temperature change for a covariate-dependent component). +- **State.** `NodeState` has a virtual age (#269); it gains what the other + events need: a condition (the conditioned life model it implies) and a + per-node model override, or a richer per-node state object that carries + them, so that `age` keeps its single meaning (operating time). +- **Incremental re-valuation.** Each component's curves (availability, + expected events) depend only on its own model, state and plan, so they + are cached by that key and an event recomputes only the components it + touched; independent components recombine through the structure + function, and dependent modules (crews, standby groups, common-cause + groups) are recomputed module by module, as 0.12's conditional runs + already split them (#189). A price change re-values without touching the + reliability at all. The live result must equal a valuation from scratch + with the same state, to the exact methods' precision. Target: about a + second per event for a 50-component plant with exact curves. +- **Attribution.** Each event's effect on the NPV is reported (the + difference between successive valuations), and the effects since any + time add up to the total change, so "why did the plan's value drop?" has + an answer by event and component. +- **Re-optimisation** (receding horizon): from the state now, the plan + that maximises NPV within crews, outage windows, budgets and + availability or PFDavg limits, warm-started from the current plan; + changes proposed only when they beat the current plan by a margin, so + the plan does not churn on sensor noise, each with its value ("bring pump + 2's replacement forward to the next outage: +$45k"). +- **Change report.** For each update: the NPV move, whether it crossed a + threshold, and whether the recommended next actions changed and why. + The application turns that into alerts. +- **Reproducibility.** The plant, its events and each valuation save to + JSON; a saved valuation reproduces exactly (with its seed where it + simulates). + +**Who does what.** + +- **SurPyval**: a degradation model's remaining life from a current level + as a life model RePyability can take (`sf`, `ff`, `qf`, `mean`, + `random`); per-unit virtual age from a repair history (there since + 0.23, #615); cheap refits or sequential updates as failures arrive, with + the covariance. (See SurPyval's roadmap.) +- **RePyability**: the plant object, the state, the valuation, the + attribution, the re-optimisation, the change report, all in model time. +- **The application**: ingesting readings and work orders, turning their + dates (and run hours read by date) into model time and the answers back + into dates, keeping each plant, running updates, and alerts. + +**Open questions.** + +- Time base: one model clock per plant. A component that runs part of the + time takes a `"duty"`, the fraction of the time it operates (#276), so a + life fitted in operating time sits on the plant's clock; turning a + utilisation measured by date into that clock is the application's. +- What a live update does when an exact route is refused (a crew-limited + plant of non-exponential units): simulate with common random numbers to + a tolerance, and say so. +- Covariate-driven condition (temperature, load) against degradation + signals: both are "condition", through different life models + (`RegressionNode` against a degradation model's remaining life). +- How far to trust a re-optimisation made on uncertain models: report the + decision's robustness (does the best action change across the parameter + draws of B7?). + +--- + +## Decision analytics on the NPV + +*Status: idea. Size: medium each. Depends on: the NPV simulator.* + +Questions that become one or two calls once plans have a value: + +- **Price-aware outage timing**: place planned outages against a forward + price curve (a function of model time), not only choose how often + (MX-10, MX-23). +- **Value of information**: what better data are worth (more teardowns, a + sensor), from the NPV lost to parameter uncertainty (MX-15; on #196 and + #200). +- **Real options**: run, refurbish, replace or life-extend as decisions + taken over time from the state now (MX-08, MX-12). +- **Fleet capital rationing**: across many plants, the portfolio of + interventions with the highest NPV for a budget. +- **Procurement break-even**: the most a more reliable part is worth, or + the life it needs at a quoted price (MX-07). +- **NPV sensitivities**: rho, and the Greeks of the NPV rather than of the + availability (0.12's #192 machinery with the NPV as the quantity), + ranked per dollar. +- **Spares and maintenance together**: stock levels and intervals chosen + jointly, since each changes the other's value (MX-14). +- **Shadow prices**: carbon or safety costs as cash flows, so "value" is + more than margin. + +--- + +## The maintenance module + +*Status: scheduled (#319). Size: large, one use case at a time.* + +`repyability.maintenance` answers a maintainer's questions about a part: +run to failure, scheduled replacement, failure finding, on-condition, and +the options at a planned replacement, with the probabilities a forecast is +made of (expected failures and costs over a window from a state, and their +distribution). Each use case is recorded in +[`repyability/maintenance/README.md`](repyability/maintenance/README.md) +before it is built: who asks, what they know and want back, how it is +solved, the answer worked by hand, and the checks. Built so far, for 0.14: +a part's failures over a window (UC-5), the NPV of skipping the next +planned replacement (UC-6), the best replacement policy with shuts (UC-7) +and when to replace a part in service next (UC-12). The maintainer chooses +which use case is next. Tasks on dates, operating schedules and meters +read by date are the application's (see "Scope" above). + +--- + +## Ongoing practice: hardening + +Hardening is continuous, not a one-off review before 1.0. It is how every +change is made: + +- **Review rounds.** A periodic persona check of the installed release, + done as users (and agents) use it: a plant engineer, a safety analyst, a + statistician, an integrator and a new user following the docs, each + answer checked independently. Each round files concrete issues, fixed in + the next. Rounds so far: 0.11 (#184), 0.12 (#213), a review of how + RePyability uses surpyval 0.24 (#266), and the code and docs review + after 0.13 (#277, #278, #279). +- **Scenario cards in the regular tests.** Each persona's study becomes a + card in `repyability/tests/scenarios`: its questions answered through + the public API, checked against an independent oracle (a closed form, + a Markov chain, an enumeration), with each question not yet answerable + as a strict xfail citing its story or issue, so a fix shows up as a + failing run until the card is updated. They run with the rest of the + suite, so they are small and exact (SurPyval's practitioner cards are + the model). The stories in `stories/` are the cards' source. +- **Every analysis on every kind of diagram.** The routes test runs each + analysis on every node kind and spec key of the test catalogue, with a + guard that fails when one is missing (#239, 0.13), and the README's + capability tables are made from it, with a test that they are current + (0.14). +- **The docs say what the code does.** The docs test checks every shown + output, not only `# ->` comments; the tests run from the installed wheel + (#167, 0.13). +- **Concurrency.** Simulations on a shared diagram from several threads + give the same seeded results (#216, 0.13). + +--- + +## Not planned + +- **Fitting lifetime models in RePyability.** Fitting stays in SurPyval; + RePyability takes fitted models (CLAUDE.md). +- **Plotting and dashboards.** These stay outside RePyability (the Reliafy + app does them); RePyability stays free of plotting dependencies. +- **Physics of failure.** Lives come from data, through SurPyval's + models, not from physical models of a failure mechanism; RePyability + does not model them. +- **Calendars and dates.** Dates, calendars, operating schedules, meters + read by date and financial years are the application's (see "Scope" + above); RePyability works in model time. +- **Moving `repyability.maintenance` out.** It stays: a part's + maintenance questions (time-based replacement and RCM's quantitative + questions) are RePyability's, in model time. + +--- + +## Done + +- **Simpler implementations** (0.13, #204): timelines from the event + loop's recording only (#205), one copy of a component's events in the + compiled loop (#206), one design of the decision-diagram search and + replay (#207), one implementation of sorting a run's changes (#208), and + random streams whose draws depend only on the seed, the stream, the + simulation and the draw (#209). +- **Hardening from the 0.12 persona check** (0.13, #213): among them, + every analysis on every kind of diagram (#239), the tests from the + installed wheel (#167) and simulations from several threads (#216). +- **The Greeks** (0.12, #197): the sensitivity measures as one family: + delta (#191), parameter sensitivity (#192), differential importance + (#193), gamma (#194), theta and Barlow–Proschan (#195, #199), vega + (#196, #200). +- **Discounted total costs and train-wise redundancy** (0.12, #184): the + first step of the NPV simulator. +- **Exact by default** (0.12, #186, #187, #189): a run's expected values + exact, or taken given its dependent modules. diff --git a/docs/api.md b/docs/api.md deleted file mode 100644 index 9369df42..00000000 --- a/docs/api.md +++ /dev/null @@ -1,164 +0,0 @@ -# API reference - -Generated from the docstrings of the public API: everything exported from the -top-level `repyability` package (`from repyability import ...`). The -installed version is `repyability.__version__`. For task-oriented -explanations and runnable examples, see the [user guide](guide/index.md). - -## System models - -::: repyability.RBD - -::: repyability.NonRepairableRBD - -::: repyability.RepairableRBD - -::: repyability.FaultTree - -::: repyability.PhasedMission - -::: repyability.Network - -## Node models - -Anything exposing `sf`/`ff` can be a node; these are the composite and helper -models provided here (see [Building an RBD](guide/building.md) and -[Redundancy models](guide/redundancy-models.md)). - -::: repyability.StandbyModel - -::: repyability.DegradingNode - -::: repyability.RepeatedNode - -::: repyability.RepeatedStandbyNode - -::: repyability.LoadSharingModel - -::: repyability.RegressionNode - -::: repyability.PerfectReliability - -::: repyability.PerfectUnreliability - -## Design inputs - -::: repyability.ComponentOption - -## Common-cause failures - -::: repyability.CCFGroup - -::: repyability.BetaFactor - -::: repyability.MGL - -## Condition-based evaluation - -::: repyability.NodeState - -## Components and maintenance - -::: repyability.NonRepairable - -::: repyability.Repairable - -::: repyability.minimal_repair_time_to_nth_failure - -## Demonstration test planning - -::: repyability.demonstration_sample_size - -::: repyability.demonstrated_reliability - -::: repyability.demonstration_test_multiple - -::: repyability.demonstration_pass_probability - -::: repyability.demonstration_plan - -::: repyability.mtbf_test_time - -::: repyability.demonstrated_mtbf - -::: repyability.mtbf_pass_probability - -::: repyability.mtbf_demonstration_plan - -## Timelines - -Up/down histories over a window (see the -[Timelines guide](guide/timelines.md)). The merge functions are in -`repyability.timelines`. - -::: repyability.Timeline - -::: repyability.Timelines - -::: repyability.timelines.series - -::: repyability.timelines.parallel - -::: repyability.timelines.k_out_of_n - -## Result types - -::: repyability.AnalysisRoute - -::: repyability.AvailabilityResult - -::: repyability.CapacityDistribution - -::: repyability.Criticalities - -::: repyability.UpDownImportance - -::: repyability.FailureCriticalityIndex - -::: repyability.RestorationCriticalityIndex - -::: repyability.CostResult - -::: repyability.ExpectedEvents - -::: repyability.RateBreakdown - -::: repyability.ExpectedCost - -::: repyability.ConfidenceInterval - -::: repyability.ControlVariate - -::: repyability.ConditionalRun - -::: repyability.UncertaintyResult - -::: repyability.UncertaintyImportance - -::: repyability.RedundancyAllocation - -::: repyability.ReliabilityRedundancyAllocation - -::: repyability.TotalCostAllocation - -::: repyability.MaintenancePlan - -::: repyability.Lever - -::: repyability.DemonstrationPlan - -::: repyability.AvailabilityAllocation - -::: repyability.SparesDemand - -::: repyability.SparesStock - -::: repyability.SimulationChunk - -::: repyability.TimelineSimulation - -::: repyability.run_shard - -::: repyability.MaintenancePolicy - -::: repyability.FailureLimitPolicy diff --git a/docs/api/components.md b/docs/api/components.md new file mode 100644 index 00000000..aad3b290 --- /dev/null +++ b/docs/api/components.md @@ -0,0 +1,11 @@ +# Components and maintenance + +::: repyability.NonRepairable + +::: repyability.Repairable + +::: repyability.minimal_repair_time_to_nth_failure + +::: repyability.Opportunity + +::: repyability.Calendar diff --git a/docs/api/demonstration.md b/docs/api/demonstration.md new file mode 100644 index 00000000..a48d5650 --- /dev/null +++ b/docs/api/demonstration.md @@ -0,0 +1,19 @@ +# Demonstration test planning + +::: repyability.demonstration_sample_size + +::: repyability.demonstrated_reliability + +::: repyability.demonstration_test_multiple + +::: repyability.demonstration_pass_probability + +::: repyability.demonstration_plan + +::: repyability.mtbf_test_time + +::: repyability.demonstrated_mtbf + +::: repyability.mtbf_pass_probability + +::: repyability.mtbf_demonstration_plan diff --git a/docs/api/index.md b/docs/api/index.md new file mode 100644 index 00000000..e065162f --- /dev/null +++ b/docs/api/index.md @@ -0,0 +1,16 @@ +# API reference + +Generated from the docstrings of the public API: everything exported from the +top-level `repyability` package (`from repyability import ...`). The +installed version is `repyability.__version__`. For task-oriented +explanations and runnable examples, see the [user guide](../guide/index.md). + +- [RBD (the base class)](rbd.md) +- [NonRepairableRBD](nonrepairable.md) +- [RepairableRBD](repairable.md) +- [Fault trees, phased missions and networks](other-systems.md) +- [Node models](nodes.md) +- [Components and maintenance](components.md) +- [Demonstration test planning](demonstration.md) +- [Timelines](timelines.md) +- [Result types](results.md) diff --git a/docs/api/nodes.md b/docs/api/nodes.md new file mode 100644 index 00000000..c353c52a --- /dev/null +++ b/docs/api/nodes.md @@ -0,0 +1,41 @@ +# Node models + +## Composite and helper nodes + +Anything exposing `sf`/`ff` can be a node; these are the composite and helper +models provided here (see [Building an RBD](../guide/building.md) and +[Redundancy models](../guide/redundancy-models.md)). + +::: repyability.StandbyModel + +::: repyability.DegradingNode + +::: repyability.RepeatedNode + +::: repyability.RepeatedStandbyNode + +::: repyability.LoadSharingModel + +::: repyability.RegressionNode + +::: repyability.PerfectReliability + +::: repyability.PerfectUnreliability + +## Design inputs + +::: repyability.ComponentOption + +## Common-cause failures + +::: repyability.CCFGroup + +::: repyability.BetaFactor + +::: repyability.MGL + +::: repyability.AlphaFactor + +## Condition-based evaluation + +::: repyability.NodeState diff --git a/docs/api/nonrepairable.md b/docs/api/nonrepairable.md new file mode 100644 index 00000000..c7803bed --- /dev/null +++ b/docs/api/nonrepairable.md @@ -0,0 +1,3 @@ +# NonRepairableRBD + +::: repyability.NonRepairableRBD diff --git a/docs/api/other-systems.md b/docs/api/other-systems.md new file mode 100644 index 00000000..4c890284 --- /dev/null +++ b/docs/api/other-systems.md @@ -0,0 +1,7 @@ +# Fault trees, phased missions and networks + +::: repyability.FaultTree + +::: repyability.PhasedMission + +::: repyability.Network diff --git a/docs/api/rbd.md b/docs/api/rbd.md new file mode 100644 index 00000000..681dee9d --- /dev/null +++ b/docs/api/rbd.md @@ -0,0 +1,7 @@ +# RBD (the base class) + +The methods every diagram has. +[`NonRepairableRBD`](nonrepairable.md) and [`RepairableRBD`](repairable.md) +list them again, with their own. + +::: repyability.RBD diff --git a/docs/api/repairable.md b/docs/api/repairable.md new file mode 100644 index 00000000..073499ed --- /dev/null +++ b/docs/api/repairable.md @@ -0,0 +1,3 @@ +# RepairableRBD + +::: repyability.RepairableRBD diff --git a/docs/api/results.md b/docs/api/results.md new file mode 100644 index 00000000..5a2b94fa --- /dev/null +++ b/docs/api/results.md @@ -0,0 +1,75 @@ +# Result types + +::: repyability.AnalysisRoute + +::: repyability.AnalysisRoutes + +::: repyability.AvailabilityResult + +::: repyability.CapacityDistribution + +::: repyability.Criticalities + +::: repyability.UpDownImportance + +::: repyability.FailureCriticalityIndex + +::: repyability.RestorationCriticalityIndex + +::: repyability.CostResult + +::: repyability.ExpectedEvents + +::: repyability.RateBreakdown + +::: repyability.ExpectedCost + +::: repyability.ConfidenceInterval + +::: repyability.ControlVariate + +::: repyability.ConditionalRun + +::: repyability.UncertaintyResult + +::: repyability.UncertaintyImportance + +::: repyability.RedundancyAllocation + +::: repyability.ReliabilityRedundancyAllocation + +::: repyability.TotalCostAllocation + +::: repyability.MaintenancePlan + +::: repyability.Lever + +::: repyability.DemonstrationPlan + +::: repyability.AvailabilityAllocation + +::: repyability.SparesDemand + +::: repyability.SparesStock + +::: repyability.SimulationChunk + +::: repyability.TimelineSimulation + +::: repyability.run_shard + +::: repyability.MaintenancePolicy + +::: repyability.FailureLimitPolicy + +::: repyability.FailureForecast + +::: repyability.SkipValue + +::: repyability.NextReplacement + +::: repyability.ReplacementChoice + +::: repyability.CalendarPolicy + +::: repyability.PlannedReplacement diff --git a/docs/api/timelines.md b/docs/api/timelines.md new file mode 100644 index 00000000..814d2f17 --- /dev/null +++ b/docs/api/timelines.md @@ -0,0 +1,15 @@ +# Timelines + +Up/down histories over a window (see the +[Timelines guide](../guide/timelines.md)). The merge functions are in +`repyability.timelines`. + +::: repyability.Timeline + +::: repyability.Timelines + +::: repyability.timelines.series + +::: repyability.timelines.parallel + +::: repyability.timelines.k_out_of_n diff --git a/docs/concepts.md b/docs/concepts.md index 6751f1b9..4e6396ec 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -1,36 +1,26 @@ # Concepts -The reference behind the numbers: what an RBD means, how each quantity is -computed, what each model assumes, and how to choose among the importance -measures. The [tutorial](tutorial.md) shows these in action and the -[user guide](guide/index.md) shows how to call them; this page explains them. +The reference behind the numbers: how RePyability computes each quantity, +what each model assumes, and which method to choose. The +[tutorial](tutorial.md) shows these in action and the +[user guide](guide/index.md) shows how to call them; this page explains how +they are worked out. !!! tip "New to the theory?" - This page is a compact summary. The [Learn](learn/index.md) course - teaches the same ideas step by step: each one worked out by hand with - small numbers, then with RePyability, with exercises. + This page is the reference for how RePyability computes each analysis. + The [Learn](learn/index.md) course teaches the ideas themselves, step by + step: each one worked out by hand with small numbers, then with + RePyability, with exercises. ## Reliability block diagrams -A **reliability block diagram** models a system as a directed graph from a -single input (source) to a single output (sink). Each intermediate node is a -component with a reliability model. The system **works** at time `t` if there -is a path of working components from input to output. - -Two structures underlie everything: - -- A **minimal path set** is a minimal set of components whose simultaneous - working guarantees the system works. The system is up iff *at least one* - path set is fully up. -- A **minimal cut set** is a minimal set of components whose simultaneous - failure guarantees the system fails. The system is down iff *at least one* - cut set is fully down. - -Series, parallel, and *k*-out-of-*n* are special cases: a series system is one -path set of every component (and each component its own cut set); a parallel -system is the reverse. A *k*-out-of-*n* node is up when at least `k` of its -incoming branches are. A component that appears in no minimal path set is -**irrelevant**: its state never decides whether the system works. +A diagram's system works at `t` when working components connect its input +to its output, and its minimal path and cut sets describe that structure +(see [Systems](learn/systems.md#reliability-block-diagrams) and +[Paths and cuts](learn/structure.md#paths-and-cuts)). A *k*-out-of-*n* node +is up when at least `k` of its incoming branches are. A component in no +minimal path set is **irrelevant**: its state never decides whether the +system works. The theory assumes a **coherent** system (repairing a component never makes it worse) with **independent** components, unless a dependency is modelled @@ -76,12 +66,9 @@ itself (a node works when it has not failed and enough of its inputs work), and the exact analyses refuse, saying why, as `analysis_routes()` reports. Raise the limit to try harder. -The identity that drives the decomposition, and the importance measures, is -**pivotal decomposition** around any node *A*: - -``` -R_sys = R_A · R_sys(A working) + (1 − R_A) · R_sys(A failed) -``` +Both stages rest on +[pivotal decomposition](learn/structure.md#pivotal-decomposition-divide-and-conquer), +which also gives the importance measures. The engine is exact *given the node reliabilities*. A node with no exact or numerical reliability (a standby or load-sharing arrangement only a @@ -106,26 +93,26 @@ T_sys = max over minimal path sets P of ( min over i in P of T_i ) `random()` draws each component's lifetime and applies this rule, through the modules (a series module fails at its first failure, a parallel one at its last, a *k*-out-of-*n* one when fewer than `k` are left) so that the path sets -are only needed for the core; `mean_time_to_failure()` is the average of many -such lifetimes. By the central -limit theorem the average is approximately normal with standard error -`s / √n` (the sample standard deviation over the square root of the number of -samples), which gives `mean_time_to_failure_interval()`. The mean is estimated -rather than integrated because the system lifetime distribution of a general -diagram, especially with composite nodes, has no convenient closed form. +are only needed for the core. `mean_time_to_failure()` integrates the exact +reliability by default. With `method="simulate"` it averages such +lifetimes, with standard error `s / √n` +([why](learn/systems.md#mean-time-to-failure-of-a-system)), which +`mean_time_to_failure_interval()` reports. ### Simulation error, and making it smaller -A simulated mean of `N` independent results has standard error `s / √N`, so -its error halves when `N` is quadrupled. Given a `tolerance`, a simulation -adds `N` results at a time until the confidence interval's half-width, -`z · s / √N`, is at most the tolerance. (The stopping point depends on the -estimated `s`, which makes a sequential rule's coverage slightly below the -nominal level when `N` is small; checking only after each batch of `N` -keeps the effect small.) +The error of a simulated mean shrinks like `1/√N` +([Lesson 6](learn/availability.md#precise-enough-sooner)). Given a +`tolerance`, a run adds `N` results at a time until the confidence +interval's half-width, `z · s / √N`, is at most the tolerance. (The +stopping point depends on the estimated `s`, which makes a sequential +rule's coverage slightly below the nominal level when `N` is small; +checking only after each batch of `N` keeps the effect small.) Two classical variance-reduction techniques make the error smaller for the -same `N`, without biasing the estimate: +same `N`, without biasing the estimate (the +[lesson](learn/availability.md#precise-enough-sooner) gives the intuition; +here is why they work): - **Antithetic variates.** A result is a function `f(U)` of uniform random numbers. `f(U)` and `f(1 − U)` have the same distribution, and when `f` is @@ -159,13 +146,10 @@ processes. ### Fault trees -A fault tree describes the same structure from the side of failure: the top -event occurs through OR gates (any input), AND gates (every input) and VOTE -gates (at least `k` of `n` inputs) over the basic events. Its logic is the -dual of a diagram's: an OR gate is a series block, an AND gate a parallel -block, and a VOTE gate on `k` of `n` failures a block needing `n − k + 1` of -`n` working; the tree's minimal cut sets are the diagram's. A tree is -evaluated by the same engine: each gate below which no event or gate is +A fault tree is the diagram's dual, with OR, AND and VOTE gates for series, +parallel and *k*-of-*n* blocks +([Lesson 3](learn/structure.md#the-same-logic-upside-down-fault-trees)). A +tree is evaluated by the same engine: each gate below which no event or gate is shared with the rest of the tree is a module with a closed form, and what the repeated events tie together is a core, solved exactly by the pivotal decomposition, on a binary decision diagram built from its gates (its cut @@ -174,9 +158,9 @@ diagram's, with the top event as the system failing. ### Parameter uncertainty -The node models are estimates, so the system reliability computed from them -is uncertain too: *epistemic* uncertainty, about the models, as opposed to -the *aleatory* variability they describe. `sf_uncertainty` propagates it by +The node models are estimates, so the system's values carry *epistemic* +uncertainty ([Lesson 2](learn/systems.md#how-sure-are-you-of-the-inputs)). +`sf_uncertainty` propagates it by Monte Carlo over the parameters: each draw gives every uncertain node a plausible model and the system reliability is computed exactly, all draws at once (the node probabilities are arrays over draws and times), and the @@ -184,13 +168,13 @@ percentiles of the draws form an uncertainty interval. A maximum-likelihood fit's own estimate of its parameters' uncertainty (the inverse Hessian of the log-likelihood, from surpyval) gives the draws on a transformed scale (log for a positive parameter, logit for one in (0, 1)), which is the delta -method's normal approximation. Nodes of one population share their -parameters and so their draws: drawing them independently averages part of -the uncertainty away. The same draws give the MTTF's, a B*X* life's and the -time to a reliability's uncertainty (`mean_uncertainty`, -`bx_life_uncertainty`, `time_to_reliability_uncertainty`): each draw's value -is the exact one for its models, the area under its reliability or the root -of its reliability less the target. A repairable system's availability and +method's normal approximation. Nodes of one population share one draw (a +tuple of nodes), as they share their parameters. The same draws give the +MTTF's, a B*X* life's and the time to a reliability's uncertainty +(`mean_uncertainty`, `bx_life_uncertainty`, +`time_to_reliability_uncertainty`): each draw's value is the exact one for +its models, the area under its reliability or the root of its reliability +less the target. A repairable system's availability and cost rate are uncertain in the same way, through its components' lives, repairs and maintenance times: each draw rebuilds the diagram with its models and works its value out as the diagram's own @@ -202,12 +186,14 @@ settle the summaries with fewer of them. ## Reliability vs availability - **Reliability** `R(t)`: the probability the system has *never* failed by - `t`. The right question for a mission or a non-repairable item. Lives on + `t` ([Lesson 1](learn/lifetimes.md#the-time-to-failure-is-a-random-variable)). + The right question for a mission or a non-repairable item. Lives on [`NonRepairableRBD`][repyability.NonRepairableRBD]. - **Availability** `A(t)`: the probability the system is *up at* `t`, - allowing for repair. The right question for a serviced, long-running - system. Lives on [`RepairableRBD`][repyability.RepairableRBD], which needs - a repairability distribution per component. + allowing for repair ([Lesson 6](learn/availability.md#a-new-component-starts-up)). + The right question for a serviced, long-running system. Lives on + [`RepairableRBD`][repyability.RepairableRBD], which needs a repairability + distribution per component. ## Importance measures — which one, and why @@ -231,9 +217,8 @@ Two more answer *design-time* and *data-targeting* questions rather than ranking at an operating point: - **Structural importance** `structural_importance`: Birnbaum with every node - reliability set to ½, i.e. the fraction of the other nodes' states in which - the node is pivotal. It is **model-free**: it depends only on the diagram, - so you can rank redundancy needs *before any data exists*. + at ½, which depends only on the diagram + ([Lesson 4](learn/importance.md#structural-importance-before-you-have-any-data)). - **Parameter sensitivity** `parameter_sensitivity`: the derivative of system reliability with respect to each node's *distribution parameters*, `∂R/∂θ = I_B · ∂R_i/∂θ`, computed numerically. Where Birnbaum says *which @@ -270,9 +255,9 @@ A rule of thumb: **Birnbaum** for "where does an improvement help most", **structural importance** at the whiteboard, and **parameter sensitivity** when deciding where to spend a testing budget. -On a repairable system the same measures are evaluated with long-run -availabilities in place of reliabilities, or with the point availabilities -from new (or from the components' current states) over time. +On a repairable system they take long-run availabilities +([Lesson 4](learn/importance.md#on-a-repairable-system)), or point +availabilities over time, from new or from the components' current states. Read together, the sensitivity measures are the system's *Greeks*, named after an option's: delta (Birnbaum), the levers' deltas (parameter @@ -284,16 +269,12 @@ runs one pumping station through all of them. ## Condition-based evaluation The measures above assume every component is new. The condition-based methods -instead take each component's *current life* `Xᵢ` and condition on it (a -repairable system's analyses over time take its components' states too: -see [From the present](#availability)): - -``` -Rᵢ(x | Xᵢ) = Rᵢ(Xᵢ + x) / Rᵢ(Xᵢ) -``` - -This is the survival of a further `x` given the component has already reached -`Xᵢ`. Feeding the conditioned per-node reliabilities through the same exact +instead take each component's *current life* `Xᵢ` (a repairable system's +analyses over time take its components' states too: see +[From the present](#availability)). Each component's reliability is +conditioned on its age, `Rᵢ(Xᵢ + x) / Rᵢ(Xᵢ)` +([Lesson 1](learn/lifetimes.md#the-exponential-failures-that-ignore-age)). +Feeding the conditioned per-node reliabilities through the same exact system computation gives `sf_given_state`; inverting it gives `remaining_life` (remaining useful life); and evaluating the importance measures at the conditioned reliabilities gives `importances_given_state`. @@ -359,9 +340,9 @@ branches): that is a single component, which fails once for every place it appears, and treating its appearances as independent copies over-states reliability. -**Cold standby.** With one unit operating and spares that do not age while -waiting, the arrangement's lifetime is the *sum* of the units' lifetimes, so -its survival function is the convolution of theirs. RePyability computes it by +**Cold standby.** With one unit operating, cold spares add their lifetimes +([Lesson 5](learn/dependence.md#cold-standby)), so the arrangement's +survival function is the convolution of theirs. RePyability computes it by numerical convolution (deterministic, no sampling). For identical Exponential units the sum is Erlang, in closed form, for any `k` operating units. @@ -370,18 +351,16 @@ the lifetime is the sum of the first `j` units' lifetimes with probability that exactly `j − 1` switches succeeded before one failed (or all succeeded): a mixture of partial sums, again computed by convolution. -**Warm and hot standby.** A dormant spare that ages at a fraction `κ` of the -operating rate (the `dormancy_factor`) accumulates *virtual age* at rate `κ` -while waiting and `1` once operating, and fails when its virtual age reaches -its baseline failure age. It can therefore fail *latent*, before it is ever -switched in. For identical Exponential units the memoryless property makes -each stage (from `j` to `j − 1` surviving units) exponential with rate -`λ (k + (j − k) κ)`, so the lifetime is **hypoexponential**; `κ = 0` gives -the cold Erlang and `κ = 1` the parallel order statistic. Hot standby -(`κ = 1`) is exactly *k*-out-of-*n* active parallel, and is worked out so -for any units. With one unit operating and any units, a spare switched in at -`τ` has aged `κτ` and runs until its failure age, so the lifetime follows a -recursion over the switch-ins, which is computed on a time grid; with two +**Warm and hot standby.** A warm spare ages at `κ` (the `dormancy_factor`) +of the operating rate +([Lesson 5](learn/dependence.md#warm-and-hot-standby)). It fails when its +virtual age reaches its baseline failure age, so it can fail before it is +switched in. Identical Exponential units give a hypoexponential lifetime, +exactly, each stage (from `j` to `j − 1` surviving units) exponential with +rate `λ (k + (j − k) κ)`. Hot standby (`κ = 1`) is *k*-out-of-*n* active +parallel, and is worked out so for any units. With one unit operating and +any units, a spare switched in at `τ` has aged `κτ` and runs until its +failure age, so the lifetime follows a recursion over the switch-ins, which is computed on a time grid; with two units, `R(t) = S₁(t) + ∫₀ᵗ f₁(u) S₂(t − (1 − κ)u) du`. With `k ≥ 2` operating units, cold, each operating position runs a renewal @@ -396,10 +375,9 @@ new), a recursion on a grid of the two; with three or more, and warm with ## Dependent failures: load sharing -Redundant units that *share a load* do not fail independently. While all are -up each carries its share; when one fails the survivors pick up the slack, -run harder, and age faster, so the failures are positively correlated, and -treating them as `n` independent parallel nodes over-counts the redundancy. +Units that share a load fail sooner as their siblings fail, so independent +parallel nodes over-count the redundancy +([Lesson 5](learn/dependence.md#a-shared-load)). A [`LoadSharingModel`][repyability.LoadSharingModel] captures the coupling as a single node. Each unit is a fitted AFT model with **load as its covariate**, @@ -435,23 +413,17 @@ dormant rate instead of a load-dependent one. ## Common-cause failures -Redundancy only buys reliability if the redundant units fail for -*independent* reasons. In practice they often share a root cause (a common -manufacturing batch, a shared power supply, one miscalibration applied to -every unit) and a single event takes them all down together. Because the -exact engine assumes independence, it **over-estimates** a redundant group; a -common-cause model injects the shared coupling. (This is the mirror image of -load sharing: there the coupling is mechanical load transfer, here it is a -shared shock.) +A shared cause can fail redundant units together, which the independent +engine over-rates ([Lesson 5](learn/dependence.md#one-cause-every-unit)); a +common-cause model adds that coupling. (This is the mirror image of load +sharing: there the coupling is load transfer, here it is a shared shock.) A [`CCFGroup`][repyability.CCFGroup] declares the coupled (symmetric) members -and the model, passed via `ccf_groups`. Two models: +and the model, passed via `ccf_groups`. Three models: -- [`BetaFactor(beta)`][repyability.BetaFactor]: a fraction `β` of each - unit's failure probability `Q` comes from a cause shared across the - **whole** group (which fails every member at once); the remaining - `(1 − β) Q` is independent. The workhorse of probabilistic-risk - assessment. +- [`BetaFactor(beta)`][repyability.BetaFactor]: a share `β` of each + member's `Q` fails the whole group at once, and the remaining `(1 − β) Q` + is independent ([Lesson 5](learn/dependence.md#the-beta-factor-model)). - [`MGL(beta, gamma, ...)`][repyability.MGL]: the **Multiple Greek Letter** model, which also resolves *partial* common causes (a cause failing some but not all of the group) through a cascade of conditional probabilities: @@ -468,6 +440,13 @@ Q_k = [ 1 / C(m−1, k−1) ] · (ρ₁ ρ₂ ⋯ ρ_k) · (1 − ρ_{k+1}) · Q being separate events, it fails with probability `Q` to first order (as in PRA's basic events). A group of `m` members takes `m − 1` letters, and `MGL(β)` on two members is exactly `BetaFactor(β)`. +- [`AlphaFactor(alphas)`][repyability.AlphaFactor]: the **alpha-factor** + model, the same multiplicities by the shares `α_1, …, α_m` of a group's + failure events that failed exactly `1, …, m` members, which event + counts estimate directly (NUREG/CR-5485): `Q_k = α_k / C(m−1, k−1) · Q` + for staggered testing, and `Q_k = k α_k / (α_t C(m−1, k−1)) · Q` with + `α_t = Σ k α_k` for non-staggered. It is the `MGL` model with the letters + those `Q_k` give. **The evaluation is exact, not a correction factor.** Each model's `decompose` splits the group's failure into an independent part plus a set of @@ -522,8 +501,6 @@ and a copy of a beta-factor group's member joins the group in a redundancy allocation. The condition-based methods would need members of different ages, and raise a clear error on a CCF RBD; `structural_importance`, being probability-free, is unaffected. -**Alpha-factor**, a data-estimable reparameterisation of the same -multiplicities, is a planned extension. A `RepairableRBD` takes `ccf_groups` too. There a member's failures are a rate, so the model splits the rate: each cause, a member's own or a shared @@ -535,48 +512,23 @@ states, exactly (see [Common-cause failures](guide/common-cause.md#repairable-sy ## Availability -A repairable component alternates between up periods (drawn from its -reliability model) and down periods (drawn from its repairability model). Each -repair restores it **as good as new**, so its history is an *alternating -renewal process*, and components do this independently of each other and of -the system's state. - -**Long-run availability.** By the renewal-reward theorem a component is up a -fraction - -``` -A_i = MTTF_i / (MTTF_i + MTTR_i) -``` - -of the time in the long run, and, the components being independent, the -system's long-run availability is the exact system probability evaluated at -the `A_i`. No simulation is needed. An instantly repaired component -(`MTTR = 0`) has `A_i = 1`. - -**Failure frequency and MUT/MDT.** In steady state a component fails -`ω_i = 1 / (MTTF_i + MTTR_i)` times per unit time. A component failure fails -the system when the component is *critical*, which happens with probability -equal to its Birnbaum importance `I_B^i` (at the availabilities), so the -system fails - -``` -ω = Σ_i I_B^i · ω_i -``` - -times per unit time (the Birnbaum/Vesely frequency formula, exact for -independent components). From it: the mean time between failures -`MTBF = 1 / ω`, the mean up time `MUT = A / ω`, and the mean down time -`MDT = (1 − A) / ω`, with `MTBF = MUT + MDT`. A nested repairable RBD -contributes its own system frequency. - -**Availability over time.** Before the long run, availability depends on -time: a new system starts up (`A(0) = 1`) and settles towards the long-run -value, possibly overshooting. A component alternates up periods `U` and down -periods `D`, as good as new after each: an alternating renewal process, whose -point availability `A_i(t)` solves the renewal equation. There is a closed -form only for exponential times; `point_availability` solves the equation -numerically, on a grid of 1,000 steps over the component's typical up time -(an error of about `4e-7`). Components that fail and are repaired +Each component alternates up and down, as good as new after each repair, +independently of the others and of the system's state +([Lesson 6](learn/availability.md#up-down-up-again)). + +**Long-run availability and frequencies.** The long-run availability is the +exact system probability at `A_i = MTTF_i / (MTTF_i + MTTR_i)`, and the +failure frequency is `ω = Σ_i I_B^i ω_i`, with `ω_i = 1 / (MTTF_i + MTTR_i)` +and `MTBF = 1 / ω`, `MUT = A / ω` and `MDT = (1 − A) / ω` from it, all exact +([derivations](learn/availability.md#a-failure-of-a-critical-component)). No +simulation is needed. An instantly repaired component (`MTTR = 0`) has +`A_i = 1`; a nested repairable RBD contributes its own system frequency. + +**Availability over time.** From new, `point_availability` solves each +component's renewal equation +([Lesson 6](learn/availability.md#a-new-component-starts-up)) numerically, +on a grid of 1,000 steps over the component's typical up time (an error of +about `4e-7`). Components that fail and are repaired independently are up or down independently at every time, so the system's `A(t)` is its system probability at the `A_i(t)`, and `mission_availability` is its mean over `[0, T]`. For a long mission that @@ -611,10 +563,11 @@ first period's end convolved with the renewal measure from new, so Under block replacement the unit's own curve runs to the first block time, which hands the block intervals' recursion the probability that it is up there (and replaced) and the repairs going on; a calendar is shifted by the -phase. A component long in service whose state is not known starts in its +state's `since_scheduled`, the time since its last scheduled replacement or +test. A component long in service whose state is not known starts in its long-run (stationary) state: off a calendar it stays at `A_i` throughout, with its events at their long-run rates; on one, it follows its settled -cycle from its phase. In the simulation, the first draw is the inverse +cycle from its `since_scheduled`. In the simulation, the first draw is the inverse transform of the conditional distribution, `qf(F(a) + u (1 − F(a))) − a`, worked through the cumulative hazard so that it keeps its precision at great ages: still one uniform per draw. @@ -624,11 +577,11 @@ component's alternating failures and repairs, merged in time order, with the system's state re-evaluated at every event) and reports the fraction of histories up at each time. The histories also give what the exact methods do not: how much the counts, downtimes and costs over the window vary, and -the criticality measures below. Each point is a proportion, so its standard error is -`√(A(1 − A)/N)`; the confidence band uses the Wilson score interval, which -stays sensible at `A = 1`. For exponential components the simulation is held -to the exact Markov solution in the test suite, and on the benchmark -diagrams to `point_availability`. +the criticality measures below. Each point's confidence band is the Wilson +score interval ([why](learn/availability.md#how-much-to-trust-a-simulation)). +For exponential components the simulation is held to the exact Markov +solution in the test suite, and on the benchmark diagrams to +`point_availability`. A nested repairable RBD runs its own history on the same clock, and the outer system sees a state change when the nested system's state changes. @@ -716,41 +669,28 @@ unit put into service at `s` is at age `t − s`, summed over its renewals. ## Costs -The long-run cost rate follows from the **renewal-reward theorem**: in the -long run, the cost per unit time is the expected cost per cycle over the -expected cycle length, and rates add over independent contributors. Each -component's corrective costs are charged at its failure frequency `ω_i`; -downtime costs are charged at the rate of time spent down: - -``` -cost rate = downtime_cost_rate · (1 − A_sys) - + Σ ω_i · (repair_cost_i + replace_cost_i) - + Σ (1 − A_i) · downtime_cost_i -``` - -A cost distribution enters through its mean, by linearity of expectation. By -the same linearity the expected cost of a finite window from new is exact: -`expected_cost` sums each category's expected events over the window (see +`expected_cost_rate` is the renewal-reward rate, each price times its rate +(`ω_i`, `1 − A_i`, `1 − A_sys`), with a cost distribution entering through +its mean ([Lesson 7](learn/costs.md#the-whole-plant-rates-times-prices)). By +linearity of expectation the expected cost of a finite window from new is +exact: `expected_cost` sums each category's expected events over the window (see availability above) times its mean cost. The cost of a window is random, -though, and its distribution has no closed form, so `cost()` simulates it: each history accumulates the charges at its failures -and the downtime it incurs. Its **spread** (standard deviation, percentiles) -is a property of the system; the **uncertainty of its mean** shrinks like -`1/√N`. As the window grows, the simulated cost per unit time converges to the -exact rate. - -The **total cost of ownership** over a horizon `H` adds the one-off cost of -buying the components, `Σ a_i`, to `H` times the long-run cost rate -(undiscounted; with a continuous `discount_rate` `r`, the present value, -`H` counting as `(1 − e^{−rH}) / r`). Redundancy that minimises it trades -copies against downtime: +though, and its distribution has no closed form, so `cost()` simulates it: +each history accumulates the charges at its failures and the downtime it +incurs. Its spread belongs to the system, while its mean's error shrinks +like `1/√N` ([Lesson 7](learn/costs.md#two-different-uncertainties)). + +The **total cost of ownership** is `Σ a_i + H ×` the cost rate +([Lesson 7](learn/costs.md#buying-redundancy)); with a continuous +`discount_rate` `r`, the present value, `H` counts as `(1 − e^{−rH}) / r`. +Redundancy that minimises it trades copies against downtime: `n_i` independently repaired active copies of component *i* each cost `a_i + H · r_i` (`r_i` its own running cost rate) and are all down `(1 − A_i)^{n_i}` of the time, so a design costs -`Σ n_i (a_i + H r_i) + H · downtime_cost_rate · (1 − A_sys)`. The total is not -monotone in the copies, but the `k+1`-th copy of a component of -unavailability `U` saves at most `H · downtime_cost_rate · U^k (1 − U)`, which -bounds the copies worth trying; the search then works as for redundancy -allocation below. +`Σ n_i (a_i + H r_i) + H · downtime_cost_rate · (1 − A_sys)`. The bound +`H · downtime_cost_rate · U^k (1 − U)` on the saving of the `k+1`-th copy of +a component of unavailability `U` limits the copies tried; the search then +works as for redundancy allocation below. ## Allocation @@ -803,11 +743,13 @@ does not beat the best design found, which keeps the exact search short. **Reliability allocation** apportions a system target among components. There are many allocations that meet a target; each rule picks one. -- *Equal apportionment* gives every component the same reliability. +- *Equal apportionment* gives every component the same reliability + ([Lesson 9](learn/design.md#equal-apportionment)). - *Proportional improvement* (ARINC-style) scales every adjustable component's failure probability by a common factor (`q_i → q_i · e^{−x w_i}` with optional weights `w_i`) and solves for `x`. The ARINC method scales - failure rates instead, which is the same for small failure probabilities. + failure rates instead, which is the same for small failure probabilities + ([Lesson 9](learn/design.md#proportional-improvement-arinc)). - *Minimization of effort* (Albert, 1958) is for a series system. If raising a component's reliability from `x` to `y` takes effort `G(x, y)`, the same function for every component, growing with `y` and adding up over @@ -854,16 +796,10 @@ most often get the largest cuts. ## Maintenance models -**Age replacement.** Replace at age `t` or at failure, whichever is first, -with a planned cost `c_p` and an unplanned cost `c_u > c_p`. Each replacement -renews the unit, so by the renewal-reward theorem the long-run cost rate is - -``` -C(t) = (c_p R(t) + c_u F(t)) / ∫₀ᵗ R(u) du -``` - -A finite optimum exists only if the unit wears out (an increasing hazard); -otherwise replacing early never pays, and the rate is `c_u / MTTF`. +**Age replacement** (`NonRepairable`) uses the renewal-reward rate +`C(t) = (c_p R(t) + c_u F(t)) / ∫₀ᵗ R(u) du`, which has a finite optimum +only for a unit that wears out; otherwise the rate is `c_u / MTTF` +([Lesson 8](learn/maintenance.md#age-replacement-the-cost-per-hour-from-first-principles)). **Minimal repair and overhaul.** A minimal repair returns the unit to the state just before it failed ("as bad as old"), so failures follow a diff --git a/docs/guide/building.md b/docs/guide/building.md index ee4326fa..0de905fe 100644 --- a/docs/guide/building.md +++ b/docs/guide/building.md @@ -45,6 +45,8 @@ A node model is anything that exposes `sf(t)` and `ff(t)`: | Model | Use it for | |---|---| | A surpyval parametric distribution (`Weibull`, `Exponential`, `LogNormal`, …) | An ordinary component with a fitted lifetime. | +| A surpyval `MixtureModel` | A population of failure modes (infant mortality and wear-out). | +| A surpyval `WienerProcess` or `GammaProcess` fit | A component that fails when its degradation reaches a threshold: its life is the time to the threshold (see [Costs](system-maintenance.md#replacing-on-the-measured-level) for replacing it on the measured level). | | A surpyval non-parametric fit (`KaplanMeier`, `NelsonAalen`, …) | Refused (`ValueError`): its curve ends at the data, and says nothing beyond it. Fit a parametric distribution in surpyval instead. | | `surpyval.FixedEventProbability` | A component with a fixed probability of failure (a demand, a mission). | | [`PerfectReliability`][repyability.PerfectReliability] / [`PerfectUnreliability`][repyability.PerfectUnreliability] | A node that never fails / has always failed (a junction, a placeholder). | @@ -154,6 +156,36 @@ plant.node_names() # ['pumps', 'valve'] A `RepairableRBD` can likewise contain `RepairableRBD` components (see [Repairable systems](repairable.md#nested-repairable-rbds)). +## Units + +A diagram works in one unit throughout: its nodes' lives, the times you +ask about, intervals, repair times, rates and costs per unit of time are +all in the unit the models were fitted in, whatever it is (hours, cycles, +kilometres, demands). Which unit makes no difference to the answers; +mixing two does, and nothing in a fitted model says which it is in yet: a +Weibull fitted to lives in cycles and one fitted to lives in hours are the +same kind of object (a unit carried by the model is asked of surpyval, +SurPyval#845). So give every node's model in one unit, and every number +you type in it too: + +- **Intervals, repair times and costs per time** are in the models' unit: + a proof-test interval of 1 means one hour if the lives are in hours, not + one year. A discount rate is per that unit too (see + [Costs](costs.md)). +- **A probability per demand** (`FixedEventProbability`) is no life: a + `RepairableRBD` simulates a unit that fails at once with that + probability or never, and refuses its long-run and time-dependent values, + which have no mean time to work from. +- **A covariate** of a [`RegressionNode`][repyability.RegressionNode] is in + the unit its model was fitted with: a temperature fitted in kelvin is + wrong given in degrees Celsius, and a load-sharing group's `load` is the + group's total. A covariate's zero is its unit's, so + `differential_importance(change="proportional")` refuses one (a uniform + change is unaffected). +- **Lives in operating time** on a diagram that runs on the calendar: a + `RepairableRBD` component that operates a fraction of the time takes a + `"duty"` (see [Repairable systems](repairable.md#operating-part-of-the-time)). + ## Validation and the structure check The constructor checks the diagram and, by default, raises `ValueError` if it diff --git a/docs/guide/capacity.md b/docs/guide/capacity.md index 60366d90..ef47b076 100644 --- a/docs/guide/capacity.md +++ b/docs/guide/capacity.md @@ -261,8 +261,9 @@ follow what the system can deliver, against a `demand`: year = pumps.availability(8760, mc_samples=500, seed=1, demand=100) # a year, in hours year.capacity[:3] # array([150. , 149.9, 149.8]) the mean capacity curve year.mean_capacity # -> 144.2 the long run: 144.23 -year.delivered_fraction # -> 0.99766 the long run: 0.99781 -year.delivered_fraction_interval().upper # -> 0.99782 +year.delivered_fraction # -> 0.99782 exact: mission_capacity(8760)'s; the long run: 0.99781 +year.sample_delivered_fraction # -> 0.99766 the simulations' own +year.delivered_fraction_interval().method # 'exact' year.capacity_time[100.0] / (500 * 8760) # -> 0.10732 the time at 100 ``` @@ -275,9 +276,15 @@ year.capacity_time[100.0] / (500 * 8760) # -> 0.10732 the time at 100 simulations. - `delivered` holds each simulation's delivered fraction: what it delivered against the demand, `min(capacity, demand)` over the window, over the - demand over the window. `delivered_fraction` is their mean, the - production availability over the window, and - `delivered_fraction_interval(confidence)` its simulation error. + demand over the window. `delivered_fraction` is the run's estimate of + the production availability over the window, taken as + `mean_availability` is (see [exact or + simulated?](simulation.md#exact-or-simulated)): exact where + `mission_capacity` works it out, as here, given the modules' histories + where those apply, and otherwise the simulations' own, which is + `sample_delivered_fraction` whatever the method; + `delivered_fraction_interval(confidence)` gives its error, and its + `method` says which it is. - `demand` defaults to the design capacity, everything up (150 here). Contracts usually name a smaller one. @@ -309,7 +316,7 @@ structure = RBD( ("pipe", "out")], capacity={"p1": 60, "p2": 40, "pipe": 80}, ) -structure.system_capacity({"p1": 0.9, "p2": 0.8, "pipe": 1.0}).mean() # -> 71.6 +structure.system_capacity({"p1": 0.9, "p2": 0.8, "pipe": 1.0}).mean # -> 71.6 ``` ## k-out-of-n nodes and demand diff --git a/docs/guide/common-cause.md b/docs/guide/common-cause.md index d602fe98..9167b7f3 100644 --- a/docs/guide/common-cause.md +++ b/docs/guide/common-cause.md @@ -20,7 +20,7 @@ and pass the groups with `ccf_groups=`: ```python import surpyval as surv from surpyval import FixedEventProbability -from repyability import BetaFactor, CCFGroup, MGL, NonRepairableRBD, PerfectReliability +from repyability import AlphaFactor, BetaFactor, CCFGroup, MGL, NonRepairableRBD, PerfectReliability # Two redundant pumps, each failing with probability 0.01 on demand edges = [("s", "p1"), ("s", "p2"), ("p1", "t"), ("p2", "t")] @@ -39,7 +39,7 @@ coupled.ff() # -> 0.001081 ten times worse: the shared cause dominates The system unreliability is ten times the independent estimate, because one shared cause (probability `β · Q = 0.001`) now fails both pumps at once. -## The two models +## The three models [`BetaFactor(beta)`][repyability.BetaFactor]: a fraction `beta` of each member's failure probability `Q` comes from a cause shared by the **whole** @@ -63,6 +63,24 @@ at least three, and so on. A group of `m` members takes exactly `m − 1` letters (`MGL(0.1, 0.3).group_size == 3`), and `MGL(beta)` on two members is exactly `BetaFactor(beta)`. +[`AlphaFactor(alphas)`][repyability.AlphaFactor]: the alpha-factor model +describes the same partial causes by the shares of a group's failure +events that failed exactly one, two, ..., all `m` members, `alpha_1, ..., +alpha_m` (summing to 1). Each share is a count of events over the total, +so it is what common-cause data sources publish (NUREG/CR-5485, the +NRC's CCF parameter estimates), where MGL's letters cannot be counted. +The probability that a cause fails a specific set of `k` members is +`alpha_k / C(m − 1, k − 1) · Q` for members tested one after another +(`testing="staggered"`, the default) and `k / C(m − 1, k − 1) · alpha_k / +alpha_t · Q`, with `alpha_t = Σ k alpha_k`, for members tested all at +once (`testing="non-staggered"`), as the two schemes count a multiple +failure differently: use the scheme the estimates were published for. A +group of `m` members takes `m` alpha factors, and the model is the `MGL` +model with the letters those probabilities give, so everything below +holds for it as for `MGL`. Its levers (for `parameter_sensitivity`, +`levers()` and the parameter uncertainty) are `alpha_2, ..., alpha_m`, +named `ccf_alpha_2`, ..., with `alpha_1` the remainder. + Partial causes matter for *k*-out-of-*n* groups. For three units where two must work, a cause that fails a pair is enough to fail the system: @@ -84,6 +102,8 @@ def two_of_three(model=None): two_of_three().ff() # -> 0.000298 independent two_of_three(BetaFactor(0.1)).ff() # -> 0.001241 every shared cause takes all three two_of_three(MGL(0.1, 0.3)).ff() # -> 0.001591 70% of shared causes take a pair +two_of_three(AlphaFactor([0.95, 0.04, 0.01])).ff() # -> 0.000969 +two_of_three(AlphaFactor([0.95, 0.04, 0.01], testing="non-staggered")).ff() # -> 0.001654 ``` With the same `beta`, the MGL group is *worse* here: a cause that fails two @@ -144,7 +164,7 @@ always strike independently. - **Disjoint.** A node can be in at most one group. - Members must be component nodes of the RBD: not the input or output node, and not a [repeated component](building.md#one-component-in-several-places). -- An `MGL` model must match its group's size. +- An `MGL` or `AlphaFactor` model must match its group's size. ## Time-dependent members @@ -258,6 +278,19 @@ with_valve.fussell_vesely()["v"] # -> 0.4808 independent: 0.9092 with_valve.risk_achievement_worth()["p1"] # -> 52.45 independent: 9.99 ``` +The joint importance of a pair conditions on both states the same way: +`R(1_i, 0_j)` is the system's reliability given member `i` works and `j` +has failed, two members of one group conditioned on their joint state, +and a node outside the groups held. With `p1` failed, `p2` is likelier to +have failed too, so the valve is worth less then, and more of a +complement of each pump; the pumps' pair is as without the group, the +valve alone remaining given their states: + +```python +with_valve.joint_importance()[("p1", "v")] # -> 0.1082 independent: 0.01 +with_valve.joint_importance()[("p1", "p2")] # -> -0.999 independent: -0.999 +``` + A member held working or broken (`working_nodes`, `broken_nodes`) is refused, as it is for `sf`. @@ -265,7 +298,8 @@ refused, as it is for `sf`. of its members. They carry one model, so each value is the derivative of the system reliability as the parameter moves for all of them at once. The parameters of the group's own model come with them, named `ccf_beta` (and, -for an `MGL` model, `ccf_gamma`, `ccf_delta`, ...): +for an `MGL` model, `ccf_gamma`, `ccf_delta`, ...; for an `AlphaFactor` +model, `ccf_alpha_2`, `ccf_alpha_3`, ..., `alpha_1` being the remainder): ```python sensitivity = with_valve.parameter_sensitivity() @@ -302,8 +336,8 @@ In `allocate_redundancy` and `redundancy_front` a copy of a member joins its group, as it would in the plant: the shared cause fails it too. A `BetaFactor` group's `beta` holds at any size, so its members can be copied: active copies of their own model, with any number of them required. An -`MGL` model's letters are for its group's size, so copying its member is -refused, as are options and cold spares for a member. The shared cause can +`MGL` or `AlphaFactor` model's parameters are for its group's size, so +copying its member is refused, as are options and cold spares for a member. The shared cause can change what is worth buying. With two pumps that fail with probability 0.05 and a valve that fails with 0.002, a third pump is the better buy if the pumps are independent, and a second valve if a fifth of their failures are @@ -339,7 +373,7 @@ group's. | The importance measures (Birnbaum, improvement potential, RAW, RRW, criticality, Fussell–Vesely) | Include the groups, exactly: a member conditioned on its state through the shock outcomes, a node outside them held. | | `parameter_sensitivity` | A group's parameters, and its model's (`ccf_beta`, ...), reported once, under the tuple of its members. | | `sf_uncertainty`, `time_to_reliability_uncertainty`, `bx_life_uncertainty`, `mean_uncertainty` | Each draw worked out with the groups; the members drawn together, and the group's model too if it is given. `mean_uncertainty` refuses a group splitting a probability, as `mean` does. | -| `allocate_redundancy`, `redundancy_front` | A member's copies join its group: a `BetaFactor` group's, active copies of the member's own model. Copies of an `MGL` group's member, and options or cold spares for a member, raise `NotImplementedError`. | +| `allocate_redundancy`, `redundancy_front` | A member's copies join its group: a `BetaFactor` group's, active copies of the member's own model. Copies of an `MGL` or `AlphaFactor` group's member, and options or cold spares for a member, raise `NotImplementedError`. | | `allocate_reliability_redundancy` | Includes the groups; a member in `uses` raises `NotImplementedError` (its reliability is the group's). | | The condition-based methods (`sf_given_state`, `remaining_life`, `importances_given_state`) | Raise `NotImplementedError`: they would need members of different ages. | | `working_nodes` / `broken_nodes` naming a group member | Raises `NotImplementedError`. | @@ -350,8 +384,8 @@ pair.mean(method="simulate", seed=0) # -> 1147.4 pair.mean() # -> 1145.8 exact, without the group ``` -Groups are saved with the RBD, their basis with them. An *alpha-factor* -model, a data-estimable reparameterisation of MGL, is a planned extension. +Groups are saved with the RBD, their basis with them (and an alpha-factor +model's testing scheme). ### Many groups @@ -363,9 +397,10 @@ that share a module, as a group of each kind of component across redundant trains does, would multiply their outcomes: past 64 combinations, their shared causes are written out as events of their own, each repeated under every member it strikes, which the diagram's decision diagram works out -with the rest (#219). A beta factor's cause, and an MGL model's by rate or -with `shocks="independent"`, are such events as they stand; an MGL -model's exclusive shocks are taken as independent causes that fail the +with the rest (#219). A beta factor's cause, and an MGL or alpha-factor +model's by rate or with `shocks="independent"`, are such events as they +stand; the exclusive shocks of an MGL or alpha-factor model are taken as +independent causes that fail the same sets of members as often, which exist unless the model leaves out a set that two pair shocks would fail together (a `gamma` of 0, say), when the groups are conditioned on together, as before. The values are the @@ -433,7 +468,7 @@ exponential lives, and either revealed failures with exponential repairs, as here, or hidden failures found by tests (with one coverage for the group), at offsets of their own if they are staggered, whose tests and repairs take no time, a fixed time or an exponential one (#220); see [the PFDavg of a -safety function](costs.md#common-cause-staggered-tests-and-test-coverage). +safety function](system-maintenance.md#common-cause-staggered-tests-and-test-coverage). `mean_availability`, `mean_unavailability`, `system_failure_frequency`, MTBF, MUT and MDT, the cost rate, `capacity_distribution`, and the interval choices built on them take the groups in. So do the importance measures: @@ -514,8 +549,8 @@ drawn.interval(0.9) # (0.9783, 0.9886) In `allocate_redundancy` a member's copies join its group, as for a non-repairable diagram: a `BetaFactor` group's, each copy alike in every way, struck by the shared cause too, and tested with its member when its failures -are hidden. Copies of an `MGL` group's member, or of a train holding one, are -refused. Each design is scored exactly by its groups' chains, which count +are hidden. Copies of an `MGL` or `AlphaFactor` group's member, or of a +train holding one, are refused. Each design is scored exactly by its groups' chains, which count how many of a member's copies are down rather than telling them apart, so a design with many copies is quick to score. The copies are repaired at once (the allocations assume as many repair crews as jobs), so a shared failure @@ -562,3 +597,6 @@ held working or broken, a member started from a current state (`state=`), a member maintained on a schedule, repaired imperfectly or in a maintenance group, and a group with limited repair crews are refused, with the reason; `analysis_routes()` says which. + +Common-cause groups can also be added to a diagram in the browser in +[Reliafy](https://reliafy.com/guides/common-cause-group). diff --git a/docs/guide/condition-based.md b/docs/guide/condition-based.md index 7df3f627..0b8fb65e 100644 --- a/docs/guide/condition-based.md +++ b/docs/guide/condition-based.md @@ -173,6 +173,23 @@ motor.remaining_life(0.9, {"m": NodeState(age=2000)}) # -> 266.7 Because the covariates live on the node, `age` keeps its one meaning (operating time survived) for every node type. +The covariates are what can be changed, so they are the node's levers +(#272): `levers()` lists each as `"covariate."` (the model's feature +name when it was fitted from a DataFrame, else its place, `"covariate.0"`), +`parameter_sensitivity` gives the system's derivative in it, and +`with_levers` builds the diagram with the component run elsewhere: + +```python +motor.parameter_sensitivity(3000)["m"] # {'covariate.0': -0.0146} per °C +cooler = motor.with_levers({("m", "covariate.0"): 60.0}) +cooler.sf(3000) # -> 0.5744 from 0.4363 at 70 °C +``` + +A covariate has no range of its own: running a component outside the +conditions its model was fitted on is extrapolation (as an accelerated life +test's use level is), which is the user's to judge. A node along a schedule +has no one value to move, and has no levers. + ### A load schedule When the conditions change over the component's life, pass a surpyval diff --git a/docs/guide/costs.md b/docs/guide/costs.md index 46b41343..47d85b31 100644 --- a/docs/guide/costs.md +++ b/docs/guide/costs.md @@ -52,7 +52,7 @@ plant.has_costs # True Every cost defaults to 0, so any subset can be priced. Costs are undiscounted (but for the [total cost](#discounting)'s optional discount rate), and every failure is repaired at the same price (preventive -replacement is [below](#preventive-maintenance)). A cost must be finite and +replacement is [below](system-maintenance.md#preventive-maintenance)). A cost must be finite and non-negative; an unknown key (such as `repair_costs`) raises `ValueError` rather than being priced at zero. @@ -81,8 +81,11 @@ no corrective cost: plant.expected_cost_rate(working_nodes=["A"]) # -> 95.1 ``` -With nothing priced, `has_costs` is `False` and the rate is `0.0` without any -work being done. +With nothing priced, not even an [acquisition cost](#the-total-cost-of-ownership), +`has_costs` is `False` and there is no cost model to evaluate: +`expected_cost_rate`, `total_cost`, `expected_cost` and `cost` refuse (a +`routes.Refused`, a `ValueError`), as every analysis refuses a diagram with +nothing it works on. ## The simulated cost distribution @@ -106,7 +109,7 @@ costs.by_component # the part of mean each costed component accounts for `antithetic`, `n_jobs` and `engine`: see [Simulation precision and speed](simulation.md)). The same result comes with `availability(...)` as `result.cost`, so one simulation gives both answers. -With nothing priced, `cost()` returns `None` and `result.cost` is `None`. +With nothing at all priced, `cost()` refuses, and `result.cost` is `None`. ### One expected value @@ -180,8 +183,8 @@ group's set-up once per stop (replacements due at one instant are one stop). It returns an [`ExpectedCost`][repyability.ExpectedCost], with `mean`, `by_category`, `by_component`, `acquisition_cost`, `total` (the two added) and `cost_rate`. It covers what the availability over time covers, -and refuses the rest with the reason; with nothing priced, every category is -0. For the spread of a window's cost, simulate it with `cost()`. +and refuses the rest with the reason, as it refuses a diagram with nothing +priced. For the spread of a window's cost, simulate it with `cost()`. ## Costs drawn from distributions @@ -226,745 +229,6 @@ fuse.expected_cost_rate() # -> 4.0 = 40 × 0.1 failures per unit time fuse.mean_availability() # -> 1.0 ``` -## Preventive maintenance - -A component's dict can also schedule preventive replacement, under the key -`"preventive"`, so that the cost and availability outputs price the trade -between preventive and corrective maintenance: - -| Key | Meaning | -|---|---| -| `interval` | Required: the replacement interval `T` (`inf`: never). | -| `policy` | `"age"` (the default): `T` after the unit was last put into service as new, so a failure restarts the clock. `"block"`: at `T, 2T, 3T, …` whatever the unit's age, skipped while it is down. `"condition"`: inspected at `T, 2T, 3T, …`, and replaced if likely to fail before the next inspection (see [below](#replacement-on-condition)). | -| `duration` | `"instant"` (the default): renewed in place, never down. Or a time-to-maintain model: the unit is down meanwhile, a *planned outage*. | -| `cost` | Charged at each preventive replacement: a number or a distribution. | - -A replacement renews the unit, so the failure it was heading for never -happens. A failure due at the same time as a replacement comes first. - -`NonRepairable.find_optimal_replacement()` finds a component's best -replacement age on its own ([Maintenance policies](maintenance.md)). At -system level the answer changes, because a planned stop of a unit with a -standby costs almost no production, while the same stop of a single point of -failure halts it. Take a pump that wears out, is repaired in about 23 h and -replaced preventively in about 7 h, alone or with a standby, with lost -production at 500 per hour: - -```python -def pump(interval, policy="age"): - return { - "reliability": surv.Weibull.from_params([1000, 2.5]), # MTTF 887 h - "repairability": surv.LogNormal.from_params([3.0, 0.5]), # 23 h - "replace_cost": 5000.0, - "preventive": { - "interval": interval, - "policy": policy, - "duration": surv.Weibull.from_params([8, 3]), # 7 h - "cost": 1000.0, - }, - } - -def alone(interval, policy="age"): - return RepairableRBD( - [("s", "p"), ("p", "t")], - {"p": pump(interval, policy)}, - downtime_cost_rate=500.0, - ) - -def with_standby(interval, policy="age"): - return RepairableRBD( - [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], - {"a": pump(interval, policy), "b": pump(interval, policy)}, - downtime_cost_rate=500.0, - ) - -alone(float("inf")).expected_cost_rate() # -> 18.0 run to failure -alone(580).expected_cost_rate() # -> 13.14 the best interval -with_standby(float("inf")).expected_cost_rate() # -> 11.3 -with_standby(500).expected_cost_rate() # -> 6.985 the best interval -``` - -| Interval (h) | 300 | 400 | 500 | 600 | 800 | never | -|---|---|---|---|---|---|---| -| Alone | 16.92 | 14.36 | 13.35 | 13.14 | 13.84 | 18.00 | -| With a standby | 8.19 | 7.21 | 6.99 | 7.15 | 8.01 | 11.30 | - -Replacement pays in both, but not at the same interval: the lone pump's -planned stops cost production, so it is best replaced less often, at about -580 h; replacing a pump with a standby costs almost no production, so the -pair is best replaced at about 500 h. Judged on its own, as `NonRepairable` -judges it (`cp=1000`, `cu=5000`), the pump is best replaced at 493 h. - -### Choosing the intervals - -`optimal_replacement_intervals` finds the best interval of every component -under age replacement at once, from the exact long-run values, rather than -sweeping them one at a time. The components are chosen together: the one -alone in the line is replaced later than the ones with a standby, because -its own replacements stop the plant: - -```python -alone(1000).optimal_replacement_intervals().intervals["p"] # -> 589.6 -plan = with_standby(1000).optimal_replacement_intervals() -plan.intervals["a"] # -> 497.1 and the same for "b" -plan.cost_rate # -> 6.985 -``` - -It can also keep the system's long-run availability to a target at the least -cost (`min_availability`), or give the most availability within a cost rate -(`max_cost_rate`): - -```python -plan = alone(1000).optimal_replacement_intervals(min_availability=0.9807) -plan.intervals["p"] # -> 604.8 a little later than the cheapest -plan.cost_rate # -> 13.139 against 13.134 -``` - -It returns a [`MaintenancePlan`][repyability.MaintenancePlan]: the intervals -(`inf` for a component better never replaced), and the system's cost rate -and availability with them. The search starts from the intervals given, and -from others around each component's mean life, and keeps the best; a target -no intervals can meet raises `ValueError`, with the best they can do. The -cost rate is usually flat near its minimum, so an interval some way from the -one found costs almost the same. - -Replacements are often made on a calendar. `allowed` gives the intervals to -choose from (#230), one list for every component or a dict of a list each -(`inf` among them for never), and every combination is tried, as -`optimal_inspection_intervals` chooses tests': - -```python -weeks = [336.0, 672.0, 1008.0, 1344.0] # every 2, 4, 6 or 8 weeks -plan = with_standby(1000).optimal_replacement_intervals(allowed=weeks) -plan.intervals # {'a': 672.0, 'b': 672.0} -plan.cost_rate # -> 7.405 against 6.985 at 497 h -``` - -The plan is the best for the long run. A new plant's units all start new, -so the first replacements of redundant units fall due together, and where -they take the system down together its first years cost more than the -plan's rate: `expected_cost` gives the cost from new, and from units of -other ages (`state`) the cost of a staggered start. - -With limited `repair_crews` a component can wait for a crew, and the exact -long-run values the search uses no longer hold, so the choice is refused. -`assume_unlimited_crews=True` (#184) chooses the intervals as if every repair -started at once, and `with_intervals(plan)` gives the diagram with them, to -simulate with its crews and see what the waiting costs: - -```python -crewed = RepairableRBD( - [("s", "a"), ("a", "b1"), ("a", "b2"), ("b1", "t"), ("b2", "t")], - {n: pump(1000) for n in ("a", "b1", "b2")}, - downtime_cost_rate=500.0, repair_crews=1, -) -plan = crewed.optimal_replacement_intervals(assume_unlimited_crews=True) -planned = crewed.with_intervals(plan) # the same crew, the plan's intervals -planned.cost(100_000.0, mc_samples=100, seed=1).mean / 100_000.0 # ~> 20.6 simulated -plan.cost_rate # -> 20.12 with a crew for every job -``` - -`with_intervals` takes a plan or a dict of intervals (and of test offsets), -and builds the copy as the diagram was built; `compare()` then sets the plan -against the schedules the diagram has. - -`expected_cost_rate` prices an age-replaced component through its renewal -cycle: it ends at a failure or a preventive replacement, whichever comes -first, with mean length `C = ∫₀ᵀ R + F(T)·MTTR + R(T)·MTTP` (`MTTP` the mean -maintenance time), so it fails `F(T) / C` times and is replaced `R(T) / C` -times per unit time, and is up `∫₀ᵀ R / C` of the time: - -``` -cost rate = … + Σ R_i(T_i) / C_i · preventive cost_i preventive actions -``` - -With instant repair and maintenance a component's own cost rate is -`NonRepairable.cost_rate(T)`. `mean_availability`, `node_availability`, the -frequencies, MUT, MDT and the importance measures account for it the same -way. - -Under block replacement the renewals are the block times at which the unit is -up: it is replaced there, and a replacement due while it is down is skipped. -Between two of them it fails and is repaired as usual, and a repair can run -over a block time. `expected_cost_rate` and the other long-run methods compute -its mean up time, failures and length between renewals numerically, from the -renewal equations of its lives and repairs within an interval, to about one -part in a million. With instant repair and replacement the cost rate is -`(c_p + c_u · M(T)) / T`, `M` the renewal function of the lives. Components -replaced at the same block times go down together, so the system's long-run -values average over the block interval (over the time the schedules take to -repeat together, for different intervals), rather than combining each -component's own average: - -```python -alone(580, "block").expected_cost_rate() # -> 14.06 against 13.14 for age -with_standby(580, "block").mean_availability() # -> 0.9901 both replaced at once -with_standby(580).mean_availability() # -> 0.9996 -``` - -The exact block-replacement values need a surpyval parametric lifetime with a -density (no units dead on arrival), and repairs that always end; otherwise -they raise `NotImplementedError`, and the simulation still applies. - -The simulation prices both policies. A replacement's cost is in -`by_category["preventive"]` (the expected cost, exact here, as the cost's -`mean` is), and a planned outage counts as downtime, but not as a failure: -`system_planned_outages` counts the times one took the system down in the -simulations. - -```python -year = alone(580).availability(t_simulation=8760.0, mc_samples=500, seed=0) -year.system_failures / year.n_simulations # -> 3.466 -year.system_planned_outages / year.n_simulations # -> 11.95 -year.cost.by_category["preventive"] # -> 11886.8 1000 each, expected -``` - -### Replacement on condition - -`"policy": "condition"` inspects the unit at every multiple of `interval`, -while it is up, and replaces it only if it is then more likely than -`"threshold"` to fail before the next inspection, given its age `a`: -`1 − R(a + T) / R(a)`, the conditional survival that `NodeState` and -`sf_given_state` use. A unit renewed a while ago is left alone, and a worn -one is replaced at the last inspection before it is likely to fail. - -| Key | Meaning | -|---|---| -| `threshold` | Required with `"condition"`: the probability, in `[0, 1]`, above which an inspection replaces the unit. | -| `inspection_cost` | Charged at each inspection: a number or a distribution. | - -`duration` and `cost` are the replacement's, as under the other policies. -An inspection takes no time, and one due while the unit is down is skipped. -Inspected weekly and replaced when more than 20% likely to fail before the -next inspection, the lone pump above costs about what the best age -replacement does: - -```python -weekly = {"interval": 168.0, "policy": "condition", "threshold": 0.2, - "duration": surv.Weibull.from_params([8, 3]), "cost": 1000.0, - "inspection_cost": 20.0} -inspected = RepairableRBD([("s", "p"), ("p", "t")], - {"p": dict(pump(580), preventive=weekly)}, - downtime_cost_rate=500.0) -inspected.expected_cost_rate() # -> 13.37 per hour -run = inspected.cost(200_000.0, mc_samples=20, seed=1) -run.cost_rate # -> 13.36 exact over the window -run.sample_mean / 200_000.0 # -> 13.27 the 20 simulations', ± 0.2 -``` - -| Threshold | 0.05 | 0.1 | 0.15 | 0.2 | 0.25 | 0.3 | -|---|---|---|---|---|---|---| -| Cost per hour | 22.10 | 15.98 | 13.47 | 13.37 | 14.02 | 14.34 | - -Judged on its age alone, a unit replaced on condition is replaced much as -under age replacement (13.14 at the best age), but on the inspection -calendar. A low threshold replaces it too young; a high one lets it fail. - -- **A threshold of 0** replaces the unit at every inspection: block - replacement at the interval. **A threshold of 1** never does: run to - failure, with inspections. -- **A constant failure rate** gives the same probability at every age, - `1 − exp(−λT)`: the unit is replaced at every inspection or at none, and - it fails as often either way. -- **Numerical in the long run** (#145). The inspections that replace the - unit are regeneration points, and a cycle between two is followed one - inspection interval at a time: the units put into service in an interval - as an alternating renewal process, as under block replacement, and the - units an inspection keeps on into the next interval by their ages. So the - long-run values, the cost rate (the inspections charged at the unit's - availability just before them) and the importance measures are - numerical, to about `1e-7`. A cycle in which the unit fails many times - before an inspection replaces it is summed to its end as a geometric - series once it falls at a steady rate. -- **Numerical over time** (#161). The same recursion, followed from new - over all the intervals rather than over one cycle, gives the availability - at any time and the expected events and cost of any window. Each - interval starts with what the inspection at its start replaced, the - repairs carried into it and the units it kept, by age, and after some - intervals it repeats from one to the next: its long-run cycle. From a - state, the unit in service at the start is decided on at its own age at - each inspection, however that falls on the grid. - -A year of the weekly-inspected pump from new, and the next 30 days of one -already 600 hours old, 100 hours after its last inspection: - -```python -from repyability import NodeState - -inspected.expected_cost(8760.0).total # -> 114491 new: less than a year at the rate, 117139 -worn = {"p": NodeState(age=600.0, phase=100.0)} -inspected.expected_cost(720.0, state=worn).total # -> 10265 against 8517 new -``` - -### Opportunistic maintenance - -Plants group their work: once a unit is down, for a failure or its planned -replacement, the crew services its neighbours too, as the line is stopped -anyway. Components that share such stops form a *maintenance group*, named -by their `"group"` key. Each failure of a member, and each scheduled -replacement, opens a *stop* of its group, at which every other member that -is working and at least its `"opportunity"` age is replaced as well, as -its own scheduled replacement would replace it: - -| Key | Meaning | -|---|---| -| `"group"` (component) | The component's maintenance group: any hashable name. | -| `opportunity` (`"preventive"`, age policy) | The age, from 0 to `interval`, from which the unit is replaced early at a stop of its group. Left out, it never is. | -| `setup_cost` (`maintenance_groups`) | Charged once per stop, however many members it renews. | -| `system_down` (`maintenance_groups`) | `True`: every outage of the system is a stop of the group as well. | - -A compressor and its motor in series wear out, and each stop of the train -costs 3,000 to set up (isolation, permits, scaffolding) besides the units' -own costs. Replaced separately at 600 h, each failure or replacement is a -stop of its own; from age 300 h, a unit is replaced at the other's stop: - -```python -def unit(scale, opportunity=None): - preventive = {"interval": 600.0, - "duration": surv.Weibull.from_params([8, 3]), # 7 h - "cost": 1000.0} - if opportunity is not None: - preventive["opportunity"] = opportunity - return { - "reliability": surv.Weibull.from_params([scale, 2.5]), - "repairability": surv.LogNormal.from_params([3.0, 0.5]), # 23 h - "replace_cost": 5000.0, - "preventive": preventive, - "group": "train", - } - -def train(opportunity=None): - return RepairableRBD( - [("s", "compressor"), ("compressor", "motor"), ("motor", "t")], - {"compressor": unit(1000, opportunity), - "motor": unit(1500, opportunity)}, - downtime_cost_rate=500.0, - maintenance_groups={"train": {"setup_cost": 3000.0}}, - ) - -train().expected_cost_rate() # -> 33.00 separate stops, exact -run = train(300).availability(200_000.0, mc_samples=20, seed=1) -run.cost.cost_rate # -> 23.48 grouped, simulated -run.opportunistic_renewals # -> {'compressor': 3337, 'motor': 3502} -``` - -| Opportunity age (h) | none | 500 | 400 | 300 | 200 | 100 | -|---|---|---|---|---|---|---| -| Cost per hour (± 0.15) | 33.00 | 29.99 | 26.82 | 23.48 | 23.20 | 23.18 | - -Grouping pays twice: the stops, and their set-ups, are fewer (5.64 an hour -of set-ups against 10.33), and the units' outages overlap, so the train is -down less (12.0 an hour of lost production against 16.7). - -- **A stop is an instant.** Work started at one instant shares one set-up, - and a member whose own failure or replacement falls due then keeps it, as - part of the stop: two units on the same schedule are replaced together, - on schedule, at one set-up. -- **Durations overlap.** Each member renewed at a stop takes its own - maintenance time, from the stop, so the set-up time each one's - maintenance includes is spent once in a series train. -- **Results.** `opportunistic_renewals` counts each component's early - renewals, summed over the simulations; the set-ups are under - `by_category["setup"]`, and each early renewal is charged the member's - preventive cost and uses a spare. -- **Exact or simulated.** A component that can be renewed early depends on - the others, so the exact long-run values, the availability over time and - the spares counts refuse it; `availability`, `cost` and `compare` - simulate it, in Python. With no `opportunity` below an interval, the - set-up cost is exact: `expected_cost_rate` charges a set-up at each - failure and each preventive replacement of a member (so `opportunity` - equal to the interval is plain age replacement, plus set-ups). It - refuses two members replaced on a clock (block replacement, or never - failing before an instant age replacement), whose replacements share - stops. -- Members cannot have hidden failures or be standby groups. - -## Hidden failures and inspection - -Some failures announce themselves: a running pump stops, and the operators -know. Others do not: a relief valve that has seized, a standby pump that will -not start or a trip that no longer trips looks just like a working one until -something tests it. Such a failure is *hidden* (or *unrevealed*): the -component is down, nobody knows, and it stays down until a periodic -*inspection* (a proof test) finds it. The key `"inspection"` in a -component's dict makes its failures hidden: - -| Key | Meaning | -|---|---| -| `interval` | Required: the time `τ` between inspections, positive and finite. The component is inspected at `τ, 2τ, 3τ, …`, or from its `offset`. | -| `duration` | `"instant"` (the default): the test takes no time. Or a time-to-test model: the component is off-line while it is tested (a planned outage), and does not age meanwhile. | -| `cost` | Charged at each inspection: a number or a distribution. | -| `offset` | The time of the first test, from 0 to less than `τ`: tests at `offset, offset + τ, …`, so that redundant components can be tested apart (staggered). By default 0, no offset: the first test at `τ`, none at the start. A positive offset puts one there, so an offset just above 0 adds a test (and its outage and cost) at the start; one within a billionth of `τ` of 0 is taken as 0. | -| `coverage` | The chance that a test finds a failure, from 0 to 1 (by default 1): its *proof-test coverage*. A failure a test misses stays hidden until a full test. | -| `full_test` | With a coverage below 1, required: the time between full tests, which find every failure, a whole multiple of `τ` (the tests at `offset` and every `full_test` after it). Often the mission time, after which the component is renewed. | - -A failure found by an inspection is repaired once the test is done (taking a -time drawn from the component's `"repairability"`), and its repair and -replace costs are charged when it is found. An inspection due while the -component is being repaired is skipped. The time a failure lies hidden counts -as downtime in every output, and each inspection's cost is in -`by_category["inspection"]`. A component can have an inspection or a -preventive schedule, but not both. - -### The average probability of failure on demand - -A protective function whose failure is hidden does not act when it is -needed, so its long-run unavailability is its *average probability of -failure on demand*, the PFDavg that safety-integrity (SIL) verification asks -for (IEC 61508 and 61511). With a constant rate `λ` of such failures, -inspected every `τ`, with instant tests and repairs, a single channel is -down, after a failure, until the next test: half an interval on average, so -`PFDavg = 1 − (1 − e^(−λτ)) / (λτ) ≈ λτ/2`. Take a shutdown valve with -dangerous undetected failures at `2 × 10⁻⁶` per hour, proof-tested yearly: - -```python -def valve(interval): - return { - "reliability": surv.Exponential.from_params([2e-6]), # dangerous, undetected - "repairability": "instant", - "inspection": {"interval": interval}, - } - -one = RepairableRBD([("s", "v"), ("v", "t")], {"v": valve(8760.0)}) -one.mean_unavailability() # -> 0.008709 about λτ/2 = 0.00876 -pair = RepairableRBD( - [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], - {"v1": valve(8760.0), "v2": valve(8760.0)}, -) -pair.mean_unavailability() # -> 1.0098e-4 about (λτ)²/3 -``` - -Two valves in parallel (1oo2), tested together, are down only when both -have failed since the last test: `PFDavg = (1/τ) ∫₀^τ (1 − e^(−λt))² dt ≈ -(λτ)²/3`. That is a third more than the `(λτ/2)²` of two channels that fail -independently in time, because both have gone untested for the same time. -The diagram supplies the voting, so 2oo3 and larger architectures follow in -the same way (see [k-out-of-n nodes](building.md#k-out-of-n-nodes)). Halving -the interval halves a single channel's PFDavg but quarters the pair's: - -| Test interval | 6 months | 1 year | 2 years | -|---|---|---|---| -| One valve | 0.00437 | 0.00871 | 0.0173 | -| Two valves (1oo2) | 2.54 × 10⁻⁵ | 1.01 × 10⁻⁴ | 3.99 × 10⁻⁴ | - -These long-run values are exact. Components inspected at the same times go -down together, so `mean_availability` and the other long-run methods average -the system's availability over one period of the inspection schedules (the -least common multiple of the intervals: components on different intervals -can be mixed), and the importance measures are ratios of those averages. -With instant tests and repairs they are in closed form for a constant -failure rate, and numerical for any other life (see [a life that wears -out](#a-life-that-wears-out)); tests and repairs that take time are -numerical too (see [tests and repairs that take -time](#tests-and-repairs-that-take-time)). Testing both valves at once, for -example, takes the whole function off-line during the test, which here -costs far more than the hidden failures: - -```python -def tested(interval, offset=0.0): - return { - "reliability": surv.Exponential.from_params([2e-6]), - "repairability": surv.LogNormal.from_params([np.log(24), 0.5]), # about a day - "inspection": { - "interval": interval, - "duration": surv.Weibull.from_params([4, 3]), # about 3.6 h - "offset": offset, - }, - } - -pair = [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")] -both = RepairableRBD(pair, {"v1": tested(8760.0), "v2": tested(8760.0)}) -both.mean_unavailability() # -> 4.28e-4 -1 - both.mission_availability(10 * 8760.0) # -> 3.95e-4 its first ten years -both.expected_events(10 * 8760.0).system_planned_outages # -> 9.0 one per test -apart = RepairableRBD(pair, {"v1": tested(8760.0), "v2": tested(8760.0, 4380.0)}) -apart.mean_unavailability() # -> 7.12e-5 -``` - -Tested half a year apart, a test takes one valve off-line while the other -holds the function, and the PFDavg falls six-fold. - -### A life that wears out - -The closed forms above need a constant failure rate. With any other life, -and instant tests and repairs, the values are numerical, exact to rounding. -A failed unit is found and renewed at the next test, so every renewal falls -on a test, and a unit renewed `m` intervals before a test is still up with -probability `R(mτ)`. In the long run a cycle lasts `S = 1 + Σ R(mτ)` -intervals (over `m ≥ 1`), and the unit is up a time `u` after a test with -probability `(R(u) + Σ R(mτ + u)) / S`. From new, the chance of a renewal -at each test follows a renewal equation over the tests, and settles to -`1/S`. A valve that wears out, with a mean life of about 15 years: - -```python -worn = surv.Weibull.from_params([150_000.0, 2.5]) # wears out; MTTF 133,090 h -wearing = RepairableRBD( - [("s", "v"), ("v", "t")], - {"v": { - "reliability": worn, - "repairability": "instant", - "inspection": {"interval": 8760.0}, - }}, -) -wearing.mean_unavailability() # -> 0.03186 about τ / (2 MTTF) = 0.0329 -1 - wearing.mission_availability(87600.0) # -> 0.01126 its first ten years -years = np.array([1, 5, 10, 20]) * 8760.0 - 1.0 # just before a test -1 - wearing.point_availability(years) -# -> [0.00082 0.01908 0.04983 0.06844] -``` - -For a short interval the long-run PFDavg is about `τ / (2 MTTF)`, whatever -the life's shape. What the wear changes is the PFD over time. A new valve -rarely fails in its first years, so its first ten years' PFD is a third of -the long run's, and the PFD just before each test climbs with its age -before it settles. The values over a window, from new or from the valve's -current state (its age, and the time since its last test), are exact too: -`expected_events`, `expected_cost` and the other [analyses over -time](repairable.md#availability-over-time-exact). - -The sums run over the intervals a unit can last. Within an interval, every -unit but the one renewed at the last test ages smoothly, so their sum is -worked out once, at Chebyshev points across the interval, and interpolated -between them, to rounding. A term that bends there (a life with a -threshold, say) is summed directly instead. A life that lasts more than -200,000 intervals (the most summed) refuses. - -### Tests and repairs that take time - -A test of a working unit takes it off-line (a planned outage) for the -test's time, during which it does not age; a failure found by a test is -repaired once the test is over, and the tests that fall in the repair are -not done. Then renewals no longer fall on the tests, and a test that can -miss a failure (see [test coverage](#common-cause-staggered-tests-and-test-coverage)) -leaves the failure for a later one. The values are still numerical, for -any life (#159). The tests that find failures are the unit's regeneration -points: from one, a new unit is put into service the test's time plus the -repair's later, and its age at its later tests is a random walk (each test -holds it back by the test's time), followed test by test on a grid of -2,000 steps or more an interval. The long run follows by renewal-reward -over that cycle (over the cycles from each place in the full tests' -period, with tests that miss), and from new, or from a state, the tests -that find failures are a renewal process on the tests. When a test is -over, and when a unit is back in service, are kept out of the grid, exact -at any time; the rest is exact to the square of the grid's step, about -1e-8. An exponential life needs no ages: its chain of states from test to -test (working, failed, missed, in a repair) is followed instead. - -```python -def valve(coverage=1.0): - inspection = { - "interval": 8760.0, - "duration": surv.ExactEventTime.from_params([8.0]), # 8 h off-line - } - if coverage < 1.0: - inspection.update(coverage=coverage, full_test=5 * 8760.0) - return RepairableRBD( - [("s", "v"), ("v", "t")], - {"v": { - "reliability": surv.Weibull.from_params([150_000.0, 2.5]), - "repairability": surv.ExactEventTime.from_params([24.0]), # a day - "inspection": inspection, - }}, - ) - -valve().mean_unavailability() # -> 0.03289 0.03186 with tests and repairs in no time -valve(0.7).mean_unavailability() # -> 0.06845 70% coverage, a full test every 5 years -valve(0.7).expected_events(10 * 8760.0).system_planned_outages # -> 8.750 -``` - -`spares_demand` and `spares_stock` count such a valve's spares on the -tests that find its failures (see [how spares are -counted](spares.md#how-it-is-computed)). Only a test that can last as long -as its interval stays simulated. A common-cause group's members' tests and -repairs may take time too, of a fixed length or an exponential one (see -[below](#common-cause-staggered-tests-and-test-coverage)). - -### Common cause, staggered tests and test coverage - -Three more terms decide a real safety function's PFDavg, and each has its -place in the diagram: - -- **Common cause.** Redundant valves of one design, in one service, fail - together more often than chance allows. Give the diagram `ccf_groups`, as - for a [non-repairable one](common-cause.md): a `BetaFactor(β)` makes a - share `β` of each valve's failures a shared cause that fails both at - once. For a redundant function the shared term usually dominates: about - `βλτ/2`, five times the pair's independent `(λτ)²/3` here. -- **Staggered tests.** An `"offset"` tests one valve half an interval after - the other: a shared failure is then found by whichever test comes first, - and the independent term falls from `(λτ)²/3` to about `5(λτ)²/24`. - Each valve is restored by its own test alone: the first test restores - its valve, and with it the pair, and the other stays down until its own - test (where practice has the first test find and restore both, the - value is a little less, the pair then not left on one valve until the - second test). -- **Test coverage.** A proof test that finds only a share `c` of the - failures (a `"coverage"`) leaves the rest hidden until a full test - (`"full_test"`), say the ten-year overhaul: about `(1 − c)λT/2` more, - `T` the full test's interval. - -```python -def proof_tested(**inspection): - return { - "reliability": surv.Exponential.from_params([2e-6]), - "repairability": "instant", - "inspection": {"interval": 8760.0, **inspection}, - } - -redundant_edges = [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")] -common = [CCFGroup(["v1", "v2"], BetaFactor(0.05))] -shared = RepairableRBD( - redundant_edges, - {"v1": proof_tested(), "v2": proof_tested()}, - ccf_groups=common, -) -shared.mean_unavailability() # -> 5.290e-4 (λτ)²/3 + βλτ/2, about -staggered = RepairableRBD( - redundant_edges, - {"v1": proof_tested(), "v2": proof_tested(offset=4380.0)}, - ccf_groups=common, -) -staggered.mean_unavailability() # -> 2.779e-4 the shared term halved -partial = RepairableRBD( - [("s", "v"), ("v", "t")], - {"v": proof_tested(coverage=0.9, full_test=87600.0)}, -) -partial.mean_unavailability() # -> 0.01642 0.9λτ/2 + 0.1λT/2, about -``` - -All three are exact with constant failure rates and instant tests and -repairs, and staggered tests and test coverage are numerical with any life -too, and with tests and repairs that take time (see [tests and repairs that -take time](#tests-and-repairs-that-take-time)). With common causes, a -group's members are a Markov chain of which of them are down, with each -member found by its own tests, and a shared failure found alike by every -test (the coverage is the group's), each member it struck restored by its -own test, as in the simulations (#237). Their tests and repairs may take no -time, a fixed time or an exponential one (#220): a working member is off -line while it is tested, and neither ages nor fails meanwhile, a failure a -test finds is repaired once the test is over, and a test that falls in a -member's own test or repair is not done, as in the simulations. IEC -61508-6's 1oo2 with an eight-hour mean repair time: - -```python -repaired = {**proof_tested(), "repairability": surv.ExactEventTime.from_params([8.0])} -mrt = RepairableRBD( - redundant_edges, {"v1": repaired, "v2": repaired}, ccf_groups=common -) -mrt.mean_unavailability() # -> 5.300e-4 IEC 61508-6: 5.316e-4 -``` - -A test of an exponential length followed by a repair of a fixed one, or -fixed lengths that run into the member's next test, are refused, saying -so: estimate those by simulation. -The importance measures take the groups in, a member's conditioned on its -state at each time, and so do the allocations, the values over time from -new (the chain followed through the tests from every member up) and the -simulations, which draw the shared causes; see [repairable -systems](common-cause.md#repairable-systems) and `analysis_routes()`. - -### Choosing the interval - -Each inspection costs `c_i`, and each unit of time the component lies failed -costs `c_d` (its `"downtime_cost"`, or the system's `downtime_cost_rate`). -Frequent tests cost more, and rare ones leave failures hidden for longer, so -the cost rate `c_i/τ + c_d·U(τ)` is least in between, near -`τ* = √(2c_i / (λc_d))`. `expected_cost_rate` prices it exactly: - -```python -def pump(interval): - return RepairableRBD( - [("s", "p"), ("p", "t")], - {"p": { - "reliability": surv.Exponential.from_params([1e-4]), - "repairability": "instant", - "downtime_cost": 500.0, # per hour failed - "inspection": {"interval": interval, "cost": 2000.0}, - }}, - ) - -pump(100.0).expected_cost_rate() # -> 22.49 testing too often -pump(283.0).expected_cost_rate() # -> 14.08 near √(2 × 2000 / (1e-4 × 500)) = 283 -pump(1000.0).expected_cost_rate() # -> 26.19 failures hidden too long -``` - -`optimal_inspection_intervals` finds it, and with a target, the cheapest -intervals that meet it. Tests are made on a calendar, and components tested -at the same times are down together, so with several components the -intervals are chosen from those allowed (`allowed`): every combination when -there are at most 2000, a local search otherwise. With one inspected -component, any interval can be chosen: - -```python -pump(100.0).optimal_inspection_intervals().intervals["p"] # -> 285.5 the formula gives 283 -``` - -For a safety function, `min_availability=1 - PFDavg target` gives the -cheapest tests that meet the target. With the valves above, each test costing -500, and tests monthly, quarterly, half-yearly, yearly or every two years: - -```python -def priced_valve(): - return { - "reliability": surv.Exponential.from_params([2e-6]), - "repairability": "instant", - "inspection": {"interval": 8760.0, "cost": 500.0}, - } - -calendar = [730.0, 2190.0, 4380.0, 8760.0, 17520.0] # hours -single = RepairableRBD([("s", "v"), ("v", "t")], {"v": priced_valve()}) -single.optimal_inspection_intervals( - allowed=calendar, min_availability=1 - 1e-3 -).intervals["v"] # -> 730.0 monthly -redundant = RepairableRBD( - [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], - {"v1": priced_valve(), "v2": priced_valve()}, -) -redundant.optimal_inspection_intervals( - allowed=calendar, min_availability=1 - 1e-3 -).intervals["v1"] # -> 17520.0 every two years -``` - -Redundancy relaxes the tests: one valve needs them monthly to keep the -PFDavg at most `10⁻³`, two in parallel every two years. The result is a -[`MaintenancePlan`][repyability.MaintenancePlan], as for -[age-replacement intervals](#choosing-the-intervals). - -When the valves are tested matters as well. Tested together, both are down -for as long as a common-cause failure stays hidden; tested half an interval -apart, it is found twice as soon (and the valve it found restored, the -other waiting for its own test). `offsets` chooses the times of the first -tests with the intervals (#184), as shares of each interval in [0, 1): a -list for every component, a dict of one per component, or `"stagger"`, the -shares `0, 1/n, …, (n − 1)/n` of `n` components, among which are tests of -one interval spread evenly over it. With a 10% common cause and a PFDavg of -at most `5 × 10⁻⁴`: - -```python -paired = RepairableRBD( - [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], - {"v1": priced_valve(), "v2": priced_valve()}, - ccf_groups=[CCFGroup(["v1", "v2"], BetaFactor(0.1))], -) -together = paired.optimal_inspection_intervals( - allowed=calendar, min_availability=1 - 5e-4) -together.intervals # {'v1': 4380.0, 'v2': 8760.0} -together.cost_rate # -> 0.1712 -apart = paired.optimal_inspection_intervals( - allowed=calendar, min_availability=1 - 5e-4, offset_shares="stagger") -apart.intervals # {'v1': 8760.0, 'v2': 8760.0} -apart.offsets # {'v1': 0.0, 'v2': 4380.0} six months apart -apart.cost_rate # -> 0.1142 a third less -``` - -Shifting every test by one time changes nothing in the long run, so the -first component's tests stay from 0, unless other tested components keep -their schedules (then its offset is chosen too). Offsets change no cost, so -of plans that cost the same the most available is chosen. The -`offset_shares` searched are shares of the interval; the result's -`offsets` are the times of the first tests, as `with_intervals(offsets=)` -takes them, and a plan has them whether they were searched or not. - ## The total cost of ownership The costs so far are running costs. Buying the system is a one-off cost: diff --git a/docs/guide/design.md b/docs/guide/design.md index f1e5d2e5..5fe67b7e 100644 --- a/docs/guide/design.md +++ b/docs/guide/design.md @@ -715,7 +715,7 @@ in the same ratio do as well. availability. The others keep theirs: components with a preventive or inspection schedule (their intervals are chosen by `optimal_replacement_intervals` and `optimal_inspection_intervals`: see - [Costs](costs.md#choosing-the-intervals)), instantly repaired ones, nested + [Costs](system-maintenance.md#choosing-the-intervals)), instantly repaired ones, nested RBDs, components with units that never fail or repairs that may never end, and those listed in `fixed`. - Components tested or block-replaced on the same calendar are down together diff --git a/docs/guide/fault-trees.md b/docs/guide/fault-trees.md index 54ff4793..43fee20d 100644 --- a/docs/guide/fault-trees.md +++ b/docs/guide/fault-trees.md @@ -192,19 +192,105 @@ array), with `Q` the top event probability and `q_e` the event's: |---|---| | `birnbaum_importance(t)` | `Q(e occurred) − Q(e did not)`: how much `Q` depends on the event. | | `criticality_importance(t)` | `I_B(e) · q_e / Q`: the share of the top event the event accounts for. | +| `improvement_potential(t)` | `Q − Q(e did not occur)`, which is `I_B(e) · q_e`: how far the top event would fall if the event could not occur. | | `fussell_vesely(t)` | The probability that some minimal cut set containing `e` has occurred, over `Q` (`method="rare_event"`: their probabilities summed, which can pass 1). | | `risk_achievement_worth(t)` | `Q(e occurred) / Q`. | | `risk_reduction_worth(t)` | `Q / Q(e did not occur)`. | +| `structural_importance()` | The fraction of the other events' states in which `e` is pivotal: the Birnbaum importance with every event at 1/2, from the gates alone. | +| `differential_importance(t)` | Each event's share of a change in `Q` when every event's probability moves ([DIM](importance.md#shares-of-a-change-differential-importance)). | +| `joint_importance(t)` | Whether preventing two events together is worth more than preventing each ([second order](importance.md#pairs-joint-importance)). | +| `barlow_proschan_importance(t)` | The probability that the event's occurring is what brings the top event about, by `t` or ever: `I_B(e)` times the event's density, integrated, over the sum. | ```python cooling.criticality_importance() # {'pump 1': 0.1597, 'pump 2': 0.1597, 'valve': 0.8319} cooling.risk_achievement_worth()["valve"] # -> 16.81 top event certain once the valve fails +cooling.improvement_potential() +# {'pump 1': 0.0095, 'pump 2': 0.0095, 'valve': 0.0495} +cooling.structural_importance() # the valve is pivotal unless both pumps have failed +# {'pump 1': 0.25, 'pump 2': 0.25, 'valve': 0.75} +timed.barlow_proschan_importance() # which event brings cooling down, over its life +# {'pump 1': 0.3625, 'pump 2': 0.3625, 'valve': 0.2750} ``` They are the same measures, with the same definitions, as the [importance measures](importance.md) of a `NonRepairableRBD`: a tree and the -diagram of the same system give the same values. +diagram of the same system give the same values. The Barlow–Proschan +importance needs the events' models (an event given as a probability has +no density, so no share), and is integrated numerically, to about `1e-9`. + +## Over a lifetime + +With lifetime models the top event has a time, and the tree gives what a +diagram gives of its lifetime (see [Reliability of a +system](reliability.md)), worked out on the tree's own evaluation: the mean +time to the top event, the area under `sf` (to about `1e-10`); the time at +which `sf` falls to a reliability, or by which the top event has occurred +in `x` percent of systems, by root-finding; and the density, hazard and +cumulative hazard, the first two by differences of the exact probability. + +```python +timed.mean() # -> 8014.98 the mean time to loss of cooling +timed.bx_life(10) # -> 3360.3 10% of systems have lost cooling by then +timed.time_to_reliability(0.5) # -> 7676.3 half have +timed.hf(5000) # -> 0.000117 the hazard rate at 5000 h +timed.Hf(5000) # -> 0.2548 the cumulative hazard, −ln sf +``` + +A tree of probabilities alone has no time to its top event: `mean`, +`bx_life` and `time_to_reliability` refuse it, saying so. A common-cause +group that splits a probability is refused by `mean` too, as a diagram +refuses it (its model assumes a small probability, while over a lifetime it +runs to 1): split the failure rate instead (`basis="rate"`). + +### Sensitivities and levers + +`parameter_sensitivity(t)` gives how far `sf(t)` moves with each parameter +of each event's model, the event's Birnbaum importance times its model's +parameter derivative (an event given as a probability has the one lever +`p`; a common-cause group's parameters, and its model's `ccf_beta`, ..., +are reported once under the tuple of its members, by differences of the +exact probability). `levers()` lists the parameters, with their values and +ranges, and `with_levers` builds the tree with some moved, as a diagram's +do ([Sensitivities: the Greeks](greeks.md)): + +```python +timed.parameter_sensitivity(5000)["valve"] +# {'alpha': 7.27e-06, 'beta': 0.1343} +[(lever.key, lever.name) for lever in timed.levers()][:2] +# [('pump 1', 'alpha'), ('pump 1', 'beta')] +timed.with_levers({("valve", "alpha"): 40000.0}).mean() # -> 8850.3 a valve lasting twice as long +``` + +### Uncertainty from fitted event models + +`sf_uncertainty` and `mean_uncertainty` draw plausible models for the +uncertain events, from their surpyval fits' covariance by default, or from +distributions over named parameters, or a list of models, and work the +top event probability, or the mean time to it, out exactly for each draw, +as a diagram's [`sf_uncertainty`](reliability.md#uncertainty-in-the-component-models) +does; the draws are those of the tree's diagram (`to_rbd()`, converted +once), so events of one population go together in a tuple, and a group's +members together. Here the pumps' Weibull scale is known only to lie +between 7000 and 9000 h: + +```python +import scipy.stats as st + +scale = {"alpha": st.uniform(7000, 2000)} +spread = timed.sf_uncertainty(5000, {("pump 1", "pump 2"): scale}, n_draws=2000, seed=1) +spread.nominal # -> 0.7751 +spread.interval(0.9) # (0.733, 0.804) +``` + +`analysis_routes()` says, without running anything, how each of the tree's +analyses is computed, as a diagram's does: exact, numerical, simulated, or +refused and why. + +```python +timed.analysis_routes()["mean"].route # 'numerical' +cooling.analysis_routes()["mean"].route # 'refused' no event has a lifetime +``` ## From diagrams to trees and back @@ -251,7 +337,8 @@ with the bridge's logic. `to_json()` / `FaultTree.from_json()` (and `to_dict` / `from_dict`) save the gates, the top event and the events, probabilities as numbers and models as RePyability saves a diagram's node models, and the common-cause -groups as a diagram saves them. +groups as a diagram saves them. A tree built by `with_levers` saves the +same way: an event given as a probability is still one. ## The model and its limits diff --git a/docs/guide/greeks.md b/docs/guide/greeks.md index d1a4a6d8..bd98d51f 100644 --- a/docs/guide/greeks.md +++ b/docs/guide/greeks.md @@ -129,7 +129,15 @@ order, each a `Lever`: whose it is and its name, its value now, the range of values it can take, whether it is discrete (one more standby unit or crew), and whether it moves a calendar that its component shares with others. Nothing is worked out, so a report can name the levers and show -their values without the sensitivities: +their values without the sensitivities. The levers are each node's own +model's parameters (and a component's maintenance options): neither +`levers()` nor `parameter_sensitivity` descends into a nested diagram or +into the units of a composite model (a `StandbyModel`, a +`LoadSharingModel`, a `DegradingNode`), whose parameters are not reached. +(One lever the sensitivities over time leave out: the `repair.q` of a +minimally repaired component, `q = 1`, the one imperfect repair the values +over time take, so the derivative in `q` does not exist there; +`with_levers` still moves it, to a diagram the simulations take.) ```python valve = {lever.name: lever for lever in station.levers() if lever.key == "valve"} @@ -344,16 +352,16 @@ ownership (`discount_rate=`, see [Costs](costs.md#discounting)). | Measure | `NonRepairableRBD` | `RepairableRBD` | `FaultTree` | |---|---|---|---| | `birnbaum_importance` | at `x` | long run, `x`, `window`, `state` | at `t` | -| `parameter_sensitivity` | at `x` | long run, `x`, `window`, `state`; `of="cost_rate"` | | -| `levers`, `with_levers` | the parameters | every lever | | +| `parameter_sensitivity` | at `x` | long run, `x`, `window`, `state`; `of="cost_rate"` | at `t` | +| `levers`, `with_levers` | the parameters | every lever | the parameters | | `differential_importance` | at `x` | long run, `x`, `window`, `state` | at `t` | | `joint_importance` | at `x` | long run, `x`, `window`, `state` | at `t` | | Theta | `reliability_rate(x)` | `availability_rate(x)`, from new or `state` | | -| `barlow_proschan_importance` | whole life, or by `x` | long run, `window`, `state` | | +| `barlow_proschan_importance` | whole life, or by `x` | long run, `window`, `state` | whole life, or by `t` | | `uncertainty_importance` | at `x`; `of=` the MTTF, a B-life, a time to a reliability | long run; `of=` the availability at `x` or over missions `x`, from new or `state`, or the cost rate | | With limited repair crews or common-cause groups, the measures follow the crews' and the groups' Markov chains: theta and the Barlow–Proschan shares split the chains' transitions by the component, or common cause, that makes -each (#199). With common-cause groups `joint_importance` is refused, as -their members cannot be held. +each (#199); `joint_importance` conditions on each pair's states through +the groups' outcomes, as the single-node measures do (#317). diff --git a/docs/guide/img/calendar-thresholds.svg b/docs/guide/img/calendar-thresholds.svg new file mode 100644 index 00000000..883143e1 --- /dev/null +++ b/docs/guide/img/calendar-thresholds.svg @@ -0,0 +1,1305 @@ + + + + + + + + image/svg+xml + + + Matplotlib v3.11.2, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/guide/importance.md b/docs/guide/importance.md index 2ac5a742..dc6c3bbd 100644 --- a/docs/guide/importance.md +++ b/docs/guide/importance.md @@ -208,6 +208,12 @@ means over it, as `mission_availability` is its mean availability: C's criticality over the first 10 time units is its share of the window's expected downtime, not the mean of its shares at each time. +At the instant of a test or a scheduled replacement, a measure is of the +state just after it, as `point_availability` is: a pair tested together is +just back from its tests, and cannot be down, so its ratio measures there +(the criticality, the risk worths) are 0/0, NaN. Evaluate just before the +instant (`8760.0 - 1e-9`, say) for the state before the test. + From a state: with A down now, in repair, B matters more for a while: ```python @@ -221,8 +227,9 @@ plant.birnbaum_importance(x=1.0, state=down)["B"] # -> 0.3886 With limited repair crews, the measures follow the crews' chain over time: the Birnbaum importance, improvement potential and risk worths hold each node working and failed in it, as in the long run, and the criticality and -Fussell–Vesely measures average over its states at each time (not yet -around nested RBDs). With common-cause groups, each time's point is split +Fussell–Vesely measures average over its states at each time; a nested +RBD, with crews of its own, enters by its own availability then, as in +`point_availability`. With common-cause groups, each time's point is split by the groups' joint states then. A component whose curve over time is not worked out (one repaired imperfectly, say) is refused, as by `point_availability`. @@ -234,8 +241,9 @@ long-run availability in each lever: each component's life and repair models' parameters (`"reliability."`, `"repairability."`), its preventive maintenance and tests (`"preventive.interval"`, `"inspection.coverage"`, ...), a standby group's, a common-cause group's -(under its members together), and the change one more standby unit or -repair crew makes: +(under its members together, but for a member's test offset, its own place +on the calendar, under the member's name), and the change one more standby +unit or repair crew makes: ```python levers = plant.parameter_sensitivity() @@ -316,7 +324,14 @@ plant.joint_importance(x=[0.5, 20.0])[("A", "C")] # array([0.0385, 0.0909]) A `FaultTree` has it too, as `−∂²P/∂q_e ∂q_f` in its events' probabilities, which is the same as its diagram's (`to_rbd()`): positive under an OR gate (complements), negative under an AND gate (substitutes). -With common-cause groups, whose members cannot be held, it is refused. + +With common-cause groups, the system given each of the pair working and +failed is conditioned on their states, as the single-node measures are +(see [Common-cause +failures](common-cause.md#importance-sensitivity-uncertainty-and-allocation)): +a member's state says something of its group's shocks, two members of one +group are conditioned on their joint state, and a node outside the groups +is held. ## Shares of a change: differential importance diff --git a/docs/guide/index.md b/docs/guide/index.md index 4de79732..7a7da092 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -5,15 +5,15 @@ takes, what it returns, and its limits. Each page is self-contained and its examples run as written. To learn the ideas step by step, with worked examples and exercises, take the [Learn](../learn/index.md) course; for a compact summary of the theory, see [Concepts](../concepts.md); for every -signature and docstring, see the [API reference](../api.md). +signature and docstring, see the [API reference](../api/index.md). | Page | What it covers | |---|---| | [Building an RBD](building.md) | Edges and node models, k-out-of-n nodes, junctions (a vote point that never fails), a component that appears in several places, nested RBDs, validation, path and cut sets, and the structural checks. | -| [Fault trees](fault-trees.md) | Static fault trees (OR, AND and VOTE gates, repeated events): the exact top event probability, minimal cut sets ranked by probability, importance measures, common-cause groups, and conversion to and from block diagrams. | +| [Fault trees](fault-trees.md) | Static fault trees (OR, AND and VOTE gates, repeated events): the exact top event probability, minimal cut sets ranked by probability, importance measures, common-cause groups, the mean time to the top event, B-lives, hazard, parameter sensitivities and the uncertainty from fitted event models, and conversion to and from block diagrams. | | [Reliability of a system](reliability.md) | `sf`/`ff`, density and hazard, conditional survival, per-node values, forcing nodes working or failed, lifetimes and the exact MTTF, inverting reliability to a time (B*X* life), and the uncertainty that uncertain component models give the system reliability. | -| [Phased missions](phased-missions.md) | Missions through phases (take-off, cruise, landing), each with its own duration and diagram over the same components: the exact mission reliability and the chance of failing in each phase, and their simulation. | -| [Networks](networks.md) | Undirected networks whose links (and nodes) fail: the exact reliability of the connection between two terminals, its minimal paths and cuts, link importance, the mean time to disconnection, and their simulation. | +| [Phased missions](phased-missions.md) | Missions through phases (take-off, cruise, landing), each with its own duration and diagram over the same components: the exact mission reliability and the chance of failing in each phase, and their simulation; each component's importance over the mission, and the mission's cut sets. | +| [Networks](networks.md) | Undirected networks whose links (and nodes) fail: the exact reliability of the connection between two terminals, its minimal paths and cuts, the importance measures, B-lives, parameter uncertainty and saving of a diagram, the mean time to disconnection, and their simulation. | | [Importance measures](importance.md) | Birnbaum, improvement potential, risk achievement and reduction worth, criticality, Fussell–Vesely, structural importance, and parameter sensitivity; on a repairable system in the long run or over time; differential and joint importance, the rate of change split by component, and Barlow–Proschan. | | [Sensitivities: the Greeks](greeks.md) | The sensitivity measures as one family, named after the option Greeks (delta, the levers' deltas, the differential importance, gamma, theta, vega), what they share, and one pumping station run through every one. | | [Condition-based evaluation](condition-based.md) | Reliability, remaining life and importance given each component's current age, and covariate-dependent components (fixed operating conditions or a load schedule). | @@ -21,7 +21,8 @@ signature and docstring, see the [API reference](../api.md). | [Common-cause failures](common-cause.md) | Beta-factor and Multiple Greek Letter groups, splitting a probability or a failure rate, in block diagrams, repairable systems (with staggered tests) and fault trees, and where they apply. | | [Repairable systems](repairable.md) | Long-run availability, failure frequency, MUT/MDT/MTBF; the exact availability over time from new and over a mission, and the unavailability to its own precision; junctions; the simulated histories and their criticality measures; shared repair crews, with exact long-run values and importance and numerical values over time for exponential components; standby groups repaired one unit at a time; imperfect repair; nested repairable RBDs, and stepping a simulation by hand. | | [System capacity](capacity.md) | How much a system can deliver: node capacities, components with several levels (derated, or degrading through stages over time), the exact distribution of the system's capacity at a time or in the long run, the probability of meeting a demand, the expected capacity, and the production availability. | -| [Costs](costs.md) | The long-run cost rate in closed form, the simulated cost distribution, costs drawn from distributions, scheduled preventive maintenance and the intervals that meet a system target, replacement on condition at periodic inspections, hidden failures found by periodic inspection (PFDavg for safety functions), and the total cost of ownership, discounted to a present value, with the redundancy that minimises it, whole trains among it. | +| [Costs](costs.md) | The long-run cost rate in closed form, the simulated cost distribution, costs drawn from distributions, and the total cost of ownership, discounted to a present value, with the redundancy that minimises it, whole trains among it. | +| [Maintaining a system](system-maintenance.md) | Scheduled preventive maintenance in a diagram and the intervals that meet a system target, replacement on condition at periodic inspections, opportunistic maintenance, and hidden failures found by periodic inspection (PFDavg for safety functions), with their intervals chosen. | | [Spares](spares.md) | How many spares each component uses over a horizon, for a system or a fleet, and the stock that meets a fill rate or a stock-out target when each spare used is reordered and arrives a lead time later; interchangeable components' spares pooled on one shelf. | | [Design and allocation](design.md) | How many redundant copies to fit (redundancy allocation), apportioning a reliability target among components (reliability allocation), and an availability target, as MTTF and MTTR targets (availability allocation). | | [Maintenance policies](maintenance.md) | Age replacement, overhaul under minimal or imperfect repair, failure-count replacement, and the expected time to the *n*-th failure. | diff --git a/docs/guide/maintenance.md b/docs/guide/maintenance.md index 03ad2990..b7c4975b 100644 --- a/docs/guide/maintenance.md +++ b/docs/guide/maintenance.md @@ -113,7 +113,142 @@ failure. To price an age- or block-replacement policy for a component in its system (where a planned stop may or may not halt production), give the component a `"preventive"` schedule in a `RepairableRBD`: see -[Costs](costs.md#preventive-maintenance). +[Costs](system-maintenance.md#preventive-maintenance). + +## Replacement over a calendar of shuts (`NonRepairable`) + +Age replacement has one planned price at every moment. In a plant, the price +depends on the date: replacing a bearing in the annual shut costs little, in a +quarterly shut more, at a weekly stop more again, on any other day more still +(or it is not allowed), and at a failure most of all. +[`optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy] +finds the long-run optimal policy over such a calendar +(opportunity-based age replacement, after Dekker et al.). The state is the +date and the part's age, and the policy is a **threshold for each date**: +replace the part if it is at least `a*(t)` old. + +Times are numbers in the life model's unit (days here), from 0: turning dates +into them is Reliafy's. + +```python +import surpyval as surv +from repyability import Calendar, NonRepairable, Opportunity + +bearing = NonRepairable(surv.Weibull.from_params([1000.0, 2.5])) +plant = Calendar( + [ + Opportunity("weekly", 3.0, period=7.0), + Opportunity("quarterly", 2.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, # by default the part's unplanned_cost + anytime=4.0, # any other day; None: not allowed; by default its planned_cost +) +policy = bearing.optimal_calendar_policy(plant) + +policy.cost_rate # -> 0.006943 a day (2.53 a year) +policy.replacement_rates # a day; a year: annual 0.962, failure 0.078, + # weekly, quarterly and any day 0 +policy.cost_rates # a day; a year: annual 0.962, failure 1.566 +policy.schedule(kinds=("quarterly", "annual")) +# -> [(0.0, 'annual', 88.375, 0.962), (time, kind, threshold, chance) +# (91.0, 'quarterly', 221.375, 0.0), +# (182.0, 'quarterly', 323.75, 0.0), +# (273.0, 'quarterly', 574.0, 0.0)] +policy.next_replacement(age=60.0, now=364.0) +# -> PlannedReplacement(time=728.0, kind='annual', price=1.0) +``` + +An [`Opportunity`][repyability.Opportunity] repeats every `period` from its +`offset`, or falls on given `dates`. + +- **What the policy does, and costs.** `replacement_rates` and `cost_rates` + give, by kind, the replacements and the cost a unit of time in the long + run; the costs sum to `cost_rate`. Here the annual shut does all the + planned work: 0.96 replacements a year at 1, with 0.08 failures a year at + 20, and no other kind is ever used. +- **Each shut's threshold, and whether it is used.** `schedule()` lists each + opportunity of a cycle as `(time, kind, threshold, chance)`: replace the + unit there if it is at least `threshold` old, which happens with `chance` + in the long run. The threshold at a shut depends on how far the next + cheaper one is (the quarterly shut a quarter before the annual waits for a + unit of 574 days, the one three quarters before it for one of 221), but at + these prices the annual shut replaces units before they reach any of them, + so each quarterly shut's chance is 0. +- **The unit in service.** `next_replacement(age, now)` says when it will be + replaced, if it does not fail first: a unit 60 days old at the annual shut + is younger than its threshold, about 88 days, so it is kept to the next + one. + +![The threshold age at each weekly, quarterly and annual shut over a year](img/calendar-thresholds.svg) + +The chart is drawn by `scripts/calendar_policy_chart.py`. The quarterly and +annual thresholds, with what the simple policies cost over ten years of the +plant ([`calendar_policy_cost`][repyability.NonRepairable.calendar_policy_cost] +prices a rule given as the age from which each kind of replacement replaces): + +| Policy | Rule | Cost over 10 years | +|---|---|---| +| **Optimal** | annual from 88 d; quarterly from 221, 324, 574 d; weekly as charted | **25.27** | +| Every annual shut | `{"annual": 0.0}` | 25.48 | +| Age replacement on any day | `{"anytime": 493.0}` (the textbook interval at 4 and 20) | 50.41 | +| Run to failure | `{}` | 82.05 | + +### How it is worked out + +`method="auto"` (the default) chooses among these: + +- **`"dp"`**: dynamic programming on a grid (its step about the mean life + over 1,000, put on the opportunities' own times). Over a periodic + calendar whose cycle (the least common multiple of its periods) has at + most 20,000 steps, relative value iteration gives the cost rate exactly for + the grid; a failure renews half a step later on average, so the grid's + error is second order in the step. With no opportunities and a planned + price on any day, it is age replacement: threshold 493 days, 0.0034620 a + day, against the textbook 493.05 and 0.0034620. +- **`"integration"`**: between opportunities in continuous time, on + quadrature nodes (32 a gap); only with no replacement on any day. It agrees + with `"dp"` as the step shrinks. +- **`"sampling"`**: for a cycle too long to sweep (co-prime periods of 7, 93, + 400 and 1,301 days repeat every 928,000 years). The cost rate is the mean + over `mc_samples` sampled times of the cycle of a window's rate, each window + an exact solve, with its standard error (`cost_rate_se`); the threshold at a + time is a solve over a window from it, deterministic. Each window takes a + second or more, so `mc_samples` (64 by default) sets the cost, and + `"auto"` warns when it falls to sampling. A sampled policy's rates by kind + (`replacement_rates`, `cost_rates`) and the schedule's chances are + estimates from a new unit run in, with no standard error: only the cost + rate has one, and the rates by kind move with the seed (by some 10% at + 8 samples). + +`policy.method` and `policy.basis` (`"exact"`, `"differenced"` or +`"sampled"`) say which was used. + +**How exact it is.** The grid's step is not an argument: it is a numerical +setting, sized from the life and put on the shuts' periods (here 0.875 days), +not a choice of when decisions are made, which the calendar says by its +opportunities. A periodic calendar keeps its period on the grid whatever +its offsets: an offset or date off the grid (a shut 0.1234 days, three +hours, into the day, as dates turned into model time give) is laid at the +grid time nearest it, within half a step, with a warning, and the policy +is the aligned calendar's. On the grid the cost rate is exact for decisions at its +times, and within about the square of the step of deciding at any moment: +for this plant 2.5272582 a year, against 2.5272583 on a grid four times as +fine. With no replacement on any day, `method="integration"` works between +the shuts in continuous time, exact but for its quadrature (2.5272576 a +year here). A plan of given dates is extended past its +last date by `tail`: `"repeat"` (repeat it) or `"anytime"` (only the +replacement on any day). Both are solved, and `policy.determined_until` is the +first time at which the two differ: the policy before it does not depend on +what comes after the plan. If at some time the ages that replace are not all +those from the youngest (a life whose hazard falls), the policy says so +(`threshold_only` is False) and warns. + +The policy prices a part alone. Given to a `RepairableRBD` component (see +[Replacement on a calendar policy](system-maintenance.md#replacement-on-a-calendar-policy)), +it is followed as worked out, in a maintenance group too, whose stops are +then chances to renew the part that the policy did not plan for: it is not +re-optimised for them. ## Overhaul under minimal repair (`Repairable`) @@ -229,4 +364,6 @@ It is `alpha · Γ(n + 1/beta) / Γ(n)`. Both policy types are small typed results: [`MaintenancePolicy`][repyability.MaintenancePolicy] (`interval`, `cost_rate`) and [`FailureLimitPolicy`][repyability.FailureLimitPolicy] -(`failure_count`, `cost_rate`). Costs are per event and undiscounted. +(`failure_count`, `cost_rate`). A calendar's policy is a +[`CalendarPolicy`][repyability.CalendarPolicy] (`cost_rate`, +`replacement_rates`, `cost_rates`, `schedule()`, `next_replacement()`). Costs are per event and undiscounted. diff --git a/docs/guide/networks.md b/docs/guide/networks.md index d3fed841..0f304332 100644 --- a/docs/guide/networks.md +++ b/docs/guide/networks.md @@ -39,7 +39,9 @@ len(grid.path_sets()) # -> 7 simple path between the terminals (its links, and its nodes that can fail); `cut_sets()` the smallest sets of links whose failure parts the terminals. Here two pairs of lines cut the load off on their own, - `{"BL", "CL"}` and `{"GA", "GB"}`. + `{"BL", "CL"}` and `{"GA", "GB"}`. `minimal_path_sets()` and + `minimal_cut_sets()` list the same sets smallest first, as a diagram's + and a fault tree's do. - **Importance.** `birnbaum_importance(x)` gives, per link, the reliability with it working less that with it failed: the two lines out of the generator and the cable into the load matter most. @@ -107,3 +109,72 @@ grid.sf(1.0, method="simulate", mc_samples=200_000, seed=1) # -> 0.98477 sim Links fail independently, as a diagram's nodes do. The network is not repairable: a link that fails stays failed. + +## Importance, lives, uncertainty and saving + +The importance measures of a block diagram (see [Importance +measures](importance.md)) are a network's too, over its links and failing +nodes, with the same definitions: `improvement_potential`, +`risk_achievement_worth`, `risk_reduction_worth`, `criticality_importance` +(`kind="failure"` or `"success"`), `fussell_vesely` (over the cut sets or +the path sets, exact or the rare-event sum), `structural_importance`, +`differential_importance`, `joint_importance` and +`barlow_proschan_importance`. Each comes from the decision diagram replayed +with an element held working or failed (the exact Fussell-Vesely from a +decision diagram of each element's cut sets), so they are exact, and a +small one keeps its precision. The lines out of the generator account for +most of the outages: + +```python +grid.criticality_importance(1.0)["GA"] # -> 0.60334 its share of the outages at a year +grid.fussell_vesely(1.0)["GA"] # -> 0.60991 +grid.risk_achievement_worth(1.0)["GB"] # -> 7.0029 the risk with GB down, relative +grid.barlow_proschan_importance()["GB"] # -> 0.26291 the outages it causes, over the whole life +``` + +`bx_life(x)` and `time_to_reliability(target)` invert `sf`: the time by +which `x` percent of connections are cut, and at which the reliability +falls to `target`. + +```python +grid.bx_life(10) # -> 2.6804 years until a tenth of the loads have been cut off +grid.time_to_reliability(0.5) # -> 7.4167 +``` + +`sf_uncertainty` and `mean_uncertainty` propagate the uncertainty of the +links' and nodes' fitted models, as a diagram's do (see [Parameter +uncertainty](reliability.md#uncertainty-in-the-component-models)): each draw gives every +uncertain element a plausible model (`"fit"` from its surpyval fit's +covariance, distributions over its parameters, or a list of models), and +the reliability or the mean is worked out exactly for it. Elements given +together in a tuple share their draws, as lines of one type should. With +the five lines' failure rate known only to lie between 0.05 and 0.15 a +year: + +```python +import scipy.stats as st + +rate = {"failure_rate": st.uniform(0.05, 0.1)} +lines = ("GA", "GB", "AC", "BC", "BL") +result = grid.sf_uncertainty(1.0, {lines: rate}, n_draws=4000, seed=1) +result.nominal # -> 0.98509 +lower, upper = result.interval(0.9) +round(lower, 4), round(upper, 4) # (0.9721, 0.9943) +mean = grid.mean_uncertainty({lines: rate}, n_draws=400, seed=1) +round(mean.nominal, 4) # -> 8.518 +``` + +A network saves and loads as a diagram does (see [Saving and +loading](saving.md#saving-and-loading)): `to_dict` and `to_json` keep the +terminals, each link's two nodes and model, and the failing nodes' models, +in surpyval's own format, so a fit's covariance round-trips; `from_dict` +and `from_json` rebuild it. `analysis_routes()` says how each analysis is +worked out, without running it: exact from the decision diagram, numerical +(`mean`, the B-lives and `barlow_proschan_importance`), simulated +(`random` and the uncertainty's draws), or refused, with the message the +method raises, for a network too large for its diagram. + +```python +Network.from_json(grid.to_json()).sf(1.0) # -> 0.98509 +grid.analysis_routes()["mean"].route # 'numerical' +``` diff --git a/docs/guide/phased-missions.md b/docs/guide/phased-missions.md index 3026bc55..b3b2f7c0 100644 --- a/docs/guide/phased-missions.md +++ b/docs/guide/phased-missions.md @@ -101,3 +101,68 @@ interval = flight.reliability_interval(1_000_000, seed=1) interval.estimate # -> 0.996079 simulated flight.reliability() # -> 0.996037 ``` + +## Importance and cut sets + +Which component matters most over the whole mission, and which +combinations of failures end it? A component is the same through every +phase, so its importance holds it across all of them: *working throughout* +(every phase's segment of its life survived) or *failed from the start* +(failed in the first phase, so in every phase). `birnbaum_importance()` is +the mission's reliability with the component working throughout less its +reliability with it failed from the start; `improvement_potential()`, +`risk_achievement_worth()`, `risk_reduction_worth()` and +`criticality_importance()` are built on the same held values, as the +diagrams' are. Each is the mission's decision diagram replayed with the +component's segments so held, exact, at the mission's end; there is no +simulated version. Here any component failed from the start fails the +take-off or the cruise, so the risk achievement worth cannot tell them +apart, but the risk reduction worth can: the navigation unit, alone in +series through the cruise, accounts for half the mission's failures, each +engine for a quarter, the gear for almost none. + +```python +raw = flight.risk_achievement_worth() +raw["nav"] == raw["gear"] # -> True +rrw = flight.risk_reduction_worth() +rrw["nav"] # -> 2.0545 +rrw["e1"] # -> 1.3191 +rrw["gear"] # -> 1.0017 +criticality = flight.criticality_importance() +criticality["nav"] # -> 0.5133 the share of failures nav accounts for +``` + +`minimal_cut_sets()` gives the combinations: each a set of events +`(component, phase)`, "the component has failed by the end of that phase", +that together fail the mission. A failure by one phase is one by every +later phase, so a cut set keeps the weakest events that still fail the +mission: an engine lost by the end of the take-off is a cut set of the +take-off, but the landing needs both engines too, so the engine lost by the +end of the landing, whenever it was lost, is the cut set listed. The +phases' own cut sets, dated, less those another implies, are the mission's +(the usual phased-mission minimisation). + +```python +cuts = flight.minimal_cut_sets() +sorted(sorted(cut) for cut in cuts) +# -> [[('e1', 'landing')], [('e2', 'landing')], [('gear', 'landing')], [('nav', 'cruise')]] +``` + +`fussell_vesely()` is the share of the mission's failure probability that +involves each component: the probability that some cut set naming it, in +any phase, has occurred, over the mission's unreliability. The union is +exact, from a decision diagram of the cut sets' events over the segments; +`method="rare_event"` sums the cut sets' probabilities instead, the +approximation many PRA tools report, which over-estimates the union when +cut sets overlap (here none share a component, so the two agree). + +```python +fv = flight.fussell_vesely() +fv["nav"] # -> 0.5143 +fv["e1"] # -> 0.2426 +flight.fussell_vesely(method="rare_event")["e1"] # -> 0.2426 +``` + +`analysis_routes()` reports each analysis as exact, or refused when the +mission's diagram (for `fussell_vesely`, the cut sets' too) would pass +`MAX_NODES`, and `reliability_interval` as simulated. diff --git a/docs/guide/reliability.md b/docs/guide/reliability.md index d98dc266..4d8ec798 100644 --- a/docs/guide/reliability.md +++ b/docs/guide/reliability.md @@ -218,7 +218,7 @@ A node's uncertainty is one of: | Given | Draws | |---|---| -| `"fit"` | The parameters, from the normal approximation of the model's maximum-likelihood fit (surpyval's `hess_inv`), on the log scale for a positive parameter and the logit scale for one in (0, 1), so every draw is valid. An offset, zero-inflation or limited-failure-population parameter keeps its fitted value. | +| `"fit"` | Every parameter the fit estimated, from the normal approximation of the model's maximum-likelihood fit (surpyval's `covariance()`), on the log scale for a positive parameter and the logit scale for one in (0, 1), so every draw is valid. That includes a limited failure population's share that ever fails (`lfp_p`) and a zero-inflated fit's share dead on arrival (`f0`). An offset keeps its fitted value, with a warning, since surpyval's covariance leaves it out (SurPyval#830). | | `{"alpha": distribution, ...}` | Each named parameter from its distribution (anything with `qf` or `ppf`: surpyval or `scipy.stats`); the others keep their values. | | A list of models | One of them, with replacement: for example refits to bootstrap resamples, or posterior draws, made in surpyval. | @@ -311,7 +311,7 @@ The pumps' fit accounts for 60% of the reliability's variance at 50 h, but `"mean"`, `"bx_life"` (`x` the percentage) or `"time_to_reliability"` (`x` the reliability). By default it is worked out by the delta method: the quantity's derivative in each input's parameters (central differences of the -exact value) with their covariance (the fit's `hess_inv`, or the variances +exact value) with their covariance (the fit's `covariance()`, or the variances of the distributions given). The inputs are independent, so each input's part is its own term, and the shares add up to 1. diff --git a/docs/guide/repairable.md b/docs/guide/repairable.md index d7da3dcd..285f530b 100644 --- a/docs/guide/repairable.md +++ b/docs/guide/repairable.md @@ -45,12 +45,14 @@ A component can be given as: (`"repair_cost"`, `"replace_cost"`, `"downtime_cost"`, and the one-off `"acquisition_cost"`; see [Costs](costs.md)), scheduled preventive replacement (`"preventive"`; see - [Costs](costs.md#preventive-maintenance)), for a component whose + [Costs](system-maintenance.md#preventive-maintenance)), for a component whose failures are hidden until a proof test finds them, periodic inspection - (`"inspection"`; see [Costs](costs.md#hidden-failures-and-inspection)), + (`"inspection"`; see [Costs](system-maintenance.md#hidden-failures-and-inspection)), its place in the queue for a repair crew (`"priority"`; see - [below](#repair-crews)), and imperfect repair (`"repair"` and - `"replace_after"`; see [below](#imperfect-repair)). Any other key raises + [below](#repair-crews)), imperfect repair (`"repair"` and + `"replace_after"`; see [below](#imperfect-repair)), and the share of the + time it operates (`"duty"`; see [below](#operating-part-of-the-time)). + Any other key raises `ValueError`, so a mistyped cost key is never silently priced at zero; - `"repairability": "instant"` for a component repaired in zero time (see [below](#instantly-repaired-components)); @@ -62,19 +64,52 @@ A component can be given as: - [`PerfectReliability`][repyability.PerfectReliability] itself, for a *junction*: a node that never fails, such as the point where two of three trains must deliver (see [below](#junctions)). A spec whose - `"reliability"` is `PerfectReliability` is one too: a what-if of a part - that never fails. + `"reliability"` is `PerfectReliability`, or a probability of failing of 0 + (`FixedEventProbability` at 0), is one too: a what-if of a part that + never fails. The constructor also takes `k`, `input_node`, `output_node` and `on_infeasible_rbd` exactly as for a [`NonRepairableRBD`](building.md), `downtime_cost_rate` (see -[Costs](costs.md)) and `repair_crews` (see [below](#repair-crews)). +[Costs](costs.md)) and `repair_crews` (see [below](#repair-crews)). Every +time is in the models' unit (see [Units](building.md#units)). Every repair restores a component to as good as new (unless it is repaired imperfectly), components fail and are repaired independently of each other (unless they wait for a repair crew), and a component keeps its own failure/repair cycle whether or not the system is up. +### Operating part of the time + +A life fitted to operating hours is the wrong clock for a component that +runs only part of the time (a duty pump, a standby generator's monthly +runs): the diagram runs on the calendar, and the component ages only while +it operates. `"duty"`, the fraction `d` of the time it operates (in +`(0, 1]`), puts its life on the calendar: its life there is its operating +life over `d`, `R(d t)`, the same kind of distribution with its scale +moved, which every exact method and simulation then takes. Its repairs, +maintenance and tests stay on the calendar, and its levers and saved spec +keep the life as given, in operating time. So is a +[state](#from-the-plant-as-it-is-now)'s `age`: on the diagram's clock, +the calendar time since it was put into service (an operating age `a` +is `NodeState(age=a / d)`). + +```python +pump = {"reliability": surv.Weibull.from_params([1000, 2]), # operating hours + "repairability": surv.Exponential.from_params([1 / 8])} +always = RepairableRBD([("s", "p"), ("p", "t")], {"p": pump}) +part = RepairableRBD([("s", "p"), ("p", "t")], {"p": {**pump, "duty": 0.4}}) +always.mean_availability() # -> 0.99105 +part.mean_availability() # -> 0.9964 +part.system_failure_frequency() * 8760 # -> 3.94 failures a year, against 9.796 +``` + +The life must be a surpyval parametric distribution of a time; a +probability per demand, a mixture, a degradation process or a nested +diagram is refused (give its life on the calendar instead). The duty is a +fixed share of the time: a component that runs only while the system runs, +or ages while idle too, needs that modelled in its life. + ### Junctions A k-out-of-n vote needs a node to vote at, and that node is often no part @@ -166,9 +201,9 @@ Both take `working_nodes`, `broken_nodes` and `method` as `mean_availability` does, and cover what it covers: age and block replacement, nested RBDs, and hidden failures, for any life, tested and repaired in no time or not, with tests that find every failure or miss -some (see [a life that wears out](costs.md#a-life-that-wears-out) and +some (see [a life that wears out](system-maintenance.md#a-life-that-wears-out) and [tests and repairs that take -time](costs.md#tests-and-repairs-that-take-time)). A test that can last as +time](system-maintenance.md#tests-and-repairs-that-take-time)). A test that can last as long as its interval raises `NotImplementedError`; simulate it. Each component's curve is computed on a grid of 1,000 steps over its typical up time: within one step of a time at which its units start or stop on a schedule (at 0, and at its @@ -274,15 +309,27 @@ availability, failures, cost and capacity over the next month?* Pass down so far, in a repair or (`maintenance=True`) in its preventive maintenance. What is left of that has the survival function `G(r + s) / G(r)`, and the unit is then new; -- **on a calendar:** `phase`, the time since its last block replacement or - test, so that the next falls `interval - phase` from now. A unit with +- **on a calendar:** `since_scheduled`, the time since its last block + replacement or test, so that the next falls `interval - since_scheduled` + from now. A unit with hidden failures is known to have been up only at its last test (or when put into service, if that was since): it may have failed since, unseen, and the next test finds it; +- **repaired imperfectly, at a virtual age:** `NodeState(age=a, + virtual_age=v)`, its virtual age `v` at its last repair and its operating + time `a` since (#269); its life left is drawn from virtual age `v + a`, + and its next repair takes the virtual age on from `v` by the whole `a` + plus that life. One down is at virtual age `v` once its repair is over. + For a unit of a fitted surpyval `GeneralizedRenewal`, `unit_states()` + gives its `virtual_age` now and `since_failure`: give + `age=since_failure` and `virtual_age=virtual_age - since_failure`. The + simulations take it (not the exact methods), for a component with no + `replace_after`, maintenance or tests; - **a nested RBD:** a dict of its own components' states; - **long in service, state unknown:** `state="stationary"` puts every - component in its long-run state (one on a calendar at its phase; for one - component, `NodeState(stationary=True, phase=...)`). + component in its long-run state (one on a calendar at its + `since_scheduled`; for one component, `NodeState(stationary=True, + since_scheduled=...)`). A component left out starts new, and every component's later units are new, as from new. @@ -336,8 +383,8 @@ before the other is back. than there are crews; or every component in its long-run state (`state="stationary"`), not one alone, as the queue ties them together. - **Not taken:** the state of a standby group (but its long-run state, - `NodeState(stationary=True)`), and the virtual age of an imperfectly - repaired component; leave them out (new). + `NodeState(stationary=True)`), and, by the exact methods, that of an + imperfectly repaired component; leave them out (new). ## Uncertain component models @@ -516,7 +563,7 @@ them with common random numbers, far more precisely than two separate runs | `n_simulations`, `time_simulated_to` | `mc_samples` and `t_simulation`. | | `criticalities` | The criticality measures (below). | | `cost` | The simulated costs, or `None` when nothing is priced (see [Costs](costs.md#the-simulated-cost-distribution)). | -| `capacity_timeline`, `capacity`, `capacity_time`, `mean_capacity`, `demand`, `delivered`, `delivered_fraction`, `delivered_fraction_interval(confidence)` | With node capacities: the mean capacity over time, the time at each capacity, and the fraction of the demand delivered (see [System capacity](capacity.md#over-a-window-simulated)). `None` without capacities. | +| `capacity_timeline`, `capacity`, `capacity_time`, `mean_capacity`, `demand`, `delivered`, `delivered_fraction`, `sample_delivered_fraction`, `delivered_fraction_interval(confidence)`, `delivered_control` | With node capacities: the mean capacity over time, the time at each capacity, and the fraction of the demand delivered, exact where `mission_capacity` works it out, as `mean_availability` is (see [System capacity](capacity.md#over-a-window-simulated)). `None` without capacities. | ```python result.mean_up_time # -> 28.08 against the exact 27.27 @@ -757,6 +804,31 @@ Under Kijima I the virtual age only grows, so a unit that is never renewed fails ever more often; under Kijima II it settles. Replacing the unit every few failures (or on a preventive schedule) bounds it. +**A model fitted to the unit's history.** A spec's `"reliability"` may be +what surpyval fits to a repairable unit's failure times (#269), and the +component is then the life and repair that model is: + +- a Poisson process, `CrowAMSAA`, `Duane` or `HPP`, is minimal repair of the + life whose cumulative hazard is the process's cumulative intensity + (Crow-AMSAA's `(t / alpha) ** beta` is a Weibull life, and a homogeneous + process an exponential one), so with repairs in no time its expected + failures are the process's own; +- a `GeneralizedRenewal` (Kijima I or II) is its life distribution with the + `"repair"` of its Kijima model and restoration factor `q`. + +```python +from surpyval import CrowAMSAA + +gearbox = CrowAMSAA.from_params([1500, 0.9]) +rbd = RepairableRBD([("s", "g"), ("g", "t")], + {"g": {"reliability": gearbox, "repairability": "instant"}}) +rbd.expected_failures(8760.0) # -> 4.8952, gearbox.cif(8760.0) +``` + +Such a spec takes no `"repair"` of its own, and is saved as the life and +repair it is. surpyval's other renewal models (ARA, ARI, G1) are refused: +their repairs are not Kijima's. + - **Costs and spares.** A repair is charged its `"repair_cost"`; a replacement, at the `N`-th failure or at any failure of a unit renewed by its repairs, its `"repair_cost"` and `"replace_cost"`, and uses a spare @@ -768,7 +840,8 @@ every few failures (or on a preventive schedule) bounds it. group, or replacement on condition, cannot be repaired imperfectly. - **Exact or simulated.** A repair does not renew the unit, so the exact long-run values, the availability over time and the spares counts refuse - it, with the reason; `availability`, `cost` and `compare` simulate it, in + it, with the reason (but minimal repair's values over time, below); + `availability`, `cost` and `compare` simulate it, in Python, with its own streams (seeds, antithetic pairs and common random numbers work as for any component). `q = 0`, and `replace_after=1` whatever `q`, are the component renewed at every failure, draw for draw. @@ -794,6 +867,35 @@ patched.expected_failures(8760.0) # -> 227.12 = (8760 / 1000) ** 2.5 patched.expected_cost(8760.0).mean # -> 113561 500 a repair ``` +- **Minimal repair whose repairs take time is numerical over a window** + (#318). The unit then fails at its life's hazard at its *operating* age, + the time it has been up, which its repairs leave as it is, and is down + for each repair. Its failures by operating age `a` are Poisson with mean + `H(a)`, and the repairs so far make reaching that age take `a` plus + their total, a compound Poisson sum: its availability and expected + failures at each time are sums over the number of repairs so far of + that Poisson probability against the distribution of their total, worked + out on a grid of its operating age (about 1,000 steps over a life, to + about 1e-7). A fitted Crow-AMSAA, Duane or HPP process with a repair + time is this component. The values over a window from new take it in, + as numerical as any component's, and a run's means are theirs by + default; its long-run values still refuse, as before. The grid is + followed to the window's end, a few FFTs of it for each repair the + unit can have in the window: a window of a few lives takes well under a + second, one of twenty lives of a fast-wearing unit a few. + +```python +repaired = RepairableRBD( + [("s", "p"), ("p", "t")], + {"p": {"reliability": surv.Weibull.from_params([1000, 2.5]), + "repairability": surv.LogNormal.from_params([3.0, 0.5]), + "repair": {"model": "kijima1", "q": 1.0}, + "repair_cost": 500.0}}, +) +repaired.expected_failures(8760.0) # -> 103.85 fewer: it is down for each repair +repaired.mission_availability(8760.0) # -> 0.7310 +``` + ## Instantly repaired components When repairs are much faster than the time scale of interest, or there is no @@ -833,10 +935,21 @@ nested.system_failure_frequency() # -> 0.03497 nested.availability(t_simulation=100.0, mc_samples=2_000, seed=0).availability[-1] # -> 0.959 ``` -Use one `RepairableRBD` object per place it appears: the same object used for -two nodes shares one simulation state and raises `ValueError` during -`availability()`. (One `NonRepairable` object *can* be reused; each node gets -its own copy.) +What does not recurse: the outer diagram counts its own components' costs +alone (a nested diagram that carries any is warned of when the outer one is +built), its `spares_demand` and `spares_stock` refuse by default with a +nested RBD, naming the components counted, and its levers and parameter +uncertainty are its own components' (a nested diagram's fits are not +drawn). Count those with the nested diagram's own methods. In +`expected_events` a nested node's failures and preventive actions are its +system's failures and planned outages, as the simulation counts them for +the node. + +Use one `RepairableRBD` object per place it appears: a nested diagram +carries its simulation state, so the same object given for two nodes is +refused when the outer diagram is built (`copy.deepcopy` it for nodes +alike). (One `NonRepairable` object *can* be reused; each node gets its own +copy.) ```python pump = NonRepairable(surv.Weibull.from_params([30, 1.5]), surv.Exponential.from_params([0.5])) @@ -873,3 +986,6 @@ drives the same machinery, with each component drawing from its own random streams. Stepped by hand, the components draw from numpy's global RNG as it is (there is no `seed` argument), so a history stepped by hand is not one of `availability()`'s. + +The same analysis runs in the browser in +[Reliafy](https://reliafy.com/guides/repairable-availability). diff --git a/docs/guide/saving.md b/docs/guide/saving.md index 1e2bf6f3..ad20eb81 100644 --- a/docs/guide/saving.md +++ b/docs/guide/saving.md @@ -34,7 +34,7 @@ path = Path(tempfile.mkdtemp()) / "plant.json" rbd.to_json(path) # written; returns None NonRepairableRBD.from_json(path).sf(30) == rbd.sf(30) # True type(RBD.from_dict(data)).__name__ # 'NonRepairableRBD': the base class dispatches on type -data["type"], data["repyability_version"] # ('NonRepairableRBD', '0.13') +data["type"], data["repyability_version"] # ('NonRepairableRBD', '0.14') ``` What is saved: @@ -69,6 +69,14 @@ into a list, and loading turns it back). Loading with the wrong class Condition-based state (`NodeState`) is not part of the RBD and is not saved. +A `FaultTree` saves the same way (`to_dict`, `to_json`, `from_dict`, +`from_json`: its gates, events and common-cause groups), and so does a +`Network` (#315): its terminals, each link's two nodes and model, and the +failing nodes' models, the models as a diagram's nodes' are saved, under +`"type": "Network"`. A model given as a probability is saved as the +`FixedEventProbability` it is taken as; one that is not surpyval's cannot +be saved. + ## Reproducibility Every Monte-Carlo method takes a `seed`: @@ -79,7 +87,7 @@ Every Monte-Carlo method takes a `seed`: | `RepairableRBD` | `availability`, `cost`, `compare` with `control_variate=False` (or where the exact methods do not reach), `spares_demand` with `method="simulate"` | | Node models | `RepeatedNode.random`, `RepeatedNode.mean` with `method="simulate"`, `RepeatedStandbyNode.random`, `StandbyModel.random`, `LoadSharingModel.random`, `DegradingNode.random`, and the `mean` of a `StandbyModel`, `LoadSharingModel` or `DegradingNode` with `method="simulate"` (or with `mc_samples` or `seed`, where it has no exact mean) | | `PhasedMission` | `reliability`, `unreliability` and `phase_failure_probabilities` with `method="simulate"`, `reliability_interval` | -| `Network` | `sf`, `ff` and `mean` with `method="simulate"`, `random` | +| `Network` | `sf`, `ff` and `mean` with `method="simulate"`, `random`, `sf_uncertainty`, `mean_uncertainty` | | `Repairable` | every simulation-backed method | surpyval samples from numpy's **global** random number generator, so a seed diff --git a/docs/guide/simulation.md b/docs/guide/simulation.md index 9308eb05..8bfa7309 100644 --- a/docs/guide/simulation.md +++ b/docs/guide/simulation.md @@ -49,8 +49,10 @@ simulations' own means are beside them, whatever the method `method` says which the means are: `"exact"`, `"conditional"`, `"control_variate"` or `"simulated"`. `control_variate=False` keeps the simulations' own means, for a quick run that should not wait for the exact -part (a few tenths of a second, more than a few hundred simulations of a -small system take). +part: the components' curves over the window, a few tenths of a second +for plain units (more than a few hundred simulations of a small system +take), and longer for maintained or tested components, whose curves +follow their schedules (see [Availability over time](repairable.md#availability-over-time-exact)). | Option | Where | What it does | |---|---|---| @@ -283,8 +285,9 @@ of 6 000 for `tolerance=0.0005`. on repair crews, and maintenance groups. And, component by component, what the exact methods over time do not take: a standby group's switching (unless its units are exponential, when its own chain follows it), whose - units then operate together; imperfect repair (but minimal repair in no - time, which the exact methods take); and inspections they do not take + units then operate together; imperfect repair (but minimal repair, + which the exact methods take, its repairs taking time or not); and + inspections they do not take (tests that can last as long as their interval), whose failures are then revealed. It keeps the rest, age and block replacement and replacement on condition diff --git a/docs/guide/spares.md b/docs/guide/spares.md index 24b76ee2..969438d2 100644 --- a/docs/guide/spares.md +++ b/docs/guide/spares.md @@ -50,7 +50,7 @@ replacement. The pumps' is not: Preventive replacement uses spares too. Replacing each pump at 1,000 hours of age nearly doubles the pumps used (see -[Costs](costs.md#preventive-maintenance) for whether it pays): +[Costs](system-maintenance.md#preventive-maintenance) for whether it pays): ```python renewed = dict(pump, preventive={"interval": 1000.0, @@ -202,9 +202,11 @@ every 1,000 hours, against 1.165), and the same stock fills more of it. A fleet's systems are taken as on block schedules of their own, out of step with each other; two block-replaced components in one part are refused, as -their block times keep step, and so is a unit that can be dead on arrival -while its repairs or block replacements may take no time, as its -replacements can then come several at one instant. A tested component's +their block times keep step. A unit that can be dead on arrival fails as it +is put in, so with repairs and block replacements in no time its +replacements come several at one instant: each replacement of a unit that +lasts brings a geometric number more, which the lasting units' count is +compounded with (#322). A tested component's spares are counted whatever its tests and repairs take, and whether or not its tests can miss a failure: the replacements still fall on the tests that find failures (#159). @@ -272,7 +274,7 @@ it is exact, with no grid: a unit renewed at a test is renewed again `k` tests later with probability `R((k − 1)τ) − R(kτ)`. With tests or repairs that take time, the chances of each cycle's length in tests come from the cycle followed test by test on a grid (see [tests and repairs that take -time](costs.md#tests-and-repairs-that-take-time)). From a random time, the +time](system-maintenance.md#tests-and-repairs-that-take-time)). From a random time, the next replacement is `j` tests on with probability `P(C ≥ j) / S`, `C` being a cycle's length in tests and `S` its mean. @@ -288,9 +290,6 @@ place in proportion to its share of the replacements. Some components' replacements are not counted this way, and the counts refuse, with the reason: -- the stock of a component under **block replacement** that can be dead on - arrival while its repairs or block replacements may take no time, as its - replacements can then come several at one instant; - a component with **hidden failures** whose tests can last as long as their interval; - a **standby group**, whose units' failures depend on each other; diff --git a/docs/guide/system-maintenance.md b/docs/guide/system-maintenance.md new file mode 100644 index 00000000..3a225c65 --- /dev/null +++ b/docs/guide/system-maintenance.md @@ -0,0 +1,855 @@ +# Maintaining a system + +!!! tip "Learning this for the first time?" + This page is the reference. The ideas behind it are taught step by step, + with worked examples and exercises, in [Lesson + 8](../learn/maintenance.md) (maintaining on purpose). + +A component of a [`RepairableRBD`][repyability.RepairableRBD] can be +replaced before it fails (on age, on a calendar, on its condition, or with +others when they stop), and a component whose failures are hidden can be +tested. This page covers each policy, what it costs and how its interval is +chosen; the costs themselves are priced as in [Costs](costs.md). For one +unit on its own, see [Maintenance policies](maintenance.md). + +## Preventive maintenance + +A component's dict can also schedule preventive replacement, under the key +`"preventive"`, so that the cost and availability outputs price the trade +between preventive and corrective maintenance: + +| Key | Meaning | +|---|---| +| `interval` | Required (but under `"calendar"`): the replacement interval `T` (`inf`: never). | +| `policy` | `"age"` (the default): `T` after the unit was last put into service as new, so a failure restarts the clock. `"block"`: at `T, 2T, 3T, …` whatever the unit's age, skipped while it is down. `"condition"`: inspected at `T, 2T, 3T, …`, and replaced if likely to fail before the next inspection (see [below](#replacement-on-condition)). `"calendar"`: at the first time the unit's age reaches a calendar policy's threshold (see [below](#replacement-on-a-calendar-policy)). | +| `calendar` | Under `"calendar"`: the [`CalendarPolicy`][repyability.CalendarPolicy] of [`NonRepairable.optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy]. | +| `duration` | `"instant"` (the default): renewed in place, never down. Or a time-to-maintain model: the unit is down meanwhile, a *planned outage*. | +| `cost` | Charged at each preventive replacement: a number or a distribution (under `"calendar"`, none: the calendar's price is charged). | + +A replacement renews the unit, so the failure it was heading for never +happens. A failure due at the same time as a replacement comes first. + +`NonRepairable.find_optimal_replacement()` finds a component's best +replacement age on its own ([Maintenance policies](maintenance.md)). At +system level the answer changes, because a planned stop of a unit with a +standby costs almost no production, while the same stop of a single point of +failure halts it. Take a pump that wears out, is repaired in about 23 h and +replaced preventively in about 7 h, alone or with a standby, with lost +production at 500 per hour: + +```python +import numpy as np +import surpyval as surv +from repyability import BetaFactor, CCFGroup, RepairableRBD + +def pump(interval, policy="age"): + return { + "reliability": surv.Weibull.from_params([1000, 2.5]), # MTTF 887 h + "repairability": surv.LogNormal.from_params([3.0, 0.5]), # 23 h + "replace_cost": 5000.0, + "preventive": { + "interval": interval, + "policy": policy, + "duration": surv.Weibull.from_params([8, 3]), # 7 h + "cost": 1000.0, + }, + } + +def alone(interval, policy="age"): + return RepairableRBD( + [("s", "p"), ("p", "t")], + {"p": pump(interval, policy)}, + downtime_cost_rate=500.0, + ) + +def with_standby(interval, policy="age"): + return RepairableRBD( + [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], + {"a": pump(interval, policy), "b": pump(interval, policy)}, + downtime_cost_rate=500.0, + ) + +alone(float("inf")).expected_cost_rate() # -> 18.0 run to failure +alone(580).expected_cost_rate() # -> 13.14 the best interval +with_standby(float("inf")).expected_cost_rate() # -> 11.3 +with_standby(500).expected_cost_rate() # -> 6.985 the best interval +``` + +| Interval (h) | 300 | 400 | 500 | 600 | 800 | never | +|---|---|---|---|---|---|---| +| Alone | 16.92 | 14.36 | 13.35 | 13.14 | 13.84 | 18.00 | +| With a standby | 8.19 | 7.21 | 6.99 | 7.15 | 8.01 | 11.30 | + +Replacement pays in both, but not at the same interval: the lone pump's +planned stops cost production, so it is best replaced less often, at about +580 h; replacing a pump with a standby costs almost no production, so the +pair is best replaced at about 500 h. Judged on its own, as `NonRepairable` +judges it (`cp=1000`, `cu=5000`), the pump is best replaced at 493 h. + +### Choosing the intervals + +`optimal_replacement_intervals` finds the best interval of every component +under age replacement at once, from the exact long-run values, rather than +sweeping them one at a time. The components are chosen together: the one +alone in the line is replaced later than the ones with a standby, because +its own replacements stop the plant: + +```python +alone(1000).optimal_replacement_intervals().intervals["p"] # -> 589.6 +plan = with_standby(1000).optimal_replacement_intervals() +plan.intervals["a"] # -> 497.1 and the same for "b" +plan.cost_rate # -> 6.985 +``` + +It can also keep the system's long-run availability to a target at the least +cost (`min_availability`), or give the most availability within a cost rate +(`max_cost_rate`): + +```python +plan = alone(1000).optimal_replacement_intervals(min_availability=0.9807) +plan.intervals["p"] # -> 604.8 a little later than the cheapest +plan.cost_rate # -> 13.139 against 13.134 +``` + +It returns a [`MaintenancePlan`][repyability.MaintenancePlan]: the intervals +(`inf` for a component better never replaced), and the system's cost rate +and availability with them. The search starts from the intervals given, and +from others around each component's mean life, and keeps the best; a target +no intervals can meet raises `ValueError`, with the best they can do. The +cost rate is usually flat near its minimum, so an interval some way from the +one found costs almost the same. + +Replacements are often made on a calendar. `allowed` gives the intervals to +choose from (#230), one list for every component or a dict of a list each +(`inf` among them for never), and every combination is tried, as +`optimal_inspection_intervals` chooses tests': + +```python +weeks = [336.0, 672.0, 1008.0, 1344.0] # every 2, 4, 6 or 8 weeks +plan = with_standby(1000).optimal_replacement_intervals(allowed=weeks) +plan.intervals # {'a': 672.0, 'b': 672.0} +plan.cost_rate # -> 7.405 against 6.985 at 497 h +``` + +The plan is the best for the long run. A new plant's units all start new, +so the first replacements of redundant units fall due together, and where +they take the system down together its first years cost more than the +plan's rate: `expected_cost` gives the cost from new, and from units of +other ages (`state`) the cost of a staggered start. + +With limited `repair_crews` a component can wait for a crew, and the exact +long-run values the search uses no longer hold, so the choice is refused. +`assume_unlimited_crews=True` (#184) chooses the intervals as if every repair +started at once, and `with_intervals(plan)` gives the diagram with them, to +simulate with its crews and see what the waiting costs: + +```python +crewed = RepairableRBD( + [("s", "a"), ("a", "b1"), ("a", "b2"), ("b1", "t"), ("b2", "t")], + {n: pump(1000) for n in ("a", "b1", "b2")}, + downtime_cost_rate=500.0, repair_crews=1, +) +plan = crewed.optimal_replacement_intervals(assume_unlimited_crews=True) +planned = crewed.with_intervals(plan) # the same crew, the plan's intervals +planned.cost(100_000.0, mc_samples=100, seed=1).mean / 100_000.0 # ~> 20.6 simulated +plan.cost_rate # -> 20.12 with a crew for every job +``` + +`with_intervals` takes a plan or a dict of intervals (and of test offsets), +and builds the copy as the diagram was built; `compare()` then sets the plan +against the schedules the diagram has. + +`expected_cost_rate` prices an age-replaced component through its renewal +cycle: it ends at a failure or a preventive replacement, whichever comes +first, with mean length `C = ∫₀ᵀ R + F(T)·MTTR + R(T)·MTTP` (`MTTP` the mean +maintenance time), so it fails `F(T) / C` times and is replaced `R(T) / C` +times per unit time, and is up `∫₀ᵀ R / C` of the time: + +``` +cost rate = … + Σ R_i(T_i) / C_i · preventive cost_i preventive actions +``` + +With instant repair and maintenance a component's own cost rate is +`NonRepairable.cost_rate(T)`. `mean_availability`, `node_availability`, the +frequencies, MUT, MDT and the importance measures account for it the same +way. + +Under block replacement the renewals are the block times at which the unit is +up: it is replaced there, and a replacement due while it is down is skipped. +Between two of them it fails and is repaired as usual, and a repair can run +over a block time. `expected_cost_rate` and the other long-run methods compute +its mean up time, failures and length between renewals numerically, from the +renewal equations of its lives and repairs within an interval, to about one +part in a million. With instant repair and replacement the cost rate is +`(c_p + c_u · M(T)) / T`, `M` the renewal function of the lives. Components +replaced at the same block times go down together, so the system's long-run +values average over the block interval (over the time the schedules take to +repeat together, for different intervals), rather than combining each +component's own average: + +```python +alone(580, "block").expected_cost_rate() # -> 14.06 against 13.14 for age +with_standby(580, "block").mean_availability() # -> 0.9901 both replaced at once +with_standby(580).mean_availability() # -> 0.9996 +``` + +The exact block-replacement values need a surpyval parametric lifetime with a +density (no units dead on arrival), and repairs that always end; otherwise +they raise `NotImplementedError`, and the simulation still applies. + +The simulation prices both policies. A replacement's cost is in +`by_category["preventive"]` (the expected cost, exact here, as the cost's +`mean` is), and a planned outage counts as downtime, but not as a failure: +`system_planned_outages` counts the times one took the system down in the +simulations. + +```python +year = alone(580).availability(t_simulation=8760.0, mc_samples=500, seed=0) +year.system_failures / year.n_simulations # -> 3.466 +year.system_planned_outages / year.n_simulations # -> 11.95 +year.cost.by_category["preventive"] # -> 11886.8 1000 each, expected +``` + +### Replacement on a calendar policy + +A part whose replacement costs differently at different times (cheap in a +shut, dearer on any other day) is best replaced on a threshold for each +time: the policy [`NonRepairable.optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy] +works out (see [Maintenance policies](maintenance.md#replacement-over-a-calendar-of-shuts-nonrepairable)). +Give it to the component, and the simulation replaces each unit at the first +time its age reaches the threshold, charging the calendar's price there; a +failure is charged the component's own `replace_cost`: + +```python +from repyability import Calendar, NonRepairable, Opportunity + +life = surv.Weibull.from_params([1000.0, 2.5]) +plant = Calendar( + [Opportunity("quarterly", 2.0, period=91.0), Opportunity("annual", 1.0, period=364.0)], + failure=20.0, anytime=4.0, +) +policy = NonRepairable(life).optimal_calendar_policy(plant) +rbd = RepairableRBD( + [("s", "bearing"), ("bearing", "t")], + {"bearing": { + "reliability": life, "repairability": "instant", "replace_cost": 20.0, + "preventive": {"policy": "calendar", "calendar": policy}, + }}, +) +rbd.cost(t_simulation=3640.0, mc_samples=500) # simulated, in Python +``` + +Only the simulations take it (in the Python loop): the exact and numerical +analyses refuse a component on a calendar policy, as its replacements depend +on when each unit was put in, and a diagram with one is not saved. In a +maintenance group (with a `"group"`, and an `"opportunity"` age of any size) +it is a member like any other: see +[Opportunistic maintenance](#opportunistic-maintenance). + +### Replacement on condition + +`"policy": "condition"` inspects the unit at every multiple of `interval`, +while it is up, and replaces it only if it is then more likely than +`"threshold"` to fail before the next inspection, given its age `a`: +`1 − R(a + T) / R(a)`, the conditional survival that `NodeState` and +`sf_given_state` use. A unit renewed a while ago is left alone, and a worn +one is replaced at the last inspection before it is likely to fail. + +| Key | Meaning | +|---|---| +| `threshold` | Required with `"condition"`: the probability, in `[0, 1]`, above which an inspection replaces the unit. | +| `inspection_cost` | Charged at each inspection: a number or a distribution. | + +`duration` and `cost` are the replacement's, as under the other policies. +An inspection takes no time, and one due while the unit is down is skipped. +Inspected weekly and replaced when more than 20% likely to fail before the +next inspection, the lone pump above costs about what the best age +replacement does: + +```python +weekly = {"interval": 168.0, "policy": "condition", "threshold": 0.2, + "duration": surv.Weibull.from_params([8, 3]), "cost": 1000.0, + "inspection_cost": 20.0} +inspected = RepairableRBD([("s", "p"), ("p", "t")], + {"p": dict(pump(580), preventive=weekly)}, + downtime_cost_rate=500.0) +inspected.expected_cost_rate() # -> 13.37 per hour +run = inspected.cost(200_000.0, mc_samples=20, seed=1) +run.cost_rate # -> 13.36 exact over the window +run.sample_mean / 200_000.0 # -> 13.27 the 20 simulations', ± 0.2 +``` + +| Threshold | 0.05 | 0.1 | 0.15 | 0.2 | 0.25 | 0.3 | +|---|---|---|---|---|---|---| +| Cost per hour | 22.10 | 15.98 | 13.47 | 13.37 | 14.02 | 14.34 | + +Judged on its age alone, a unit replaced on condition is replaced much as +under age replacement (13.14 at the best age), but on the inspection +calendar. A low threshold replaces it too young; a high one lets it fail. + +- **A threshold of 0** replaces the unit at every inspection: block + replacement at the interval. **A threshold of 1** never does: run to + failure, with inspections. +- **A constant failure rate** gives the same probability at every age, + `1 − exp(−λT)`: the unit is replaced at every inspection or at none, and + it fails as often either way. +- **Numerical in the long run** (#145). The inspections that replace the + unit are regeneration points, and a cycle between two is followed one + inspection interval at a time: the units put into service in an interval + as an alternating renewal process, as under block replacement, and the + units an inspection keeps on into the next interval by their ages. So the + long-run values, the cost rate (the inspections charged at the unit's + availability just before them) and the importance measures are + numerical, to about `1e-7`. A cycle in which the unit fails many times + before an inspection replaces it is summed to its end as a geometric + series once it falls at a steady rate. +- **Numerical over time** (#161). The same recursion, followed from new + over all the intervals rather than over one cycle, gives the availability + at any time and the expected events and cost of any window. Each + interval starts with what the inspection at its start replaced, the + repairs carried into it and the units it kept, by age, and after some + intervals it repeats from one to the next: its long-run cycle. From a + state, the unit in service at the start is decided on at its own age at + each inspection, however that falls on the grid. + +A year of the weekly-inspected pump from new, and the next 30 days of one +already 600 hours old, 100 hours after its last inspection: + +```python +from repyability import NodeState + +inspected.expected_cost(8760.0).total # -> 114491 new: less than a year at the rate, 117139 +worn = {"p": NodeState(age=600.0, since_scheduled=100.0)} +inspected.expected_cost(720.0, state=worn).total # -> 10265 against 8517 new +``` + +#### Replacing on the measured level + +A unit whose life is a degradation process fitted in surpyval +(`surpyval.WienerProcess.fit` or `surpyval.GammaProcess.fit`, with its +failure `threshold`) can be replaced on what an inspection measures rather +than on its age (#271): `"level"` in place of `"threshold"`. Each +inspection reads the unit's level, and replaces the unit if the level is at +or past `"level"`; a unit kept goes on from the level found. + +| Key | Meaning | +|---|---| +| `level` | With `"condition"`, in place of `threshold`: the degradation level at or past which an inspection replaces the unit, below the process's failure `threshold`. The life must be a surpyval Wiener or gamma process. | + +```python +g = np.random.default_rng(7) +hours = np.tile(np.arange(1.0, 11.0) * 30.0, 12) # 12 units, 10 readings +units = np.repeat(np.arange(12), 10) +worn = np.concatenate([np.cumsum(g.gamma(2.0, 0.25, 10)) for _ in range(12)]) +wear = surv.GammaProcess.fit(hours, worn, units, threshold=8.0) + +def liner(level): + preventive = {"interval": 60.0, "policy": "condition", "level": level, + "cost": 5.0, "inspection_cost": 0.5} + spec = {"reliability": wear, "repair_cost": 50.0, "preventive": preventive, + "repairability": surv.Exponential.from_params([1 / 20])} + return RepairableRBD([("s", "a"), ("a", "t")], {"a": spec}) + +liner(6.0).cost(5000.0, mc_samples=400, seed=1).mean # -> 95.06 +``` + +Run to failure, the liner costs about 402 over the 5000 hours. Replaced at +a level of 7.5, 7, 6 or 5 it costs about 250, 133, 95 and 103: a lower +level means fewer failures but more replacements. + +- **By simulation only.** The level a unit is found at depends on the + levels before it, which only the simulation follows: the exact and + numerical methods (the long run, over time, the importance measures) + refuse such a component, and `availability()` and `cost()` simulate it, + on the Python engine. A start state cannot give the level either, so a + run from one refuses it. +- **How a level is drawn.** Between inspections a unit's failure is drawn + from the level last measured (surpyval's first-passage quantile from it). + If that failure falls before the next inspection, the unit fails then. + Otherwise the inspection's level is drawn given that the unit has not + failed by then: for a gamma process, a gamma increment truncated at the + threshold; for a Wiener process, the path's level killed at the threshold + (the method of images). surpyval does not give this level yet + (SurPyval#836), so RePyability works it out from the fitted parameters. +- **Without a level**, a degradation process is a life like any other, its + first-passage time to the threshold, and every exact method takes it. + +### Opportunistic maintenance + +Plants group their work: once a unit is down, for a failure or its planned +replacement, the crew services its neighbours too, as the line is stopped +anyway. Components that share such stops form a *maintenance group*, named +by their `"group"` key. Each failure of a member, and each scheduled +replacement, opens a *stop* of its group, at which every other member that +is working and at least its `"opportunity"` age is replaced as well, as +its own scheduled replacement would replace it: + +| Key | Meaning | +|---|---| +| `"group"` (component) | The component's maintenance group: any hashable name. | +| `opportunity` (`"preventive"`, age or calendar policy) | The age, from 0 to `interval` (any age under a calendar policy), from which the unit is replaced early at a stop of its group. Left out, it never is. | +| `setup_cost` (`maintenance_groups`) | Charged once per stop, however many members it renews. | +| `system_down` (`maintenance_groups`) | `True`: every outage of the system is a stop of the group as well. | + +A compressor and its motor in series wear out, and each stop of the train +costs 3,000 to set up (isolation, permits, scaffolding) besides the units' +own costs. Replaced separately at 600 h, each failure or replacement is a +stop of its own; from age 300 h, a unit is replaced at the other's stop: + +```python +def unit(scale, opportunity=None): + preventive = {"interval": 600.0, + "duration": surv.Weibull.from_params([8, 3]), # 7 h + "cost": 1000.0} + if opportunity is not None: + preventive["opportunity"] = opportunity + return { + "reliability": surv.Weibull.from_params([scale, 2.5]), + "repairability": surv.LogNormal.from_params([3.0, 0.5]), # 23 h + "replace_cost": 5000.0, + "preventive": preventive, + "group": "train", + } + +def train(opportunity=None): + return RepairableRBD( + [("s", "compressor"), ("compressor", "motor"), ("motor", "t")], + {"compressor": unit(1000, opportunity), + "motor": unit(1500, opportunity)}, + downtime_cost_rate=500.0, + maintenance_groups={"train": {"setup_cost": 3000.0}}, + ) + +train().expected_cost_rate() # -> 33.00 separate stops, exact +run = train(300).availability(200_000.0, mc_samples=20, seed=1) +run.cost.cost_rate # -> 23.48 grouped, simulated +run.opportunistic_renewals # -> {'compressor': 3337, 'motor': 3502} +``` + +| Opportunity age (h) | none | 500 | 400 | 300 | 200 | 100 | +|---|---|---|---|---|---|---| +| Cost per hour (± 0.15) | 33.00 | 29.99 | 26.82 | 23.48 | 23.20 | 23.18 | + +Grouping pays twice: the stops, and their set-ups, are fewer (5.64 an hour +of set-ups against 10.33), and the units' outages overlap, so the train is +down less (12.0 an hour of lost production against 16.7). + +- **A stop is an instant.** Work started at one instant shares one set-up, + and a member whose own failure or replacement falls due then keeps it, as + part of the stop: two units on the same schedule are replaced together, + on schedule, at one set-up. +- **Durations overlap.** Each member renewed at a stop takes its own + maintenance time, from the stop, so the set-up time each one's + maintenance includes is spent once in a series train. +- **Results.** `opportunistic_renewals` counts each component's early + renewals, summed over the simulations; the set-ups are under + `by_category["setup"]`, and each early renewal is charged the member's + preventive cost and uses a spare. +- **A component on a [calendar policy](#replacement-on-a-calendar-policy) + is a member like any other.** Each of its planned replacements is a stop + of its group, and at a stop it is renewed early from its `opportunity` + age, charged the calendar's price for a planned replacement on that day + (the opportunity's if one falls then, else the price on any day; where + the calendar allows none, its own `replace_cost`, as a failure is); the + new unit's next planned replacement then counts from that day. The + policy is the one worked out for the part alone: it is not re-optimised + with the group's stops as opportunities, so the thresholds assume the + calendar's prices, and the group's stops are extra chances it did not + plan for. +- **Exact or simulated.** A component that can be renewed early depends on + the others, so the exact long-run values, the availability over time and + the spares counts refuse it; `availability`, `cost` and `compare` + simulate it, in Python. With no `opportunity` below an interval, the + set-up cost is exact: `expected_cost_rate` charges a set-up at each + failure and each preventive replacement of a member (so `opportunity` + equal to the interval is plain age replacement, plus set-ups). It + refuses two members replaced on a clock (block replacement, or never + failing before an instant age replacement), whose replacements share + stops. +- Members cannot have hidden failures or be standby groups. + +## Hidden failures and inspection + +Some failures announce themselves: a running pump stops, and the operators +know. Others do not: a relief valve that has seized, a standby pump that will +not start or a trip that no longer trips looks just like a working one until +something tests it. Such a failure is *hidden* (or *unrevealed*): the +component is down, nobody knows, and it stays down until a periodic +*inspection* (a proof test) finds it. The key `"inspection"` in a +component's dict makes its failures hidden: + +| Key | Meaning | +|---|---| +| `interval` | Required: the time `τ` between inspections, positive and finite. The component is inspected at `τ, 2τ, 3τ, …`, or from its `offset`. | +| `duration` | `"instant"` (the default): the test takes no time. Or a time-to-test model: the component is off-line while it is tested (a planned outage), and does not age meanwhile. | +| `cost` | Charged at each inspection: a number or a distribution. | +| `offset` | The time of the first test, from 0 to less than `τ`: tests at `offset, offset + τ, …`, so that redundant components can be tested apart (staggered). By default 0, no offset: the first test at `τ`, none at the start. A positive offset puts one there, so an offset just above 0 adds a test (and its outage and cost) at the start; one within a billionth of `τ` of 0 is taken as 0. | +| `coverage` | The chance that a test finds a failure, from 0 to 1 (by default 1): its *proof-test coverage*. A failure a test misses stays hidden until a full test. | +| `full_test` | With a coverage below 1, required: the time between full tests, which find every failure, a whole multiple of `τ` (the tests at `offset` and every `full_test` after it). Often the mission time, after which the component is renewed. | + +A failure found by an inspection is repaired once the test is done (taking a +time drawn from the component's `"repairability"`), and its repair and +replace costs are charged when it is found. An inspection due while the +component is being repaired is skipped. The time a failure lies hidden counts +as downtime in every output, and each inspection's cost is in +`by_category["inspection"]`. A component can have an inspection or a +preventive schedule, but not both. + +### The average probability of failure on demand + +A protective function whose failure is hidden does not act when it is +needed, so its long-run unavailability is its *average probability of +failure on demand*, the PFDavg that safety-integrity (SIL) verification asks +for (IEC 61508 and 61511). With a constant rate `λ` of such failures, +inspected every `τ`, with instant tests and repairs, a single channel is +down, after a failure, until the next test: half an interval on average, so +`PFDavg = 1 − (1 − e^(−λτ)) / (λτ) ≈ λτ/2`. Take a shutdown valve with +dangerous undetected failures at `2 × 10⁻⁶` per hour, proof-tested yearly: + +```python +def valve(interval): + return { + "reliability": surv.Exponential.from_params([2e-6]), # dangerous, undetected + "repairability": "instant", + "inspection": {"interval": interval}, + } + +one = RepairableRBD([("s", "v"), ("v", "t")], {"v": valve(8760.0)}) +one.mean_unavailability() # -> 0.008709 about λτ/2 = 0.00876 +pair = RepairableRBD( + [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], + {"v1": valve(8760.0), "v2": valve(8760.0)}, +) +pair.mean_unavailability() # -> 1.0098e-4 about (λτ)²/3 +``` + +Two valves in parallel (1oo2), tested together, are down only when both +have failed since the last test: `PFDavg = (1/τ) ∫₀^τ (1 − e^(−λt))² dt ≈ +(λτ)²/3`. That is a third more than the `(λτ/2)²` of two channels that fail +independently in time, because both have gone untested for the same time. +The diagram supplies the voting, so 2oo3 and larger architectures follow in +the same way (see [k-out-of-n nodes](building.md#k-out-of-n-nodes)). Halving +the interval halves a single channel's PFDavg but quarters the pair's: + +| Test interval | 6 months | 1 year | 2 years | +|---|---|---|---| +| One valve | 0.00437 | 0.00871 | 0.0173 | +| Two valves (1oo2) | 2.54 × 10⁻⁵ | 1.01 × 10⁻⁴ | 3.99 × 10⁻⁴ | + +These long-run values are exact. Components inspected at the same times go +down together, so `mean_availability` and the other long-run methods average +the system's availability over one period of the inspection schedules (the +least common multiple of the intervals: components on different intervals +can be mixed), and the importance measures are ratios of those averages. +With instant tests and repairs they are in closed form for a constant +failure rate, and numerical for any other life (see [a life that wears +out](#a-life-that-wears-out)); tests and repairs that take time are +numerical too (see [tests and repairs that take +time](#tests-and-repairs-that-take-time)). Testing both valves at once, for +example, takes the whole function off-line during the test, which here +costs far more than the hidden failures: + +```python +def tested(interval, offset=0.0): + return { + "reliability": surv.Exponential.from_params([2e-6]), + "repairability": surv.LogNormal.from_params([np.log(24), 0.5]), # about a day + "inspection": { + "interval": interval, + "duration": surv.Weibull.from_params([4, 3]), # about 3.6 h + "offset": offset, + }, + } + +pair = [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")] +both = RepairableRBD(pair, {"v1": tested(8760.0), "v2": tested(8760.0)}) +both.mean_unavailability() # -> 4.28e-4 +1 - both.mission_availability(10 * 8760.0) # -> 3.95e-4 its first ten years +both.expected_events(10 * 8760.0).system_planned_outages # -> 9.0 one per test +apart = RepairableRBD(pair, {"v1": tested(8760.0), "v2": tested(8760.0, 4380.0)}) +apart.mean_unavailability() # -> 7.12e-5 +``` + +Tested half a year apart, a test takes one valve off-line while the other +holds the function, and the PFDavg falls six-fold. + +### A life that wears out + +The closed forms above need a constant failure rate. With any other life, +and instant tests and repairs, the values are numerical, exact to rounding. +A failed unit is found and renewed at the next test, so every renewal falls +on a test, and a unit renewed `m` intervals before a test is still up with +probability `R(mτ)`. In the long run a cycle lasts `S = 1 + Σ R(mτ)` +intervals (over `m ≥ 1`), and the unit is up a time `u` after a test with +probability `(R(u) + Σ R(mτ + u)) / S`. From new, the chance of a renewal +at each test follows a renewal equation over the tests, and settles to +`1/S`. A valve that wears out, with a mean life of about 15 years: + +```python +worn = surv.Weibull.from_params([150_000.0, 2.5]) # wears out; MTTF 133,090 h +wearing = RepairableRBD( + [("s", "v"), ("v", "t")], + {"v": { + "reliability": worn, + "repairability": "instant", + "inspection": {"interval": 8760.0}, + }}, +) +wearing.mean_unavailability() # -> 0.03186 about τ / (2 MTTF) = 0.0329 +1 - wearing.mission_availability(87600.0) # -> 0.01126 its first ten years +years = np.array([1, 5, 10, 20]) * 8760.0 - 1.0 # just before a test +1 - wearing.point_availability(years) +# -> [0.00082 0.01908 0.04983 0.06844] +``` + +For a short interval the long-run PFDavg is about `τ / (2 MTTF)`, whatever +the life's shape. What the wear changes is the PFD over time. A new valve +rarely fails in its first years, so its first ten years' PFD is a third of +the long run's, and the PFD just before each test climbs with its age +before it settles. The values over a window, from new or from the valve's +current state (its age, and the time since its last test), are exact too: +`expected_events`, `expected_cost` and the other [analyses over +time](repairable.md#availability-over-time-exact). + +The sums run over the intervals a unit can last. Within an interval, every +unit but the one renewed at the last test ages smoothly, so their sum is +worked out once, at Chebyshev points across the interval, and interpolated +between them, to rounding. A term that bends there (a life with a +threshold, say) is summed directly instead. A life that lasts more than +200,000 intervals (the most summed) refuses. + +### Tests and repairs that take time + +A test of a working unit takes it off-line (a planned outage) for the +test's time, during which it does not age; a failure found by a test is +repaired once the test is over, and the tests that fall in the repair are +not done. Then renewals no longer fall on the tests, and a test that can +miss a failure (see [test coverage](#common-cause-staggered-tests-and-test-coverage)) +leaves the failure for a later one. The values are still numerical, for +any life (#159). The tests that find failures are the unit's regeneration +points: from one, a new unit is put into service the test's time plus the +repair's later, and its age at its later tests is a random walk (each test +holds it back by the test's time), followed test by test on a grid of +2,000 steps or more an interval. The long run follows by renewal-reward +over that cycle (over the cycles from each place in the full tests' +period, with tests that miss), and from new, or from a state, the tests +that find failures are a renewal process on the tests. When a test is +over, and when a unit is back in service, are kept out of the grid, exact +at any time; the rest is exact to the square of the grid's step, about +1e-8. An exponential life needs no ages: its chain of states from test to +test (working, failed, missed, in a repair) is followed instead. + +```python +def valve(coverage=1.0): + inspection = { + "interval": 8760.0, + "duration": surv.ExactEventTime.from_params([8.0]), # 8 h off-line + } + if coverage < 1.0: + inspection.update(coverage=coverage, full_test=5 * 8760.0) + return RepairableRBD( + [("s", "v"), ("v", "t")], + {"v": { + "reliability": surv.Weibull.from_params([150_000.0, 2.5]), + "repairability": surv.ExactEventTime.from_params([24.0]), # a day + "inspection": inspection, + }}, + ) + +valve().mean_unavailability() # -> 0.03289 0.03186 with tests and repairs in no time +valve(0.7).mean_unavailability() # -> 0.06845 70% coverage, a full test every 5 years +valve(0.7).expected_events(10 * 8760.0).system_planned_outages # -> 8.750 +``` + +`spares_demand` and `spares_stock` count such a valve's spares on the +tests that find its failures (see [how spares are +counted](spares.md#how-it-is-computed)). Only a test that can last as long +as its interval stays simulated. A common-cause group's members' tests and +repairs may take time too, of a fixed length or an exponential one (see +[below](#common-cause-staggered-tests-and-test-coverage)). + +### Common cause, staggered tests and test coverage + +Three more terms decide a real safety function's PFDavg, and each has its +place in the diagram: + +- **Common cause.** Redundant valves of one design, in one service, fail + together more often than chance allows. Give the diagram `ccf_groups`, as + for a [non-repairable one](common-cause.md): a `BetaFactor(β)` makes a + share `β` of each valve's failures a shared cause that fails both at + once. For a redundant function the shared term usually dominates: about + `βλτ/2`, five times the pair's independent `(λτ)²/3` here. +- **Staggered tests.** An `"offset"` tests one valve half an interval after + the other: a shared failure is then found by whichever test comes first, + and the independent term falls from `(λτ)²/3` to about `5(λτ)²/24`. + Each valve is restored by its own test alone: the first test restores + its valve, and with it the pair, and the other stays down until its own + test (where practice has the first test find and restore both, the + value is a little less, the pair then not left on one valve until the + second test). +- **Test coverage.** A proof test that finds only a share `c` of the + failures (a `"coverage"`) leaves the rest hidden until a full test + (`"full_test"`), say the ten-year overhaul: about `(1 − c)λT/2` more, + `T` the full test's interval. + +```python +def proof_tested(**inspection): + return { + "reliability": surv.Exponential.from_params([2e-6]), + "repairability": "instant", + "inspection": {"interval": 8760.0, **inspection}, + } + +redundant_edges = [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")] +common = [CCFGroup(["v1", "v2"], BetaFactor(0.05))] +shared = RepairableRBD( + redundant_edges, + {"v1": proof_tested(), "v2": proof_tested()}, + ccf_groups=common, +) +shared.mean_unavailability() # -> 5.290e-4 (λτ)²/3 + βλτ/2, about +staggered = RepairableRBD( + redundant_edges, + {"v1": proof_tested(), "v2": proof_tested(offset=4380.0)}, + ccf_groups=common, +) +staggered.mean_unavailability() # -> 2.779e-4 the shared term halved +partial = RepairableRBD( + [("s", "v"), ("v", "t")], + {"v": proof_tested(coverage=0.9, full_test=87600.0)}, +) +partial.mean_unavailability() # -> 0.01642 0.9λτ/2 + 0.1λT/2, about +``` + +All three are exact with constant failure rates and instant tests and +repairs, and staggered tests and test coverage are numerical with any life +too, and with tests and repairs that take time (see [tests and repairs that +take time](#tests-and-repairs-that-take-time)). With common causes, a +group's members are a Markov chain of which of them are down, with each +member found by its own tests, and a shared failure found alike by every +test (the coverage is the group's), each member it struck restored by its +own test, as in the simulations (#237). Their tests and repairs may take no +time, a fixed time or an exponential one (#220): a working member is off +line while it is tested, and neither ages nor fails meanwhile, a failure a +test finds is repaired once the test is over, and a test that falls in a +member's own test or repair is not done, as in the simulations. IEC +61508-6's 1oo2 with an eight-hour mean repair time: + +```python +repaired = {**proof_tested(), "repairability": surv.ExactEventTime.from_params([8.0])} +mrt = RepairableRBD( + redundant_edges, {"v1": repaired, "v2": repaired}, ccf_groups=common +) +mrt.mean_unavailability() # -> 5.300e-4 IEC 61508-6: 5.316e-4 +``` + +A test of an exponential length followed by a repair of a fixed one, or +fixed lengths that run into the member's next test, are refused, saying +so: estimate those by simulation. +The importance measures take the groups in, a member's conditioned on its +state at each time, and so do the allocations, the values over time from +new (the chain followed through the tests from every member up) and the +simulations, which draw the shared causes; see [repairable +systems](common-cause.md#repairable-systems) and `analysis_routes()`. + +### Choosing the interval + +Each inspection costs `c_i`, and each unit of time the component lies failed +costs `c_d` (its `"downtime_cost"`, or the system's `downtime_cost_rate`). +Frequent tests cost more, and rare ones leave failures hidden for longer, so +the cost rate `c_i/τ + c_d·U(τ)` is least in between, near +`τ* = √(2c_i / (λc_d))`. `expected_cost_rate` prices it exactly: + +```python +def pump(interval): + return RepairableRBD( + [("s", "p"), ("p", "t")], + {"p": { + "reliability": surv.Exponential.from_params([1e-4]), + "repairability": "instant", + "downtime_cost": 500.0, # per hour failed + "inspection": {"interval": interval, "cost": 2000.0}, + }}, + ) + +pump(100.0).expected_cost_rate() # -> 22.49 testing too often +pump(283.0).expected_cost_rate() # -> 14.08 near √(2 × 2000 / (1e-4 × 500)) = 283 +pump(1000.0).expected_cost_rate() # -> 26.19 failures hidden too long +``` + +`optimal_inspection_intervals` finds it, and with a target, the cheapest +intervals that meet it. Tests are made on a calendar, and components tested +at the same times are down together, so with several components the +intervals are chosen from those allowed (`allowed`): every combination when +there are at most 2000, a local search otherwise. With one inspected +component, any interval can be chosen: + +```python +pump(100.0).optimal_inspection_intervals().intervals["p"] # -> 285.5 the formula gives 283 +``` + +For a safety function, `min_availability=1 - PFDavg target` gives the +cheapest tests that meet the target. With the valves above, each test costing +500, and tests monthly, quarterly, half-yearly, yearly or every two years: + +```python +def priced_valve(): + return { + "reliability": surv.Exponential.from_params([2e-6]), + "repairability": "instant", + "inspection": {"interval": 8760.0, "cost": 500.0}, + } + +calendar = [730.0, 2190.0, 4380.0, 8760.0, 17520.0] # hours +single = RepairableRBD([("s", "v"), ("v", "t")], {"v": priced_valve()}) +single.optimal_inspection_intervals( + allowed=calendar, min_availability=1 - 1e-3 +).intervals["v"] # -> 730.0 monthly +redundant = RepairableRBD( + [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], + {"v1": priced_valve(), "v2": priced_valve()}, +) +redundant.optimal_inspection_intervals( + allowed=calendar, min_availability=1 - 1e-3 +).intervals["v1"] # -> 17520.0 every two years +``` + +Redundancy relaxes the tests: one valve needs them monthly to keep the +PFDavg at most `10⁻³`, two in parallel every two years. The result is a +[`MaintenancePlan`][repyability.MaintenancePlan], as for +[age-replacement intervals](#choosing-the-intervals). + +When the valves are tested matters as well. Tested together, both are down +for as long as a common-cause failure stays hidden; tested half an interval +apart, it is found twice as soon (and the valve it found restored, the +other waiting for its own test). `offsets` chooses the times of the first +tests with the intervals (#184), as shares of each interval in [0, 1): a +list for every component, a dict of one per component, or `"stagger"`, the +shares `0, 1/n, …, (n − 1)/n` of `n` components, among which are tests of +one interval spread evenly over it. With a 10% common cause and a PFDavg of +at most `5 × 10⁻⁴`: + +```python +paired = RepairableRBD( + [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], + {"v1": priced_valve(), "v2": priced_valve()}, + ccf_groups=[CCFGroup(["v1", "v2"], BetaFactor(0.1))], +) +together = paired.optimal_inspection_intervals( + allowed=calendar, min_availability=1 - 5e-4) +together.intervals # {'v1': 4380.0, 'v2': 8760.0} +together.cost_rate # -> 0.1712 +apart = paired.optimal_inspection_intervals( + allowed=calendar, min_availability=1 - 5e-4, offset_shares="stagger") +apart.intervals # {'v1': 8760.0, 'v2': 8760.0} +apart.offsets # {'v1': 0.0, 'v2': 4380.0} six months apart +apart.cost_rate # -> 0.1142 a third less +``` + +Shifting every test by one time changes nothing in the long run, so the +first component's tests stay from 0, unless other tested components keep +their schedules (then its offset is chosen too). Offsets change no cost, so +of plans that cost the same the most available is chosen. The +`offset_shares` searched are shares of the interval; the result's +`offsets` are the times of the first tests, as `with_intervals(offsets=)` +takes them, and a plan has them whether they were searched or not. diff --git a/docs/guide/timelines.md b/docs/guide/timelines.md index 4b915721..e15cb361 100644 --- a/docs/guide/timelines.md +++ b/docs/guide/timelines.md @@ -203,8 +203,12 @@ bool(TimelineSimulation.join(halves).system == year) # True `Timelines` of one unit join the same way: `Timelines([first, second])`. Each history keeps every change, about 25 bytes of memory each: for a -summary of a long run, `availability` keeps less. A diagram with -common-cause groups is refused, as the simulations refuse it. +summary of a long run, `availability` keeps less. With common-cause +groups, a shared cause fails the members it strikes one after another at +one instant, each a failure of the member's own: a system failure it makes +is the member's whose fall took the system down (the last the structure +could spare), not the group's, and `failures_by_cause()` counts it under +that member. ### From the plant as it is now diff --git a/docs/index.md b/docs/index.md index 65bcbf84..2604ceb7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -16,7 +16,9 @@ Scope notes: fit your failure data in surpyval and pass the models in. - **Visualisation lives in the Reliafy app, not here.** RePyability returns numbers and typed result objects; plotting and dashboards are a separate - layer. + layer. [Reliafy](https://reliafy.com) is a hosted, browser-based app built + on RePyability and SurPyval, for building reliability block diagrams and + analysing their reliability and availability without writing code. ## Install @@ -52,11 +54,13 @@ rbd.mean_time_to_failure() # -> 93.82 mean time to failure, exactly rbd.birnbaum_importance(50) # per-node Birnbaum importance at t = 50 ``` -Minimal path and cut sets are available too: +Minimal path and cut sets are available too, listed smallest first: ```python -rbd.get_min_path_sets(include_in_out_nodes=False) -rbd.get_min_cut_sets() +rbd.minimal_path_sets() +# [frozenset({'pump1', 'valve'}), frozenset({'pump2', 'valve'})] +rbd.minimal_cut_sets() +# [frozenset({'valve'}), frozenset({'pump1', 'pump2'})] ``` You can force nodes working or failed to explore conditional behaviour: @@ -114,7 +118,8 @@ result.criticalities.iou.up # intersection-over-union importance (system up) | Dependence | Beta-factor and Multiple Greek Letter common-cause groups, splitting a probability or a failure rate, in block diagrams, repairable systems and fault trees | [Common-cause failures](guide/common-cause.md) | | Availability | Long-run availability, failure frequency, MUT/MDT/MTBF; exact availability over time and over a mission, and the unavailability to its own precision; simulated histories with criticality measures; shared repair crews, exact for exponential components in the long run and numerical over time; repairable standby groups | [Repairable systems](guide/repairable.md) | | Capacity | The exact distribution of how much a system can deliver, from its components' capacities (with several levels, or degrading through stages over time), at a time or in the long run; the probability of meeting a demand, the expected capacity and the production availability | [System capacity](guide/capacity.md) | -| Cost | Exact long-run cost rate; simulated cost distributions with percentiles; costs drawn from distributions; scheduled preventive maintenance (age or block replacement) priced exactly at system level, and its intervals chosen for a cost or availability target; replacement on condition at periodic inspections, numerical in the long run and over time; hidden failures found by periodic inspection (PFDavg); total cost of ownership, discounted to a present value, and the redundancy that minimises it, whole trains among it | [Costs](guide/costs.md) | +| Cost | Exact long-run cost rate; simulated cost distributions with percentiles; costs drawn from distributions; total cost of ownership, discounted to a present value, and the redundancy that minimises it, whole trains among it | [Costs](guide/costs.md) | +| Maintaining a system | Scheduled preventive maintenance (age or block replacement) priced exactly at system level, and its intervals chosen for a cost or availability target; replacement on condition at periodic inspections, numerical in the long run and over time; opportunistic maintenance; hidden failures found by periodic inspection (PFDavg) | [Maintaining a system](guide/system-maintenance.md) | | Spares | The distribution of each component's replacements over a horizon, for a system or a fleet, and the stock that meets a fill rate or a stock-out target for a replenishment lead time; interchangeable components' spares pooled on one shelf | [Spares](guide/spares.md) | | Design | Optimal redundancy allocation within a budget (of one or several resources) or to a target, with a choice of component types, k-out-of-n nodes and cold standby spares, and the whole cost-reliability trade-off; reliability-redundancy allocation; reliability allocation by equal and ARINC-style apportionment, minimum effort (Albert) and cost-based (Mettas) methods; availability allocation, to MTTF and MTTR targets or the cheapest mix of the two | [Design and allocation](guide/design.md) | | Maintenance | Age replacement; overhaul under minimal or imperfect repair; replace at the *N*-th failure | [Maintenance policies](guide/maintenance.md) | @@ -137,4 +142,4 @@ result.criticalities.iou.up # intersection-over-union importance (system up) - **[Concepts](concepts.md)**: the theory: path and cut sets, the exact engine, choosing an importance measure, conditioning, standby and load sharing, common cause, availability, cost, allocation and maintenance models. -- **[API reference](api.md)**: every public class and method. +- **[API reference](api/index.md)**: every public class and method. diff --git a/docs/learn/availability.md b/docs/learn/availability.md index 16862f34..d155bb56 100644 --- a/docs/learn/availability.md +++ b/docs/learn/availability.md @@ -585,7 +585,7 @@ takes components down on purpose. That downtime counts against availability and ends up periods, but it is not a failure: `system_failure_frequency()` leaves it out, and the simulation counts it separately, in `system_planned_outages`. The guide shows how to schedule it in [Preventive -maintenance](../guide/costs.md#preventive-maintenance). +maintenance](../guide/system-maintenance.md#preventive-maintenance). ## Pitfalls @@ -798,7 +798,9 @@ mean availability over its first 5 hours to within ±0.0005? lost production, over the long run and over a finite window. The user guide's [Repairable systems](../guide/repairable.md) page has every option of `RepairableRBD`, including conditioning on components held working or -broken, and [Concepts](../concepts.md#availability) summarises the theory. +broken, and [Concepts](../concepts.md#availability) says how RePyability +computes each value: the curve's grid, a mission's mean, and the plant as +it is now. To go the other way, from an availability target for this plant to the MTTF or MTTR each pump and the valve needs, see [Availability allocation](../guide/design.md#availability-allocation). diff --git a/docs/learn/costs.md b/docs/learn/costs.md index f1dc4869..3c2bb99a 100644 --- a/docs/learn/costs.md +++ b/docs/learn/costs.md @@ -656,7 +656,7 @@ the mean be overrun, and is 1.138 million the worst case? ```python np.mean(year.samples > year.percentile(90)) # -> 0.1 - np.mean(year.samples > year.mean) # -> 0.5 + np.mean(year.samples > year.mean) # -> 0.48 year.samples.max() # -> 1195381.3 ``` @@ -753,6 +753,6 @@ parallel worth buying (a) as priced, at 1000 per hour of lost production; - [Costs](../guide/costs.md) in the user guide: every pricing option, costs drawn from distributions, instant repair, preventive maintenance and the total cost of ownership. -- [Concepts: Costs](../concepts.md#costs) for the theory in brief, and +- [Concepts: Costs](../concepts.md#costs) for how RePyability computes them, and [Repairable systems](../guide/repairable.md) for the simulation that `cost()` runs. diff --git a/docs/learn/dependence.md b/docs/learn/dependence.md index 47545447..05bc8049 100644 --- a/docs/learn/dependence.md +++ b/docs/learn/dependence.md @@ -302,11 +302,12 @@ How `StandbyModel` gets its answer depends on the case: | Cold, `k=2` or more identical units working together | Numerical: renewal counts of the working positions | | Cold, `k=2`, any units | Numerical: a recursion over the switch-ins | | Warm, one unit working at a time: any lifetimes | Numerical: a recursion over the switch-ins | -| Anything else: warm with `k=2` or more, or `k=3` or more different units working together, cold | Simulation: pass `seed=0` for repeatable results; `sf` then returns a one-element array | +| Anything else: warm with `k=2` or more, or `k=3` or more different units working together, cold | None: `is_simulated` is `True`, its `sf`, `ff`, `cs` and `mean()` raise `NotImplementedError`, and the system's simulations draw its lifetimes | An imperfect switch is supported for cold standby. The [guide](../guide/redundancy-models.md#standby-cold-warm-and-hot) has every -option. +option, and [Concepts](../concepts.md#standby-and-repeated-nodes) says how +each case is computed. !!! note "Wearing-out units gain even more" The running example's pumps wear out (Weibull, with scale 100 h and @@ -359,8 +360,10 @@ group.is_simulated # False: exact for identical exponential units A [`LoadSharingModel`][repyability.LoadSharingModel] takes one fitted AFT model per unit, the total `load` $L$, and `k`, the number of units the group needs; while $s$ units survive, each carries $L/s$. For identical -exponential units the result is exact; otherwise it is simulated (pass -`seed`). +exponential units the result is exact, and for other identical units +numerical. Different units have no exact or numerical reliability +(`is_simulated` is `True`): the system's simulations draw their lifetimes +(see [Concepts](../concepts.md#dependent-failures-load-sharing)). **By hand**, the stage picture applies again. A `RegressionNode` holds a fitted regression model at a fixed load, so it gives one unit's mean life at @@ -478,7 +481,10 @@ groups as `ccf_groups`. The library conditions on whether each shared cause strikes, as the formula does, and evaluates each case exactly, so it returns the exact $Q_{\text{sys}}$ above. `BetaFactor(0)` gives back the independent answer and `BetaFactor(1)` a pair no better than one pump. A group's members -must carry identical models. +must carry identical models. Splitting a probability suits a mission or a +test interval, where each $q$ is small; over a whole life, +`basis="rate"` splits the rate instead (see +[Concepts](../concepts.md#common-cause-failures)). ### A third unit barely helps @@ -516,7 +522,7 @@ some causes fail only some units (two pumps of three), which matters when the group needs more than one unit. The **Multiple Greek Letter** (MGL) model generalises the beta factor with a parameter per level ($\gamma$ for the chance that a shared failure takes a third unit, and so on); see -[Common-cause failures](../guide/common-cause.md#the-two-models). +[Common-cause failures](../guide/common-cause.md#the-three-models). ## Pitfalls @@ -676,5 +682,5 @@ Every option is in the guide: [Redundancy models](../guide/redundancy-models.md) (standby with several working units, repeated nodes, load sharing) and [Common-cause failures](../guide/common-cause.md) (the MGL model, and which -methods honour a group). The theory is summarised in +methods honour a group). How RePyability computes each is in [Concepts](../concepts.md#standby-and-repeated-nodes). diff --git a/docs/learn/design.md b/docs/learn/design.md index 215b6060..ecdc6dcd 100644 --- a/docs/learn/design.md +++ b/docs/learn/design.md @@ -268,7 +268,8 @@ items behave like this (they are "knapsack" problems), so "best value first" is a heuristic, not a method. Use it only when the problem is too large for the exact search; RePyability's exact method uses a dynamic program over the nodes when they are in series, and a pruned search otherwise, and stops with an -explanation if a problem is too large. +explanation if a problem is too large. [Concepts](../concepts.md#allocation) +describes both searches, and the cost function of the cost-based allocation. ### The whole trade-off: the Pareto front @@ -461,6 +462,6 @@ This is the last lesson. From here: fitted models to a costed, maintained design; - the [User guide](../guide/index.md) covers every method and option, with [Design and allocation](../guide/design.md) for this lesson's; -- the [Glossary](glossary.md) and [Concepts](../concepts.md) summarise the - ideas; and the [course index](index.md) maps the lessons, if you want to +- the [Glossary](glossary.md) defines the terms, and [Concepts](../concepts.md) + says how RePyability computes each analysis; and the [course index](index.md) maps the lessons, if you want to revisit one. diff --git a/docs/learn/index.md b/docs/learn/index.md index 00def012..6f10355d 100644 --- a/docs/learn/index.md +++ b/docs/learn/index.md @@ -26,7 +26,7 @@ You will need: the course uses. - **Python:** enough to run the examples and change the numbers. Every code block on these pages runs as written, in order, and the numbers quoted in - them are checked automatically when the documentation is built. + them are checked automatically by the test suite. ## The lessons diff --git a/docs/learn/maintenance.md b/docs/learn/maintenance.md index e4acb88f..6b577fa9 100644 --- a/docs/learn/maintenance.md +++ b/docs/learn/maintenance.md @@ -409,7 +409,7 @@ shutdown for several parts, or with staggered block times. computed numerically, to about one part in a million: a repair can run over a block time, and components replaced at the same times go down together. The simulation (`availability` or `cost`) gives the spread over - a finite window. See [Costs](../guide/costs.md#preventive-maintenance) in + a finite window. See [Costs](../guide/system-maintenance.md#preventive-maintenance) in the user guide for every detail of the schedule. ## Failures nobody sees @@ -468,7 +468,7 @@ pair.mean_unavailability() # -> 1.01e-4 about (λτ)²/3 = 1.02e-4 Redundancy did far more than halve the PFDavg here: it cut it by a factor of 86, and halving the test interval would quarter it again. Tests with a duration, and repairs that take time, are simulated (see -[Costs](../guide/costs.md#hidden-failures-and-inspection)). +[Costs](../guide/system-maintenance.md#hidden-failures-and-inspection)). A real safety function needs three more terms, each of which the diagram takes. The two valves share a design and a service, so some failures strike @@ -480,7 +480,7 @@ found by whichever test comes first. And a test that finds only a share `c` of the failures (a `"coverage"`) leaves the rest hidden until a full test (`"full_test"`), adding about $(1 - c)\lambda T/2$ for full tests every $T$. All three are exact: see -[Costs](../guide/costs.md#common-cause-staggered-tests-and-test-coverage). +[Costs](../guide/system-maintenance.md#common-cause-staggered-tests-and-test-coverage). **Choosing the test interval.** Each test costs $c_i$, and each hour the part lies failed costs $c_d$. The cost rate is then about @@ -503,7 +503,9 @@ gearbox as old as it was ("minimal repair"). Then failures come faster and faster, and the maintenance question becomes *when to overhaul or replace the whole unit*. The [`Repairable`](../guide/maintenance.md#overhaul-under-minimal-repair-repairable) class answers it for one unit, with the same trade-off between planned and -unplanned costs. +unplanned costs; [Concepts](../concepts.md#maintenance-models) gives its +models (minimal repair, the Barlow–Hunter overhaul and Kijima's imperfect +repair). ## Pitfalls @@ -660,5 +662,5 @@ should it be tested, and what does that cost per hour? system to designing one. - In the user guide, [Maintenance policies](../guide/maintenance.md) covers `NonRepairable` and `Repairable` in full, and - [Costs](../guide/costs.md#preventive-maintenance) the preventive schedule + [Costs](../guide/system-maintenance.md#preventive-maintenance) the preventive schedule of a `RepairableRBD`. diff --git a/docs/learn/systems.md b/docs/learn/systems.md index 0c60cb6d..99d991d1 100644 --- a/docs/learn/systems.md +++ b/docs/learn/systems.md @@ -751,3 +751,6 @@ from the lesson, and then make both uncertain at once. [MTTF intervals](../guide/reliability.md#lifetimes-and-mean-time-to-failure), time to a reliability) and [Redundancy models](../guide/redundancy-models.md) (standby, repeated and load-sharing units). +- [Concepts](../concepts.md#lifetimes-by-simulation) says how RePyability + simulates lifetimes, makes the simulation error smaller and propagates + [parameter uncertainty](../concepts.md#parameter-uncertainty). diff --git a/docs/tutorial.md b/docs/tutorial.md index 9a767f82..63b8429d 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -365,7 +365,7 @@ discrete-event simulation: result = rep.availability(t_simulation=20000, mc_samples=2000, seed=0) result.availability[-1] # -> 0.999 availability at 20 000 h result.criticalities.failure_criticality_index.per_system_failure["ctrl"] -# -> 0.922 the controller caused 92% of the outages +# -> 0.916 the controller caused 92% of the outages ``` The controller again: a second one (step 6) is the obvious investment. See @@ -396,4 +396,7 @@ transient input you supply at evaluation time. - **[Concepts](concepts.md)**: the theory behind these numbers: path and cut sets, the importance-measure family and when to use each, conditioning, availability and cost, and the dependent-failure models from step 9. -- **[API reference](api.md)**: the generated signatures and docstrings. +- **[API reference](api/index.md)**: the generated signatures and docstrings. + +The same kind of diagram can be built and analysed in the browser in +[Reliafy](https://reliafy.com/guides/build-an-rbd). diff --git a/mkdocs.yml b/mkdocs.yml index 3510e9d3..8cc3bea6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -37,7 +37,10 @@ plugins: members_order: source show_root_heading: true show_root_full_path: false - show_source: true + # The source is on GitHub: inlined, it made the page 5.7 MB. + show_source: false + # Each class's page lists what it inherits (RBD's methods). + inherited_members: true separate_signature: true merge_init_into_class: true filters: ["!^_"] @@ -104,6 +107,7 @@ nav: - Repairable systems: guide/repairable.md - System capacity: guide/capacity.md - Costs: guide/costs.md + - Maintaining a system: guide/system-maintenance.md - Spares: guide/spares.md - Design and allocation: guide/design.md - Maintenance policies: guide/maintenance.md @@ -112,4 +116,14 @@ nav: - Simulation precision and speed: guide/simulation.md - Saving, reproducibility and performance: guide/saving.md - Concepts: concepts.md - - API reference: api.md + - API reference: + - Overview: api/index.md + - RBD (the base class): api/rbd.md + - NonRepairableRBD: api/nonrepairable.md + - RepairableRBD: api/repairable.md + - Fault trees, missions, networks: api/other-systems.md + - Node models: api/nodes.md + - Components and maintenance: api/components.md + - Demonstration testing: api/demonstration.md + - Timelines: api/timelines.md + - Result types: api/results.md diff --git a/pyproject.toml b/pyproject.toml index 5b65e226..e561c58b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -42,14 +42,13 @@ dependencies = [ # Deliberately uncapped: RePyability consumes a small surface of surpyval # (sf/ff/qf/random/mean, extras/with_params, regression sf/sf_tvc, # to_dict/from_dict). A surpyval release reaches every fresh install at - # once, so the upstream workflow runs the tests against surpyval's - # development branch to show a break before it is released. CI also tests - # this minimum. 0.23 names the limited-failure proportion lfp_p (p goes - # in its next release), takes success_run's level as alpha_ci, and - # pickles its fits (for n_jobs' worker processes), as RePyability relies - # on; 0.22 named a distribution's parameters parameter_names and - # computes the Normal and LogNormal quantiles directly. - "surpyval>=0.23", + # once, so run the tests against surpyval's development branch before it + # is released, to show a break first. CI also tests this minimum. 0.24 + # gives a mixture a precise tail (sf) and has conditional_gaps read a + # limited failure population's lfp_p, not a mixture's p; 0.23 named the + # proportion lfp_p, takes success_run's level as alpha_ci, and pickles + # its fits (for n_jobs' worker processes), as RePyability relies on. + "surpyval>=0.24", "tqdm>=4.64", ] @@ -59,13 +58,20 @@ dependencies = [ fast = ["numba>=0.60"] dev = [ "black==26.3.1", - "isort>=5.12", - "flake8>=6.0", + # Pinned with black and mypy, and in step with .pre-commit-config.yaml. + "isort==9.0.1", + "flake8==7.4.1", "flake8-pyproject>=1.2", # Pinned (like black) so a new mypy release can't fail CI on code that # type-checks locally; bump deliberately. "mypy==2.3.0", "pytest>=7.0", + # The suite runs on every core (pytest -n auto), and its coverage is + # collected from each worker. + "pytest-xdist>=3.5", + # A test that hangs fails, named, rather than stopping its CI job. + "pytest-timeout>=2.3", + "pytest-cov>=5.0", "coverage>=7.0", "pre-commit>=3.0", ] @@ -90,9 +96,11 @@ Issues = "https://github.com/derrynknife/RePyability/issues" # cannot silently rot. testpaths = ["repyability"] addopts = "--doctest-modules" +# Far past any test's time: a test that hangs fails, named. +timeout = 1800 # A surpyval deprecation that RePyability or its tests run into fails them -# ("... will be removed in v0.23"), so that the upstream workflow, on -# surpyval's development branch, shows it before the name is removed. +# ("... will be removed in v0.23"), so that a run on surpyval's development +# branch shows it before the name is removed. filterwarnings = [ "error:.*will be removed in v[0-9]:DeprecationWarning:repyability", ] @@ -108,6 +116,7 @@ include = ["repyability*"] [tool.setuptools.package-data] "repyability" = ["py.typed"] "repyability.tests" = ["*.json"] +"repyability.maintenance" = ["README.md", "CALENDAR_DECISIONS.md"] # For Black pre-commit hook [tool.black] @@ -124,6 +133,8 @@ line_length = 79 # mypy [tool.mypy] exclude = ['repyability/tests/'] +# Check the bodies of functions without annotations too. +check_untyped_defs = true [[tool.mypy.overrides]] module = [ diff --git a/repyability/__init__.py b/repyability/__init__.py index c9d4f6c2..5d3348dc 100644 --- a/repyability/__init__.py +++ b/repyability/__init__.py @@ -41,10 +41,20 @@ mtbf_test_time, ) from repyability.fault_tree import FaultTree -from repyability.maintenance import FailureLimitPolicy, MaintenancePolicy +from repyability.maintenance import ( + Calendar, + CalendarPolicy, + FailureLimitPolicy, + MaintenancePolicy, + NextReplacement, + Opportunity, + PlannedReplacement, + ReplacementChoice, + SkipValue, +) from repyability.network import Network from repyability.non_repairable import NonRepairable -from repyability.rbd.ccf import MGL, BetaFactor, CCFGroup +from repyability.rbd.ccf import MGL, AlphaFactor, BetaFactor, CCFGroup from repyability.rbd.chunks import SimulationChunk from repyability.rbd.degrading_node import DegradingNode from repyability.rbd.helper_classes import ( @@ -74,6 +84,7 @@ ExpectedCost, ExpectedEvents, FailureCriticalityIndex, + FailureForecast, Lever, MaintenancePlan, RateBreakdown, @@ -88,7 +99,7 @@ UncertaintyResult, UpDownImportance, ) -from repyability.rbd.routes import AnalysisRoute +from repyability.rbd.routes import AnalysisRoute, AnalysisRoutes from repyability.rbd.shards import run_shard from repyability.rbd.standby_node import StandbyModel from repyability.repairable import ( @@ -121,6 +132,7 @@ "RegressionNode", "BetaFactor", "MGL", + "AlphaFactor", "CCFGroup", "minimal_repair_time_to_nth_failure", # Demonstration test planning @@ -134,11 +146,15 @@ "mtbf_pass_probability", "mtbf_demonstration_plan", "ComponentOption", + # Maintenance planning + "Opportunity", + "Calendar", # Up/down histories "Timeline", "Timelines", # Result types "AnalysisRoute", + "AnalysisRoutes", "AvailabilityResult", "CapacityDistribution", "ConditionalRun", @@ -160,6 +176,12 @@ "RestorationCriticalityIndex", "MaintenancePolicy", "FailureLimitPolicy", + "FailureForecast", + "SkipValue", + "NextReplacement", + "ReplacementChoice", + "CalendarPolicy", + "PlannedReplacement", "MaintenancePlan", "RateBreakdown", "AvailabilityAllocation", diff --git a/repyability/_version.py b/repyability/_version.py index 639c5fa1..1e8b0e30 100644 --- a/repyability/_version.py +++ b/repyability/_version.py @@ -5,4 +5,4 @@ dependencies) via ``[tool.setuptools.dynamic]`` in ``pyproject.toml``. """ -__version__ = "0.13" +__version__ = "0.14" diff --git a/repyability/demonstration.py b/repyability/demonstration.py index 5847b0a9..38d30fae 100644 --- a/repyability/demonstration.py +++ b/repyability/demonstration.py @@ -556,7 +556,7 @@ def demonstration_plan( A plan's consumer's risk is the chance that a design of the target ``reliability`` passes it, at most ``1 - confidence``; its producer's risk is the chance that a design of ``good_reliability``, which should - pass, fails it, at most ``producer_risk`` (#184). A test allowing no + pass, fails it, at most ``producer_risk``. A test allowing no failures keeps the first with the fewest units, but often fails a good design: allowing failures, with more units, keeps both. For each number of failures allowed, from none, the fewest units that keep the @@ -720,7 +720,7 @@ def mtbf_demonstration_plan( The total test time and the failures it allows, for a time-terminated test, such that a design of the target ``mtbf`` passes with a chance of at most ``1 - confidence`` (the consumer's risk) and one of - ``good_mtbf`` fails with a chance of at most ``producer_risk`` (#184): + ``good_mtbf`` fails with a chance of at most ``producer_risk``: the fixed-length test plans of MIL-HDBK-781, for a discrimination ratio ``good_mtbf / mtbf``. For each number of failures allowed, from none, the shortest test that keeps the consumer's risk (``mtbf_test_time``) diff --git a/repyability/fault_tree.py b/repyability/fault_tree.py index 6be274c1..533037a1 100644 --- a/repyability/fault_tree.py +++ b/repyability/fault_tree.py @@ -25,9 +25,20 @@ which multiply with the shared events, are found from it only when asked for. Nothing is approximated: the top event probability and the importance measures are exact, repeated events included. + +The analyses over a lifetime are the diagram's, on the tree's own +evaluation (#314): the mean time to the top event, the B-lives, the density +and hazard, the Barlow-Proschan importance and the parameter sensitivities +call the functions ``rbd/non_repairable_rbd.py`` shares with +``NonRepairableRBD``, with the tree's probabilities in place of the +diagram's; the uncertainty from fitted event models is drawn by the +diagram of the tree (``to_rbd()``, converted once), whose draws are the +events'. """ +import functools import json +import warnings from typing import ( TYPE_CHECKING, Any, @@ -40,12 +51,12 @@ Optional, Sequence, Tuple, - Union, ) import numpy as np from numpy.typing import ArrayLike +from repyability.rbd import _measures from repyability.rbd._ordered_bdd import OrderedBDD from repyability.rbd.modular import ( KOON, @@ -54,10 +65,18 @@ SERIES, Decomposition, ) -from repyability.utils.checks import no_distribution, real_array +from repyability.rbd.routes import AnalysisRoute, Refused +from repyability.utils.checks import ( + no_distribution, + one_of, + real_array, + simulation_options, +) +from repyability.utils.wrappers import outside_level if TYPE_CHECKING: # pragma: no cover from repyability.rbd.non_repairable_rbd import NonRepairableRBD + from repyability.rbd.results import Lever, UncertaintyResult #: The kinds of gate. GATE_KINDS = ("or", "and", "vote") @@ -172,11 +191,12 @@ class FaultTree: gate's input. ccf_groups : list of CCFGroup, optional Common-cause groups over basic events (keyword only), as a - ``NonRepairableRBD`` takes them (#184): each group's members, + ``NonRepairableRBD`` takes them: each group's members, events with the same model, occur together through its shared - causes as well as on their own, split by its ``BetaFactor`` or - ``MGL`` model at every ``t``. The top event probability and the - importance measures sum over the groups' shock outcomes, exactly + causes as well as on their own, split by its ``BetaFactor``, + ``MGL`` or ``AlphaFactor`` model at every ``t``. The top event + probability and the importance measures sum over the groups' + shock outcomes, exactly as the diagram's do; the cut sets stay sets of basic events (their probabilities, in ``ranked_cut_sets``, take the groups in). By default none. @@ -300,72 +320,79 @@ def _validated_ccf(self, groups) -> list: """The common-cause groups, checked: each a ``CCFGroup`` of basic events, an event in one group at most, and the members of a group with one model (a symmetric group, as the models assume).""" - from repyability.rbd.ccf import CCFGroup, as_groups + from repyability.rbd.ccf import as_groups, checked_groups from repyability.rbd.serialisation import serialise_model groups = as_groups(groups) if not groups: return [] - seen: set = set() - for group in groups: - if not isinstance(group, CCFGroup): + + def check_member(member, group): + if member not in self.events: raise ValueError( - "ccf_groups must hold CCFGroup instances, got " - f"{type(group).__name__}." + f"CCF group member {member!r} is not a basic event " + "of the tree." ) - for member in group.members: - if member not in self.events: - raise ValueError( - f"CCF group member {member!r} is not a basic event " - "of the tree." - ) - if member in seen: - raise ValueError( - f"Event {member!r} appears in more than one CCF " - "group." - ) - seen.add(member) + + def saved(member): # Compared as saved, as the diagram compares its members (and, # as there, not at all for a model that cannot be saved). - try: - specs = [ - ( - self.events[m] - if isinstance(self.events[m], float) - else serialise_model(self.events[m]) - ) - for m in group.members - ] - except Exception: - continue - if any(spec != specs[0] for spec in specs[1:]): - raise ValueError( - f"CCF group {list(group.members)} is not symmetric: its " - "members must have the same model." - ) - return list(groups) + event = self.events[member] + return ( + event if isinstance(event, float) else serialise_model(event) + ) + + return checked_groups( + groups, + check_member, + saved, + lambda group: ( + f"CCF group {list(group.members)} is not symmetric: its " + "members must have the same model." + ), + noun="Event", + ) - def _evaluation(self, p: dict, q: dict, size: int, expand: bool = True): + def _evaluation( + self, + p: dict, + q: dict, + size: int, + expand: bool = True, + groups: Optional[list] = None, + structure: Optional[Decomposition] = None, + ): """The tree's probabilities at the events' probabilities of not having occurred ``p`` and of having occurred ``q``, with the - common-cause groups each conditioned on within the smallest module - holding its members, or (with ``expand``) written out as shock - events where those would multiply (#219, see - ``rbd._ccf_modules``).""" + common-cause groups (the tree's, or ``groups`` in their place, as + the sensitivities move a group's model) each conditioned on within + the smallest module holding its members, or (with ``expand``) + written out as shock events where those would multiply (#219, see + ``rbd._ccf_modules``); over ``structure`` (the decomposition's + dual, for the path sets) in place of the decomposition.""" from repyability.rbd import _ccf_modules + if groups is None: + groups = self.ccf_groups plans = self.__dict__.setdefault("_ccf_plans", {}) - if expand not in plans: - plans[expand] = _ccf_modules.Plan( - self._decomposition, - self.ccf_groups, - self._ccf_expanded if expand else None, + key = (expand, _ccf_modules.Plan.key(groups), id(structure)) + if key not in plans: + plans[key] = _ccf_modules.Plan( + self._decomposition if structure is None else structure, + groups, + ( + functools.partial(self._ccf_expanded, groups=groups) + if expand + else None + ), ) return _ccf_modules.Evaluation( - plans[expand], self.ccf_groups, p, q, size, self._ccf_check + plans[key], groups, p, q, size, self._ccf_check ) - def _ccf_expanded(self, chosen: dict) -> Optional[Decomposition]: + def _ccf_expanded( + self, chosen: dict, groups: Optional[list] = None + ) -> Optional[Decomposition]: """The tree's structure with the causes of the groups ``chosen`` (each group's, as the members each fails) written out as events of their own (#219): each member's place taken by an OR gate of its @@ -374,11 +401,13 @@ def _ccf_expanded(self, chosen: dict) -> Optional[Decomposition]: tree is then too entangled to work out.""" from repyability.rbd._ccf_modules import Failed, Shock + if groups is None: + groups = self.ccf_groups gates = self.gates events: Dict[Hashable, Any] = {e: 0.5 for e in self.events} failed: Dict[Hashable, Any] = {} for g, sets in chosen.items(): - for member in self.ccf_groups[g].members: + for member in groups[g].members: strikes = [ Shock(g, k) for k, struck in enumerate(sets) @@ -392,9 +421,7 @@ def _ccf_expanded(self, chosen: dict) -> Optional[Decomposition]: gates[name] = (*head, [failed.get(x, x) for x in inputs]) for member, gate in failed.items(): g = next( - i - for i, group in enumerate(self.ccf_groups) - if member in group.members + i for i, group in enumerate(groups) if member in group.members ) gates[gate] = ( "or", @@ -422,6 +449,101 @@ def _ccf_check(self, index: int, group, Q: np.ndarray) -> None: self._ccf_warned.add(index) validity_warning(group, largest) + def _require_time_varying(self) -> None: + """Raise if no event's probability depends on time, so that the + time to a reliability is undefined.""" + if self.is_fixed: + raise Refused( + "The top event probability does not vary with time (every " + "event is a fixed probability); the time to a reliability " + "is undefined." + ) + + def _require_varying_lifetimes( + self, what: str = "lifetimes to average" + ) -> None: + """Raise if every event is a probability: the top event then + occurs at the start or never, so the system has no lifetimes (no + ``what``: none to average, no failure density over time).""" + if self.is_fixed: + raise Refused( + "The top event probability does not vary with time (every " + f"event is a fixed probability): the system has no {what}." + ) + + def _require_lifetimes(self) -> None: + """Raise unless the exact mean time to the top event is defined: + its probability must vary with time, and hold over whole + lifetimes, which common-cause groups that split a probability do + not (as ``NonRepairableRBD.mean`` refuses them).""" + self._require_varying_lifetimes() + if any(g.model.basis == "probability" for g in self.ccf_groups): + raise NotImplementedError( + "The exact mean time to the top event does not account for " + "common-cause (CCF) groups that split a failure " + "probability (the default basis): their models assume it " + "is small (see CCFGroup), while over a whole lifetime it " + "runs to 1. Split the failure rate instead " + "(BetaFactor(beta, basis='rate'), or MGL(..., " + "basis='rate')), which holds over the whole life; the " + "diagram of the tree, to_rbd(), estimates the MTTF with " + "the common cause left out, mean(method='simulate')." + ) + + def _event_model(self, event: Hashable) -> Any: + """An event's model: as given, a probability as the + ``FixedEventProbability`` of it (as ``to_rbd`` draws it), whose + one parameter, ``p``, is its lever.""" + from surpyval import FixedEventProbability + + model = self.events[event] + if isinstance(model, float): + return FixedEventProbability.from_params(model) + return model + + def _as_event(self, event: Hashable, model: Any) -> Any: + """A moved ``model`` of ``event`` as the event is given: the + probability of a ``FixedEventProbability`` for an event given as + one (see ``with_levers``).""" + if isinstance(self.events[event], float): + return float(np.ravel(model.params)[0]) + return model + + def _lifetime_models(self) -> list: + """The events' lifetime models (a probability has none).""" + return [m for m in self.events.values() if not isinstance(m, float)] + + def _time_scale(self) -> float: + """The shortest typical failure time over the events' models (see + ``NonRepairableRBD._time_scale``), to size a step at time 0 by; + NaN if none has one.""" + from repyability.rbd._measures import model_scale + + scales = [model_scale(m) for m in self._lifetime_models()] + scales = [x for x in scales if np.isfinite(x) and x > 0.0] + return min(scales) if scales else float("nan") + + @property + def _init_args(self) -> dict: + """The constructor's arguments, as ``with_levers`` rebuilds the + tree from them.""" + return { + "gates": self.gates, + "events": dict(self.events), + "top": self.top, + "ccf_groups": list(self.ccf_groups) or None, + } + + def _rbd(self) -> "NonRepairableRBD": + """The diagram of the tree (``to_rbd``), converted once: the + uncertainty analyses draw the events' models through it, its + nodes being the events (a repeated event's later places repeats of + its node) and its common-cause groups the tree's own.""" + rbd = self.__dict__.get("_rbd_cache") + if rbd is None: + rbd = self.__dict__["_rbd_cache"] = self.to_rbd() + return rbd + @property def gates(self) -> Dict[Hashable, tuple]: """``{gate name: gate}``: ``("or", inputs)``, ``("and", inputs)`` or @@ -572,15 +694,19 @@ def _core(self, position: Dict[Hashable, int]) -> Tuple[list, int]: # -- evaluation -------------------------------------------------------- - def _times(self, t) -> Tuple[np.ndarray, bool]: + def _times(self, t) -> Tuple[np.ndarray, tuple]: + """The times ``t`` as a row (1-d), and their shape: ``()`` for a + number, which ``_out`` gives a float back for. None stands for any + time (1.0) when no event's probability depends on time.""" if t is None: if not self.is_fixed: raise ValueError( - "t is required: an event's probability depends on time." + "t is required: this fault tree is time-varying (an " + "event's probability depends on time)." ) t = 1.0 - scalar = np.ndim(t) == 0 - return np.atleast_1d(real_array(t, "t")), scalar + times = real_array(t, "t") + return times.reshape(-1), times.shape def _event_probabilities(self, t: np.ndarray) -> Tuple[dict, dict]: """Each event's probability of not having occurred (``p``) and of @@ -618,13 +744,22 @@ def _independent_top(self, p: dict, q: dict, size: int) -> np.ndarray: return np.broadcast_to(np.asarray(fails, dtype=float), (size,)) @staticmethod - def _out(values, scalar: bool): + def _out(values, shape: tuple): + """``values`` (a row per time, or a dict of rows) shaped as the + times were (see ``_times``): a float for a number.""" + + def shaped(row): + row = np.asarray(row, dtype=float).reshape(-1) + if shape == (): + return float(row[0]) + size = int(np.prod(shape, dtype=np.int64)) + if row.size != size: + row = np.broadcast_to(row, (size,)).copy() + return row.reshape(shape) + if isinstance(values, dict): - return { - k: (float(np.asarray(v).reshape(-1)[0]) if scalar else v) - for k, v in values.items() - } - return float(np.asarray(values).reshape(-1)[0]) if scalar else values + return {k: shaped(v) for k, v in values.items()} + return shaped(values) def top_event_probability(self, t: Optional[ArrayLike] = None): """The probability that the top event has occurred. @@ -668,9 +803,9 @@ def top_event_probability(self, t: Optional[ArrayLike] = None): >>> round(tree.top_event_probability(), 4) # 0.01 + 0.99 * 0.1 ** 2 0.0199 """ - t, scalar = self._times(t) + t, shape = self._times(t) p, q = self._event_probabilities(t) - return self._out(self._top(p, q, len(t)), scalar) + return self._out(self._top(p, q, len(t)), shape) def ff(self, t: Optional[ArrayLike] = None): """The probability that the top event has occurred by ``t``: the @@ -714,7 +849,7 @@ def sf(self, t: Optional[ArrayLike] = None): >>> round(tree.sf(), 4), round(tree.ff(), 4) (0.98, 0.02) """ - t, scalar = self._times(t) + t, shape = self._times(t) p, q = self._event_probabilities(t) size = len(t) if self.ccf_groups: @@ -724,7 +859,405 @@ def sf(self, t: Optional[ArrayLike] = None): p, q, shape=size, works=True, fails=False ) values = np.broadcast_to(np.asarray(total, dtype=float), (size,)) - return self._out(values, scalar) + return self._out(values, shape) + + def _sf_and_ff(self, times: np.ndarray) -> Tuple[np.ndarray, np.ndarray]: + """The probabilities that the top event has not occurred and that + it has by the ``times``, each worked out in its own right (so a + small one keeps its precision), from one evaluation.""" + p, q = self._event_probabilities(times) + size = len(times) + if self.ccf_groups: + works, fails = self._evaluation(p, q, size).system() + else: + works, fails = self._decomposition.probabilities( + p, q, shape=size, works=True, fails=True + ) + return ( + np.broadcast_to(np.asarray(works, dtype=float), (size,)), + np.broadcast_to(np.asarray(fails, dtype=float), (size,)), + ) + + def Hf(self, t: Optional[ArrayLike] = None): + """The cumulative hazard ``H(t) = -ln R(t)`` of the system the tree + describes, ``R`` its [`sf`][repyability.FaultTree.sf]: exact (so + it honours the common-cause groups), ``+inf`` where the top event + is certain. While the top event is less likely than not it is + ``-log1p(-F(t))`` from its probability ``F``, so that a small one + keeps its precision, as ``NonRepairableRBD.Hf`` has it. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when no event's + probability depends on time. + + Returns + ------- + float or numpy.ndarray + The cumulative hazard: a float for a number ``t``, an array + for an array. + + Raises + ------ + ValueError + If ``t`` is left out and an event's probability depends on + time. + + Examples + -------- + One exponential event's cumulative hazard is its rate times ``t``: + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Exponential.from_params([0.01])}, + ... ) + >>> round(tree.Hf(50), 6) + 0.5 + """ + from repyability.rbd._measures import cumulative_hazard + + times, shape = self._times(t) + return self._out(cumulative_hazard(*self._sf_and_ff(times)), shape) + + def _density(self, times: np.ndarray, dx: float) -> np.ndarray: + """The density of the time to the top event at ``times``, by + central differences of the exact probabilities (see ``df``).""" + from repyability.rbd._measures import ( + density_by_differences, + difference_steps, + ) + + return density_by_differences( + self._sf_and_ff, + times, + difference_steps(times, dx, self._time_scale), + ) + + def df(self, t: Optional[ArrayLike] = None, dx: float = 1e-6): + """The density of the time to the top event, ``f(t) = -dR/dt``. + + By a central finite difference of the exact + [`sf`][repyability.FaultTree.sf] (so it honours the common-cause + groups), with step ``dx * |t|``, relative to ``t`` (at 0, ``dx`` + of the shortest of the events' typical failure times), as + ``NonRepairableRBD.df`` takes the diagram's: the difference is of + the top event probability while it is the smaller, which keeps + its precision, the lower point is clipped at 0, and a negative + result (rounding) is clipped to 0. It assumes the probability is + smooth near ``t``, which does not hold where an event's model has a + kink or a step; a tree of probabilities alone has a density of 0. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when no event's + probability depends on time. + dx : float, optional + The relative step of the difference, by default 1e-6. + + Returns + ------- + float or numpy.ndarray + The density: a float for a number ``t``, an array for an + array. + + Raises + ------ + ValueError + If ``t`` is left out and an event's probability depends on + time. + + Examples + -------- + One exponential event: ``f(t) = rate * exp(-rate * t)``. + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Exponential.from_params([0.01])}, + ... ) + >>> round(tree.df(50), 6) + 0.006065 + """ + times, shape = self._times(t) + return self._out(self._density(times, dx), shape) + + def hf(self, t: Optional[ArrayLike] = None, dx: float = 1e-6): + """The hazard rate of the top event, ``h(t) = f(t) / R(t)``: the + numerical density of [`df`][repyability.FaultTree.df] over the + exact [`sf`][repyability.FaultTree.sf] (both honour the + common-cause groups), ``+inf`` where the top event is certain. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when no event's + probability depends on time. + dx : float, optional + The relative step of the difference, passed to ``df``, by + default 1e-6. + + Returns + ------- + float or numpy.ndarray + The hazard rate: a float for a number ``t``, an array for an + array. + + Raises + ------ + ValueError + If ``t`` is left out and an event's probability depends on + time. + + Examples + -------- + One exponential event has a constant hazard rate: + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Exponential.from_params([0.01])}, + ... ) + >>> round(tree.hf(50), 6) + 0.01 + """ + times, shape = self._times(t) + sf, _ = self._sf_and_ff(times) + density = self._density(times, dx) + return self._out( + np.divide( + density, sf, out=np.full_like(density, np.inf), where=sf > 0 + ), + shape, + ) + + def mean( + self, + method: str = "exact", + *, + mc_samples: Optional[int] = None, + seed=None, + ) -> float: + """The mean time to the top event: the MTTF of the system the tree + describes. + + Exact to about ``1e-10``, relative: the area under the probability + that the top event has not occurred, ``integral from 0 to infinity + of R(t) dt`` with ``R`` the exact [`sf`][repyability.FaultTree.sf], + integrated by adaptive quadrature on pieces that start at the + events' models' kinks, as ``NonRepairableRBD.mean`` integrates the + diagram's (it is as exact as the events' models' own + probabilities: see ``analysis_routes``). ``inf`` if the top event + may never occur (an event of a limited failure population). An + event given as a probability has occurred, or not, from the start + and for good, as ``sf`` takes it. A common-cause group that splits + the failure rate (``basis="rate"``) is included; one splitting a + probability is refused, as the diagram refuses it. + + ``method="simulate"`` estimates it from ``mc_samples`` lifetimes + of the tree's diagram ([`random`][repyability.FaultTree.random]): + the draws are the diagram's, ``to_rbd().mean(method="simulate")``. + + Parameters + ---------- + method : str, optional + ``"exact"`` (the default) or ``"simulate"``. + mc_samples : int, optional + With ``method="simulate"``, the number of simulated lifetimes, + by default 100_000. + seed : int, optional + With ``method="simulate"``, seeds the draws, by default None. + + Returns + ------- + float + The mean time to the top event (``inf`` if it may never + occur). + + Raises + ------ + ValueError + If every event is a probability, so that the top event has no + time; if ``method`` is unknown, or ``mc_samples`` or ``seed`` + is given with ``method="exact"``. + NotImplementedError + With a common-cause group that splits a failure probability + (the default basis): its model assumes it is small (see + [`CCFGroup`][repyability.CCFGroup]), while over a whole + lifetime it runs to 1. + + Examples + -------- + Either of two exponential events, with rates 0.01 and 0.03, is a + failure rate of 0.04: a mean time of 25. + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> E = surv.Exponential.from_params + >>> tree = FaultTree( + ... {"top": ("or", ["a", "b"])}, {"a": E([0.01]), "b": E([0.03])} + ... ) + >>> round(tree.mean(), 8) + 25.0 + >>> round(tree.mean(method="simulate", mc_samples=10_000, seed=1), 1) + 25.3 + """ + from repyability.rbd._mean_lifetime import ( + mean_lifetime, + model_kinks, + model_knots, + ) + + one_of("method", method, ("exact", "simulate")) + if method == "simulate": + return self._rbd().mean( + method="simulate", mc_samples=mc_samples, seed=seed + ) + simulation_options("mean()", {"mc_samples": mc_samples, "seed": seed}) + self._require_lifetimes() + models = self._lifetime_models() + return mean_lifetime( + lambda t: self.sf(t), + [model_knots(m) for m in models], + [model_kinks(m) for m in models], + ) + + def random(self, size: int, seed=None) -> np.ndarray: + """``size`` random times to the top event (Monte-Carlo): the + lifetimes of the tree's diagram, + [`NonRepairableRBD.random`][repyability.NonRepairableRBD.random] + of ``to_rbd()`` (converted once), so the draws are the diagram's + (an event given as a probability occurs at the start, with its + probability, or never; a common-cause group that splits a + probability is left out, as there). + + Parameters + ---------- + size : int + The number of lifetimes. + seed : int, optional + Seeds the draws, by default None. + + Returns + ------- + numpy.ndarray + ``size`` times to the top event (``inf`` where it never + occurs). + + Raises + ------ + ValueError + If ``size`` is not a positive integer. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> E = surv.Exponential.from_params + >>> tree = FaultTree( + ... {"top": ("or", ["a", "b"])}, {"a": E([0.01]), "b": E([0.03])} + ... ) + >>> times = tree.random(5, seed=1) + >>> times.shape, bool((times == tree.to_rbd().random(5, seed=1)).all()) + ((5,), True) + """ + return self._rbd().random(size, seed=seed) + + def time_to_reliability(self, target, upper_bound: Optional[float] = None): + """The time at which the probability that the top event has not + occurred falls to ``target``: the inverse of + [`sf`][repyability.FaultTree.sf] (so it honours the common-cause + groups), found by Brent's method between 0 and an upper bound + found by doubling from 1 unless given, as + ``NonRepairableRBD.time_to_reliability`` finds the diagram's. + + Parameters + ---------- + target : float or array_like + The reliability to solve for, in (0, 1), or several. + upper_bound : float, optional + A time at which the reliability is below ``target``; by + default found by doubling. + + Returns + ------- + float or numpy.ndarray + The time at which ``sf(t) == target``: an array of one for + each target, if several are given. + + Raises + ------ + ValueError + If ``target`` is not in (0, 1); if every event is a + probability (the reliability is constant in time); if + ``target`` exceeds the reliability at ``t = 0``, so it is + never reached; if no upper bound is found (after 1000 + doublings); or if the reliability at ``upper_bound`` is still + above ``target``. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Weibull.from_params([100, 2])}, + ... ) + >>> round(tree.time_to_reliability(0.9), 2) + 32.46 + """ + from repyability.rbd._measures import invert_reliability + + self._require_time_varying() + return invert_reliability( + lambda t: float(self.sf(t)), target, upper_bound + ) + + def bx_life(self, x, upper_bound: Optional[float] = None): + """The Bx life: the time by which the top event has occurred in + ``x`` percent of systems, ``time_to_reliability(1 - x / 100)``. + ``bx_life(10)`` is the B10 life. + + Parameters + ---------- + x : float or array_like + The percentage in which the top event has occurred (10 for the + B10 life, not a fraction), in + (0, 100), or several. + upper_bound : float, optional + As for ``time_to_reliability``. + + Returns + ------- + float or numpy.ndarray + The time (one for each ``x``, if several are given). + + Raises + ------ + ValueError + If ``x`` is not in (0, 100), or as for + ``time_to_reliability``. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Weibull.from_params([100, 2])}, + ... ) + >>> round(tree.bx_life(10), 2) + 32.46 + """ + x = real_array(x, "x") + if not np.all((0.0 < x) & (x < 100.0)): + raise ValueError("x must be a percentage in (0, 100).") + return self.time_to_reliability(1.0 - x / 100.0, upper_bound) def occurs(self, events: Collection[Hashable]) -> bool: """Whether the top event occurs when exactly ``events`` have. @@ -755,7 +1288,7 @@ def occurs(self, events: Collection[Hashable]) -> bool: (False, True) """ # A bare string is one event's name, not its characters (#225). - occurred = {events} if isinstance(events, str) else set(events) + occurred: set = {events} if isinstance(events, str) else set(events) unknown = occurred - set(self.events) if unknown: raise ValueError(f"Unknown event(s) {sorted(map(str, unknown))}.") @@ -898,23 +1431,35 @@ def ranked_cut_sets( # -- importance -------------------------------------------------------- - def _conditioned(self, t) -> Tuple[np.ndarray, dict, dict, dict, bool]: - """The top event probability, and for each event the top event - probability given that it has occurred and given that it has not, - with the events' probabilities of occurring.""" - times, scalar = self._times(t) + def _held(self, p: dict, q: dict, size: int) -> Tuple[Any, dict, dict]: + """The top event probability of independent events, and for each + event that probability given that it has occurred and given that + it has not.""" + top = self._independent_top(p, q, size) + occurred, not_occurred = {}, {} + for e in self.events: + p_e, q_e = dict(p), dict(q) + p_e[e], q_e[e] = np.zeros(size), np.ones(size) + occurred[e] = self._independent_top(p_e, q_e, size) + p_e[e], q_e[e] = np.ones(size), np.zeros(size) + not_occurred[e] = self._independent_top(p_e, q_e, size) + return top, occurred, not_occurred + + def _conditioned( + self, t + ) -> Tuple[np.ndarray, np.ndarray, dict, dict, dict, dict, tuple]: + """The probabilities that the top event has not occurred and that + it has; for each event, the top event probability given that it + has occurred and given that it has not; the events' probabilities + of not having occurred and of having (a group member's, its + chances of each in its group); and the times' shape.""" + times, shape = self._times(t) p, q = self._event_probabilities(times) size = len(times) if not self.ccf_groups: - top = self._top(p, q, size) - occurred, not_occurred = {}, {} - for e in self.events: - p_e, q_e = dict(p), dict(q) - p_e[e], q_e[e] = np.zeros(size), np.ones(size) - occurred[e] = self._top(p_e, q_e, size) - p_e[e], q_e[e] = np.ones(size), np.zeros(size) - not_occurred[e] = self._top(p_e, q_e, size) - return top, occurred, not_occurred, q, scalar + works, _ = self._sf_and_ff(times) + top, occurred, not_occurred = self._held(p, q, size) + return works, top, occurred, not_occurred, p, q, shape # With common-cause groups, as the diagram's measures: a member's # occurrence says something of the others', so the top event's # probability given it is a sum over its group's shock outcomes, @@ -922,35 +1467,26 @@ def _conditioned(self, t) -> Tuple[np.ndarray, dict, dict, dict, bool]: # other groups conditioned on within their modules (#219); an event # outside the groups is held, as without them. evaluation = self._evaluation(p, q, size) - _, top_sum = evaluation.system() - joint_in: Dict[Hashable, Any] = {} - joint_out: Dict[Hashable, Any] = {} - chance_in: Dict[Hashable, Any] = {} - chance_out: Dict[Hashable, Any] = {} - grouped = set() + works_sum, top_sum = evaluation.system() + occurred, not_occurred, up, down = {}, {}, dict(p), dict(q) for e in self.events: - values, joint = evaluation.joints(e) - joint_in[e], joint_out[e] = values["down_bad"], values["down_ok"] - if joint: - grouped.add(e) - chance_in[e], chance_out[e] = values["fails"], values["works"] - occurred, not_occurred, marginal = {}, {}, dict(q) - with np.errstate(divide="ignore", invalid="ignore"): - for e in self.events: - if e in grouped: - occurred[e] = joint_in[e] / chance_in[e] - not_occurred[e] = joint_out[e] / chance_out[e] - marginal[e] = np.asarray(chance_in[e], dtype=float) - else: - occurred[e], not_occurred[e] = joint_in[e], joint_out[e] + given = evaluation.given((e,)) + (chance_up, _, not_occurred[e]), (chance_down, _, occurred[e]) = ( + given[True,], + given[False,], + ) + if e in evaluation.group_of: + up[e] = np.asarray(chance_up, dtype=float) + down[e] = np.asarray(chance_down, dtype=float) + works = np.broadcast_to(np.asarray(works_sum, dtype=float), (size,)) top = np.broadcast_to(np.asarray(top_sum, dtype=float), (size,)) - return top, occurred, not_occurred, marginal, scalar + return works, top, occurred, not_occurred, up, down, shape def birnbaum_importance(self, t: Optional[ArrayLike] = None) -> dict: """Birnbaum importance of each basic event. In the guide's Greeks it is *delta*: how far the system moves with each - basic event (see [Sensitivities: the Greeks](guide/greeks.md)). + basic event (see [Sensitivities: the Greeks](../guide/greeks.md)). ``P(top | e occurred) - P(top | e did not)``: how much the top event probability depends on the event, the rate at which it rises with @@ -979,24 +1515,33 @@ def birnbaum_importance(self, t: Optional[ArrayLike] = None) -> dict: >>> {e: round(v, 4) for e, v in tree.birnbaum_importance().items()} {'pump 1': 0.095, 'pump 2': 0.095, 'valve': 0.99} """ - _, occurred, not_occurred, _, scalar = self._conditioned(t) + _, _, occurred, not_occurred, _, _, shape = self._conditioned(t) return self._out( - {e: occurred[e] - not_occurred[e] for e in self.events}, scalar + {e: occurred[e] - not_occurred[e] for e in self.events}, shape ) - def criticality_importance(self, t: Optional[ArrayLike] = None) -> dict: + def criticality_importance( + self, t: Optional[ArrayLike] = None, kind: str = "failure" + ) -> dict: """Criticality importance of each basic event. - ``I_B(e) * P(e) / P(top)``: the probability that the event has - occurred and is critical, given that the top event has occurred, so - the share of the top event it accounts for. ``nan`` where the top - event cannot occur. + ``kind="failure"`` (the default): ``I_B(e) * P(e) / P(top)``, the + probability that the event has occurred and is critical, given + that the top event has occurred, so the share of the top event it + accounts for; ``nan`` where the top event cannot occur. + ``kind="success"``: ``I_B(e) * (1 - P(e)) / (1 - P(top))``, the + probability that the event has not occurred and is critical, given + that the top event has not, as + [`NonRepairableRBD.criticality_importance`][repyability.NonRepairableRBD.criticality_importance] + defines both forms; ``nan`` where the top event is certain. Parameters ---------- t : array_like, optional Time/s, a number or an array. May be left out when no event's probability depends on time. + kind : str, optional + ``"failure"`` (the default) or ``"success"``. Returns ------- @@ -1004,6 +1549,12 @@ def criticality_importance(self, t: Optional[ArrayLike] = None) -> dict: ``{event: importance}``: floats for a number ``t``, arrays for an array. + Raises + ------ + ValueError + If ``kind`` is neither "failure" nor "success", or ``t`` is + left out and an event's probability depends on time. + Examples -------- >>> from repyability import FaultTree @@ -1015,13 +1566,13 @@ def criticality_importance(self, t: Optional[ArrayLike] = None) -> dict: >>> {e: round(v, 4) for e, v in tree.criticality_importance().items()} {'pump 1': 0.1597, 'pump 2': 0.1597, 'valve': 0.8319} """ - top, occurred, not_occurred, q, scalar = self._conditioned(t) - with np.errstate(divide="ignore", invalid="ignore"): - out = { - e: (occurred[e] - not_occurred[e]) * q[e] / top - for e in self.events - } - return self._out(out, scalar) + one_of("kind", kind, ("failure", "success")) + works, top, occurred, not_occurred, p, q, shape = self._conditioned(t) + system, member = (works, p) if kind == "success" else (top, q) + birnbaum = {e: occurred[e] - not_occurred[e] for e in self.events} + return self._out( + _measures.criticality_importance(birnbaum, member, system), shape + ) def risk_achievement_worth(self, t: Optional[ArrayLike] = None) -> dict: """Risk achievement worth of each basic event. @@ -1052,10 +1603,10 @@ def risk_achievement_worth(self, t: Optional[ArrayLike] = None) -> dict: >>> {e: round(v, 2) for e, v in tree.risk_achievement_worth().items()} {'pump 1': 2.44, 'pump 2': 2.44, 'valve': 16.81} """ - top, occurred, _, _, scalar = self._conditioned(t) + _, top, occurred, _, _, _, shape = self._conditioned(t) with np.errstate(divide="ignore", invalid="ignore"): out = {e: occurred[e] / top for e in self.events} - return self._out(out, scalar) + return self._out(out, shape) def risk_reduction_worth(self, t: Optional[ArrayLike] = None) -> dict: """Risk reduction worth of each basic event. @@ -1087,13 +1638,16 @@ def risk_reduction_worth(self, t: Optional[ArrayLike] = None) -> dict: >>> {e: round(v, 3) for e, v in tree.risk_reduction_worth().items()} {'pump 1': 1.19, 'pump 2': 1.19, 'valve': 5.95} """ - top, _, not_occurred, _, scalar = self._conditioned(t) + _, top, _, not_occurred, _, _, shape = self._conditioned(t) with np.errstate(divide="ignore", invalid="ignore"): out = {e: top / not_occurred[e] for e in self.events} - return self._out(out, scalar) + return self._out(out, shape) def fussell_vesely( - self, t: Optional[ArrayLike] = None, method: str = "exact" + self, + t: Optional[ArrayLike] = None, + fv_type: str = "c", + method: str = "exact", ) -> dict: """Fussell-Vesely importance of each basic event. @@ -1105,12 +1659,20 @@ def fussell_vesely( sets' probabilities instead (the rare-event approximation of their union, as many PRA tools report it), which can exceed 1 when the events are not rare. ``nan`` where the top event cannot occur. + With ``fv_type="p"`` the minimal path sets (see + [`minimal_path_sets`][repyability.FaultTree.minimal_path_sets]) + take the place of the cut sets, as the diagram's ``fv_type="p"`` + does: the probability that every event of some path set + containing the event has occurred. Parameters ---------- t : array_like, optional Time/s, a number or an array. May be left out when no event's probability depends on time. + fv_type : str, optional + ``"c"`` (the cut sets, the default) or ``"p"`` (the path + sets). method : str, optional ``"exact"`` (the default) or ``"rare_event"``. @@ -1123,8 +1685,9 @@ def fussell_vesely( Raises ------ ValueError - If ``method`` is not "exact" or "rare_event", or ``t`` is left - out and an event's probability depends on time. + If ``fv_type`` is not "c" or "p", ``method`` is not "exact" or + "rare_event", or ``t`` is left out and an event's probability + depends on time. Examples -------- @@ -1137,45 +1700,57 @@ def fussell_vesely( >>> {e: round(v, 4) for e, v in tree.fussell_vesely().items()} {'pump 1': 0.1681, 'pump 2': 0.1681, 'valve': 0.8403} """ - if method not in ("exact", "rare_event"): + if fv_type not in ("c", "p"): raise ValueError( - f"method must be 'exact' or 'rare_event', got {method!r}." + "fv_type must be either 'c' (cut-set) or 'p' (path-set), " + f"fv_type={fv_type!r} was given." ) + one_of("method", method, ("exact", "rare_event")) from repyability.rbd._ccf_modules import ( expected_product, group_outcomes, ) - times, scalar = self._times(t) + times, shape = self._times(t) p, q = self._event_probabilities(times) size = len(times) top = self._top(p, q, size) share = {e: np.zeros(size) for e in self.events} # The common-cause groups each conditioned on within the smallest - # module holding its members (#219). + # module holding its members (#219). The path sets are the dual + # structure's cut sets. if method == "exact": + structure = self._decomposition + if fv_type == "p": + structure = structure.dual() failed = ( - self._evaluation(p, q, size, expand=False).failed_cut_sets() + self._evaluation( + p, q, size, expand=False, structure=structure + ).failed_cut_sets() if self.ccf_groups - else self._decomposition.failed_cut_sets(p, q, shape=size) + else structure.failed_cut_sets(p, q, shape=size) ) for e, probability in failed.items(): share[e] = share[e] + probability else: outcomes = group_outcomes(self.ccf_groups, p, q, self._ccf_check) - for cut in self.minimal_cut_sets(): - probability = expected_product(cut, q, outcomes) - for e in cut: + sets = ( + self.minimal_cut_sets() + if fv_type == "c" + else self.minimal_path_sets() + ) + for members in sets: + probability = expected_product(members, q, outcomes) + for e in members: share[e] = share[e] + probability - with np.errstate(divide="ignore", invalid="ignore"): - out = {e: share[e] / top for e in self.events} - return self._out(out, scalar) + return self._out(_measures.fussell_vesely(share, top), shape) def differential_importance( self, t: Optional[ArrayLike] = None, *, change: str = "uniform", + kind: str = "failure", groups: Optional[Mapping[Hashable, Collection[Hashable]]] = None, ) -> dict: """Each basic event's share of the change in the top event @@ -1183,7 +1758,7 @@ def differential_importance( importance measure (DIM, Borgonovo & Apostolakis, 2001). In the guide's Greeks it is *DIM*, the shares of a change (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``DIM_e = dP/dq_e dq_e / sum_f dP/dq_f dq_f``, with ``q_e`` the events' probabilities, so the shares add up to 1, and a group's @@ -1204,6 +1779,11 @@ def differential_importance( probability depends on time. change : str, optional ``"uniform"`` (the default) or ``"proportional"``. + kind : str, optional + With ``change="proportional"``, the criticality importance + shared out: ``"failure"`` (the default) or ``"success"`` (see + ``criticality_importance``), as + ``NonRepairableRBD.differential_importance`` takes it. groups : dict, optional ``{name: events}``: each group's share, the sum of its events', instead of each event's. @@ -1217,9 +1797,9 @@ def differential_importance( Raises ------ ValueError - If ``change`` is not "uniform" or "proportional", a group names - an unknown event, or ``t`` is left out and an event's - probability depends on time. + If ``change`` is not "uniform" or "proportional", ``kind`` is + not "failure" or "success", a group names an unknown event, or + ``t`` is left out and an event's probability depends on time. Examples -------- @@ -1243,19 +1823,20 @@ def differential_importance( f"by as much) or 'proportional' (each by the same fraction " f"of itself), got {change!r}." ) + one_of("kind", kind, ("failure", "success")) if change == "uniform": values = self.birnbaum_importance(t) else: - values = self.criticality_importance(t) + values = self.criticality_importance(t, kind=kind) return shares(values, groups, scalar=t is None or np.ndim(t) == 0) def joint_importance(self, t: Optional[ArrayLike] = None) -> dict: - """The joint (second-order) importance of each pair of basic events - (#194): whether preventing the two together is worth more than + """The joint (second-order) importance of each pair of basic events: + whether preventing the two together is worth more than preventing each. In the guide's Greeks it is *gamma*: complements or substitutes (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``JRI(e, f) = -d2P / dq_e dq_f``, ``P`` the top event probability and ``q`` the events' probabilities: how much event ``f``'s @@ -1266,7 +1847,11 @@ def joint_importance(self, t: Optional[ArrayLike] = None) -> dict: preventing the other worth more. Negative, they are substitutes, as under an AND gate: either one prevented keeps the gate from occurring. Exact, from the top event probability with both events - held, as the tree is multilinear in the events. + held, as the tree is multilinear in the events. With common-cause + groups, the top event probability given each of the pair occurring + and not is conditioned on their states, as ``birnbaum_importance`` + conditions on one (two members of one group on their joint state), + and an event outside the groups is held. Parameters ---------- @@ -1286,8 +1871,6 @@ def joint_importance(self, t: Optional[ArrayLike] = None) -> dict: ValueError If ``t`` is left out and an event's probability depends on time. - NotImplementedError - With common-cause groups, whose members cannot be held. Examples -------- @@ -1303,18 +1886,21 @@ def joint_importance(self, t: Optional[ArrayLike] = None) -> dict: >>> round(joint[("pump 1", "valve")], 4) # complements 0.1 """ - if self.ccf_groups: - raise NotImplementedError( - "The joint importance holds pairs of events occurring and " - "not, which a common-cause group's members cannot be (the " - "causes they share would still strike the others): it is " - "not worked out with common-cause groups, as yet." - ) - times, scalar = self._times(t) + from repyability.rbd.rbd import Pairs + + times, shape = self._times(t) p, q = self._event_probabilities(times) size = len(times) - ones, zeros = np.ones(size), np.zeros(size) events = list(self.events) + if self.ccf_groups: + from repyability.rbd._ccf_modules import joint_importance + + evaluation = self._evaluation(p, q, size) + pairs = Pairs.of( + events, lambda e, f: joint_importance(evaluation, e, f) + ) + return Pairs(self._out(pairs, shape)) + ones, zeros = np.ones(size), np.zeros(size) def importance(p_: dict, q_: dict, of: list) -> dict: out = {} @@ -1326,8 +1912,6 @@ def importance(p_: dict, q_: dict, of: list) -> dict: out[f] = occurred - self._top(p_f, q_f, size) return out - from repyability.rbd.rbd import Pairs - pairs = Pairs() for k, e in enumerate(events): rest = events[k + 1 :] # noqa: E203 @@ -1340,18 +1924,820 @@ def importance(p_: dict, q_: dict, of: list) -> dict: prevented = importance(p_e, q_e, rest) for f in rest: pairs[Pairs.oriented(e, f)] = prevented[f] - occurring[f] - return Pairs(self._out(pairs, scalar)) + return Pairs(self._out(pairs, shape)) - # -- conversion -------------------------------------------------------- + def structural_importance(self) -> Dict[Hashable, float]: + """The structural importance of each basic event: the fraction of + the states of the other events in which it is pivotal, the top + event occurring when it has and not when it has not. - def to_rbd(self) -> "NonRepairableRBD": - """The reliability block diagram of the same system. + It is the Birnbaum importance with every event's probability at + 1/2, so it depends on the gates alone, not on the events' models + or the common-cause groups: useful at design time, before any + data. It is the diagram's ``structural_importance`` (see + ``to_rbd``). - The diagram says when the system works: an OR gate becomes a series - block of its inputs, an AND gate a parallel block, and a VOTE gate - needing ``k`` of ``n`` failures a block needing ``n - k + 1`` of its - ``n`` inputs working, through a perfect junction node. Each event - becomes a node with the event's model (a probability ``p`` becomes a + Returns + ------- + dict + ``{event: importance}``, each in [0, 1]. + + Examples + -------- + The valve is pivotal unless both pumps have failed (three states + of four), and a pump when the other has failed and the valve has + not (one of four): + + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["valve", "flow"]), + ... "flow": ("and", ["pump 1", "pump 2"])}, + ... {"pump 1": 0.1, "pump 2": 0.1, "valve": 0.05}, + ... ) + >>> tree.structural_importance() + {'pump 1': 0.25, 'pump 2': 0.25, 'valve': 0.75} + """ + half = {e: np.full(1, 0.5) for e in self.events} + _, occurred, not_occurred = self._held(half, half, 1) + return { + e: float(np.ravel(occurred[e] - not_occurred[e])[0]) + for e in self.events + } + + def improvement_potential(self, t: Optional[ArrayLike] = None) -> dict: + """The improvement potential of each basic event: ``P(top) - P(top + | e did not occur)``, how far the top event probability would fall + if the event could not occur. It is ``I_B(e) * q_e``, the Birnbaum + importance times the event's probability, and is worked out as + that product, so a small one keeps its precision; with common-cause + groups, a member's is conditioned through the groups' outcomes, as + ``birnbaum_importance`` is. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when no event's + probability depends on time. + + Returns + ------- + dict + ``{event: potential}``: floats for a number ``t``, arrays for + an array. + + Raises + ------ + ValueError + If ``t`` is left out and an event's probability depends on + time. + + Examples + -------- + A valve that could not fail would take the top event from 0.0595 + to the pumps' 0.01: + + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["valve", "flow"]), + ... "flow": ("and", ["pump 1", "pump 2"])}, + ... {"pump 1": 0.1, "pump 2": 0.1, "valve": 0.05}, + ... ) + >>> {e: round(v, 4) for e, v in tree.improvement_potential().items()} + {'pump 1': 0.0095, 'pump 2': 0.0095, 'valve': 0.0495} + """ + _, _, occurred, not_occurred, _, q, shape = self._conditioned(t) + return self._out( + {e: (occurred[e] - not_occurred[e]) * q[e] for e in self.events}, + shape, + ) + + def _rate_parts(self, t: np.ndarray) -> Dict[Any, np.ndarray]: + """Each event's part in the rate of change of the probability that + the top event has not occurred, at the times ``t``, and each + common-cause group's under the tuple of its members (see + ``NonRepairableRBD.reliability_rate`` and ``rate_parts``): a + probability has no density, so no part.""" + from repyability.rbd._measures import density as model_density + from repyability.rbd._measures import rate_parts + + _, _, occurred, not_occurred, _, _, _ = self._conditioned(t) + size = len(t) + + def density(event, x: np.ndarray) -> np.ndarray: + model = self.events[event] + if isinstance(model, float): + return np.zeros(x.shape) + return model_density(model, x) + + return rate_parts( + {e: occurred[e] - not_occurred[e] for e in self.events}, + density, + t, + self.ccf_groups, + inputs=self._event_probabilities, + fails=lambda p, q: self._top(p, q, size), + scale=self._time_scale, + ) + + def barlow_proschan_importance( + self, t: Optional[ArrayLike] = None + ) -> dict: + """Each basic event's Barlow-Proschan importance: the probability + that the top event is caused by the event's occurring (Barlow & + Proschan, 1975), given that it has occurred by ``t``, or ever. + + In the guide's Greeks it is *theta*, integrated: who caused the + failures (see [Sensitivities: the Greeks](../guide/greeks.md)). + + An event's occurring causes the top event when the event is + critical then, so the probability that the top event has occurred + by ``t`` through event ``e`` is ``integral_0^t I_B^e(s) f_e(s) + ds``, its Birnbaum importance times its model's density, and its + share of the sum over the events is its importance: the shares add + up to 1. A common-cause group's part is its members' together, + under the tuple of their names (the top event probability's change + with the members' lives moved on, by differences). The integrals + are by adaptive quadrature to about ``1e-9``, as + ``NonRepairableRBD.barlow_proschan_importance`` works out the + diagram's. An event given as a probability has no density, so no + share; with every event one, the top event occurs at the start or + never, and the measure is refused. + + Parameters + ---------- + t : array_like, optional + Time/s by which the top event has occurred, a number or an + array; by default over the whole life. + + Returns + ------- + dict + ``{event: share}`` (and each common-cause group's): floats for + a number ``t`` or the whole life, arrays for an array. NaN + where the top event cannot have occurred. + + Raises + ------ + ValueError + For a negative time, or if every event is a probability (the + system then has no lifetimes). + + Examples + -------- + Either of two exponential events, with rates 1 and 3: the second + causes three quarters of the top events. + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> E = surv.Exponential.from_params + >>> tree = FaultTree( + ... {"top": ("or", ["a", "b"])}, {"a": E([1.0]), "b": E([3.0])} + ... ) + >>> shares = tree.barlow_proschan_importance() + >>> {e: round(v, 6) for e, v in shares.items()} + {'a': 0.25, 'b': 0.75} + """ + from repyability.rbd._measures import barlow_proschan, life_end + + self._require_varying_lifetimes( + "lifetimes, so no failure density over time to share out" + ) + + def sf(x: np.ndarray) -> np.ndarray: + return np.asarray(self.sf(np.atleast_1d(x)), dtype=float).ravel() + + return barlow_proschan( + self._rate_parts, + self._lifetime_models(), + t, + lambda knots: life_end(sf, knots), + ) + + def parameter_sensitivity( + self, t: Optional[ArrayLike] = None, rel_step: float = 1e-5 + ) -> Dict[Any, Dict[str, Any]]: + """The sensitivity of the probability that the top event has not + occurred (``sf``) to each parameter of the events' models. + + In the guide's Greeks it is the levers' *deltas* (see + [Sensitivities: the Greeks](../guide/greeks.md)). + + For event ``e`` with parameter ``theta``, ``d sf / d theta = + I_B(e) * d sf_e(t; theta) / d theta``: its Birnbaum importance + times how far its model's reliability moves with the parameter, + the latter by a central difference with step ``rel_step * + |theta|`` (``rel_step`` if ``theta`` is 0), the model rebuilt with + ``from_params`` (one-sided where one side is not a valid value, nan + where neither is), as ``NonRepairableRBD.parameter_sensitivity`` + works out the diagram's. An event given as a probability has one + parameter, ``p``, the probability (as a ``FixedEventProbability`` + names it). The members of a common-cause group carry one model, + so its parameters are reported once, under the tuple of the + members, each moved for every member at once, and so are the + group's own (``"ccf_beta"``, and an [`MGL`][repyability.MGL] + model's ``"ccf_gamma"``, ...), each by a central difference of + the exact probabilities with the groups. ``levers()`` lists the + parameters, with their values and ranges, and ``with_levers`` + builds the tree with them moved. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when no event's + probability depends on time. + rel_step : float, optional + The relative step of the differences, by default 1e-5. + + Returns + ------- + dict + ``{event: {parameter: sensitivity}}``, by surpyval's parameter + names, a common-cause group's under the tuple of its members: + floats for a number ``t``, arrays for an array. + + Raises + ------ + ValueError + If ``t`` is left out and an event's probability depends on + time. + + Examples + -------- + One exponential event: ``R = exp(-rate t)``, so ``dR/d rate = -t + exp(-rate t)``, about -9.048 at ``t = 10`` with rate 0.01: + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Exponential.from_params([0.01])}, + ... ) + >>> sens = tree.parameter_sensitivity(10) + >>> {name: round(v, 3) for name, v in sens["c"].items()} + {'failure_rate': -9.048} + """ + from repyability.rbd._model_utils import lever_spec + from repyability.rbd.ccf import CCFGroup + from repyability.rbd.non_repairable_rbd import ( + _dsf_dparam, + ccf_group_sensitivity, + ) + + times, shape = self._times(t) + size = len(times) + _, _, occurred, not_occurred, _, _, _ = self._conditioned(times) + p, q = self._event_probabilities(times) + groups = list(self.ccf_groups) + group_of = {m: g for g in groups for m in g.members} + out: Dict[Any, Dict[str, Any]] = {} + for event in self.events: + spec = lever_spec(self._event_model(event)) + if event not in group_of: + if spec is None: + continue + birnbaum = occurred[event] - not_occurred[event] + out[event] = { + name: self._out( + birnbaum * _dsf_dparam(spec, j, times, rel_step), + shape, + ) + for j, name in enumerate(spec.names) + } + continue + group = group_of[event] + if event != group.members[0]: + continue + index = next(i for i, g in enumerate(groups) if g is group) + + def system(members=None, model=None, group=group, index=index): + given, failing = p, q + if members is not None: + works, fails = members + given = {**p, **{m: works for m in group.members}} + failing = {**q, **{m: fails for m in group.members}} + trial = groups + if model is not None: + trial = list(groups) + trial[index] = CCFGroup(group.members, model) + up, down = self._evaluation( + given, failing, size, groups=trial + ).system() + return ( + np.broadcast_to(np.asarray(up, dtype=float), (size,)), + np.broadcast_to(np.asarray(down, dtype=float), (size,)), + ) + + values = ccf_group_sensitivity( + system, group, spec, times, rel_step + ) + out[tuple(group.members)] = { + name: self._out(value, shape) for name, value in values.items() + } + return out + + def levers(self) -> List["Lever"]: + """The parameters ``parameter_sensitivity`` moves, in the order it + reports them: each event's model's, by surpyval's names (``p`` + for an event given as a probability), and each common-cause + group's under the tuple of its members (the one model its members + carry, then its own, ``"ccf_beta"``, ...), each a + [`Lever`][repyability.Lever] with its value and range, as a + diagram's ``levers()`` lists them. Nothing is worked out; + [`with_levers`][repyability.FaultTree.with_levers] sets them. The + levers are the events' models' own parameters: those of the units + inside a composite model (a ``StandbyModel``, a + ``LoadSharingModel``, a ``DegradingNode``) or of a nested diagram + are not reached (see the guide's Greeks). + + Returns + ------- + list of Lever + The levers, in ``parameter_sensitivity``'s order. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c", "d"])}, + ... {"c": surv.Weibull.from_params([100, 2]), "d": 0.05}, + ... ) + >>> for lever in tree.levers(): + ... print(lever.key, lever.name, lever.value, lever.bounds) + c alpha 100.0 (0.0, inf) + c beta 2.0 (0.0, inf) + d p 0.05 (0.0, 1.0) + """ + from repyability.rbd import _sensitivity + + return _sensitivity.public_levers(self) + + def with_levers(self, values) -> "FaultTree": + """A copy of this tree with parameters at new values, each moved + as ``parameter_sensitivity`` moves it: an event's model rebuilt + with the parameter changed (an event given as a probability keeps + being one), a common-cause group's members' one model for every + member, a group's own (``"ccf_beta"``, ...) in its model. + ``levers()`` lists the parameters. + + Parameters + ---------- + values : dict + Each parameter's new value, by its + [`Lever`][repyability.Lever] (from ``levers()``) or by its + ``(key, name)``, as ``parameter_sensitivity`` reports it. + + Returns + ------- + FaultTree + The new tree, built as this one was with the parameters + moved; this one is unchanged. + + Raises + ------ + ValueError + For a parameter the tree does not have (with the closest + name), or a value its model refuses (one outside the lever's + ``bounds``). + TypeError + If ``values`` is not a dict, a lever is neither a ``Lever`` + nor a ``(key, name)``, or a value is not a number. + + Examples + -------- + Twice the Weibull scale: at 100 the top event has not occurred + with probability ``exp(-(100 / 200) ** 2)``, about 0.7788, where + it had ``exp(-1)``: + + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("or", ["c"])}, + ... {"c": surv.Weibull.from_params([100, 2])}, + ... ) + >>> longer = tree.with_levers({("c", "alpha"): 200.0}) + >>> round(longer.sf(100), 4) + 0.7788 + """ + from repyability.rbd import _sensitivity + + return _sensitivity.with_levers(self, values) + + # -- uncertainty ------------------------------------------------------- + + def sf_uncertainty( + self, + t: Optional[ArrayLike] = None, + uncertainty: Optional[Dict[Hashable, Any]] = None, + *, + n_draws: int = 1000, + seed=None, + sampling: str = "random", + ) -> "UncertaintyResult": + """The probability that the top event has not occurred, over + plausible event models (parameter uncertainty). + + As ``NonRepairableRBD.sf_uncertainty`` does for a diagram's + reliability (see there for the kinds of uncertainty): each of the + ``n_draws`` draws gives every uncertain event a plausible model, + from the normal approximation of its surpyval fit (``"fit"``), + from distributions over named parameters, or from a list of + models, and the probability at ``t`` is worked out exactly for the + draw, with the common-cause groups (whose models may be drawn too, + the [`CCFGroup`][repyability.CCFGroup] given as a key). The draws + are the diagram's of the tree (``to_rbd``, converted once): the + events are its nodes, so they are keyed as its nodes are, events + of one population together in a tuple, and a group's members + together. An event given as a probability has no fit, so it is + not drawn by default; its ``p`` may be given a distribution. + + Parameters + ---------- + t : array_like, optional + Time/s, a number or an array. May be left out when every + event's probability, drawn or not, is fixed. + uncertainty : dict, optional + ``{event or tuple of events: uncertainty}``, and ``{CCFGroup: + uncertainty}``, as for ``NonRepairableRBD.sf_uncertainty``. By + default every event whose model is a surpyval fit with a + parameter covariance, as ``"fit"``. + n_draws : int, optional + The number of draws, by default 1000. + seed : int, optional + Seeds the draws, for reproducible results. + sampling : str, optional + ``"random"`` (the default), or ``"sobol"`` for the points of a + scrambled Sobol sequence. + + Returns + ------- + UncertaintyResult + The probability of every draw (``samples``), the nominal value + with the events' own models, and summaries such as + ``interval`` (see + [`UncertaintyResult`][repyability.UncertaintyResult]). + + Raises + ------ + ValueError + As for ``NonRepairableRBD.sf_uncertainty``: an unknown event + or parameter, an uncertainty that cannot be drawn, a group's + members not given together, or ``t`` left out for lifetime + models. + + Examples + -------- + Cooling is lost if both pumps fail or the valve does. The pumps + are of one type, whose exponential failure rate is known only to + lie between 1 and 3 per 1000 h, so they share the one draw: + + >>> import scipy.stats as st + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> pump = surv.Exponential.from_params([0.002]) + >>> tree = FaultTree( + ... {"no cooling": ("or", ["no flow", "valve"]), + ... "no flow": ("and", ["pump 1", "pump 2"])}, + ... {"pump 1": pump, "pump 2": pump, "valve": 0.01}, + ... ) + >>> rate = {"failure_rate": st.uniform(0.001, 0.002)} + >>> result = tree.sf_uncertainty( + ... 500, {("pump 1", "pump 2"): rate}, n_draws=10_000, seed=1 + ... ) + >>> round(result.nominal, 4) + 0.5944 + >>> lower, upper = result.interval(0.9) + >>> round(lower, 3), round(upper, 3) + (0.409, 0.813) + """ + return self._rbd().sf_uncertainty( + t, uncertainty, n_draws=n_draws, seed=seed, sampling=sampling + ) + + def mean_uncertainty( + self, + uncertainty: Optional[Dict[Hashable, Any]] = None, + *, + n_draws: int = 1000, + seed=None, + sampling: str = "random", + ) -> "UncertaintyResult": + """The mean time to the top event over plausible event models + (parameter uncertainty). + + As [`sf_uncertainty`][repyability.FaultTree.sf_uncertainty] does + for the probability: each draw gives every uncertain event a + plausible model, and the mean time to the top event is worked out + exactly for the draw, as [`mean`][repyability.FaultTree.mean] + works it out. The draws are the diagram's of the tree (``to_rbd``, + converted once), as for ``sf_uncertainty``. + + Parameters + ---------- + uncertainty : dict, optional + ``{event or tuple of events: uncertainty}``, and ``{CCFGroup: + uncertainty}``, as for ``sf_uncertainty``. By default every + event whose model is a surpyval fit with a parameter + covariance, as ``"fit"``. + n_draws : int, optional + The number of draws, by default 1000. + seed : int, optional + Seeds the draws, for reproducible results. + sampling : str, optional + ``"random"`` (the default), or ``"sobol"``. + + Returns + ------- + UncertaintyResult + The mean time of every draw (``samples``; ``inf`` for a draw + in which the top event may never occur), the exact mean with + the events' own models (``nominal``), and summaries such as + ``interval``. + + Raises + ------ + ValueError + As for ``sf_uncertainty``, or if every event is a probability + (the top event has no time). + NotImplementedError + With a common-cause group that splits a failure probability, + as for ``mean``. + + Examples + -------- + Two pumps of one type, both of which must fail, whose exponential + failure rate is known only to lie between 1 and 3 per 1000 h: + + >>> import scipy.stats as st + >>> import surpyval as surv + >>> from repyability import FaultTree + >>> pump = surv.Exponential.from_params([0.002]) + >>> tree = FaultTree( + ... {"no flow": ("and", ["pump 1", "pump 2"])}, + ... {"pump 1": pump, "pump 2": pump}, + ... ) + >>> rate = {"failure_rate": st.uniform(0.001, 0.002)} + >>> result = tree.mean_uncertainty( + ... {("pump 1", "pump 2"): rate}, seed=1 + ... ) + >>> round(result.nominal, 6) # 1.5 / 0.002 + 750.0 + >>> lower, upper = result.interval(0.9) + >>> round(lower), round(upper) + (515, 1352) + """ + self._require_lifetimes() + return self._rbd().mean_uncertainty( + uncertainty, n_draws=n_draws, seed=seed, sampling=sampling + ) + + # -- how each analysis is computed --------------------------------------- + + def analysis_routes(self) -> Dict[str, AnalysisRoute]: + """How each analysis of this tree is computed, found without + running it: exactly, numerically, by simulation, or not at all. + + As a diagram's + [`analysis_routes`][repyability.NonRepairableRBD.analysis_routes]: + the gates are always evaluated exactly, but a value is only as + exact as the events' probabilities it is made of, so an event + whose model's reliability is numerical makes the analyses built + on it numerical, and one with no reliability makes them refuse. A + refusal is found by the check the method itself runs, and its + reason is the message it would raise. + + Returns + ------- + dict[str, AnalysisRoute] + For each public analysis, by method name: its ``route`` + (``"exact"``, ``"numerical"``, ``"simulated"`` or + ``"refused"``), the ``reason``, and the ``events`` that decide + it, as ``nodes`` (see + [`AnalysisRoute`][repyability.AnalysisRoute]). + + Examples + -------- + A tree of probabilities has no time to the top event: + + >>> from repyability import FaultTree + >>> tree = FaultTree( + ... {"top": ("and", ["a", "b"])}, {"a": 0.1, "b": 0.2} + ... ) + >>> routes = tree.analysis_routes() + >>> routes["top_event_probability"].route + 'exact' + >>> routes["mean"].route + 'refused' + >>> routes["sf_uncertainty"].route + 'simulated' + """ + from repyability.rbd import routes as r + from repyability.rbd._model_utils import parametric_spec + + events = { + e: ( + (r.EXACT, "a probability") + if isinstance(m, float) + else (r.model_route(m)) + ) + for e, m in self.events.items() + } + out: Dict[str, AnalysisRoute] = {} + + def give(names, route) -> None: + for name in names: + out[name] = route + + def built(route, reason): + return r.with_nodes(route, reason, events, "probabilities") + + grouped = bool(self.ccf_groups) + summed = ( + ", summed over the common-cause groups' outcomes." + if grouped + else "." + ) + through = ( + " A common-cause group's member is conditioned on through the " + "groups' outcomes." + if grouped + else "" + ) + give( + ("top_event_probability", "ff", "sf", "Hf"), + built( + r.EXACT, + "The gates over the events' probabilities, exactly" + summed, + ), + ) + give( + ("df", "hf"), + built( + r.NUMERICAL, + "The exact top event probability, differentiated by central " + "differences.", + ), + ) + fixed = r.refusal(self._require_time_varying) + give( + ("time_to_reliability", "bx_life"), + ( + r.refused(fixed) + if fixed + else built( + r.NUMERICAL, + "The exact top event probability, inverted by " + "root-finding.", + ) + ), + ) + give( + ( + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + ), + built( + r.EXACT, + "The exact top event probability, with each event occurred " + "and not." + through, + ), + ) + give( + ("ranked_cut_sets",), + built( + r.EXACT, + "Each minimal cut set's exact probability of its events " + "occurring together" + summed, + ), + ) + give( + ("fussell_vesely",), + built( + r.EXACT, + "The exact probability that a minimal cut set (fv_type='p': " + "path set) containing each event has occurred, over the top " + "event probability (method='rare_event' sums the sets' " + "probabilities instead)" + summed, + ), + ) + give( + ("joint_importance",), + built( + r.EXACT, + "Each event's exact Birnbaum importance with the other " + "event held not occurring, less with it held occurring." + + through, + ), + ) + give( + ("differential_importance",), + built( + r.EXACT, + "The exact Birnbaum importance of each event (or, for a " + "proportional change, its criticality importance), shared " + "out." + through, + ), + ) + out["structural_importance"] = AnalysisRoute( + r.EXACT, "From the gates alone." + ) + differenced = sorted( + str(e) + for e, m in self.events.items() + if not isinstance(m, float) and parametric_spec(m) is None + ) + densities = r.refusal( + lambda: self._require_varying_lifetimes( + "lifetimes, so no failure density over time to share out" + ) + ) + out["barlow_proschan_importance"] = ( + r.refused(densities) + if densities + else built( + r.NUMERICAL, + "Each event's Birnbaum importance times its model's failure " + "density" + + ( + f" (differences of the probability for {differenced})" + if differenced + else "" + ) + + ( + ", a common-cause group's part by differences of the " + "top event probability with its members' lives moved on" + if grouped + else "" + ) + + ", integrated over time by adaptive quadrature (to about " + "1e-9).", + ) + ) + out["parameter_sensitivity"] = built( + r.NUMERICAL, + "The exact Birnbaum importance times each parameter's " + "derivative, by differences." + + ( + " A common-cause group's parameters, and its model's, by " + "differences of the exact top event probability." + if grouped + else "" + ), + ) + lifetimes = r.refusal(self._require_lifetimes) + out["mean"] = ( + r.refused(lifetimes) + if lifetimes + else built( + r.NUMERICAL, + "The exact probability that the top event has not occurred, " + "integrated over time by adaptive quadrature (to about " + "1e-10, relative); method='simulate' takes the mean of the " + "diagram's simulated lifetimes.", + ) + ) + out["random"] = AnalysisRoute( + r.SIMULATED, + "Monte-Carlo lifetimes of the tree's diagram (to_rbd), each " + "event's drawn from its model.", + ) + drawn = ( + " The common-cause groups are in each draw, and their models " + "may be drawn too." + if grouped + else "" + ) + out["sf_uncertainty"] = built( + r.SIMULATED, + "The events' model parameters drawn from their uncertainty, and " + "the exact top event probability for each draw." + drawn, + ) + out["mean_uncertainty"] = ( + r.refused(lifetimes) + if lifetimes + else built( + r.SIMULATED, + "The events' model parameters drawn from their uncertainty, " + "and the exact mean time to the top event for each draw " + "(the area under its probability of not having occurred)." + + drawn, + ) + ) + return r.AnalysisRoutes(sorted(out.items())) + + # -- conversion -------------------------------------------------------- + + def to_rbd(self) -> "NonRepairableRBD": + """The reliability block diagram of the same system. + + The diagram says when the system works: an OR gate becomes a series + block of its inputs, an AND gate a parallel block, and a VOTE gate + needing ``k`` of ``n`` failures a block needing ``n - k + 1`` of its + ``n`` inputs working, through a perfect junction node. Each event + becomes a node with the event's model (a probability ``p`` becomes a ``FixedEventProbability`` of ``p``); junctions are ``PerfectReliability`` nodes named after their gates. @@ -1475,7 +2861,7 @@ def from_rbd(cls, rbd) -> "FaultTree": group with one that can: the tree keeps it, for the group's causes to strike it with the others, as an event its logic makes irrelevant (the top event ``OR(G, AND(G, member))``, which is - ``G``, #237). + ``G``). Parameters ---------- @@ -1495,6 +2881,12 @@ def from_rbd(cls, rbd) -> "FaultTree": ValueError If the diagram cannot fail (a direct input-to-output edge). + Warns + ----- + UserWarning + If the diagram has node capacities, which the tree leaves out + (a tree has no capacity analysis). + Examples -------- >>> from surpyval import FixedEventProbability @@ -1528,6 +2920,13 @@ def from_rbd(cls, rbd) -> "FaultTree": "The diagram cannot fail (its input joins its output " "directly), so it has no top event." ) + if rbd.capacity: + warnings.warn( + "The diagram's node capacities are left out of the fault " + "tree: its top event is the system failing, with no " + "capacity analysis (the diagram keeps its own).", + stacklevel=outside_level(), + ) taken = set(rbd.reliabilities) def fresh(base) -> Any: @@ -1816,7 +3215,3 @@ def _model_is_fixed(model) -> bool: from repyability.rbd.non_repairable_rbd import NonRepairableRBD return NonRepairableRBD._model_is_fixed(model) - - -# A gate as given: ("or", inputs), ("and", inputs) or ("vote", k, inputs). -GateSpec = Union[Tuple[str, Sequence[Hashable]], Tuple[str, int, Sequence]] diff --git a/repyability/maintenance/CALENDAR_DECISIONS.md b/repyability/maintenance/CALENDAR_DECISIONS.md new file mode 100644 index 00000000..65baa258 --- /dev/null +++ b/repyability/maintenance/CALENDAR_DECISIONS.md @@ -0,0 +1,366 @@ +# Calendar replacement policy: decisions to take or verify + +This file accompanies the brief *"Long-run optimal replacement policy over an +outage calendar"* (use case UC-13 in `README.md`), built in full: + +- `Opportunity`, `Calendar`: `_calendar.py`. +- Phase 1, `CalendarPolicy` and `NonRepairable.optimal_calendar_policy` / + `calendar_policy_cost`: `_replacement.py`. +- Phase 2: `_integration.py`. +- Phase 3 and the routing: `_replacement.py`. +- The guide section with its chart: `docs/guide/maintenance.md`, drawn by + `scripts/calendar_policy_chart.py`. +- The `RepairableRBD` hook. + +Each item below is a place where I chose something the brief left open, +departed from it, or could not check it. Each says what was done and what +you need to decide. Items are grouped; **★** marks those that most need your +eye. + +## A. Things I could not verify + +1. **★ The prototype was not available.** `calendar_replacement.py` was in + neither the repo nor the session. As a result, phase 1 tests 4, 5 and 7 + check numbers I worked out myself on a plant of my own. That plant is a + Weibull life of scale 1000 days and shape 2.5, a weekly stop at 3, a + quarterly shut at 2, an annual shut at 1, any day at 4 and a failure at + 20. The prototype's own figures are not checked: + - 492 days against 493; + - DP 17.1, replace at every annual shut 17.6, age replacement 29.6, run to + failure 37.5; + - "$4 against $3 a week later". + + *Decide:* commit the prototype, or its calendar and prices, and I will add + its figures as tests. + + What was checked on my plant, over ten years: + + | Policy | Cost | + |---|---| + | Optimal | 25.27 | + | Replace at every annual shut | 25.48 | + | Age replacement on any day | 50.41 | + | Run to failure | 82.05 | + + The skip test `replace(annual, 60) is False` and + `replace(annual, 300) is True` holds. The annual threshold is 88 days. +2. **Test 6 (horizon independence).** On my plant: + - year 3's thresholds from 10-, 20- and 40-year solves are identical, as + asserted; + - from a 5-year solve they are identical too, where the brief expected up + to about 10 days of end effect; + - from a 4-year solve they differ by up to 14 days. + + The test asserts at most 10 days for 5 years, which holds. +3. **The literature reference** in the docstrings is "Dekker et al., early + 1990s", as the brief gives it, with no full citation. *Decide:* give the + reference you want quoted. + +## B. Where I departed from the brief + +4. **★ A failure renews half a step later on average.** The brief's + recursion puts the unit fitted after a failure in step `n` at age 0 at + `n+1`: `(1−p)(c_f + V(0))`. The cost rate is then low by about half a step + for each failure: + - 1.7e-4 relative for the collapse test at a step of 1 day, which fails + the brief's own 1e-5 target; + - the threshold comes out at 493 days rather than the prototype's 492. + + I use `(1−p)(c_f + ½(V(0) + V(1)))` instead: the new unit is 0 or 1 step + old, half each, which is a mean of half a step, as a failure uniform in + the step gives. The error becomes second order: + - 1.3e-7 at a step of 1 day, 3e-6 at 2 days, 5e-6 at 4 days; + - threshold 493 days, against the textbook 493.05. + + *Decide:* keep this, or revert to the brief's recursion and relax test 1 + to about 2e-4. +5. **The step is not an argument (decided).** It is a numerical + setting, not a choice of when decisions are made; the calendar says + that, by its opportunities. It is sized from the models, as CLAUDE.md + requires, so that no result depends on the unit of time: + - about the mean life over 1,000 (`STEPS_PER_LIFE`); + - reduced to a whole fraction of the largest step on which every + period (and a plan's repeat) lies, so that a periodic calendar keeps + its period on the grid whatever its offsets; the offsets and dates + are put on it too, unless that makes it far finer (below a tenth of + the life's step), when each is laid at the grid time nearest it + (decision 10). Decided in the 0.14 persona review: an offset of + 0.1234 days (three hours, what a date turned into model time + gives) on a period of 180 had made the step life-sized, which does + not divide 180, so the calendar lost its period, was sampled (100 + times slower) and its schedule drifted; + - for the plant in days this gives 0.875 days. + + The tests set it through the internal solver + (`_replacement.optimal_calendar_policy(..., step=)`). +6. **The cycle limit is 20,000 steps** (`MAX_CYCLE`), not "about 20 years of + days" (7,300). Relative value iteration on 20,000 steps is a few seconds a + sweep at worst. *Decide:* the limit. +7. **The settings are module constants, not arguments.** These are: + - `MAX_CYCLE`, `TOLERANCE`, `MAX_LIVES`, `STEPS_PER_LIFE`, `FAR`, `SPAN`, + `SETTLE_GAPS` and `SETTLE_SPANS` in `_replacement.py`; + - `NODES`, `AGES`, `MAX_AGES` and `MAX_OPPORTUNITIES` in + `_integration.py`. + + This follows the package's other modules. The method takes only `step`, + `max_age`, `method`, `tail`, `mc_samples` and `seed`. *Decide:* whether any + should be arguments. +8. **The method has more arguments than the brief's signature:** `tail`, + `mc_samples` and `seed`. `mc_samples` and `seed` are the package's names + for a sampled run (CLAUDE.md), default 64. +9. **What `"part"` means on a `Calendar`.** The brief makes `anytime=None` + mean "not allowed" and also says `cp` is the default. Both are kept with a + sentinel: + - `Calendar(opportunities, failure="part", anytime="part")`; + - `"part"` takes the part's `unplanned_cost` / `planned_cost`; + - `anytime="part"` on a part with no planned cost allows no replacement + on any day; + - `None` means not allowed; + - a calendar that allows no planned replacement at all (no + opportunities, and none on any day either way) is refused by + `optimal_calendar_policy`, as `optimal_replacement_policy` refuses + with no costs (decided in the 0.14 persona review: it had returned + running to failure, silently). `calendar_policy_cost` prices a rule + on any calendar. + + *Decide:* keep the sentinel, or change the defaults. +10. **Rounding dates onto the grid.** Each date is rounded to its nearest + grid time on its own, so periods do not drift, and a date moves by half a + step at most. The brief's "warn if a date moved by more than step/2" + therefore cannot fire. I warn instead when an opportunity's times move by + more than 1% of the step. For example, an annual shut every 365.25 days + on a 1-day grid warns: "moved by up to 0.5". The schedule and + `next_replacement` give the grid times the opportunities are laid at + (a shut at 0.1234 is listed at 0.0), which the warning says. *Decide:* + the threshold for the warning, or no warning. +11. **`Calendar.planned_cost(times, step=None)` takes a step.** Without one, + only exact times match an opportunity. With one, an opportunity counts at + the grid time nearest it, as the policy sees it. +12. **`Calendar.period` is in time.** It is the least common multiple of the + periods, each read as a fraction with a denominator of at most 10⁶. For + example, 365.25 alone gives 365.25, and 7 with 365.25 gives 10,227. The + grid's own cycle, in steps, is kept separately. An `Opportunity`'s times + run both ways from its offset (`offset + k·period` for every whole `k`). + +## C. Choices the brief left open + +13. **★ How a dated plan goes on after its last date.** + - **`tail="repeat"` (the default)** repeats the plan from its first date + to its last, closed by the mean gap between its dates: + `(last − first) · m/(m−1)` for `m` dates, or the date itself when there + is only one. Repeats run forward only; nothing is put before the plan. + Dates at 364, 728 and 1,092 thus give exactly the annual calendar's + rate. + - **`tail="anytime"`** leaves only replacement on any day after the + plan, alongside the calendar's periodic opportunities. + - **Both tails are solved.** `determined_until` is the first time at + which their thresholds, or the kinds of replacement open, differ. + - **The cost rate of a dated calendar always comes from the repeated + plan**, even with `tail="anytime"`, because only the repeated plan + defines a long run. + + *Decide:* the repeat period, and what `cost_rate` should mean for a plan. +14. **★ How `method="auto"` routes.** + - A periodic calendar with a short cycle goes to relative value + iteration (exact on the grid). + - A periodic calendar with a long cycle goes to sampling, with a + warning that says so (the cycle has more than `MAX_CYCLE` steps, and + `mc_samples` sets the cost: each window is an exact solve of a + second or more; from the 0.14 persona review, where the docs' + co-prime example took 95 s at the default 64 samples with no + notice). A sampled policy's rates by kind and the schedule's chances + are estimates from a unit run in with no standard error (they moved + by 12% between seeds at 8 samples); only `cost_rate_se` has one. + The method is unchanged. + - **A dated plan goes to dynamic programming, not sampling.** The brief + routes "non-periodic" calendars to sampling, but sampling uniform + phases fits only periodic schedules; a plan's dates are fixed. + - `"auto"` never picks integration, which must be asked for. + - `method="dp"` on a long cycle differences a long finite horizon + (`basis="differenced"`) rather than refusing. + - `method="sampling"` on a dated plan is refused. +15. **What sampling samples.** It draws one uniform time over the whole + cycle and shifts every schedule by it, rather than drawing an + independent uniform phase for each schedule. This is right for any + periods, and the same thing for co-prime ones. Each window is differenced + over the longer of the longest period and the mean life. Each window + then runs on for a settling length of 2 × the longest gap plus 3 × the + age a unit outlives with chance 1e-3. On the co-prime calendar + (7/93/400/1,301 days), a sample takes about 0.2 s, and doubling the + settling length changes the rate by less than its standard error. +16. **How a sampled or long-horizon policy gives its thresholds.** + The threshold at a time is worked out in cached blocks of steps, each solved over + a window long enough for its end not to reach the block. No random + numbers are involved, so it is deterministic. +17. **★ Pricing a given rule (`calendar_policy_cost`).** A rule is a dict of + the age from which each kind replaces (opportunity names, plus + `"anytime"`); kinds left out never replace. + - **Where kinds coincide,** the kind with the youngest age in the rule + replaces, and the cheapest wins a tie. + - **The cost rate** comes from relative value iteration with the + decisions fixed. The brief suggests the stationary distribution of a + one-cycle matrix instead; both give the same `g`, and this reuses the + same code. + - **The return value is a float.** + + *Decide:* the name, the dict form, and whether it should return a result + that carries its basis. +18. **What the policy gives (revised after review).** + - **No per-time `threshold(t)` or `replace(t, age)`:** dropped as of + no use. Most of their values, at the weekly stops and on ordinary + days, are never reached. + - **`replacement_rates` and `cost_rates`:** by kind, the long-run + replacements and cost a unit of time, failures as `"failure"`. The + costs sum to `cost_rate`. + - On a periodic calendar, both are exact for the grid. They come + from the long-run share of units at each age, carried forward + from a new unit until the ages at the start of a cycle settle. + - For a sampled, long or dated calendar, they are estimated from a + new unit run in over a settling length. + - **`schedule(kinds=(), until=None)`:** lists `(time, kind, threshold, + chance)`, where `chance` is the long-run chance of a replacement + there, or from a new unit at 0 where the calendar does not repeat. + A shut the policy never uses has a chance of 0. + - By default it lists the opportunities, not the days on which only + the any-day replacement is open. + - `until` defaults to the period, or to the plan's end; a sampled + policy needs it given. + - **`next_replacement(age, now=0)`:** when the unit in service will + be replaced if it does not fail first, at which opportunity and + price (`PlannedReplacement`). A unit of age 0 has just been put in, + so it is not replaced again at that time. A unit past the oldest + age the thresholds were worked out to (`settings["max_age"]`, the + age a unit outlives with a chance of 1e-9) is past every threshold, + and is replaced at the next opportunity, or on any day now where + the calendar allows one (decided in the 0.14 persona review: it had + answered `inf`, "the unit all but sure to fail first", for a unit + given as in service). `inf` now means no planned replacement comes + (after a plan's last date with none on any day). + + *Decide:* the names, and whether rates by kind should be given per + unit of time (as now) or per cycle. +19. **A replace set that is not a threshold.** When the ages that replace + are not all those from the youngest (for example a log-normal life), + the schedule's threshold is the youngest age that replaces, + `threshold_only` is False, and a warning is given. The rates by kind + then take every age from the youngest, so they no longer sum exactly + to the cost rate. +20. **The NPV hook.** `_sweep` takes `beta`, a discount per step that + defaults to 1. There is no public API yet, as the brief asks. + +## D. Phase 2 (integration) + +21. **The quadrature.** + - **Product integration:** each piece's increase in the distribution, + times the mean of the integrand at its ends. It is exact in the + density, so a density that is infinite at 0 is no problem. + - **Not used:** Simpson or Gauss rules, `numerical_convolution` and + `_quadrature`. + - **32 pieces per gap** (`NODES`). + - **Convergence:** relative value iteration, not policy iteration with a + linear solve. + + It agrees with the grid as the step shrinks (2e-6 at a step of 1 day). +22. **The age grid is aligned with the shortest gap.** Its spacing is a whole + fraction of the shortest gap. Without that, daily opportunities came out + 0.3% *below* age replacement, which is impossible. With it they match to + 3e-9. It needs as many ages as the gap demands: integration refuses age + grids beyond 20,000 points, and cycles beyond 20,000 opportunities. The + daily test takes about 8 s. +23. **Dated plans in phase 2** are supported with `tail="repeat"` only. The + policy is given from the plan's first date onwards. + +## E. The `RepairableRBD` hook (deliverable 6) + +24. **★ How a component uses the policy.** A component takes + `"preventive": {"policy": "calendar", "calendar": }`. + - At each renewal, the event loop asks the policy for the first time at + which the unit's age reaches the threshold, counting from when the + unit was put in, and schedules the replacement then. + - **Planned replacements are charged the calendar's price** for that + opportunity, so the spec takes no `"cost"` (and no `"interval"`). + - **Failures are charged the diagram's own `replace_cost`** and + `repair_cost`. The calendar's failure price only shapes the policy. + - **The policy is applied as given, not re-optimised.** It was optimised + for a part replaced at once. If the diagram's repairs or maintenance + take time, its outages are not what the policy assumed. + - **The policy carries the life it was worked out for** (`life`, its + saved form as the diagrams save a model, None where the life does + not save), and the spec refuses a component whose life's saved form + differs, naming both; a policy with no saved form passes. Decided in + the 0.14 persona review: a policy worked out for another bearing + had been accepted silently, and simulated at a cost rate 19% off. + + *Decide:* whether the failure price should come from the calendar. +25. **★ It is simulation only.** + - Every exact or numerical analysis refuses it, through the same check + as an opportunistic group member (`_require_no_opportunities`). That + covers availability over time, long-run values, the exact twin, + expected cost, spares and so on. + - **This holds even for a component alone,** whose `g` the policy itself + gives exactly. + - **numba refuses it** (`_compiled.unsupported`), so it runs in the + Python loop. + - **Saving it is refused** (`to_json` raises). + - **A state's phase is refused,** because a calendar's times are + absolute; an age is taken. + - **Checked:** the catalogue has a diagram for it; the routes, levers and + capability tests pass; and the simulated cost of a lone bearing + matches the policy's `g` within 3 standard errors over 40 years. + + *Decide:* whether the analyses of a lone calendar component should use + the policy's `g` (exact) rather than refuse. +26. **Maintenance groups (decided and done, #349).** The brief says the + hook "ties into the existing opportunistic-maintenance groups". A + calendar component in a group is a member like any other, under the + group's existing rules, and its policy is not re-optimised: + - Its planned replacements are stops of its group, as an age or block + replacement is: each other member working and at least its + `"opportunity"` age is renewed early there, and the group's + `setup_cost` is charged once a stop. + - At a stop another member opens (or the system going down, with + `system_down`), it is renewed early if at least its own + `"opportunity"` age (allowed under the calendar policy, any + finite age from 0: the policy has no single interval to bound it + by). The renewal is charged the calendar's price for a planned + replacement on the day (`CalendarPolicy._planned_price`: the + opportunity's if one falls then, else the price on any day); where + the calendar allows none, the component's own `replace_cost`, as a + failure's replacement is charged (other members' early renewals + are charged their preventive cost, which a calendar component has + none of). The new unit's next planned replacement counts from the + stop (`_renewal`). + - Not re-optimised: the policy's thresholds are those worked out for + the part alone, with the calendar's prices; the group's stops are + chances it did not plan for. Re-optimising with the stops (random + times, at the group's price) would need the other members' failure + processes in the policy's state, a different problem. + - Checked (`test_maintenance_calendar_rbd.py`): timelines worked by + hand on both kinds of stop, the price on the day against + `Calendar.planned_cost`, a plain simulation of the grouped rule + against the event loop, and that a conditional run and the compiled + engine leave the group whole to the Python loop. + +## F. Around the build + +27. **UC-7's `replacement_policies` and UC-12's + `optimal_next_replacement`.** Decided: `replacement_policies` is + removed (a special case of `optimal_calendar_policy`), with the + `older_than` rule it added to `condition_cycle`; + `optimal_replacement_policy` is kept. Done (#348): + `optimal_next_replacement(policy=)` takes the `cost_rate` of a + `CalendarPolicy` (or a `MaintenancePolicy`) as its baseline; the + age replacement's stays the default. +28. **The chart.** It is an SVG committed under `docs/guide/img/`, drawn by + `scripts/calendar_policy_chart.py` with matplotlib, outside the package + (which keeps no plotting dependency). Run the script again after a change + to the policy. +29. **CLAUDE.md.** The scope note gains a line: a calendar of opportunities + in model time is RePyability's, and laying it on real dates is Reliafy's. +30. **Failures that were already on dev, fixed here:** + - `test_every_export_is_a_node_or_said_not_to_be` had been failing + since `ReplacementPlan` was added; it is now listed as not a node, + with `Opportunity` and `Calendar`; + - `test_api_docstrings` had been failing for the cost parameters of + `NonRepairable`, `ReplacementPlan` and `replacement_policies`; this was + fixed in #340. diff --git a/repyability/maintenance/README.md b/repyability/maintenance/README.md new file mode 100644 index 00000000..6f82e2d8 --- /dev/null +++ b/repyability/maintenance/README.md @@ -0,0 +1,931 @@ +# Maintenance: use cases + +This is the working record of RePyability's maintenance questions (#319), +built **one use case at a time**: each use case is a question someone +actually asks, written down before any code, with how it is answered and +what was learned. + +**RePyability is about timelines; Reliafy is about calendars** (the scope +note in `CLAUDE.md`). Every use case here is asked in model time, from a +state: ages, how far into an interval, windows of time from now. Dates, +calendars, operating schedules, meters read by date and financial years +are Reliafy's, which turns them into model time, calls RePyability, and +turns the answers back into dates. Reliafy's record +(`docs/maintenance-scheduling.md`) holds the calendar side of the same +use cases, under the same numbers. + +## The use cases, by number + +| # | The question here, in model time | State | +|---|---|---| +| UC-1 | A system's expected maintenance work in each of a run of windows from its state | Answered by hand | +| UC-2, UC-3, UC-4 | Tasks on dates and monthly rules: calendar only | Reliafy's | +| UC-5 | A part's chance of failing, expected failures and their cost over a window, from its age | **Built**: `NonRepairable.failures` | +| UC-6 | Right after a failure, the value of skipping the next planned replacement | **Built**: `NonRepairable.skip_next_replacement` | +| UC-7 | The cheapest replacement policy when replacing at a shut is cheaper than between shuts | Built as `replacement_policies`, then removed: UC-13 answers it | +| UC-8 | Each policy's cost over a window from a state, and its spread | Answered by hand | +| UC-9 | The cost in each of a run of windows from a state, and how often each outcome happens | Answered by hand | +| UC-10 | Tasks by use on an operating schedule: calendar only | Reliafy's | +| UC-11 | A part renewed at every service and at failure: its risk before each service, from its state, in running hours | Answered by hand | +| UC-12 | When to replace this part next: now, at a scheduled outage, at a planned replacement, or at its failure? | **Built**: `NonRepairable.optimal_next_replacement` | +| UC-13 | The long-run optimal replacement policy over a calendar of shuts with prices of their own | **Built**: `NonRepairable.optimal_calendar_policy` | + +## How a use case is added + +1. **Write it down first**, using the template below, with the status + *proposed*. Say who asks, in their own words, what they know and what + they want back. +2. **Answer it by hand with what RePyability has today**, if it can be + answered at all. Record the script and its output under *Today*. This + shows exactly what is missing, and gives the numbers the new code must + reproduce. +3. **Decide how it is solved**: the method, and whether it is exact, + numerical or simulated (exact by default, simulation on request, as + everywhere in RePyability). Note the choices made and why. +4. **Build the smallest thing that answers it**, with tests that check it + against the hand-worked answer and an independent check (a closed + form, the simulation, or a sum that must hold). Mark it *built*, with + the pull request. + +Keep to the project's rules (`CLAUDE.md`): +- Models come in already fitted; fitting stays in surpyval. +- Results are plain data (`results._ResultMapping`, with `to_dict()`), + never plots: the Reliafy app draws them. +- Model time, not dates: Reliafy turns dates into ages and windows. +- Inputs are checked where they are given (`utils.checks`). + +## Template + +```markdown +### UC-n: + +- **Status:** proposed | answered by hand | built (#PR) +- **Who asks, and why:** +- **What they know:** (the diagram, the components' states now, the plan, costs, ...) +- **What they want back:** (the shape of the answer: a table by window, a number) +- **How it is solved:** (the method; exact, numerical or simulated; the existing code it builds on) +- **Today:** (how close RePyability gets now: a script and its output) +- **Checks:** (how we know the answer is right) +- **Learned / open questions:** +``` + +## Use cases + +### UC-1: "What maintenance work is coming on this system, in each of the next twelve windows of 730 hours, from its state now?" + +(First asked month by month; months, as dates, are Reliafy's, and a +window of 730 hours stands for a month here.) + +- **Status:** answered by hand with today's RePyability; not built. +- **Who asks, and why:** a planner building the next year's work and + budget for one system already in service. +- **What they know:** the system's diagram, each component's fitted life + and repair models and its maintenance (age replacement, block + replacement, proof tests of hidden failures), and each component's + state now: its age, and how far it is into its block or test interval. +- **What they want back:** in each window, the expected number of + preventive replacements, corrective repairs and tests, by component, + and of planned system outages. +- **How it is solved:** `RepairableRBD.expected_events(t, state=...)` + gives each count over `[0, t)` from the state, exactly (no simulation), + counting the events due at exact times (an age, a block replacement) + exactly. A window's expected count is the difference of the counts at + its two ends, as expectations add over windows. +- **Today (by hand):** + + ```python + import numpy as np + import surpyval as surv + from repyability import NodeState, RepairableRBD + + window = 730.0 # hours + fixed = surv.ExactEventTime.from_params + rbd = RepairableRBD( + [("s", "pump"), ("pump", "valve"), ("valve", "sensor"), ("sensor", "t")], + { + "pump": { # replaced at age 6000 h, 4 h of work + "reliability": surv.Weibull.from_params([9000.0, 2.5]), + "repairability": surv.LogNormal.from_params([np.log(8.0), 0.5]), + "preventive": {"interval": 6000.0, "duration": fixed([4.0])}, + }, + "valve": { # replaced every 8760 h whatever happens, 6 h of work + "reliability": surv.Weibull.from_params([20000.0, 1.5]), + "repairability": surv.Exponential.from_params([1 / 12.0]), + "preventive": {"interval": 8760.0, "policy": "block", + "duration": fixed([6.0])}, + }, + "sensor": { # hidden failures, proof-tested every 4380 h + "reliability": surv.Exponential.from_params([1 / 50000.0]), + "repairability": "instant", + "inspection": {"interval": 4380.0}, + }, + }, + ) + state = { + "pump": NodeState(age=5200.0), + "valve": NodeState(age=7000.0, since_scheduled=7000.0), + "sensor": NodeState(age=3000.0, since_scheduled=3000.0), + } + ends = window * np.arange(1, 13) + events = rbd.expected_events(ends, state=state) + by_window = lambda cumulative: np.diff(cumulative, prepend=0.0) + by_window(events.node_preventive["pump"]) + ``` + + Expected counts in each window of 730 hours (rounded): + + | Window | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | + |---|---|---|---|---|---|---|---|---|---|---|---|---| + | Pump replaced (age 6000 h) | 0 | 0.90 | 0 | 0 | 0 | 0 | 0 | 0 | 0.05 | 0.65 | 0.00 | 0.01 | + | Valve replaced (block) | 0 | 0 | 1.00 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | + | Sensor tested | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | + | Pump repaired | 0.09 | 0.01 | 0.01 | 0.02 | 0.03 | 0.04 | 0.05 | 0.07 | 0.07 | 0.03 | 0.01 | 0.02 | + | Valve repaired | 0.03 | 0.03 | 0.02 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 | 0.03 | 0.03 | 0.03 | + | Sensor failure found at a test | 0 | 0.08 | 0 | 0 | 0 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 | + | System planned outages | 0 | 0.83 | 0.99 | 0 | 0 | 0 | 0 | 0 | 0.05 | 0.63 | 0.00 | 0.01 | + + How to read it: + - **The pump** reaches age 6000 h in window 2 and is replaced then, + unless it failed first: 0.90 is the chance it survives the 800 h from + 5200 h to 6000 h. Its replacement then reaches 6000 h in windows 9 + to 12, depending on when it was put in. + - **The valve** is replaced every 8760 hours whatever happens (block + replacement), so its replacement is certain, in window 3. + - **The sensor** is tested in windows 2 and 8. +- **Checks:** the windows add up to `expected_events` over the whole + span. The block replacements and tests land where their phases put + them, worked out by hand. The age replacement's first chance matches + `R(6000) / R(5200)`. +- **Learned / open questions:** + - **A task and a system outage are different counts.** In window 2 the + pump's replacement is 0.90 likely, but the system's planned outages + are 0.83. The sensor's failures are hidden, so with probability about + 0.07 it has already failed unseen, and the series system is already + down when the pump work starts: that work causes no new outage. The + forecast must say which it reports, and explain this. + - **Differencing a numerical count can give tiny negatives** + (`-1e-12`) in a window with no events, to be shown as 0, without + changing the sums. + - By hand it means differencing and picking the node dictionaries + apart. What RePyability could give is the counts in consecutive + windows from a state in one call; months, as dates, are Reliafy's. + + +### UC-5: "My bearing is 224 days old and is replaced when it fails. What's the chance it fails in the next 82 days, how many replacements should I expect, and at $400 each, what will they cost?" + +(First asked with dates: installed on 1 March 2026, from 11 October to +the end of 2026. Turning dates into an age and a window is Reliafy's; the +question here is the model-time one it becomes.) + +- **Status:** built: `NonRepairable(..., unplanned_cost=)` and its + `failures(t, age=)`, giving a `FailureForecast`. +- **Who asks, and why:** a planner with a non-repairable part (a bearing) + run to failure and replaced, budgeting ahead. +- **What they know:** the bearing's life, a Weibull with a scale of 365 + days and a shape of 2 (fitted in surpyval); the current one is 224 + days old and working; a failed one is replaced with a new one, for + $400. +- **The answer:** over the next 82 days, **a 27.8% chance it fails, 0.283 + replacements expected (a new bearing can fail too), $113.20.** +- **How it is solved:** in model time (RePyability is about timelines): + - the chance it fails: the life conditioned on the age reached, + `1 - R(age + t) / R(age)`, exact; + - the expected replacements: the renewal count from that age, as + `RepairableRBD.expected_events` works it out for the part as a + component started from its state (exact, to about 1e-7; each + replacement taking the part's `time_to_replace`). +- **As built:** + + ```python + bearing = NonRepairable(surv.Weibull.from_params([365.0, 2.0]), unplanned_cost=400.0) + forecast = bearing.failures(82.0, age=224.0) + forecast.probability, forecast.expected_failures, forecast.expected_cost + # 0.2784, 0.2830, 113.20 + ``` + + First built with dates, as a `Component` (a name, a life, an installed + date and a cost each failure); that overlapped `NonRepairable`, which + is the part (its life, replacement time and costs), and dates are + Reliafy's, so it became a method of the part in model time, and + `NonRepairable` takes `planned_cost` and `unplanned_cost` by name. +- **Checks:** `test_non_repairable_failures.py`: the UC-5 answers against + the closed form and a plain simulation; an exponential life (its rate + times the window, whatever its age); Weibull units of several ages and + windows, and replacements that take time, against the simulation; + several windows at once; the input checks and the costs. + +### UC-6: "Right after a failure, what's the NPV of skipping the next planned replacement?" + +- **Status:** built: `NonRepairable.skip_next_replacement` and + `SkipValue` (`repyability/maintenance/_skip.py`, #350). +- **Who asks, and why:** the planner of UC-5, now replacing the bearing on + a plan as well as when it fails. A planned replacement costs less than + an unplanned one, but replaces a working bearing. Right after a + failure, the new bearing makes the next planned replacement look + wasteful: is it worth skipping? +- **What they know (the maintainer chose these numbers):** the bearing of + UC-5 (Weibull, scale 365 days, shape 2); a plan every 180 days; $150 + for a planned replacement and $400 for one at a failure; a discount + rate of 8% a year. It is asked right after a failure, a new bearing + just in. +- **What they want back:** the two options compared, each a + present-value cost, and the NPV of skipping, as what it saves + (positive: skip it), net of the failures it adds. +- **The option the maintainer named: does the plan reset?** + - **The plan resets** (age replacement): the next planned replacement + is an interval after the last replacement, planned or not, so right + after a failure it is 180 days away. Skipping it, the bearing runs + to a failure or to 360 days. Every replacement starts the same + process afresh, so the two choices are the same process from the + bearing's replacement on, but at different times: the NPV needs the + discounted value of the plan from new. + - **The plan does not reset** (fixed intervals, block replacement): + planned replacements fall every 180 days whatever happens. Right + after a failure, the next may be any number of days away, up to 180; + a failure just before it leaves a nearly new bearing to be replaced. + Skipping it, the bearing runs on to a failure or the following one. + Both choices replace at the following one, so they differ only + before it. +- **The answer:** + - **Resetting plan: skipping saves $26.56.** + - **Plan on fixed intervals:** it depends on the days to the next + planned replacement: + + | Days to the next planned replacement | 1 | 10 | 30 | 60 | 90 | 120 | 150 | 180 | + |---|---|---|---|---|---|---|---|---| + | NPV of skipping ($) | 149.06 | 140.73 | 123.08 | 99.06 | 78.21 | 60.68 | 46.42 | 35.26 | + + Skipping pays every time: a 180-day plan is too frequent for this + bearing. `NonRepairable.optimal_replacement_policy()` puts the best + age at about 298 days (undiscounted). On a plan every 365 days the + answer changes sign: skipping pays when the next is less than + about 261 days away (the failure came at least about 104 days after + the last planned replacement), and costs up to about $2 otherwise. +- **How it is solved:** exactly, with no simulation: + - **Resetting:** renewal-reward with discounting. With `M(s)` the + expected discounted length of a cycle cut off at age `s` (the + integral of `exp(-r x) R(x)` to `s`) and `A(s)` its expected + discounted cost, the plan from new is worth `A(T) / (r M(T))`, and + `NPV = A(T) - A(2T) + A(T) (M(2T) - M(T)) / M(T)`. The discount rate + cancels from the last term, so it holds undiscounted too (the + classic comparison: the cycle cost saved, less the cost rate over + the time added). `M` is integrated by the Gauss-Kronrod rule that + the MTTF uses (`_mean_lifetime._integrate`). + - **Fixed intervals:** `NPV = planned exp(-r d) + exp(-r d) F(T) - (F(d + T) - + F(d))`, `d` the days to the next planned replacement and `F(t)` the expected + discounted cost of a new bearing's failures (and its replacements') + in `[0, t)`: `RepairableRBD.expected_cost(t, discount_rate=r)`. + - **The two costs compared.** On fixed intervals, each choice's cost + to the following planned replacement: done, `planned exp(-r d) + + exp(-r d) F(T) + F(d)`; skipped, `F(d + T)`. On a resetting plan with + a discount rate, the present value of the whole future under each: + done, the plan from new, `A(T) / (r M(T))`; skipped, the longer first + cycle and then the plan from new, `A(2T) + (1 - r M(2T)) A(T) / (r + M(T))`. Undiscounted those are infinite, and `replace` and `skip` + are `inf` (one meaning for the fields whatever the rate: the whole + future's present value; from the 0.14 persona review, where they + had shown the first cycles' comparison terms, 0 and -26.33); only + the NPV is finite, the classic comparison: each first cycle's cost + less its length at the plan's cost rate `g = A(T) / M(T)`, `A(T) - + g M(T)` (0, the plan's own) against `A(2T) - g M(2T)`. Either way + the difference is the NPV above. A rate that discounts the costs + away (0.08 given as a rate a day) is warned of on both plans, by + the check a `RepairableRBD`'s costs make (#231). + - **The discount rate** is continuous, per unit of the life's time, as + `RepairableRBD.expected_cost` takes it (0 by default): for 8% a year + with a life in days, `r = log(1.08) / 365.25`, so a cost a year away + is worth `1 / 1.08`. Reliafy turns a rate a year into it. +- **Today (before this use case):** the fixed-interval plan by hand from three + `expected_cost` calls and the formula above; the resetting plan not + at all (`NonRepairable` prices age replacement only in the long run, + undiscounted). +- **Design, as built:** a method of the part, in the life's time unit, + with the part's own costs: + + ```python + import math + from repyability import NonRepairable + + bearing = NonRepairable(surv.Weibull.from_params([365.0, 2.0]), planned_cost=150, unplanned_cost=400) + a_day = math.log(1.08) / 365.25 + value = bearing.skip_next_replacement(180, resets=True, discount_rate=a_day) + value.replace, value.skip, value.npv, value.best # 5735.39, 5708.84, 26.56, "skip" + + value = bearing.skip_next_replacement(180, resets=False, time_to_next=30, discount_rate=a_day) + value.npv # 123.08 + ``` + + `resets` has no default: the two plans answer differently, so it is + always said. A plan on fixed intervals needs `time_to_next`; a + resetting one refuses it (its next planned replacement is always + `every` away). `SkipValue` carries `resets`, `every`, `time_to_next`, + `discount_rate`, `replace`, `skip`, `npv = replace - skip` and `best`. + It was first built as a `ReplacementPlan(life, every, planned_cost, + unplanned_cost, resets=)` with a `skip_next(discount_rate, + days_to_next=)` in days and a rate a year, and reworked before release + (#350): UC-12's `optimal_next_replacement` answers the long-run, + undiscounted question of a part with no plan in force, and UC-6 the + discounted one about a plan in force, so each keeps a method of the + part. +- **Checks:** `test_maintenance_skip.py` checks: + - the answers against plain simulations of both plans (the resetting + one over 300 years, each choice's present value too); + - the two costs of a fixed plan against the part's failures over the + windows, and of a resetting plan against an exponential life's closed + forms; + - an exponential life against both plans' closed forms, at 0%, 8% and + 50% a year; + - the resetting plan with no discounting against the classic + undiscounted comparison, and a tiny rate next to it; + - the 180-day plan's NPVs falling with the time to the next one, and + the 365-day plan's change of sign; + - the same answer with the life in hours and the rate an hour; + - the input checks, and that the costs are the part's own. +- **Learned / open questions:** + - **The plan itself may be the question.** With these costs a 180-day + plan is too frequent, so skipping always pays. Comparing the plan + with the best one (discounted), or the latest day at which skipping + a fixed-interval replacement still pays, are natural next use cases. + - **Model time, not dates.** "Right after a failure" needs only the + time to the next planned replacement; dates, and a rate a year, are + Reliafy's. + - **Replacements take no time.** + - **The API** (#350): the maintainer did not like `skip_next` on a plan + object of its own. It is now a comparison of the two options, each a + present-value cost, from an explicit state (right after a failure), + on the part (`NonRepairable`). Deferring, or moving to a shut, are + UC-12's and UC-13's. + +### UC-7: "Replacing the bearing at a shut (every 180 days) costs $100, a planned replacement between shuts $150, and one at a failure $400. Which replacement policy is cheapest in the long run?" + +- **Status:** built as `replacement_policies(life, shut_every=, + shut_cost=, planned_cost=, failure_cost=)` (#339), then removed before + release: UC-13's `NonRepairable.optimal_calendar_policy` answers it (one + shut type, a planned replacement on any day, a failure), finding the best + policy of any form rather than the best of four families. The record + below is kept for what it found. +- **Who asks, and why:** the planner of UC-6. The plant shuts every 180 + days; replacing in a shut is cheaper than a planned replacement that + needs its own outage, which is cheaper than a failure. +- **What they know:** the bearing of UC-5 (Weibull, scale 365 days, + shape 2), the shut interval and the three prices. +- **What they want back:** each policy's long-run cost a day, and the + best. +- **The policies** (the last holds the others): + 1. run to failure; + 2. age replacement between shuts at age τ (the plan resets); + 3. at every shut, or every second shut (block replacement); + 4. at each shut, only if the bearing is at least *a* days old; + 5. both: at a shut if at least *a* old, or between shuts at age τ. +- **The answer:** + + | Policy | Cost a day | + |---|---| + | Run to failure | 1.237 | + | Age replacement between shuts, best τ about 300 days | 1.117 | + | Every second shut | 1.098 | + | Every shut | 1.056 | + | **At each shut, if at least about 138 days old** | **0.990** | + + Any *a* from 120 to 180 days costs within 0.1% of the best. A + replacement between shuts never pays at these prices (policy 5's best + τ is "never"): the shuts come often enough. +- **How it is solved:** + - policies 1 to 3: a one-component `RepairableRBD`'s + `expected_cost_rate()`, exact; + - policy 4: exactly too, through replacement on condition, which + replaces at an inspection a unit more likely than a threshold to fail + before the next. For a hazard that rises (as this Weibull's), that is + a unit at least *a* old, at the threshold `1 - R(a + 180) / R(a)`; + the best *a* by a one-dimensional search; + - policy 5: simulated by hand (RePyability's policies do not combine a + condition at shuts with an age between them). +- **Checks:** every cost rate against a plain simulation of the policy + (which runs about 0.001 a day low, from stopping at 150,000 days: run + to failure 1.2354 against the exact 1.2366). +- **Learned:** + - **It agrees with UC-6.** With the shut price, UC-6's value of + skipping the next shut right after a failure changes sign when the + shut is about 145 days away (discounted at 0.021% a day, 8% a year; + 142 undiscounted), next to the best *a*. + - **An age, not a condition.** A rule "at least *a* old at a shut" + should be given as an age for any life: where the hazard does not + rise, an age and a condition threshold differ. + - Policy 5 exactly needs a Markov chain on how far into the shut + interval each new bearing went in, on a grid, if a case needs it. +- **As built:** the maintainer asked for the tools to find the best policy + of four families: run to failure, every *d* days between shuts, every + *z*-th shut, and every *x*-th shut as long as older than *y*. + + ```python + import surpyval as surv + from repyability import replacement_policies + + found = replacement_policies( + surv.Weibull.from_params([365.0, 2.0]), + shut_every=180.0, shut_cost=100.0, planned_cost=150.0, failure_cost=400.0, + ) + found.run_to_failure # 1.2366 a day + found.between_shuts # at age 297.6, 1.1170 + found.every_shut # every shut (z = 1), 1.0555 + found.at_a_shut_if_older # every shut (x = 1) if at least 137.9 old, 0.9903 + found.best # the last; found.saving, 0.2463 a day + ``` + + - **One calculation for the shut families:** replacement on condition at + inspections every *x* shuts, its rule an age (`older_than`, new to + `_condition_replacement.condition_cycle`; 0 for every shut), exact for + any life, not only one whose hazard rises. Between shuts: + `NonRepairable.optimal_replacement_policy` at the planned price. + - **The search:** every number of shuts, and for each the age on a grid + of 8 a shut interval, refined around the best, up to the age a unit + outlives with a chance of 0.1%. About 10 seconds for the bearing. + - **Learned while building:** a rule that seldom replaces (every sixth + shut, if older than 1,080 days) has cycles of many thousands of + failures, and its cost a day is good only to a few parts in 10,000. + With a shut dearer than a failure, such a rule "beat" running to + failure by 0.01%, which cannot be (a planned replacement dearer than + a failure never pays). So a planned policy is chosen only if it saves + more than 0.1% against running to failure. + - **Checks** (`test_maintenance_shuts.py`): the UC-7 figures; every shut + against RePyability's block replacement (to 1e-6); three shut + policies against a plain simulation; the age rule against the + threshold rule for a rising hazard; an exponential life (no planned + replacement pays); a shut dearer than a failure; the answer the same + in days and in hours; JSON; the input checks. + +### UC-8: "From the bearing's state now, what will each policy cost over the next *t* days, and how much could that vary?" + +(First asked over a timeline of dates; the windows here are the days +from now to the end of each quarter, as Reliafy would turn them.) + +- **Status:** answered by hand with today's RePyability; not built. +- **Who asks, and why:** the planner of UC-7: a long-run cost a day says + which policy is cheapest, but a budget starts from the bearing as it + is, and needs to know how far the spend can stray from the mean. +- **What they know:** UC-7's bearing, prices and policies; its state: 224 + days old, the next shut in 21 days (159 days into the 180-day shut + interval). +- **What they want back:** the cost over windows of 82, 447, 813 and 1,086 + days from now: its mean and its spread (10th, 50th, 90th percentiles). +- **The answer** (dollars): + + **The best policy** (at a shut if at least 138 days old): + + | Window (days) | Mean | 10% | 50% | 90% | + |---|---|---|---|---| + | 82 | 133 | 100 | 100 | 100 | + | 447 | 487 | 300 | 300 | 900 | + | 813 | 846 | 500 | 800 | 1,400 | + | 1,086 | 1,090 | 600 | 1,000 | 1,600 | + + **Over 1,086 days:** + + | Policy | Mean | 10% to 90% | Long run, a year | + |---|---|---|---| + | At a shut if at least 138 days old | 1,090 | 600 to 1,600 | 362 | + | Every shut | 1,155 | 600 to 1,800 | 386 | + | Run to failure | 1,373 | 800 to 2,000 | 452 | + + - **The first window costs more than the long run** (about $590 a year + over the first 82 days, against $362): the bearing is old, and is + almost sure to be replaced at the shut in 21 days. + - **The spend comes in steps** ($100 at a shut, $400 at a failure), so + the percentiles jump. + - **The best policy lowers the mean more than the spread.** +- **How it is solved:** + - **the mean:** `RepairableRBD.expected_cost(t, state=...)`, exact, from + `NodeState(age=224, since_scheduled=159)`; + - **the spread:** simulated, as a spread must be: + `RepairableRBD.cost(t, state=..., seed=...)` for each window with one + seed. Simulation *r* draws the same lives whatever the window, so the + runs to each window are the same histories cut off there. +- **Checks:** the simulated means agree with the exact ones within $2 + (20,000 simulations each). +- **Learned:** each window's spread must come from the same simulations, + or the 90th percentiles of longer windows need not even rise; the + seeded streams give that for free. What RePyability could give in one + call: a policy's cost over several windows from a state, its mean exact + and its distribution from one run. + +### UC-9: "What will the bearing cost in each of a run of consecutive windows from now, and how often does each outcome happen in each?" + +(First asked by financial year, July to June, from 11 October 2026; the +windows here, 0 to 263, 263 to 629, 629 to 994 and 994 to 1,359 days +from now, are those years as Reliafy would turn them. Summing by +financial year is Reliafy's.) + +- **Status:** answered by hand with today's RePyability; not built. +- **Who asks, and why:** the planner of UC-8, passing a budget on: each + window's own spend, not the running total, and what could happen in + it. +- **What they know:** UC-8's bearing, its state and the best policy of + UC-7. +- **The answer:** the share of simulations with each outcome in each + window (1,000,000 plain simulations; RePyability's own agree within + about 0.3 points): + + | What happens in the window | Cost | 0–263 | 263–629 | 629–994 | 994–1,359 | + |---|---|---|---|---|---| + | No failure, one shut replacement | $100 | – | 0.9% | 1.2% | 1.5% | + | No failure, two shut replacements | $200 | 70.7% | 59.1% | 58.6% | 58.2% | + | One failure, no shut replacement | $400 | – | 0.5% | 0.7% | 0.8% | + | One failure, one shut replacement | $500 | 22.1% | 27.2% | 26.9% | 26.4% | + | One failure, two shut replacements | $600 | 3.0% | 2.9% | 3.2% | 3.5% | + | Two failures, no shut replacement | $800 | 1.4% | 4.8% | 4.6% | 4.5% | + | Two failures, one shut replacement | $900 | 2.5% | 3.5% | 3.6% | 3.8% | + | Three or more failures, or two and both shuts | $1,000+ | 0.3% | 1.2% | 1.2% | 1.2% | + | Mean (exact) | | $307 | $359 | $359 | $359 | + + The chance of more than $200 (any failure) is 29% in the first window + and 40% in each full one; of more than $600, 4% and 9%. +- **How it is solved:** as UC-8, at each window's end: the means by + differencing `expected_cost(t, state=...)`, exact; each simulation's + cost in a window as its cost to the window's end less its cost to its + start, from one seed (`cost(...).samples`), so every window's outcomes + are of the same histories. +- **Checks:** no simulation spends less than $0 in a window; the + simulated means agree with the exact ones within $4. +- **Learned:** + - **Costs in steps need their outcomes, not percentiles.** Here the + 10th, 50th and 90th percentiles read $200, $200, $600, and hide that + 60% of full windows cost exactly $200: give each outcome and its + share, and the chance of exceeding a budget. + - **Ranges do not add.** The four windows' 90th percentiles sum to + $2,300; the 90th percentile of their total is $2,000. + - What RePyability could give: each simulation's cost in each of a run + of windows, from one run (by hand it took a run per window's end). + +### UC-11: "My pump's seal is replaced at every 500-hour service and whenever it fails. It has run 340 hours, and the next service is in 160. What's the chance it fails before each service, what will failures and replacements cost over the next 944 running hours, and is replacing it at every service worth it?" + +(It comes from Reliafy's side: the pump's on and off times give its +running hours, so the seal's age (340 h, since it went in at the last +service), how far the pump is into its service interval (340 of 500 h) +and, through the operating schedule, how many running hours lie ahead +(944 h to the end of the year, at 16 hours a weekday). Turning the +answers back into dates is Reliafy's too.) + +- **Status:** answered by hand with today's RePyability; design to agree. +- **Who asks, and why:** a planner whose part is renewed by the + maintenance as well as at failures, so its risk starts again at each + service: what is the risk before the next one, and is the plan right? +- **What they know:** + - the seal's life in running hours, fitted in surpyval from the pump's + history: a Weibull with a scale of 6,000 h and a shape of 2.2; + - the plan: a new seal at every service, every 500 running hours on + the pump's meter (block replacement: the services do not move when a + seal fails), for $200; a new seal at a failure, at once, for $900; + - the state: the seal is 340 h old, and the next service is in 160 h. +- **What they want back:** the chance it fails before each service, the + expected failures, replacements and cost over the hours ahead, and + whether the plan is worth it. +- **The answer:** + - **Before the next service** (160 h): a **0.24%** chance it fails, + `1 - R(500) / R(340)`. + - **Before each later service:** a new seal goes in at each, so the + chance is the same in every interval, **0.42%**, `1 - R(500)`. + - **Over the next 944 running hours** (two services): **0.0078 failures + expected, 2 seals at services, $407** ($7 of failures, $400 of + planned seals). + - **Without the replacements at services**, the same seal from 340 h + would fail in those 944 h with a chance of 3.1% ($28 expected); over + 5,000 h, 54%. + - **The plan is not worth it.** Renewing at every service costs $0.41 a + running hour; the best age replacement (`NonRepairable(..., + planned_cost=200, unplanned_cost=900).optimal_replacement_policy()`) + replaces at about 3,180 h, for $0.12 a running hour. With services + every 500 h, the natural policy is UC-7's: renew at a service only if + the seal is older than some age. +- **How it is solved:** in running hours, by today's methods: + - the chances: the life conditioned on the age, exact; + - the expected failures, planned replacements and cost: a + one-component `RepairableRBD` with a block replacement every 500 h + (`"preventive": {"interval": 500, "policy": "block", "cost": 200}`), + `replace_cost` 900, started from `NodeState(age=340, since_scheduled=340)`: + `expected_events(944, state=...)` and `expected_cost(944, state=...)`, + exact; + - the long-run cost: its `expected_cost_rate()`. +- **Today (by hand):** + + ```python + import surpyval as surv + from repyability import NodeState, RepairableRBD + + life = surv.Weibull.from_params([6000.0, 2.2]) # running hours + rbd = RepairableRBD( + [("s", "seal"), ("seal", "t")], + {"seal": {"reliability": life, "repairability": "instant", + "replace_cost": 900.0, + "preventive": {"interval": 500.0, "policy": "block", "cost": 200.0}}}, + ) + state = {"seal": NodeState(age=340.0, since_scheduled=340.0)} + rbd.expected_events(944.0, state=state).node_failures # {'seal': 0.0078} + rbd.expected_cost(944.0, state=state).mean # 407.06 + 1 - life.sf(500) / life.sf(340) # 0.0024, before the next service + ``` + +- **Checks:** the expected failures over 944 h against a plain simulation + of the seal and its services (0.0080 ± 0.0001, against 0.0078). +- **Proposed design (to agree with the maintainer):** a part's risk and + costs under a plan that renews it, on the part, in model time, beside + `failures`: the plan (renew every so many hours, keeping to the + interval or resetting at each renewal) and the state (the part's age, + how far into the interval), giving the chance of a failure before each + of the next renewals, and the expected failures, renewals and cost + over a window. It shares its open questions with UC-6's rework (the + plan as arguments or an object, the state given explicitly). +- **Learned:** + - **Renewal at each service resets the risk:** the chance before a + service is small and the same each time, where without it the risk + grows with the seal's age. + - **The answer can show the plan is wrong:** here a renewal at every + service costs over three times the best age replacement, which is + the question to ask next (UC-7's shape, at services). + - **History in, risk out:** Reliafy's history gives the age and the + time since the last service in running hours, RePyability the risk + and cost in running hours, and the schedule turns them into dates. + +### UC-12: "The bearing is 200 days old. Replacing it now costs $150; a planned replacement any time costs $150 too, and one at a failure $400. Outages are scheduled in 150 and 330 days, at which replacing it costs $100. Do I replace it now?" + +- **Status:** built: `NonRepairable.optimal_next_replacement(age, + now_cost=, scheduled_outages=, policy=)` (see *As built*); `policy=`, + the baseline as a `CalendarPolicy`, added by #348. +- **Who asks, and why:** a planner deciding on one part, today, from its + age and the outages scheduled. The maintainer asked for this first, as + the clean starting point for the shut questions (UC-7's policies are + rules for every part; this is one decision for this part). +- **What they know:** + - the part's life (UC-5's bearing: Weibull, scale 365 days, shape 2) + and its age now; + - the price of replacing it now (an outage today, an opportunity, or a + planned job), of a planned replacement at any time, and of one at a + failure; + - the scheduled outages (shuts, turnarounds), each as a time from now + and what replacing the part at it costs (Reliafy turns the dates + into times). +- **What they want back:** replace now or not; what each choice costs; + and the age at which replacing now becomes the best choice. +- **The choices**, for this part only: replace it now; keep it to one + of the scheduled outages and replace it there; keep it to a planned + replacement at a time between (the best time, found continuously); + or run it to failure. In each, a failure first means a replacement at + the failure price. +- **How it is solved:** each choice's expected cost until this part is + replaced, less the value of the days it serves: + + `cost(u, price) = price · S(u) + failure · (1 - S(u)) - g · ∫₀ᵘ S`, + + where `S(u) = R(age + u) / R(age)` is the chance it is still working + `u` from now, and `g`, the **baseline**, is what a day of the part's + service is worth: the long-run cost a day of how the position is run. + By default the age replacement's + (`optimal_replacement_policy().cost_rate`, 1.117 a day for the + bearing); given `policy=`, a `CalendarPolicy` (UC-13) or a + `MaintenancePolicy`, its `cost_rate` (#348). Replacing now costs its + price, and serves no days of this part. The best choice is the + smallest. +- **Why the days are valued:** keeping the part means a new one, and + every part after it, starts later; the days the old one serves are days + a new one would otherwise serve, at the baseline's cost a day. Without + `g`, a bearing 6 days old would be replaced: kept to the outage in 150 + days, 83% reach it ($100) and 17% fail ($400), $150 on average, as + replacing now costs. With it, the 141 days it serves on average are + worth 141 × 1.117 = $157.5, so keeping it costs -$7.5 against $150. +- **With no outages it is the textbook age replacement.** The best later + replacement of a part of age *a* is at 297.6 - *a* days from now, and + replacing now is the best from 297.6 days old: the optimal interval + exactly. The baseline is that calculation's own cost rate: the optimal + interval is the age at which a day's added risk, (400 - 150) × hazard, + reaches it. +- **The answer:** + + | Choice | Age 100 | Age 200 | Age 300 | + |---|---|---|---| + | Replace now ($150) | 150.0 | 150.0 | **150.0** | + | At the outage in 150 days ($100) | **55.3** | **110.0** | 154.7 | + | At the outage in 330 days ($100) | 84.5 | 156.5 | 206.4 | + | Planned, at the best time ($150) | 86.3 (in 198 days) | 133.9 (in 98) | 150.0 (now) | + | Run to failure | 128.0 | 186.1 | 226.0 | + + At 200 days old: **keep it to the outage in 150 days**. + + Each later choice has its own break-even age, past which replacing now + beats it; replacing now is the best once it beats them all: + + | Replacing now beats... | from age | + |---|---| + | running to failure | 133.3 days | + | the outage in 330 days | 189.3 days | + | the outage in 150 days | 288.4 days | + | a planned replacement later | 297.6 days | + + So **replace now from 297.6 days old**, the optimal age (from 288.4, + a planned replacement within days is cheaper by very little). +- **When cheap shuts keep coming, the baseline is the calendar policy's + (#348).** With a shut every 180 days at $100 (UC-13's + `optimal_calendar_policy`), a day of the bearing's service is worth + 0.990 rather than 1.117, so keeping it is dearer at every age and + replacing now beats each later choice from a younger age. At 200 days + old the outage in 150 days is still the best (124.6, against 150.0 + now, 178.5 at the outage in 330 days, 142.8 planned in 64 days and + 210.4 run to failure); the break-even ages: + + | Replacing now beats... | from age, age replacement's rate | with the shuts' | + |---|---|---| + | running to failure | 133.3 days | 86.4 days | + | the outage in 330 days | 189.3 days | 153.0 days | + | the outage in 150 days | 288.4 days | 256.1 days | + | a planned replacement later | 297.6 days | 263.9 days | + + So with the shuts kept coming, **replace now from 263.9 days old**. + The policy of a calendar with no opportunities (replacement on any + day at the planned price) is age replacement, and gives the answer + above: its rate is exact for its grid, and the break-even ages agree + to 1e-4 days. +- **As built:** + + ```python + import surpyval as surv + from repyability import NonRepairable + + bearing = NonRepairable( + surv.Weibull.from_params([365.0, 2.0]), + planned_cost=150.0, unplanned_cost=400.0, + ) + found = bearing.optimal_next_replacement( + 200.0, scheduled_outages=[(150.0, 100.0), (330.0, 100.0)] + ) + found.best # the outage in 150 days, 110.0 + found.now # False + found.choices # now, each outage, planned (in 97.6 days), failure + found.replace_from_age # 297.6: replacing now beats every later choice + found.cost_rate # 1.117 a day: the baseline + + from repyability import Calendar, Opportunity + + shuts = bearing.optimal_calendar_policy( + Calendar([Opportunity("shut", 100.0, period=180.0)]) + ) + found = bearing.optimal_next_replacement( + 200.0, scheduled_outages=[(150.0, 100.0), (330.0, 100.0)], + policy=shuts, + ) + found.cost_rate # 0.990 a day: the shuts' policy's + found.replace_from_age # 263.9 + ``` + + - **The maintainer's choices:** the name (`optimal_replacement_policy` + is the rule for a part in general, a hypothetical one, and + `optimal_next_replacement` the next replacement of the one in + service); the outages as `scheduled_outages`, (time from now, price) + pairs and nothing else; and the baseline the optimal policy: the + replacement policy's, or the policy given (`policy=`, a + `CalendarPolicy` or a `MaintenancePolicy`; its cost rate alone is + refused, naming the policy to give; #348). A first version also + took the regular shuts (`shut_every=`, `shut_cost=`) to value a day + at `replacement_policies`' best (0.990 a day with shuts every 180 + days, which moves the break-even against the outage in 150 days to + 256 days): that doubled up on the outages, and assumed what comes + after the listed ones; UC-13's calendar policy now says what the + shuts are worth, and the outages listed stay the ones scheduled. + - `now_cost` is the planned price unless given (an outage today, or a + chance while the machine is apart, is cheaper). **A planned + replacement is one later than now** (from the 0.14 persona review): + where no later time beats replacing at once (a part past its + optimal age), replacing at once is replacing now, at `now_cost`, so + the planned choice is then replacing now (time 0, at `now_cost`, + never the planned price at time 0 that `now_cost` contradicts), + `now` is True exactly when the best choice is at time 0, and + `replace_from_age` follows. A part 400 days old with `now_cost=500` + runs to failure (254.5, against 500 now, from no age); with + `now_cost=100` it is replaced now. A planned replacement with a + lead time is a scheduled outage at that time. + - **Each choice's break-even age** (`ReplacementChoice.break_even_age`) + is the youngest at which replacing now costs no more than it, on a + grid of 64 ages up to the one a new part reaches with a chance of + 0.1%, then halved down to; `replace_from_age` is the largest. A + planned replacement's best time is found on a grid of 2,001 times up + to the time the part outlives with a chance of 0.1%, refined; if none + beats running to failure, its time is `inf`. + - **Lives:** those with a survival function in closed form (surpyval's + parametric distributions, mixtures, degradation processes); a + non-parametric estimate or a `StandbyModel` is refused. + - **Checks** (`test_maintenance_next.py`): the figures above; with no + outages, the best planned time is the optimal age less the part's, + and replacing now the best from the optimal age (to 1e-4); every + choice against a plain simulation of it, at either baseline; the + policy of a calendar with no opportunities gives the default's + answer (to 1e-3), the shuts' policy the figures above, every choice + dearer and beaten younger, and a break-even age is where the choice + costs what replacing now does; the age replacement policy given is + the default; a cost rate, a calendar or anything but a policy is + refused; an exponential life (each + outage's cost `price · exp(-λu)`, running to failure 0 and best); + a planned replacement at once is replacing now, at `now_cost` above + and below the planned price, and `now` is True exactly when the + best choice is at time 0; a + cheaper chance now breaks even younger; the answer the same in days + and in hours; JSON; the input checks. +- **Learned / open questions:** + - It is a break-even age for each later choice, as the maintainer + framed it: the decision is "replace now" once the part is older than + the largest. + - **Still to check:** the decision against the timeline calculation + (the best decision for every age and day, worked back from past the + last scheduled outage), to measure what valuing a new part the same + whenever it is fitted leaves out. + +### UC-13: "Replacing the bearing costs 1 in the annual shut, 2 in a quarterly shut, 3 at a weekly stop, 4 on any other day and 20 at a failure. What is the best replacement policy in the long run?" + +- **Status:** built (the maintainer's brief, "long-run optimal replacement + policy over an outage calendar"): `NonRepairable.optimal_calendar_policy`, + `calendar_policy_cost`, `Opportunity`, `Calendar`, `CalendarPolicy`. + Decisions to take or verify: `repyability/maintenance/CALENDAR_DECISIONS.md`. +- **Who asks, and why:** a reliability engineer setting a part's + replacement rule in a plant whose shuts make replacing cheap at some + times and dear at others. Age replacement assumes one planned price at + every moment; here the price depends on the time. +- **What they know:** the part's life (Weibull, scale 1000 days, shape + 2.5); the shuts, each with its period, offset (or dates) and price; the + price on any other day (or none); the price at a failure. +- **What they want back:** the long-run cost a day and what it is made + of (replacements and cost by kind of shut, and failures), which shuts + the policy uses and the age from which each replaces, when the unit in + service will be replaced, and what simpler rules cost. +- **How it is solved:** the state is the time and the age; dynamic + programming on a grid (relative value iteration over a periodic + calendar's cycle, exact for the grid; a long finite horizon otherwise), + integration between opportunities in continuous time (no replacement on + any day), and, for a cycle too long to sweep, the cost rate sampled over + times of the cycle, each a window solved exactly. +- **The answer:** 0.006943 a day (25.27 over ten years); the annual shut + replaces a unit of 88 days or more, the quarterly shuts one of 222, 324 + and 574 days (the last a quarter before the annual), the weekly stops + older ones still, and no unit kept reaches the age the dearer day would + replace it at. Every annual shut costs 25.48 over ten years, age + replacement on any day 50.41, running to failure 82.05. +- **Checks** (`test_maintenance_calendar.py`, + `test_maintenance_calendar_methods.py`): age replacement with no + opportunities (to 1e-5); the cycle's rate against a long horizon's + (to 1e-6); the policy simulated plainly (within 2 standard errors); the + phase effect; skipping a young unit; the policy over 10, 20 and 40 + years (the same) and 5 (within 10 days); the simple policies; a life + whose replacing ages are no threshold (flagged); repeated plans against + the periodic calendar; integration against the grid, the daily limit + (age replacement to 1e-6) and long gaps; sampling against the exact + rate, a co-prime calendar, its determinism; the routes; units. +- **Learned:** + - The brief's recursion renews a failed unit at the next step, low by + about half a step a failure (1.7e-4 for this bearing at a step of a + day). Renewing it half a step on average (as new or a step old, half + each) makes the error second order (1.3e-7). + - At prices like these the annual shut does nearly all the work: the + quarterly and weekly stops matter only for a unit a failure renewed + early in the year, so a calendar with a dearer failure (40) is the + one that tells the methods apart. + - **What the policy does, rather than a threshold for every time** (the + maintainer, after seeing it used): `threshold(t)` and `replace(t, + age)` were dropped as of no use (most times' thresholds, the weekly + stops' and the dearer day's, are never reached). In their place: + `replacement_rates` and `cost_rates` by kind (the long-run chance of + each step's replacement, forward from a new unit until the ages at a + cycle's start settle; exact for the grid, the costs summing to the + cost rate to 1e-11, and for the quadrature with integration); + `schedule()` gives each shut's long-run chance of a replacement (0 for + a shut never used); and `next_replacement(age, now)` says when the + unit in service will be replaced, at which shut and price. For the + plant: 0.962 replacements a year at the annual shut, 0.078 failures, + no other kind used. + - **From the 0.14 persona review** (a planner using the bearing of + UC-12, with shuts every 180 days at 100): + - **The step is sized from the periods, not the offsets.** A shut at + an offset of 0.1234 days (three hours into the day, what a date + turned into model time gives) had the step sized from the gcd of + the period and the offset, 1/5000, far below the life's step, so + the life-sized step was taken, which does not divide 180: the + calendar lost its period, `"auto"` sampled it (57 s for 0.5 s, + the cost rate 0.04% off, no period) and the schedule drifted + (0.0, 180.17, 360.03). The step is now a whole fraction of the + periods (and a plan's repeat), the offsets and dates on it too + only where that keeps it near the life's; an offset off the grid + is laid at the grid time nearest it (within half a step, with the + warning), and the policy is the aligned calendar's: the same step, + cost rate, thresholds and chances, `dp`, exact, period 180. + - **`"auto"` says when it samples** (the cycle has more than 20,000 + steps; `mc_samples` sets the cost), and the docs say that a + sampled policy's rates by kind and the schedule's chances are + estimates from a unit run in with no standard error (they move by + some 10% between seeds at 8 samples); only `cost_rate_se` has one. + - **A calendar allowing no planned replacement is refused** + (`Calendar()` on a part with no `planned_cost`, or + `Calendar(anytime=None)`), as `optimal_replacement_policy` refuses + with no costs, rather than return running to failure. + - **A unit past the oldest age worked out is replaced at the next + opportunity** (`next_replacement(5000.0, now=0.0)` gives the shut + at 0, not `inf`): it is past every threshold. + - **The policy carries the life it was worked out for** (`life`, its + saved form), and a `RepairableRBD` component with another life is + refused, naming both: a policy worked out for the Weibull(1000, + 2.5) bearing given to the Weibull(365, 2) one simulated at a cost + rate 19% off the one worked out for it, silently. + diff --git a/repyability/maintenance.py b/repyability/maintenance/__init__.py similarity index 72% rename from repyability/maintenance.py rename to repyability/maintenance/__init__.py index c8bee66d..e013260b 100644 --- a/repyability/maintenance.py +++ b/repyability/maintenance/__init__.py @@ -1,10 +1,18 @@ -"""Shared result types for component maintenance-policy optimisations.""" +"""Maintenance: the single-unit optimisers' result types, and the choices +about a part's next replacement (#319). + +RePyability is about timelines: the calendar side of maintenance planning +(tasks on dates, operating schedules, meters read by date) is Reliafy's. +``README.md`` in this package records the use cases, the question each +answers and how it is solved.""" from dataclasses import dataclass +from repyability.rbd.results import _ResultMapping + @dataclass(frozen=True) -class MaintenancePolicy: +class MaintenancePolicy(_ResultMapping): """An optimal preventive-maintenance policy for a single component. Returned by the ``optimal_replacement_policy()`` method of @@ -13,7 +21,8 @@ class MaintenancePolicy: ``optimal_overhaul_policy()`` method of [`Repairable`][repyability.Repairable] (periodic overhaul: repair each failure and renew the unit every ``interval``). It is an immutable - (frozen) dataclass. + (frozen) dataclass, read as a mapping too, with ``to_dict()`` for + JSON. Attributes ---------- @@ -43,13 +52,14 @@ class MaintenancePolicy: @dataclass(frozen=True) -class FailureLimitPolicy: +class FailureLimitPolicy(_ResultMapping): """An optimal failure-limit (replace-at-N-th-failure) policy. Returned by the ``optimal_failure_limit_policy()`` method of [`Repairable`][repyability.Repairable]: rather than renewing at a fixed *age*, the unit is repaired on each failure and replaced at the - ``failure_count``-th failure. It is an immutable (frozen) dataclass. + ``failure_count``-th failure. It is an immutable (frozen) dataclass, + read as a mapping too, with ``to_dict()`` for JSON. Attributes ---------- @@ -81,3 +91,19 @@ class FailureLimitPolicy: failure_count: int cost_rate: float + + +# After the policies, which the modules it imports may import from here. +from repyability.maintenance._calendar import ( # noqa: E402 + Calendar, + Opportunity, +) +from repyability.maintenance._next import ( # noqa: E402 + NextReplacement, + ReplacementChoice, +) +from repyability.maintenance._replacement import ( # noqa: E402 + CalendarPolicy, + PlannedReplacement, +) +from repyability.maintenance._skip import SkipValue # noqa: E402 diff --git a/repyability/maintenance/_calendar.py b/repyability/maintenance/_calendar.py new file mode 100644 index 00000000..66543c0a --- /dev/null +++ b/repyability/maintenance/_calendar.py @@ -0,0 +1,463 @@ +"""The opportunities to replace a part, in model time (#319, UC-13 of this +package's ``README.md``): shuts on a period (weekly, quarterly, annual, +each with its own price and offset) or on given times, a planned +replacement on any day at a dearer price, and a failure at the dearest. + +Times are numbers in the life model's unit, from 0; turning dates into +them is Reliafy's (``CLAUDE.md``: RePyability is about timelines). A +calendar is laid on a grid of a step for the dynamic programme +(``_Grid``): each date to its nearest grid time, so a date moves by half a +step at most and the dates of a period keep no drift. +""" + +import math +import warnings +from dataclasses import dataclass +from fractions import Fraction +from typing import Optional, Tuple + +import numpy as np + +from repyability.utils import checks +from repyability.utils.wrappers import outside_level + +#: The name of a planned replacement on any day, among the kinds of +#: replacement a schedule lists; no opportunity may take it. +ANYTIME = "anytime" + +#: A price taken from the part: its unplanned cost (a failure) or its +#: planned cost (a replacement on any day). +PART = "part" + +#: The largest denominator a period is read with, as a fraction: a period +#: of 365.25 is 1461/4, of 30.4375 is 487/16. +_DENOMINATOR = 1_000_000 + +#: The share of a step a date may move, laid on the grid, before a +#: warning says so. +_MOVED = 0.01 + + +def _fraction(value: float) -> Fraction: + """``value`` as the fraction it is read as, for periods' common + multiple.""" + return Fraction(value).limit_denominator(_DENOMINATOR) + + +def _common_multiple(values) -> Fraction: + """The least common multiple of positive fractions.""" + fractions = [_fraction(v) for v in values] + numerator = math.lcm(*(f.numerator for f in fractions)) + denominator = math.gcd(*(f.denominator for f in fractions)) + return Fraction(numerator, denominator) + + +def _price(value, name: str) -> float: + """``value`` as a price: a finite number, at least 0.""" + if not (checks.is_number(value) and 0.0 <= float(value) < math.inf): + raise ValueError( + f"{name} must be a number, at least 0; got {value!r}." + ) + return float(value) + + +@dataclass(frozen=True) +class Opportunity: + """A kind of scheduled opportunity to replace a part: a shut (weekly, + quarterly, annual, ...) at which replacing it costs ``cost``. + + Its times are either every ``period`` from ``offset`` (on and on, both + ways: ``offset + k * period`` for every whole ``k``), or the given + ``dates``. Times are in the life model's unit, from 0. + + Parameters + ---------- + name : str + The opportunity's name, as the policy's schedule lists it (not + ``"anytime"``, which names a replacement on any day). + cost : float + What replacing the part at it costs, at least 0. + period : float, optional + The time from one to the next, more than 0. + offset : float, optional + The time of one of them, by default 0. Only with ``period``. + dates : array_like, optional + Its times, instead of ``period``: finite numbers, put in order. + + Examples + -------- + >>> from repyability import Opportunity + >>> weekly = Opportunity("weekly", 3.0, period=7.0) + >>> annual = Opportunity("annual", 1.0, period=364.0, offset=182.0) + >>> turnarounds = Opportunity("turnaround", 0.5, dates=[400.0, 1130.0]) + >>> weekly.times(0.0, 30.0).tolist() + [0.0, 7.0, 14.0, 21.0, 28.0] + """ + + name: str + cost: float + period: Optional[float] = None + offset: float = 0.0 + dates: Optional[Tuple[float, ...]] = None + + def __post_init__(self): + if not isinstance(self.name, str) or not self.name: + raise TypeError( + f"An opportunity's name must be text; got {self.name!r}." + ) + if self.name == ANYTIME: + raise ValueError( + f"{ANYTIME!r} names a planned replacement on any day (the " + "calendar's anytime price): give the opportunity another " + "name." + ) + object.__setattr__( + self, "cost", _price(self.cost, f"The cost of {self.name!r}") + ) + if (self.period is None) == (self.dates is None): + raise ValueError( + f"Opportunity {self.name!r}: give a period (with an " + "offset) or dates, one of the two." + ) + if self.period is not None: + if not ( + checks.is_number(self.period) + and 0.0 < float(self.period) < math.inf + ): + raise ValueError( + f"Opportunity {self.name!r}: the period must be a " + f"number, more than 0; got {self.period!r}." + ) + if not ( + checks.is_number(self.offset) + and math.isfinite(float(self.offset)) + ): + raise ValueError( + f"Opportunity {self.name!r}: the offset must be a " + f"finite number; got {self.offset!r}." + ) + object.__setattr__(self, "period", float(self.period)) + object.__setattr__(self, "offset", float(self.offset)) + else: + if self.offset != 0.0: + raise ValueError( + f"Opportunity {self.name!r}: an offset applies only to " + "a period; dates give the times themselves." + ) + dates = checks.real_array(self.dates, "dates") + if dates.ndim != 1 or dates.size == 0: + raise ValueError( + f"Opportunity {self.name!r}: dates must be a list of " + f"times, at least one; got {self.dates!r}." + ) + if not np.all(np.isfinite(dates)): + raise ValueError( + f"Opportunity {self.name!r}: dates must be finite; got " + f"{self.dates!r}." + ) + object.__setattr__( + self, "dates", tuple(float(d) for d in np.unique(dates)) + ) + + @property + def periodic(self) -> bool: + """Whether its times are on a period (not given dates).""" + return self.period is not None + + def times(self, start: float, end: float) -> np.ndarray: + """Its times from ``start`` up to ``end``. + + Parameters + ---------- + start : float + The first time to give, if one falls there. + end : float + The time to give them up to (not including). + + Returns + ------- + numpy.ndarray + Its times, in order. + """ + if self.period is None: + dates = np.asarray(self.dates, float) + return dates[(dates >= start) & (dates < end)] + first = math.ceil((start - self.offset) / self.period) + last = math.ceil((end - self.offset) / self.period) + times = self.offset + np.arange(first, last) * self.period + return times[(times >= start) & (times < end)] + + +class Calendar: + """The prices of replacing a part over time: its scheduled + opportunities, a planned replacement on any day, and a failure. + + Parameters + ---------- + opportunities : list of Opportunity + The scheduled opportunities, each with its own name. + failure : float or "part", optional + What a replacement at a failure costs. By default ``"part"``: the + part's ``unplanned_cost``. + anytime : float, "part" or None, optional + What a planned replacement on any day costs. By default + ``"part"``: the part's ``planned_cost``, if it has one (else no + replacement on any day). None: no replacement but at the + opportunities (and at failures). + + Examples + -------- + >>> from repyability import Calendar, Opportunity + >>> calendar = Calendar( + ... [ + ... Opportunity("weekly", 3.0, period=7.0), + ... Opportunity("annual", 1.0, period=364.0), + ... ], + ... failure=40.0, + ... anytime=4.0, + ... ) + >>> calendar.periodic, calendar.period + (True, 364.0) + >>> calendar.planned_cost([0.0, 3.0, 7.0]).tolist() + [1.0, 4.0, 3.0] + """ + + def __init__(self, opportunities=(), failure=PART, anytime=PART): + if isinstance(opportunities, Opportunity): + opportunities = [opportunities] + opportunities = tuple(opportunities) + for opportunity in opportunities: + if not isinstance(opportunity, Opportunity): + raise TypeError( + "A calendar's opportunities must each be an " + f"Opportunity; got {opportunity!r}." + ) + names = [o.name for o in opportunities] + repeated = sorted({n for n in names if names.count(n) > 1}) + if repeated: + raise ValueError( + f"Opportunity names must differ; {repeated} repeat." + ) + self.opportunities = opportunities + self.failure = ( + PART if _is_part(failure) else _price(failure, "failure") + ) + if anytime is None or _is_part(anytime): + self.anytime = anytime + else: + self.anytime = _price(anytime, "anytime") + + def __repr__(self) -> str: + return ( + f"Calendar({list(self.opportunities)!r}, failure=" + f"{self.failure!r}, anytime={self.anytime!r})" + ) + + @property + def periodic(self) -> bool: + """Whether every opportunity is on a period (no given dates), so + that the calendar repeats.""" + return all(o.periodic for o in self.opportunities) + + @property + def period(self) -> Optional[float]: + """The time after which a periodic calendar repeats: the least + common multiple of its periods (each read as a fraction), which + may be very long; None if it has dates, or no opportunities.""" + if not self.periodic or not self.opportunities: + return None + return float(_common_multiple(o.period for o in self.opportunities)) + + @property + def plan_end(self) -> Optional[float]: + """The last of the given dates; None if it has none.""" + dated = [o.dates[-1] for o in self.opportunities if not o.periodic] + return max(dated) if dated else None + + def _repeat(self) -> Optional[float]: + """The period a plan of given dates repeats with past its last + date: from its first date to its last, and the mean gap between + them (the date itself, with one); None with no dates.""" + dated = sorted( + d for o in self.opportunities if not o.periodic for d in o.dates + ) + if not dated: + return None + first, last = dated[0], dated[-1] + if len(dated) == 1: + if not last > 0.0: + raise ValueError( + "A plan of one date repeated after it must have it " + f"after 0; it is {last:g}." + ) + return last + return (last - first) * len(dated) / (len(dated) - 1) + + def planned_cost(self, times, step: Optional[float] = None) -> np.ndarray: + """The cheapest planned replacement at each of ``times``. + + Parameters + ---------- + times : float or array_like + The times, in the life's unit. + step : float, optional + A grid's step: an opportunity then counts at the grid time + nearest it, as the policy lays it. Without one, only at its + time itself. + + Returns + ------- + numpy.ndarray + The price of the cheapest planned replacement open at each + time (an opportunity there, or the calendar's own ``anytime`` + price), ``inf`` where none is. + """ + times = checks.real_array(times, "times") + out = np.full(times.shape, math.inf) + if checks.is_number(self.anytime): + out[:] = float(self.anytime) + flat = np.ravel(times) + if flat.size == 0: + return out + for opportunity in self.opportunities: + near = opportunity.times( + float(flat.min()) - (step or 0.0) - 1.0, + float(flat.max()) + (step or 0.0) + 1.0, + ) + if near.size == 0: + continue + at = np.clip(np.searchsorted(near, flat), 1, max(near.size - 1, 1)) + distance = ( + np.minimum( + np.abs(near[at - 1] - flat), np.abs(near[at] - flat) + ) + if near.size > 1 + else np.abs(near[0] - flat) + ) + if step is None: + hit = distance <= 1e-12 * np.maximum(1.0, np.abs(flat)) + else: + hit = distance <= 0.5 * float(step) * (1.0 + 1e-12) + cheaper = hit & (opportunity.cost < out.reshape(-1)) + out.reshape(-1)[cheaper] = opportunity.cost + return out + + def _prices(self, part) -> Tuple[float, Optional[float]]: + """The failure and anytime prices, with the part's for + ``"part"``.""" + if _is_part(self.failure): + if not hasattr(part, "cu"): + raise ValueError( + "The calendar takes the failure price from the part, " + "which has none: give the calendar a failure price, or " + "the part an unplanned_cost." + ) + failure = float(part.cu) + else: + failure = float(self.failure) + if _is_part(self.anytime): + anytime = float(part.cp) if hasattr(part, "cp") else None + else: + anytime = self.anytime + return failure, anytime + + +def _is_part(value) -> bool: + return isinstance(value, str) and value == PART + + +class _Grid: + """A calendar laid on a grid of ``step`` from time 0: the cheapest + planned price at each grid time, and which kind of replacement it is. + + ``tail`` extends given dates past the plan: ``"repeat"`` repeats them + every ``repeat`` (``Calendar._repeat``), forward only; ``"anytime"`` + leaves only the replacement on any day after them; None leaves them as + they are. ``cycle`` is the number of steps after which the grid + repeats, from ``cycle_start`` (None if it does not). + """ + + def __init__(self, calendar, anytime, step, tail=None): + self.calendar, self.anytime, self.step = calendar, anytime, step + self.names = [o.name for o in calendar.opportunities] + [ANYTIME] + self.tail = tail + end = calendar.plan_end + self.plan_end = end + self.repeat = calendar._repeat() if tail == "repeat" else None + # The grid index from which the prices repeat every cycle: the + # plan's first date, for a repeated plan. + self.cycle_start = 0 + if self.repeat is not None: + first = min( + d + for o in calendar.opportunities + if not o.periodic + for d in o.dates + ) + self.cycle_start = max(int(np.rint(first / step)), 0) + periods = [o.period for o in calendar.opportunities if o.periodic] + if self.repeat is not None: + periods.append(self.repeat) + self.cycle: Optional[int] + if calendar.opportunities and ( + calendar.periodic or self.repeat is not None + ): + self.cycle = int( + _common_multiple([p / step for p in periods] + [1.0]) + ) + elif not calendar.opportunities: + self.cycle = 1 + else: + self.cycle = None + self._warn_moved() + + def _warn_moved(self) -> None: + """Warn of an opportunity whose times are not on the grid.""" + for o in self.calendar.opportunities: + if o.periodic: + times = o.offset + np.arange(64) * o.period + else: + times = np.asarray(o.dates, float) + moved = np.abs(times - np.rint(times / self.step) * self.step) + if moved.max() > _MOVED * self.step: + warnings.warn( + f"The times of opportunity {o.name!r} are not on the " + f"grid of step {self.step:g}: each is taken at the " + f"grid time nearest it (moved by up to " + f"{moved.max():.3g}).", + stacklevel=outside_level(), + ) + + def _dates(self, o, start: float, end: float) -> np.ndarray: + if o.periodic: + return o.times(start, end) + dates = np.asarray(o.dates, float) + if self.repeat is None: + return dates[(dates >= start) & (dates < end)] + # Repeated forward from the plan, not before it. + first = max(math.floor((start - dates.max()) / self.repeat), 0) + last = math.ceil((end - dates.min()) / self.repeat) + shifts = np.arange(first, last + 1) * self.repeat + times = (dates[None, :] + shifts[:, None]).ravel() + return times[(times >= start) & (times < end)] + + def prices(self, start: int, n: int) -> Tuple[np.ndarray, np.ndarray]: + """The cheapest planned price at grid times ``start`` to + ``start + n - 1`` (``inf`` where there is none), and the index in + ``names`` of the kind of replacement it is (-1 where none).""" + step = self.step + costs = np.full(n, math.inf) + kinds = np.full(n, -1, dtype=np.int64) + anytime = self.anytime + if anytime is not None: + costs[:] = anytime + kinds[:] = len(self.names) - 1 + low, high = (start - 0.5) * step, (start + n - 0.5) * step + for k, o in enumerate(self.calendar.opportunities): + times = self._dates(o, low - step, high + step) + index = np.rint(times / step).astype(np.int64) - start + index = index[(index >= 0) & (index < n)] + cheaper = o.cost < costs[index] + costs[index[cheaper]] = o.cost + kinds[index[cheaper]] = k + return costs, kinds diff --git a/repyability/maintenance/_integration.py b/repyability/maintenance/_integration.py new file mode 100644 index 00000000..6ce8d7d7 --- /dev/null +++ b/repyability/maintenance/_integration.py @@ -0,0 +1,510 @@ +"""Phase 2 of the calendar policy (#319, UC-13): the same problem as the +dynamic programme on a grid (``_replacement``), with continuous time +between opportunities and no replacement on any day (a free boundary +inside a gap, left to the grid). + +At opportunity ``k`` (time ``t_k``, price ``c_k``, gap ``Δ`` to the next), +with ``V_{k+1}`` the cost to go at the next by the part's age: + + G(s) = ∫_s^{t_{k+1}} f(u - s) [c_f + G(u)] du + + R(t_{k+1} - s) V_{k+1}(t_{k+1} - s) + W_k(a) = ∫_0^Δ f(a + τ) / R(a) [c_f + G(t_k + τ)] dτ + + R(a + Δ) / R(a) V_{k+1}(a + Δ) + V_k(a) = min(W_k(a), c_k + W_k(0)) + +``G(s)`` is the cost to go of a new unit put in at ``s`` in the gap (after +a failure), a Volterra equation solved backward on ``NODES + 1`` nodes: +each integral by product integration, the distribution's increase over a +piece times the mean of the integrand's ends (exact in the density, so a +density unbounded at 0 is integrated as well as any). ``V_k`` lives on an +age grid of ``AGES`` points, linearly interpolated. A gap's matrices +depend on its length alone, so gaps of one length share them. + +Over a periodic calendar, relative value iteration across one cycle (as +``_replacement._relative_values``); a plan of given dates is repeated +past its last date. The threshold at an opportunity is where +``W_k(a) = c_k + W_k(0)``, by bisection. +""" + +import math +from fractions import Fraction +from typing import Dict + +import numpy as np + +from repyability.maintenance._calendar import ( + ANYTIME, + _common_multiple, + _fraction, +) +from repyability.maintenance._replacement import ( + FAR, + MAX_LIVES, + TOLERANCE, + CalendarPolicy, + _age_where, + _mean, + _survival, +) + +#: Quadrature pieces in each gap between opportunities. +NODES = 32 +#: Points of the age grid the cost to go is kept on, at least (more where +#: the shortest gap needs them: the spacing is a whole fraction of it). +AGES = 801 +#: The most points the age grid may have. +MAX_AGES = 20_000 +#: The most opportunities a cycle may have. +MAX_OPPORTUNITIES = 20_000 +#: Bisection steps for a threshold. +HALVINGS = 60 + + +def _cycle(calendar, repeat) -> tuple: + """The opportunities over one cycle: its length, and their times in + ``[0, length)``, prices and kinds (a time with several takes the + cheapest).""" + periods = [o.period for o in calendar.opportunities if o.periodic] + if repeat is not None: + periods.append(repeat) + length = float(_common_multiple(periods)) + times, prices, kinds = [], [], [] + for k, o in enumerate(calendar.opportunities): + if o.periodic: + count = Fraction(_fraction(length)) / _fraction(o.period) + if count > MAX_OPPORTUNITIES: + raise NotImplementedError( + "method='integration' needs a cycle of at most " + f"{MAX_OPPORTUNITIES} opportunities; this one has more: " + "use method='dp' or 'sampling'." + ) + local = o.offset % o.period + np.arange(int(count)) * o.period + else: + dates = np.asarray(o.dates, float) % repeat + copies = int(round(length / repeat)) + local = ( + dates[None, :] + repeat * np.arange(copies)[:, None] + ).ravel() + local = np.mod(local, length) + times.append(local) + prices.append(np.full(local.size, o.cost)) + kinds.append(np.full(local.size, k)) + t = np.concatenate(times) + c = np.concatenate(prices) + kind = np.concatenate(kinds) + order = np.lexsort((c, t)) + t, c, kind = t[order], c[order], kind[order] + tolerance = 1e-9 * length + keep = np.concatenate([[True], np.diff(t) > tolerance]) + if t.size > 1 and length - t[-1] + t[0] <= tolerance: + keep[-1] = False + names = [o.name for o in calendar.opportunities] + [ANYTIME] + return length, t[keep], c[keep], kind[keep], names + + +class _Gap: + """A gap's quadrature, by its length: the matrices that give ``G`` at + its nodes and ``W`` on the age grid from the cost to go at its end.""" + + def __init__(self, length, sf, ages, failure): + self.length = length + x = np.linspace(0.0, length, NODES + 1) + self.x = x + m = NODES + # G at node i: a new unit at x_i, to the end at x_m. + left = x[None, :-1] - x[:, None] + right = x[None, 1:] - x[:, None] + weights = np.where( + left >= 0.0, + sf(np.clip(left, 0.0, None)) - sf(np.clip(right, 0.0, None)), + 0.0, + ) + a = np.eye(m + 1) + a[:m, :m] -= 0.5 * weights[:m] + a[:m, 1:] -= 0.5 * weights[:m] + self.a_inverse = np.linalg.inv(a) + self.failed_new = failure * weights[:m].sum(axis=1) + self.remaining = length - x[:m] + self.lasting_new = sf(self.remaining) + # W on the age grid: a unit of each age kept through the gap. + self.ages = ages + self.failure = failure + self.sf = sf + self.q, self.lasting, self.later = self._kept(ages) + # Forward, the same operators transposed: a new unit at each node + # is, at the gap's end, of each age with ``ends`` (how G takes the + # cost to go there), after ``renewals`` failures on the way. + reach = np.zeros((m + 1, ages.size)) + reach[:m] = self.lasting_new[:, None] * _spread(self.remaining, ages) + reach[m, 0] = 1.0 + self.ends = self.a_inverse @ reach + once = np.zeros(m + 1) + once[:m] = weights[:m].sum(axis=1) + renewals = self.a_inverse @ once + self.fails = self.q @ (1.0 + 0.5 * (renewals[:-1] + renewals[1:])) + self.kept_to = _nearest(self.later, ages) + + def _kept(self, ages): + held = self.sf(ages) + reached = self.sf(ages[:, None] + self.x[None, :]) + with np.errstate(divide="ignore", invalid="ignore"): + scaled = np.where( + held[:, None] > 0.0, reached / held[:, None], 0.0 + ) + q = scaled[:, :-1] - scaled[:, 1:] + dead = held <= 0.0 + q[dead] = 0.0 + q[dead, 0] = 1.0 + return q, scaled[:, -1], ages + self.length + + def new_unit(self, after: np.ndarray, grid: np.ndarray) -> np.ndarray: + """``G`` at the gap's nodes, given the cost to go ``after`` (on + the age grid ``grid``) at its end.""" + rhs = np.empty(NODES + 1) + rhs[:NODES] = self.failed_new + self.lasting_new * np.interp( + self.remaining, grid, after + ) + rhs[NODES] = after[0] + return self.a_inverse @ rhs + + def kept(self, g_nodes, after, grid, ages=None) -> np.ndarray: + """``W`` at ``ages`` (the age grid by default).""" + if ages is None: + q, lasting, later = self.q, self.lasting, self.later + else: + q, lasting, later = self._kept(ages) + middle = self.failure + 0.5 * (g_nodes[:-1] + g_nodes[1:]) + return q @ middle + lasting * np.interp(later, grid, after) + + def forward(self, share: np.ndarray): + """The chance of the unit in place being of each age at the gap's + end, from ``share`` at its start, and the expected failures in + it.""" + pieces = share @ self.q + nodes = np.zeros(NODES + 1) + nodes[:-1] += 0.5 * pieces + nodes[1:] += 0.5 * pieces + kept = share * self.lasting + j, part = self.kept_to + size = share.size + later = nodes @ self.ends + later += np.bincount(j, kept * (1.0 - part), minlength=size) + later += np.bincount(j + 1, kept * part, minlength=size) + return later, float(share @ self.fails) + + +def _nearest(points: np.ndarray, grid: np.ndarray): + """Where linear interpolation on ``grid`` takes each of ``points`` + from, as ``np.interp`` does (a point past the grid's end from its last + age): the index below it, and its share of the way to the next.""" + x = np.clip(points, grid[0], grid[-1]) + j = np.clip(np.searchsorted(grid, x, side="right") - 1, 0, grid.size - 2) + return j, (x - grid[j]) / (grid[j + 1] - grid[j]) + + +def _spread(points: np.ndarray, grid: np.ndarray) -> np.ndarray: + """The linear interpolation's weights of ``points`` on ``grid``, as a + matrix (each row sums to 1).""" + j, part = _nearest(points, grid) + out = np.zeros((points.size, grid.size)) + rows = np.arange(points.size) + out[rows, j] = 1.0 - part + out[rows, j + 1] += part + return out + + +class _OpportunityTable: + """The thresholds at the opportunities of a cycle, from ``origin`` + (a repeated plan's first date: none before it).""" + + def __init__( + self, + length, + times, + thresholds, + kinds, + names, + origin=0.0, + prices=None, + chance=None, + ): + self.length, self.times = length, times + self.thresholds, self.kinds, self.names = thresholds, kinds, names + self.origin = origin + self.prices = ( + np.zeros(times.size) if prices is None else np.asarray(prices) + ) + self.chance = ( + np.zeros(times.size) if chance is None else np.asarray(chance) + ) + + def _index(self, t) -> np.ndarray: + """The index in ``times`` of the opportunity at each of ``t`` + (within a tolerance), -1 where none falls, or before + ``origin``.""" + t = np.asarray(t, float) + index = np.full(t.shape, -1, np.int64) + if self.times.size == 0: + return index + local = np.mod(t, self.length) + where = np.searchsorted(self.times, local) + tolerance = 1e-9 * self.length + for shift in (0, -1): + i = np.clip(where + shift, 0, self.times.size - 1) + hit = np.abs(self.times[i] - local) <= tolerance + hit |= np.abs(self.times[i] - local - self.length) <= tolerance + hit |= np.abs(self.times[i] - local + self.length) <= tolerance + index = np.where(hit, i, index) + before = t < self.origin - tolerance + return np.where(before, -1, index) + + def at(self, t): + index = self._index(t) + if self.times.size == 0: + return np.full(index.shape, math.inf), index + found = index >= 0 + i = np.maximum(index, 0) + out = np.where(found, self.thresholds[i], math.inf) + kinds = np.where(found, self.kinds[i], -1) + return out, kinds + + def price_at(self, t: float) -> float: + """The price of the opportunity at ``t``, ``inf`` if none falls + there (this table prices no replacement on any day).""" + index = int(self._index(t)) + return math.inf if index < 0 else float(self.prices[index]) + + def next_planned(self, installed, after, oldest): + """The first opportunity at or after ``after`` (strictly after, if + the unit was put in then) at which a unit put in at ``installed`` + is replaced, its price and kind; ``(inf, 0.0, None)`` if none + does. A unit past ``oldest`` (the oldest age worked out) is past + every threshold, so the opportunities are searched to where the + unit reaches that age, or a cycle ahead if it already has.""" + if self.times.size == 0: + return math.inf, 0.0, None + start = max(after, self.origin) + first = math.floor((start - self.times[-1]) / self.length) + end = max(installed + oldest, start) + self.length + last = math.ceil((end - self.times[0]) / self.length) + cycles = np.arange(first, last + 1) * self.length + times = (cycles[:, None] + self.times[None, :]).ravel() + thresholds = np.tile(self.thresholds, cycles.size) + prices = np.tile(self.prices, cycles.size) + kinds = np.tile(self.kinds, cycles.size) + tolerance = 1e-9 * self.length + later = ( + (times > after + tolerance) + if after <= installed + else (times >= after - tolerance) + ) + due = later & (times >= self.origin - tolerance) + due &= times - installed >= thresholds - tolerance + if not due.any(): + return math.inf, 0.0, None + i = int(np.argmax(due)) + return float(times[i]), float(prices[i]), self.names[kinds[i]] + + def schedule(self, until, wanted): + out = [] + cycles = int(math.ceil(until / self.length)) + for c in range(cycles): + for t, a, k, p in zip( + self.times, self.thresholds, self.kinds, self.chance + ): + time = c * self.length + t + name = self.names[k] + if ( + time >= until + or time < self.origin - 1e-9 * self.length + or (wanted is not None and name not in wanted) + ): + continue + out.append((float(time), name, float(a), float(p))) + return out + + +def integrated_policy( + part, calendar, failure, *, max_age=None, tail="repeat", life=None +): + """The calendar policy between opportunities in continuous time (see + the module's docstring).""" + sf = _survival(part) + mean = _mean(part, sf) + if not calendar.opportunities: + policy = CalendarPolicy( + cost_rate=failure / mean, + cost_rate_se=None, + replacement_rates={"failure": 1.0 / mean}, + cost_rates={"failure": failure / mean}, + method="integration", + basis="exact", + step=None, + period=None, + determined_until=None, + threshold_only=True, + settings={"opportunities": 0}, + life=life, + ) + table = _OpportunityTable( + 1.0, np.array([]), np.array([]), np.array([], np.int64), [ANYTIME] + ) + return policy._bind(table, [ANYTIME], None, 0.0) + repeat = None + if not calendar.periodic: + if tail != "repeat": + raise NotImplementedError( + "method='integration' repeats a plan of given dates past " + "its last date; for tail='anytime' use method='dp'." + ) + repeat = calendar._repeat() + length, times, prices, kinds, names = _cycle(calendar, repeat) + oldest = ( + float(max_age) if max_age is not None else _age_where(sf, FAR, mean) + ) + gaps = np.diff(np.concatenate([times, [times[0] + length]])) + # The age grid's spacing a whole fraction of the shortest gap, so that + # a gap of a whole number of them lands a kept unit on the grid. + shortest = float(gaps.min()) + spacing = shortest / math.ceil(shortest / (oldest / (AGES - 1))) + points = int(math.ceil(oldest / spacing)) + 1 + if points > MAX_AGES: + raise NotImplementedError( + "method='integration' needs an age grid of at most " + f"{MAX_AGES} points; gaps of {shortest:g} over ages to " + f"{oldest:g} need {points}: use method='dp'." + ) + grid = np.arange(points) * spacing + shapes: Dict[float, _Gap] = {} + + def gap(width): + key = round(width, 9) + if key not in shapes: + shapes[key] = _Gap(width, sf, grid, failure) + return shapes[key] + + def sweep(h, keep_values=False): + after = h + kept: list = [None] * times.size + for k in range(times.size - 1, -1, -1): + shape = gap(gaps[k]) + g_nodes = shape.new_unit(after, grid) + w = shape.kept(g_nodes, after, grid) + if keep_values: + kept[k] = (shape, g_nodes, after, w) + after = np.minimum(w, prices[k] + w[0]) + return after, kept + + h = np.zeros(grid.size) + limit = max(1, int(MAX_LIVES * oldest / length)) + low = high = math.nan + converged = False + sweeps = 0 + for sweeps in range(1, limit + 1): + v, _ = sweep(h) + change = v - h + low, high = float(change.min()), float(change.max()) + h = v - v[0] + if high - low <= TOLERANCE * max(abs(high), abs(low), 1e-300): + converged = True + break + _, kept = sweep(h, keep_values=True) + thresholds = np.empty(times.size) + upper = True + for k, (shape, g_nodes, after, w) in enumerate(kept): + replace = prices[k] + w[0] + act = replace < w + if not act.any(): + thresholds[k] = math.inf + continue + i = int(np.argmax(act)) + if not act[i:].all(): + upper = False + if i == 0: + thresholds[k] = 0.0 + continue + lo, hi = grid[i - 1], grid[i] + for _ in range(HALVINGS): + middle = 0.5 * (lo + hi) + value = shape.kept(g_nodes, after, grid, np.array([middle]))[0] + if replace < value: + hi = middle + else: + lo = middle + thresholds[k] = hi + replaced, failures = _stationary(kept, prices, grid, length, oldest) + rates = {name: 0.0 for name in names if name != ANYTIME} + spent = dict(rates) + for k in range(times.size): + rates[names[kinds[k]]] += float(replaced[k] / length) + spent[names[kinds[k]]] += float(replaced[k] * prices[k] / length) + rates["failure"] = float(failures / length) + spent["failure"] = float(failures * failure / length) + policy = CalendarPolicy( + cost_rate=0.5 * (low + high) / length, + cost_rate_se=None, + replacement_rates=rates, + cost_rates=spent, + method="integration", + basis="exact", + step=None, + period=length, + determined_until=None, + threshold_only=upper, + settings={ + "sweeps": sweeps, + "converged": converged, + "cost_rate_bounds": (low / length, high / length), + "nodes": NODES, + "ages": int(grid.size), + "max_age": oldest, + "opportunities": int(times.size), + "tail": tail if repeat is not None else None, + "repeat": repeat, + }, + life=life, + ) + origin = 0.0 + if repeat is not None: + origin = min( + d + for o in calendar.opportunities + if not o.periodic + for d in o.dates + ) + table = _OpportunityTable( + length, times, thresholds, kinds, names, origin, prices, replaced + ) + return policy._bind(table, names, None, oldest) + + +def _stationary(kept, prices, grid, length, oldest): + """The long-run chance of a replacement at each opportunity of the + cycle, and the expected failures over it: forward across the cycle + (each opportunity replacing the ages its last sweep did, then its + gap) from a new unit, until the ages at its start settle.""" + share = np.zeros(grid.size) + share[0] = 1.0 + + def across(share): + replaced = np.zeros(len(kept)) + failures = 0.0 + for k, (shape, _, _, w) in enumerate(kept): + act = prices[k] + w[0] < w + moved = float(share[act].sum()) + share = np.where(act, 0.0, share) + share[0] += moved + replaced[k] = moved + share, failed = shape.forward(share) + failures += failed + return share, replaced, failures + + limit = max(1, int(MAX_LIVES * oldest / length)) + for _ in range(limit): + later, _, _ = across(share) + settled = float(np.abs(later - share).sum()) <= TOLERANCE + share = later + if settled: + break + _, replaced, failures = across(share) + return replaced, failures diff --git a/repyability/maintenance/_next.py b/repyability/maintenance/_next.py new file mode 100644 index 00000000..2f304fb7 --- /dev/null +++ b/repyability/maintenance/_next.py @@ -0,0 +1,353 @@ +"""When to replace a part in service next (#319, UC-12 of this package's +``README.md``): now, at one of the scheduled outages, at a planned +replacement between them, or at its failure. + +Each choice is costed from the part's age ``a`` to its replacement: its +expected cost, less the time it serves valued at ``g``, the long-run cost +a unit of time of how the position is run (the baseline): the part's +[`optimal_replacement_policy`][repyability.NonRepairable.optimal_replacement_policy], +or the policy given, a +[`CalendarPolicy`][repyability.CalendarPolicy] where cheap shuts keep +coming (#348), + + cost(u, price) = price S(u) + failure (1 - S(u)) - g ∫₀ᵘ S, + +with ``S(u) = R(a + u) / R(a)``, the chance the part is working ``u`` from +now (replacing now costs its price). The time the part serves is time a +new part would otherwise serve, which ``g`` values: this is the relative +value of the renewal-reward process, a new part taken to be worth the same +whenever it is fitted. With no outages and the age replacement's rate, +replacing now is the best from the optimal replacement age exactly. + +A planned replacement is one later than now, costed at its best time, on +a grid of ``_GRID`` times up to the time the part outlives with a chance +of ``_FAR``, refined between the best one's neighbours. Where its cost is +lowest as soon as possible (no later time beats replacing at once, as +for a part past its optimal age), replacing at once is replacing now, +and costs what that does, ``now_cost``: the planned choice is then +replacing now (time 0, at ``now_cost``), never a planned price at time +0 that ``now_cost`` contradicts; ``now`` is True exactly when the best +choice is at time 0. A planned replacement with a lead time is a +scheduled outage at that time. Each later choice's break-even age (the +youngest at which replacing now costs no more) is found on a grid of +``_AGES`` ages up to the age a new part reaches with a chance of ``_FAR``, +then halved down to its first. +""" + +import math +from dataclasses import dataclass +from typing import List, Optional + +import numpy as np +from scipy.integrate import cumulative_trapezoid, quad +from scipy.optimize import minimize_scalar + +from repyability.rbd.results import _ResultMapping +from repyability.utils import checks + +#: The chance of a part outliving the latest time a planned replacement is +#: tried at, and the oldest age a break-even age is looked for at. +_FAR = 1e-3 +#: The times a planned replacement is tried at, before the best is refined. +_GRID = 2001 +#: The ages a break-even age is looked for at, before it is halved down to. +_AGES = 64 +#: Halvings of a break-even age's bracket. +_HALVINGS = 50 + + +@dataclass +class ReplacementChoice(_ResultMapping): + """One choice of when to replace a part next: an entry of + [`NextReplacement`][repyability.NextReplacement]. + + Attributes + ---------- + kind : str + ``"now"``, ``"outage"`` (at one of the scheduled outages), + ``"planned"`` (a planned replacement at its best time) or + ``"failure"`` (run to failure). + time : float + When, from now: 0 for now, the outage's time, the planned + replacement's best time (0 where no later time beats replacing + at once, which is then replacing now, at ``now_cost``; ``inf`` + if none beats running to failure), ``inf`` at failure. + price : float + What the replacement costs if the part has not failed by then (at + failure, the failure price). + cost : float + The expected cost until the part is replaced (at the failure price + if it fails first), less the time it serves at ``cost_rate``: the + lowest is the best. + break_even_age : float or None + The youngest age at which replacing now costs no more than this + choice (``inf``: no age up to the one a new part reaches with a + chance of 0.1%). None for replacing now. + """ + + kind: str + time: float + price: float + cost: float + break_even_age: Optional[float] + + +@dataclass +class NextReplacement(_ResultMapping): + """When to replace a part in service next: returned by + [`NonRepairable.optimal_next_replacement`][repyability.NonRepairable.optimal_next_replacement]. + + Attributes + ---------- + age : float + The part's age now. + best : ReplacementChoice + The cheapest choice (on a tie, the earlier in ``choices``). + now : bool + Whether the best is to replace it now. + choices : list of ReplacementChoice + Every choice: now, each outage by its time, a planned replacement at + its best time, and run to failure. + replace_from_age : float + The age from which replacing now beats every later choice: the + largest break-even age. + cost_rate : float + The value of a unit of the part's service, the baseline: the + long-run cost a unit of time of the policy given, or of the part's + [`optimal_replacement_policy`][repyability.NonRepairable.optimal_replacement_policy]. + """ + + age: float + best: ReplacementChoice + now: bool + choices: List[ReplacementChoice] + replace_from_age: float + cost_rate: float + + +def _amount(value, name: str) -> float: + """``value`` as a cost: a finite number, at least 0.""" + if not (checks.is_number(value) and 0.0 <= float(value) < math.inf): + raise ValueError( + f"{name} must be a number, at least 0; got {value!r}." + ) + return float(value) + + +def _time(value, name: str) -> float: + """``value`` as a time from now: a finite number, more than 0.""" + if not (checks.is_number(value) and 0.0 < float(value) < math.inf): + raise ValueError( + f"{name} must be a time from now, a number more than 0; " + f"got {value!r}." + ) + return float(value) + + +def _coming(outages) -> list: + """The scheduled outages as (time, price) pairs, by time.""" + if isinstance(outages, (str, bytes)) or not hasattr(outages, "__iter__"): + raise TypeError( + "scheduled_outages must be a list of (time from now, price) " + f"pairs; got {outages!r}." + ) + coming = [] + for outage in outages: + pair = () if isinstance(outage, (str, bytes)) else tuple(outage) + if len(pair) != 2: + raise TypeError( + "each scheduled outage must be a (time from now, price) " + f"pair; got {outage!r}." + ) + coming.append( + ( + _time(pair[0], "the time of a scheduled outage"), + _amount(pair[1], "the price at a scheduled outage"), + ) + ) + return sorted(coming) + + +def _beyond(lasting, scale: float) -> float: + """The time by which ``lasting`` (a chance of still working) falls to + ``_FAR``: doubled from ``scale`` past it, then halved down to it.""" + low, high = 0.0, scale + for _ in range(200): + if lasting(high) <= _FAR: + break + low, high = high, 2.0 * high + else: + raise ValueError( + "The life never falls to a chance of 0.1% of working (some units " + "never fail), so a part may be kept for good." + ) + for _ in range(40): + middle = 0.5 * (low + high) + if lasting(middle) > _FAR: + low = middle + else: + high = middle + return high + + +class _Life: + """A part's life, priced: each choice's cost from an age.""" + + def __init__(self, sf, scale: float, failed: float, rate: float): + self.sf, self.scale = sf, scale + self.failed, self.rate = failed, rate + + def survival(self, x: float) -> float: + with np.errstate(invalid="ignore"): + return float(np.ravel(self.sf(x))[0]) + + def keep(self, age: float, u: float, price: float) -> float: + """The cost of keeping a part ``age`` old to ``u`` from now, then + replacing it at ``price``.""" + held = self.survival(age) + lasting = self.survival(age + u) / held + cost = price * lasting + self.failed * (1.0 - lasting) + if self.rate > 0.0: + with np.errstate(invalid="ignore"): + served = quad(self.survival, age, age + u, limit=200)[0] + cost -= self.rate * served / held + return cost + + def planned(self, age: float, price: float) -> tuple: + """The best time from now for a planned replacement of a part + ``age`` old at ``price``, and its cost: ``(0.0, None)`` where no + later time beats replacing it at once (its cost is lowest as + soon as possible), ``inf`` where none beats running to + failure.""" + held = self.survival(age) + later = _beyond(lambda u: self.survival(age + u) / held, self.scale) + times = np.linspace(0.0, later, _GRID) + with np.errstate(invalid="ignore"): + lasting = np.ravel(np.asarray(self.sf(age + times), float)) / held + costs = price * lasting + self.failed * (1.0 - lasting) + costs -= self.rate * cumulative_trapezoid(lasting, times, initial=0.0) + i = int(np.argmin(costs)) + if i == _GRID - 1: + # Still falling where the part is all but sure to have failed: + # no planned replacement beats running it to failure. + return math.inf, self.keep(age, math.inf, price) + best_time, best = float(times[i]), self.keep(age, times[i], price) + refined = minimize_scalar( + lambda u: self.keep(age, u, price), + bounds=(times[max(i - 1, 0)], times[min(i + 1, _GRID - 1)]), + method="bounded", + options={"xatol": later * 1e-9}, + ) + if refined.success and refined.fun < best: + best_time, best = float(refined.x), float(refined.fun) + if not best < price or best_time <= later * 1e-9: + # Lowest at once (at time 0 it costs the price): replacing now. + return 0.0, None + return best_time, best + + +def _break_even(beaten, oldest: float) -> float: + """The youngest age at which ``beaten(age)`` holds: on a grid of + ``_AGES`` ages up to ``oldest``, then halved down to the first.""" + ages = np.linspace(0.0, oldest, _AGES) + first = next((i for i, at in enumerate(ages) if beaten(float(at))), None) + if first is None: + return math.inf + if first == 0: + return 0.0 + low, high = float(ages[first - 1]), float(ages[first]) + for _ in range(_HALVINGS): + middle = 0.5 * (low + high) + if beaten(middle): + high = middle + else: + low = middle + return high + + +def _baseline(part, policy) -> float: + """The long-run cost a unit of time of ``policy`` (a + ``CalendarPolicy`` or a ``MaintenancePolicy``), or of the part's + optimal replacement policy if none is given.""" + from repyability.maintenance import CalendarPolicy, MaintenancePolicy + + if policy is None: + policy = part.optimal_replacement_policy() + elif not isinstance(policy, (CalendarPolicy, MaintenancePolicy)): + what = "its cost rate" if checks.is_number(policy) else "this" + raise TypeError( + "policy must be the policy whose long-run cost rate values the " + "time the part serves: a CalendarPolicy " + "(optimal_calendar_policy) or a MaintenancePolicy " + f"(optimal_replacement_policy), not {what}; got {policy!r}." + ) + return _amount(policy.cost_rate, "the cost rate of policy") + + +def optimal_next_replacement( + part, age, *, now_cost=None, scheduled_outages=(), policy=None +) -> NextReplacement: + """The work of ``NonRepairable.optimal_next_replacement`` (see + there).""" + from repyability.rbd._model_utils import model_mean + + part._require_costs() + if part.model_parameterization not in ("parametric", "degradation"): + raise NotImplementedError( + "optimal_next_replacement needs a life with a survival function " + "in closed form (a surpyval parametric distribution, mixture or " + "degradation process); this part's is " + f"{part.model_parameterization}." + ) + if not (checks.is_number(age) and 0.0 <= float(age) < math.inf): + raise ValueError(f"age must be a number, at least 0; got {age!r}.") + age = float(age) + now = part.cp if now_cost is None else _amount(now_cost, "now_cost") + coming = _coming(scheduled_outages) + rate = _baseline(part, policy) + mean = model_mean(part.reliability) + life = _Life(part.reliability_function, mean, part.cu, rate) + if not life.survival(age) > 0.0: + raise ValueError( + f"The life gives no chance of a unit working at age {age:g}." + ) + + # Each later choice: its kind, time (a planned replacement's is its + # best) and price. + later = [("outage", time, price) for time, price in coming] + later += [("planned", 0.0, part.cp), ("failure", math.inf, part.cu)] + + def chosen(kind: str, time: float, price: float, at: float) -> tuple: + """A choice's time, price and cost for a part ``at`` old. A + planned replacement that pays as soon as possible is replacing + now, at now_cost.""" + if kind == "planned": + when, cost = life.planned(at, price) + return (0.0, now, now) if cost is None else (when, price, cost) + return time, price, life.keep(at, time, price) + + oldest = _beyond(life.survival, mean) + tie = 1e-10 * max(now, part.cu) + choices = [ReplacementChoice("now", 0.0, now, now, None)] + evens: List[float] = [] + for kind, time, price in later: + when, at_price, cost = chosen(kind, time, price, age) + even = _break_even( + lambda at: now <= chosen(kind, time, price, at)[2] + tie, oldest + ) + evens.append(even) + choices.append(ReplacementChoice(kind, when, at_price, cost, even)) + # On a tie, the earlier choice; a planned replacement that never pays + # is running to failure. + best = min( + choices, + key=lambda c: (c.cost, c.kind == "planned" and c.time == math.inf), + ) + return NextReplacement( + age, + best, + best.kind == "now", + choices, + max(evens), + rate, + ) diff --git a/repyability/maintenance/_replacement.py b/repyability/maintenance/_replacement.py new file mode 100644 index 00000000..a59898c3 --- /dev/null +++ b/repyability/maintenance/_replacement.py @@ -0,0 +1,1246 @@ +"""The long-run optimal replacement of a part over a calendar of +opportunities (#319, UC-13 of this package's ``README.md``): opportunity- +based age replacement (Dekker et al., early 1990s), with deterministic +periodic opportunities at prices of their own. + +A planned replacement's price depends on the time (a cheap shut, a dearer +day, or none), so the state is the time and the part's age, and the best +policy is a threshold for each time: replace if the age is at least +``a*(t)``. It is worked out on a grid of ``step`` by dynamic programming +(``_sweep``): from each step, keep the part (it survives the step with a +chance ``p_j`` at age index ``j``, or fails and is replaced, as new by the +next step, at the failure price) or replace it now at the step's price. + +- **A finite horizon** (``_sweep`` from zero values): backward over the + steps. It gives the thresholds over any calendar, and the cost rate by + differencing the cost to go of a new part at two times. +- **A periodic calendar** (``_relative_values``): relative value iteration + over one cycle, the cost rate exact for the grid, its bounds the span of + a sweep's change. +- **A long cycle or none** (``_sampled_rate``): the cost rate is the mean, + over uniformly sampled times of the cycle, of the rate differenced over + a window from each; each window an exact solve. The policy at a time is + a solve over a window from it (deterministic, cached). +- **A given rule** (``_evaluate``): its cost rate, by the same iteration + with the decisions fixed, to price naive policies. + +A failure is replaced by the next step, so the grid's cost rate is low by +about ``step / 2`` of each failure's time over the mean cycle: O(step). +Phase 2 (``_integration``) works between opportunities in continuous time. +""" + +import math +import warnings +from dataclasses import dataclass, field +from fractions import Fraction +from typing import Any, Callable, ClassVar, Dict, Optional, cast + +import numpy as np + +from repyability.maintenance._calendar import ( + ANYTIME, + Calendar, + _fraction, + _Grid, +) +from repyability.rbd.results import _ResultMapping +from repyability.utils import checks +from repyability.utils.wrappers import outside_level + +#: Default grid: about this many steps to the mean life, put on the +#: opportunities' own grid where it can be. +STEPS_PER_LIFE = 1000 +#: The chance of a unit outliving the oldest age on the grid. +FAR = 1e-9 +#: The chance of a unit outliving the age taken as a life's span, for the +#: windows the policy at a time is worked out over. +SPAN = 1e-3 +#: The most steps a cycle may have for relative value iteration; a longer +#: one is sampled. +MAX_CYCLE = 20_000 +#: Relative value iteration stops when the span of a sweep's change is +#: within this share of it. +TOLERANCE = 1e-10 +#: The most steps relative value iteration may take, in lives of the age +#: grid (sweeps times the cycle). +MAX_LIVES = 400 +#: A window's settling length: this many of the longest gap between +#: planned options, and of a life's span. +SETTLE_GAPS = 2 +SETTLE_SPANS = 3 + +METHODS = ("auto", "dp", "integration", "sampling") +TAILS = ("repeat", "anytime") + + +class _Ages: + """A part's life on the age grid: ``p[j]``, the chance a unit of age + ``j * step`` survives the next step (the oldest age kept as the + oldest).""" + + def __init__(self, part, step: float, max_age: Optional[float]): + self.step = step + self.sf = _survival(part) + self.mean = _mean(part, self.sf) + if max_age is None: + max_age = _age_where(self.sf, FAR, self.mean) + self.span = _age_where(self.sf, SPAN, self.mean) + size = int(math.ceil(max_age / step)) + ages = np.arange(size + 1) * step + held = self.sf(ages) + later = self.sf(ages + step) + with np.errstate(divide="ignore", invalid="ignore"): + p = np.where(held > 0.0, later / held, 0.0) + self.p = np.clip(np.nan_to_num(p), 0.0, 1.0) + self.size = size + + +def _survival(part) -> Callable[[np.ndarray], np.ndarray]: + """The part's survival function, over an array of ages.""" + sf = part.reliability_function + + def survival(ages): + ages = np.asarray(ages, float) + with np.errstate(all="ignore"): + try: + out = np.asarray(sf(ages), float) + if out.shape != ages.shape: + raise ValueError + except (TypeError, ValueError): + out = np.array([float(np.ravel(sf(a))[0]) for a in ages]) + return np.clip(np.nan_to_num(out, nan=0.0), 0.0, 1.0) + + return survival + + +def _mean(part, sf) -> float: + from repyability.rbd._model_utils import MODEL_ERRORS, model_mean + + try: + mean = float(model_mean(part.reliability)) + except MODEL_ERRORS: + mean = math.nan + if not 0.0 < mean < math.inf: + mean = _age_where(sf, 0.5, 1.0) + return mean + + +def _age_where(sf, chance: float, scale: float) -> float: + """The age a unit outlives with ``chance``: doubled from ``scale`` + past it, then halved down to it.""" + low, high = 0.0, scale + for _ in range(200): + if float(sf(np.array([high]))[0]) <= chance: + break + low, high = high, 2.0 * high + else: + raise ValueError( + "The life never falls to a chance of working of " + f"{chance:g} (some units never fail): give max_age." + ) + for _ in range(60): + middle = 0.5 * (low + high) + if float(sf(np.array([middle]))[0]) > chance: + low = middle + else: + high = middle + return high + + +def _common_divisor(values) -> Optional[float]: + """The largest step every one of ``values`` (read as fractions, 0 + left out) is a whole number of; None with none.""" + fractions = [_fraction(abs(v)) for v in values if v != 0.0] + if not fractions: + return None + return float( + Fraction( + math.gcd(*(f.numerator for f in fractions)), + math.lcm(*(f.denominator for f in fractions)), + ) + ) + + +def _default_step(calendar: Calendar, mean: float) -> float: + """About ``mean / STEPS_PER_LIFE``, but a whole fraction of the + largest step every period (and the period a plan of given dates + repeats with) lies on, so that a periodic calendar keeps its period + on the grid whatever its offsets; and of the offsets and dates too, + unless that makes the step far finer (below a tenth of the life's), + when each is laid at the grid time nearest it (``_Grid``, which + warns). An offset of a few hours on a period of days is what dates + turned into model time give, and must not cost the period.""" + base = mean / STEPS_PER_LIFE + periods = [o.period for o in calendar.opportunities if o.periodic] + if not calendar.periodic: + periods.append(calendar._repeat()) + others = [o.offset for o in calendar.opportunities if o.periodic] + others += [ + d for o in calendar.opportunities if not o.periodic for d in o.dates + ] + for common in ( + _common_divisor(periods + others), + _common_divisor(periods), + ): + if common is not None and common >= base / 10.0: + return common / math.ceil(common / base) + return base + + +def _sweep( + values: np.ndarray, + costs: np.ndarray, + p: np.ndarray, + failure: float, + beta: float = 1.0, + fixed: Optional[np.ndarray] = None, + record: Optional[np.ndarray] = None, +): + """Backward over the steps of ``costs``, from ``values`` (the cost to + go, by age index, after the last): each step, keep the part or + replace it at the step's price (``inf``: no replacement). + + ``beta`` discounts a step (1: none). ``fixed`` gives, for a fixed + rule, the age index from which each step replaces (-1: never), + instead of the best. ``record`` gets the cost to go of a new part at + each step. + + Returns the values at the first step, each step's first replacing age + index (-1: none), and whether every step's replacing ages were all + those from its first (a threshold). + """ + first = np.full(costs.size, -1, dtype=np.int64) + upper = True + v = values + shifted = np.empty_like(values) + for n in range(costs.size - 1, -1, -1): + shifted[:-1] = v[1:] + shifted[-1] = v[-1] + keep = p * shifted + keep += (1.0 - p) * (failure + 0.5 * (v[0] + v[1])) + if beta != 1.0: + keep *= beta + cost = costs[n] + if cost < math.inf: + replace = cost + keep[0] + if fixed is None: + act = replace < keep + if act.any(): + i = int(np.argmax(act)) + first[n] = i + if upper and not act[i:].all(): + upper = False + v = np.minimum(keep, replace) + else: + i = int(fixed[n]) + first[n] = i + if i >= 0: + keep[i:] = replace + v = keep + else: + v = keep + if record is not None: + record[n] = v[0] + return v, first, upper + + +@dataclass +class _Cycle: + """Relative value iteration's result over one cycle.""" + + rate: float + low: float + high: float + first: np.ndarray + upper: bool + sweeps: int + converged: bool + + +def _relative_values( + costs: np.ndarray, + ages: _Ages, + failure: float, + fixed: Optional[np.ndarray] = None, +) -> _Cycle: + """The average cost rate over a periodic calendar of ``costs`` (one + cycle of steps), by relative value iteration: sweep the cycle from the + last relative values until the span of the change is within + ``TOLERANCE`` of it. The rate is exact for the grid; ``low`` and + ``high`` bound it.""" + h = np.zeros(ages.size + 1) + length = costs.size * ages.step + limit = max(1, int(MAX_LIVES * (ages.size + 1) / costs.size)) + low = high = math.nan + for sweep in range(1, limit + 1): + v, first, upper = _sweep(h, costs, ages.p, failure, fixed=fixed) + change = v - h + low, high = float(change.min()), float(change.max()) + h = v - v[0] + if high - low <= TOLERANCE * max(abs(high), abs(low), 1e-300): + return _Cycle( + 0.5 * (low + high) / length, + low / length, + high / length, + first, + upper, + sweep, + True, + ) + warnings.warn( + f"The cost rate did not settle in {limit} sweeps of the cycle: it " + f"lies between {low / length:.6g} and {high / length:.6g}.", + stacklevel=outside_level(), + ) + return _Cycle( + 0.5 * (low + high) / length, + low / length, + high / length, + first, + upper, + limit, + False, + ) + + +def _window(grid: _Grid, start: int, n: int, ages: _Ages, failure: float): + """A finite-horizon solve over ``n`` steps from grid index ``start``: + the cost to go of a new part at each step, each step's first + replacing age index and the kinds, and whether they are + thresholds.""" + costs, kinds = grid.prices(start, n) + record = np.empty(n) + _, first, upper = _sweep( + np.zeros(ages.size + 1), costs, ages.p, failure, record=record + ) + return record, first, kinds, upper + + +def _propagate(share: np.ndarray, first: np.ndarray, p: np.ndarray): + """Forward over the steps of ``first`` (each step's first replacing + age index, -1 for none) from ``share``, the chance of the unit in + place being of each age index: the share at the end, the chance of a + planned replacement at each step, and the expected failures. A failed + unit is renewed as the backward sweep has it: by the next step, as + new or a step old, half each.""" + share = share.copy() + replaced = np.zeros(first.size) + failures = 0.0 + later = np.empty_like(share) + for n in range(first.size): + i = first[n] + if i >= 0: + moved = float(share[i:].sum()) + share[i:] = 0.0 + share[0] += moved + replaced[n] = moved + failed = share * (1.0 - p) + lost = float(failed.sum()) + failures += lost + later[0] = 0.0 + later[1:] = share[:-1] * p[:-1] + later[-1] += share[-1] * p[-1] + later[0] += 0.5 * lost + later[1] += 0.5 * lost + share, later = later, share + return share, replaced, failures + + +def _stationary(first: np.ndarray, ages: _Ages): + """The long-run chance of a planned replacement at each step of a + cycle, and the expected failures over it: forward over the cycle from + a new unit until the ages at its start settle (within + ``TOLERANCE``).""" + share = np.zeros(ages.size + 1) + share[0] = 1.0 + limit = max(1, int(MAX_LIVES * (ages.size + 1) / first.size)) + for _ in range(limit): + later, _, _ = _propagate(share, first, ages.p) + settled = float(np.abs(later - share).sum()) <= TOLERANCE + share = later + if settled: + break + _, replaced, failures = _propagate(share, first, ages.p) + return replaced, failures + + +def _summary(names, anytime, kinds, costs, replaced, failures, failure, time): + """Each kind of replacement's rate and cost a unit of time, failures + as ``"failure"``: from the chance of a planned replacement at each + step (``replaced``), the steps' kinds and prices, and the failures, + over ``time``.""" + shown = [n for n in names if n != ANYTIME or anytime is not None] + rates = dict.fromkeys(shown, 0.0) + spent = dict.fromkeys(shown, 0.0) + for k, name in enumerate(names): + if name not in rates: + continue + here = kinds == k + rates[name] = float(replaced[here].sum() / time) + spent[name] = float((replaced[here] * costs[here]).sum() / time) + rates["failure"] = float(failures / time) + spent["failure"] = float(failures * failure / time) + return rates, spent + + +class _Thresholds: + """Where a policy's thresholds come from: ``at(index)`` gives the + first replacing age index and the kind of replacement at grid + indices; ``horizon`` is the steps within which a step that replaces + comes, if any does (a cycle; a window's settling length).""" + + horizon: int + + def at(self, index: np.ndarray): # pragma: no cover - interface + raise NotImplementedError + + +class _CycleTable(_Thresholds): + """Thresholds repeating every cycle, and the long-run chance of a + planned replacement at each step.""" + + def __init__(self, first, kinds, chance): + self.first, self.kinds, self.chance = first, kinds, chance + self.horizon = first.size + + def chances(self, n: int) -> np.ndarray: + return self.chance[np.mod(np.arange(n), self.first.size)] + + def at(self, index): + where = np.mod(index, self.first.size) + return self.first[where], self.kinds[where] + + +class _WindowTable(_Thresholds): + """Thresholds from finite-horizon solves: a table from 0, and past it + blocks of steps, each solved over a window long enough for its end + not to reach the block's decisions (cached).""" + + def __init__(self, grid, ages, failure, settle, first, kinds): + self.grid, self.ages, self.failure = grid, ages, failure + self.settle = settle + self.first, self.kinds = first, kinds + self.block = max(settle, 1024) + self.blocks: Dict[int, tuple] = {} + self.horizon = max(settle, 1) + + def _block(self, b: int): + if b not in self.blocks: + _, f, k, _ = _window( + self.grid, + b * self.block, + self.block + self.settle, + self.ages, + self.failure, + ) + self.blocks[b] = (f[: self.block], k[: self.block]) + return self.blocks[b] + + def chances(self, n: int) -> np.ndarray: + """The chance of a planned replacement at each of the first ``n`` + steps, from a new unit at 0.""" + first, _ = self.at(np.arange(n)) + start = np.zeros(self.ages.size + 1) + start[0] = 1.0 + return _propagate(start, first, self.ages.p)[1] + + def at(self, index): + index = np.asarray(index, np.int64) + first = np.empty(index.shape, np.int64) + kinds = np.empty(index.shape, np.int64) + table = self.first.size + inside = (index >= 0) & (index < table) + first[inside] = self.first[index[inside]] + kinds[inside] = self.kinds[index[inside]] + outside = ~inside + if outside.any(): + blocks = np.floor_divide(index, self.block) + for b in np.unique(blocks[outside]): + f, k = self._block(int(b)) + here = outside & (blocks == b) + first[here] = f[index[here] - b * self.block] + kinds[here] = k[index[here] - b * self.block] + return first, kinds + + +#: Steps a policy looks ahead at once for a unit's next planned +#: replacement. +_CHUNK = 1024 + + +@dataclass(frozen=True) +class PlannedReplacement(_ResultMapping): + """A unit's next planned replacement on a calendar policy: returned by + [`CalendarPolicy.next_replacement`][repyability.CalendarPolicy.next_replacement]. + + Attributes + ---------- + time : float + When, in the calendar's time (``inf``: no planned replacement + comes, as after the last date of a plan with none on any day). + kind : str or None + The opportunity it falls at (``"anytime"`` for a day with no + shut); None if none. + price : float or None + What the replacement costs there; None if none. + """ + + time: float + kind: Optional[str] + price: Optional[float] + + +@dataclass(frozen=True) +class CalendarPolicy(_ResultMapping): + """The long-run optimal replacement policy of a part over a calendar + of opportunities. Returned by + [`NonRepairable.optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy]. + + At each opportunity the policy replaces the part if it is at least a + threshold age, which depends on the time: a shut shortly before a + cheaper one waits for an older part than one long before it. + ``schedule()`` lists the thresholds with the long-run chance of a + replacement at each, ``next_replacement(age)`` says when a unit in + service will be replaced, and ``replacement_rates`` and + ``cost_rates`` say what the policy does and costs in the long run, + by kind. + + Attributes + ---------- + cost_rate : float + The long-run average cost a unit of time, ``g``, whatever the + start. + cost_rate_se : float or None + Its standard error, where it is estimated by sampling (None where + it is worked out). + replacement_rates : dict + Replacements a unit of time in the long run, by kind: each + opportunity's name, ``"anytime"`` (a day with no shut, where the + calendar allows one) and ``"failure"``. A kind the policy never + uses has 0. Exact for the grid where the cost rate is + (``basis`` ``"exact"``); otherwise, with the schedule's chances, + estimates from a new unit run in, with no standard error (only + ``cost_rate_se`` has one): a sampled policy's move with the seed. + cost_rates : dict + What each kind costs a unit of time in the long run (its rate + times its price); they sum to ``cost_rate``. + method : str + ``"dp"`` (dynamic programming on the grid), ``"integration"`` + (between opportunities in continuous time) or ``"sampling"`` + (sampled windows of a long cycle). + basis : str + ``"exact"`` (exact for the grid, or for the quadrature), + ``"differenced"`` (from a long finite horizon) or ``"sampled"``; + the rates by kind are estimated from a new unit run in, where the + cost rate is not exact. + step : float or None + The grid's step, sized from the life and put on the opportunities' + times (None for integration, which has none). + period : float or None + The time after which the thresholds repeat (None if they do not, + or too long to keep). + determined_until : float or None + For a calendar with given dates: the first time at which the + policy depends on how the calendar goes on after its last date + (repeating it, or leaving only replacement on any day). None for a + periodic calendar. + threshold_only : bool + Whether, at every time worked out, the ages that replace are all + those from the first (else the threshold is the youngest that + replaces, and some older ones do not: a life whose hazard falls; + the rates by kind then take every age from the youngest). + settings : dict + How it was worked out: the calendar's tail, sweeps, bounds, + samples, windows. + life : dict or None + The saved form of the life it was worked out for (as the + diagrams save a model), which a + [`RepairableRBD`][repyability.RepairableRBD] component given the + policy must have; None where the life does not save. + + Examples + -------- + A bearing with a Weibull life (scale 1000 days, shape 2.5), shuts + every 91 days at 3 and every 364 days at 1, no replacement between + them, and 20 at a failure: + + >>> import surpyval as surv + >>> from repyability import Calendar, NonRepairable, Opportunity + >>> bearing = NonRepairable(surv.Weibull.from_params([1000.0, 2.5])) + >>> calendar = Calendar( + ... [ + ... Opportunity("quarterly", 3.0, period=91.0), + ... Opportunity("annual", 1.0, period=364.0), + ... ], + ... failure=20.0, + ... anytime=None, + ... ) + >>> policy = bearing.optimal_calendar_policy(calendar) + >>> policy.method, policy.basis, policy.period + ('dp', 'exact', 364.0) + >>> round(policy.cost_rate * 364, 3) + 2.527 + >>> {k: round(v * 364, 3) for k, v in policy.replacement_rates.items()} + {'quarterly': 0.0, 'annual': 0.962, 'failure': 0.078} + >>> policy.next_replacement(age=300.0, now=200.0) + PlannedReplacement(time=364.0, kind='annual', price=1.0) + """ + + # Bound after construction (``_bind``), not fields: where the + # thresholds come from, the kinds' names, the calendar's grid and the + # oldest age worked out. + _table: ClassVar[Any] = None + _names: ClassVar[Any] = None + _grid: ClassVar[Any] = None + _oldest: ClassVar[Any] = None + + cost_rate: float + cost_rate_se: Optional[float] + replacement_rates: Dict[str, float] + cost_rates: Dict[str, float] + method: str + basis: str + step: Optional[float] + period: Optional[float] + determined_until: Optional[float] + threshold_only: bool + settings: Dict[str, Any] = field(default_factory=dict) + life: Optional[dict] = None + + def _bind(self, table, names, grid=None, oldest=None) -> "CalendarPolicy": + object.__setattr__(self, "_table", table) + object.__setattr__(self, "_names", names) + object.__setattr__(self, "_grid", grid) + object.__setattr__(self, "_oldest", oldest) + return self + + def _next_planned(self, installed: float, after: float): + """When a unit put in at ``installed`` is next replaced on the + policy, at or after ``after`` (strictly after, if put in then): + its time, price and kind, ``(inf, 0.0, None)`` if no step + replaces it. A unit past the oldest age the policy was worked out + to is past every threshold, so it is replaced at the first step + that replaces at all: the search runs to where the unit reaches + that age, or a horizon ahead (a cycle, a window's settling + length) if it already has. What a ``RepairableRBD`` component on + a ``"calendar"`` preventive policy asks at each of its + renewals.""" + if self._grid is None: + return self._table.next_planned(installed, after, self._oldest) + step = cast(float, self.step) + if after <= installed: + n = int(math.floor(after / step + 1e-9)) + 1 + else: + n = int(math.ceil(after / step - 1e-9)) + limit = int(math.ceil((installed + self._oldest) / step)) + 1 + limit = max(limit, n + self._table.horizon) + while n <= limit: + index = np.arange(n, min(n + _CHUNK, limit + 1)) + first, kinds = self._table.at(index) + ages = index * step - installed + due = (first >= 0) & (ages >= first * step - 1e-9 * step) + if due.any(): + at = int(np.argmax(due)) + i = int(index[at]) + price, _ = self._grid.prices(i, 1) + return i * step, float(price[0]), self._names[kinds[at]] + n += _CHUNK + return math.inf, 0.0, None + + def _planned_price(self, t: float) -> float: + """The price of a planned replacement at time ``t``, whatever the + unit's age: the opportunity's if one falls there (on a grid, the + one laid at the grid time nearest ``t``), else the calendar's + price on any day; ``inf`` where the calendar allows none. What a + ``RepairableRBD`` component on a ``"calendar"`` preventive policy + is charged when a stop of its maintenance group renews it early + (#349).""" + if self._grid is None: + return self._table.price_at(t) + step = cast(float, self.step) + price, _ = self._grid.prices(int(np.rint(t / step)), 1) + return float(price[0]) + + def _threshold(self, t): + """The threshold age at time (or times) ``t``: ``inf`` where no + replacement is open, or none pays (for the tests and the + diagrams).""" + times = checks.real_array(t, "t") + index = ( + times + if self.step is None + else np.rint(times / self.step).astype(np.int64) + ) + values, _ = self._table.at(np.ravel(index)) + if self.step is not None: + values = np.where(values >= 0, values * self.step, math.inf) + out = np.asarray(values, float).reshape(np.shape(index)) + return float(out) if out.ndim == 0 else out + + def next_replacement(self, age, now=0.0) -> PlannedReplacement: + """When the unit in service, ``age`` old at time ``now``, will be + replaced on the policy, if it does not fail first: the first + time, at or after now, at which a planned replacement is open + and the unit is at least its threshold. A unit older than the + oldest age the policy was worked out to (``settings["max_age"]``) + is past every threshold, and is replaced at the next + opportunity, or on any day now where the calendar allows one. + + Parameters + ---------- + age : float + The unit's age now, at least 0 (0: just put in, so not + replaced again now). + now : float, optional + The time now, in the calendar's time, by default 0. + + Returns + ------- + PlannedReplacement + When, at which opportunity, and at what price. + """ + if not (checks.is_number(age) and 0.0 <= float(age) < math.inf): + raise ValueError(f"age must be a number, at least 0; got {age!r}.") + if not (checks.is_number(now) and math.isfinite(float(now))): + raise ValueError(f"now must be a finite number; got {now!r}.") + now = float(now) + time, price, kind = self._next_planned(now - float(age), now) + if kind is None: + return PlannedReplacement(math.inf, None, None) + return PlannedReplacement(float(time), kind, float(price)) + + def schedule(self, kinds=(), until=None) -> list: + """The planned replacements open from time 0 up to ``until``, with + each one's threshold and the chance the policy replaces the unit + there. + + Parameters + ---------- + kinds : tuple of str, optional + The kinds to list, by name; by default every opportunity. + ``"anytime"`` among them lists the days with only a replacement + on any day open too. + until : float, optional + The time to list them up to (not including). By default the + period over which the thresholds repeat, or the end of a + calendar's given dates; give it where there is neither. + + Returns + ------- + list of (float, str, float, float) + Each as ``(time, kind, threshold, chance)``: replace the unit + there if it is at least ``threshold`` old, which happens with + ``chance`` in the long run (a periodic calendar's), or from a + new unit at 0 (given dates, or a cycle too long to keep). An + opportunity the policy never uses has a chance of 0. + """ + if until is None: + until = self.period or self.settings.get("table_until") + if until is None: + raise ValueError( + "The policy does not repeat: give until, the time to " + "list the opportunities up to." + ) + if not (checks.is_number(until) and 0.0 < float(until) < math.inf): + raise ValueError( + f"until must be a number, more than 0; got {until!r}." + ) + wanted = set(kinds) if kinds else None + unknown = (wanted or set()) - set(self._names) + if unknown: + raise ValueError( + f"Unknown kinds {sorted(unknown)}: the calendar's are " + f"{self._names}." + ) + if self.step is None: + return self._table.schedule(float(until), wanted) + n = int(math.ceil(float(until) / self.step - 1e-9)) + first, kind = self._table.at(np.arange(n)) + chance = self._table.chances(n) + out = [] + for i in np.flatnonzero(kind >= 0): + name = self._names[kind[i]] + if wanted is None and name == ANYTIME: + continue + if wanted is not None and name not in wanted: + continue + age = first[i] * self.step if first[i] >= 0 else math.inf + out.append( + (float(i * self.step), name, float(age), float(chance[i])) + ) + return out + + +def _resolve_step(step, calendar, part) -> float: + if step is None: + sf = _survival(part) + return _default_step(calendar, _mean(part, sf)) + if not (checks.is_number(step) and 0.0 < float(step) < math.inf): + raise ValueError(f"step must be a number, more than 0; got {step!r}.") + return float(step) + + +def _settle(grid: _Grid, ages: _Ages) -> int: + """A window's settling length, in steps: the policy at a time depends + on the calendar this far ahead of it, no further.""" + calendar = grid.calendar + gaps = [o.period for o in calendar.opportunities if o.periodic] + if grid.repeat is not None: + gaps.append(grid.repeat) + if grid.plan_end is not None: + gaps.append(abs(grid.plan_end)) + longest = max(gaps, default=0.0) + length = SETTLE_GAPS * longest + SETTLE_SPANS * ages.span + return int(math.ceil(length / grid.step)) + + +def optimal_calendar_policy( + part, + calendar, + *, + step=None, + max_age=None, + method="auto", + tail="repeat", + mc_samples=64, + seed=None, +) -> CalendarPolicy: + """The work of ``NonRepairable.optimal_calendar_policy`` (see + there).""" + if not isinstance(calendar, Calendar): + raise TypeError(f"calendar must be a Calendar; got {calendar!r}.") + checks.one_of("method", method, METHODS) + checks.one_of("tail", tail, TAILS) + if max_age is not None and not ( + checks.is_number(max_age) and 0.0 < float(max_age) < math.inf + ): + raise ValueError( + f"max_age must be a number, more than 0; got {max_age!r}." + ) + failure, anytime = calendar._prices(part) + if not calendar.opportunities and anytime is None: + allowed = ( + "its anytime is None" + if calendar.anytime is None + else "the part has no planned_cost for its anytime" + ) + raise ValueError( + "The calendar allows no planned replacement (no opportunities, " + f"and none on any day: {allowed}), so there is no policy to " + "find but running to failure: give the part a planned_cost, or " + "the calendar opportunities or an anytime price." + ) + dated = not calendar.periodic + life = _saved_life(part) + if method == "integration": + from repyability.maintenance._integration import integrated_policy + + if anytime is not None: + raise NotImplementedError( + "method='integration' works between opportunities, with no " + "replacement on any day; this calendar has one (anytime " + f"{anytime:g}): use method='dp', or anytime=None." + ) + return integrated_policy( + part, calendar, failure, max_age=max_age, tail=tail, life=life + ) + step = _resolve_step(step, calendar, part) + ages = _Ages(part, step, None if max_age is None else float(max_age)) + grid = _Grid(calendar, anytime, step, tail=tail if dated else None) + names = grid.names + settle = _settle(grid, ages) + short = grid.cycle is not None and grid.cycle <= MAX_CYCLE + if method == "auto": + method = "sampling" if (not dated and not short) else "dp" + if method == "sampling": + warnings.warn( + f"The calendar repeats only after {grid.cycle:,} steps of " + f"the grid, more than {MAX_CYCLE:,}, so the cost rate is " + f"sampled over mc_samples ({mc_samples}) windows of it, " + "each an exact solve that may take a second or more: " + "mc_samples sets the cost, and cost_rate_se says how close " + "the rate is; the rates by kind are estimates with no " + "standard error. method='dp' differences one long horizon " + "instead.", + stacklevel=outside_level(), + ) + if method == "sampling": + if dated: + raise NotImplementedError( + "method='sampling' samples times of a periodic calendar; " + "this one has given dates: use method='dp'." + ) + if ( + mc_samples is None + or not checks.is_whole(mc_samples) + or int(mc_samples) < 2 + ): + raise ValueError( + f"mc_samples must be a whole number, at least 2; got " + f"{mc_samples!r}." + ) + return _sampled_policy( + grid, ages, failure, settle, int(mc_samples), seed, names, life + ) + if not dated and short: + return _cycle_policy(grid, ages, failure, names, life) + if not dated: + return _differenced_policy(grid, ages, failure, settle, names, life) + return _dated_policy( + part, calendar, anytime, ages, failure, settle, tail, names, life + ) + + +def _saved_life(part) -> Optional[dict]: + """The saved form of the part's life, for the policy to carry (a + ``RepairableRBD`` component given the policy must have that life); + None where it does not save.""" + from repyability.rbd.serialisation import serialise_model + + try: + return serialise_model(part.reliability) + except (NotImplementedError, TypeError, ValueError): + return None + + +def _cycle_policy(grid, ages, failure, names, life) -> CalendarPolicy: + costs, kinds = grid.prices(0, grid.cycle) + cycle = _relative_values(costs, ages, failure) + _warn_not_threshold(cycle.upper) + replaced, failures = _stationary(cycle.first, ages) + length = grid.cycle * grid.step + rates, spent = _summary( + names, grid.anytime, kinds, costs, replaced, failures, failure, length + ) + policy = CalendarPolicy( + cost_rate=cycle.rate, + cost_rate_se=None, + replacement_rates=rates, + cost_rates=spent, + method="dp", + basis="exact", + step=grid.step, + period=length, + determined_until=None, + threshold_only=cycle.upper, + settings={ + "sweeps": cycle.sweeps, + "converged": cycle.converged, + "cost_rate_bounds": (cycle.low, cycle.high), + "cycle_steps": grid.cycle, + "max_age": ages.size * ages.step, + }, + life=life, + ) + table = _CycleTable(cycle.first, kinds, replaced) + return policy._bind(table, names, grid, ages.size * ages.step) + + +def _differenced(record, n1: int, n2: int, step: float) -> float: + return float((record[n1] - record[n2]) / ((n2 - n1) * step)) + + +def _run_in(grid, ages, failure, start, settle, span): + """A window from grid index ``start``: a new unit run in over + ``settle`` steps, then ``span`` steps, and a settling length for the + end. The cost rate differenced over the ``span`` steps, and each kind's + replacements and failures over them (from the new unit run in).""" + record, first, kinds, upper = _window( + grid, start, settle + span + settle, ages, failure + ) + rate = _differenced(record, settle, settle + span, grid.step) + new = np.zeros(ages.size + 1) + new[0] = 1.0 + settled, _, _ = _propagate(new, first[:settle], ages.p) + window = slice(settle, settle + span) + _, replaced, failures = _propagate(settled, first[window], ages.p) + costs, _ = grid.prices(start + settle, span) + return rate, replaced, failures, kinds[window], costs, upper + + +def _differenced_policy( + grid, ages, failure, settle, names, life +) -> CalendarPolicy: + """A periodic calendar too long to sweep, by a long finite horizon: + the cost rate differenced over ``2 * settle`` steps, after a settling + length.""" + span = 2 * max(settle, 1) + rate, replaced, failures, kinds, costs, upper = _run_in( + grid, ages, failure, 0, settle, span + ) + _warn_not_threshold(upper) + rates, spent = _summary( + names, + grid.anytime, + kinds, + costs, + replaced, + failures, + failure, + span * grid.step, + ) + policy = CalendarPolicy( + cost_rate=rate, + cost_rate_se=None, + replacement_rates=rates, + cost_rates=spent, + method="dp", + basis="differenced", + step=grid.step, + period=None, + determined_until=None, + threshold_only=upper, + settings={ + "differenced_over": span * grid.step, + "settle": settle * grid.step, + "table_until": span * grid.step, + "max_age": ages.size * ages.step, + }, + life=life, + ) + _, first, kinds, _ = _window(grid, 0, span + settle, ages, failure) + table = _WindowTable( + grid, ages, failure, settle, first[:span], kinds[:span] + ) + return policy._bind(table, names, grid, ages.size * ages.step) + + +def _sampled_rate(grid, ages, failure, settle, samples, rng): + """The cost rate as the mean, over uniformly sampled times of the + cycle, of the rate differenced over a window from each (the cost to + go of a new part there, less at ``span`` steps on, over the time + between), and its standard error; and each kind's rate, from a new + unit run in before each window.""" + calendar = grid.calendar + longest = max( + [o.period for o in calendar.opportunities if o.periodic] + [ages.mean] + ) + span = int(math.ceil(longest / grid.step)) + cycle = grid.cycle if grid.cycle is not None else 2**62 + rates = np.empty(samples) + by_kind = np.zeros(len(grid.names)) + spent = np.zeros(len(grid.names)) + failures = 0.0 + for i in range(samples): + start = int(rng.integers(0, cycle)) - settle + rate, replaced, failed, kinds, costs, _ = _run_in( + grid, ages, failure, start, settle, span + ) + rates[i] = rate + open_ = kinds >= 0 + np.add.at(by_kind, kinds[open_], replaced[open_]) + np.add.at(spent, kinds[open_], replaced[open_] * costs[open_]) + failures += failed + time = samples * span * grid.step + shown = [n for n in grid.names if n != ANYTIME or grid.anytime is not None] + kinds_rates = { + n: float(by_kind[k] / time) + for k, n in enumerate(grid.names) + if n in shown + } + kinds_costs = { + n: float(spent[k] / time) + for k, n in enumerate(grid.names) + if n in shown + } + kinds_rates["failure"] = float(failures / time) + kinds_costs["failure"] = float(failures * failure / time) + return ( + float(rates.mean()), + float(rates.std(ddof=1) / math.sqrt(samples)), + kinds_rates, + kinds_costs, + ) + + +def _sampled_policy( + grid, ages, failure, settle, samples, seed, names, life +) -> CalendarPolicy: + rng = np.random.default_rng(checks.seed(seed)) + rate, error, rates, spent = _sampled_rate( + grid, ages, failure, settle, samples, rng + ) + _, first, kinds, upper = _window(grid, 0, 1 + settle, ages, failure) + policy = CalendarPolicy( + cost_rate=rate, + cost_rate_se=error, + replacement_rates=rates, + cost_rates=spent, + method="sampling", + basis="sampled", + step=grid.step, + period=None, + determined_until=None, + threshold_only=upper, + settings={ + "mc_samples": samples, + "settle": settle * grid.step, + "cycle": None if grid.cycle is None else grid.cycle * grid.step, + "max_age": ages.size * ages.step, + }, + life=life, + ) + table = _WindowTable(grid, ages, failure, settle, first[:1], kinds[:1]) + return policy._bind(table, names, grid, ages.size * ages.step) + + +def _dated_policy( + part, calendar, anytime, ages, failure, settle, tail, names, life +) -> CalendarPolicy: + """A calendar with given dates: the policy over the plan and a + settling length past it, solved with the chosen tail; with the other + tail too, for the first time the two differ. The cost rate, and each + kind's, are the repeated plan's.""" + step = ages.step + end = max(calendar.plan_end, 0.0) + table = int(math.ceil(end / step)) + settle + tables = {} + for name in TAILS: + grid = _Grid(calendar, anytime, step, tail=name) + _, first, kinds, upper = _window( + grid, 0, table + settle, ages, failure + ) + tables[name] = (grid, first[:table], kinds[:table], upper) + grid, first, kinds, upper = tables[tail] + other = tables["anytime" if tail == "repeat" else "repeat"] + differ = np.flatnonzero((first != other[1]) | (kinds != other[2])) + until = (differ[0] if differ.size else table) * step + repeated = tables["repeat"][0] + if repeated.cycle is not None and repeated.cycle <= MAX_CYCLE: + costs, cycle_kinds = repeated.prices( + repeated.cycle_start, repeated.cycle + ) + cycle = _relative_values(costs, ages, failure) + replaced, failures = _stationary(cycle.first, ages) + rate, basis = cycle.rate, "exact" + time = repeated.cycle * step + else: + span = 2 * max(settle, 1) + rate, replaced, failures, cycle_kinds, costs, _ = _run_in( + repeated, ages, failure, 0, settle, span + ) + basis = "differenced" + time = span * step + rates, spent = _summary( + names, anytime, cycle_kinds, costs, replaced, failures, failure, time + ) + _warn_not_threshold(upper) + policy = CalendarPolicy( + cost_rate=rate, + cost_rate_se=None, + replacement_rates=rates, + cost_rates=spent, + method="dp", + basis=basis, + step=step, + period=None, + determined_until=float(until), + threshold_only=upper, + settings={ + "tail": tail, + "repeat": repeated.repeat, + "table_until": end + step, + "settle": settle * step, + "max_age": ages.size * ages.step, + }, + life=life, + ) + window = _WindowTable(grid, ages, failure, settle, first, kinds) + return policy._bind(window, names, grid, ages.size * ages.step) + + +def _warn_not_threshold(upper: bool) -> None: + if not upper: + warnings.warn( + "At some times the ages that replace are not all those from " + "the youngest (a life whose hazard falls): the threshold given " + "is the youngest age that replaces, and some older ones are " + "kept.", + stacklevel=outside_level(), + ) + + +def _rule_prices(grid: _Grid, rule: dict, start: int, n: int): + """For a fixed rule (the age from which each kind replaces), each + step's price and first replacing age index, taking at each step the + kind with the youngest age in the rule (the cheapest at a tie).""" + step = grid.step + costs = np.full(n, math.inf) + first = np.full(n, -1, dtype=np.int64) + best = np.full(n, math.inf) + calendar = grid.calendar + options = [] + if grid.anytime is not None and ANYTIME in rule: + options.append((ANYTIME, grid.anytime, np.arange(n))) + low, high = (start - 0.5) * step, (start + n - 0.5) * step + for o in calendar.opportunities: + if o.name not in rule: + continue + times = grid._dates(o, low - step, high + step) + index = np.rint(times / step).astype(np.int64) - start + options.append((o.name, o.cost, index[(index >= 0) & (index < n)])) + for name, price, index in options: + age = float(rule[name]) + i = int(math.ceil(age / step - 1e-9)) + better = (age < best[index]) | ( + (age == best[index]) & (price < costs[index]) + ) + chosen = index[better] + best[chosen] = age + costs[chosen] = price + first[chosen] = i + return costs, first + + +def calendar_policy_cost( + part, calendar, rule, *, step=None, max_age=None, tail="repeat" +) -> float: + """The work of ``NonRepairable.calendar_policy_cost`` (see there).""" + if not isinstance(calendar, Calendar): + raise TypeError(f"calendar must be a Calendar; got {calendar!r}.") + checks.one_of("tail", tail, TAILS) + if not isinstance(rule, dict): + raise TypeError( + "rule must be a dict of the age from which each kind of " + f"replacement replaces the part; got {rule!r}." + ) + names = [o.name for o in calendar.opportunities] + [ANYTIME] + for name, age in rule.items(): + if name not in names: + raise ValueError( + f"Unknown kind {name!r} in the rule: the calendar's are " + f"{names}." + ) + if not (checks.is_number(age) and 0.0 <= float(age)): + raise ValueError( + f"The age for {name!r} must be a number, at least 0 (inf " + f"for never); got {age!r}." + ) + rule = {k: float(v) for k, v in rule.items() if float(v) < math.inf} + failure, anytime = calendar._prices(part) + if ANYTIME in rule and anytime is None: + raise ValueError( + "The rule replaces on any day, which the calendar does not " + "allow (its anytime is None)." + ) + step = _resolve_step(step, calendar, part) + ages = _Ages(part, step, None if max_age is None else float(max_age)) + dated = not calendar.periodic + grid = _Grid(calendar, anytime, step, tail="repeat" if dated else None) + if grid.cycle is not None and grid.cycle <= MAX_CYCLE: + costs, first = _rule_prices(grid, rule, grid.cycle_start, grid.cycle) + return _relative_values(costs, ages, failure, fixed=first).rate + settle = _settle(grid, ages) + span = max(settle, 1) + n = 2 * span + settle + costs, first = _rule_prices(grid, rule, 0, n) + record = np.empty(n) + _sweep( + np.zeros(ages.size + 1), + costs, + ages.p, + failure, + fixed=first, + record=record, + ) + return _differenced(record, 0, 2 * span, step) diff --git a/repyability/maintenance/_skip.py b/repyability/maintenance/_skip.py new file mode 100644 index 00000000..1e6d129a --- /dev/null +++ b/repyability/maintenance/_skip.py @@ -0,0 +1,255 @@ +"""The value of skipping the next planned replacement right after a +failure (#319, UC-6 of this package's ``README.md``). + +A part is replaced when it fails (unplanned, at its ``unplanned_cost``) +and on a plan (planned, at its ``planned_cost``), in no time either way. +Right after a failure a new unit is in, and the question is whether to +skip the next planned replacement. Each choice is costed in present value, +at a continuous discount rate ``r`` a unit of the life's time (0 by +default: undiscounted), and the NPV of skipping is what it saves: the cost +of doing the replacement, less that of skipping it. + +**A plan that resets** (age replacement): the next planned replacement is +``every`` after the last replacement, planned or not. Each replacement +starts the same process afresh, so a cycle's discounted cost ``A(s)`` and +discounted length ``M(s)`` (cut off at age ``s``) give the value of the +plan from new, ``V = A(T) / (r M(T))`` (renewal-reward, discounted), and +skipping makes the first cycle's cut-off ``2 T`` rather than ``T``:: + + NPV = A(T) - A(2T) + A(T) (M(2T) - M(T)) / M(T) + +with ``M(s)`` the integral of ``exp(-r x) R(x)`` to ``s``, and ``A(s) = +unplanned L(s) + planned exp(-r s) R(s)``, ``L(s) = 1 - exp(-r s) R(s) - +r M(s)`` the discounted chance of a failure first. The discount rate +cancels from the last term, which holds with no discounting too. The two +present values compared are those of the whole future: ``V`` if the +replacement is done, and ``A(2T) + (1 - r M(2T)) V`` if it is skipped +(the longer first cycle, then the plan from new). Undiscounted both are +infinite (``replace`` and ``skip`` are ``inf``), and only their +difference is finite, the classic comparison: each choice's first +cycle's cost less its length at the plan's cost rate ``g = A(T) / +M(T)``, ``A(T) - g M(T)`` (0, the plan's own) against ``A(2T) - g +M(2T)``. Either way the difference is the NPV above. A discount rate +that discounts the costs away (a rate a year with a life in days, #231) +is warned of as a ``RepairableRBD``'s costs warn of it. + +**A plan that does not reset** (block replacement, on a calendar): +planned replacements fall on fixed times, the next ``d`` after the +failure. Done, a new unit goes in at ``d``; skipped, the unit in now runs +on. Both replace at the following time, ``d + T``, so the two differ only +before it, and are compared to it:: + + NPV = planned exp(-r d) + exp(-r d) F(T) - (F(d + T) - F(d)) + +with ``F(t)`` the expected discounted cost of the failures of a new unit +and its replacements in ``[0, t)``: a +[`RepairableRBD`][repyability.RepairableRBD]'s ``expected_cost`` of a +component replaced in no time (exact, no simulation). +""" + +import math +from dataclasses import dataclass + +import numpy as np + +from repyability.rbd.results import _ResultMapping +from repyability.utils import checks + + +def _time(value, name: str) -> float: + """``value`` as a time: a finite number, more than 0.""" + if not (checks.is_number(value) and 0.0 < float(value) < math.inf): + raise ValueError( + f"{name} must be a time, a number more than 0, in the life's " + f"time unit; got {value!r}." + ) + return float(value) + + +@dataclass +class SkipValue(_ResultMapping): + """Whether to skip the next planned replacement of a part, right after + a failure: returned by + [`NonRepairable.skip_next_replacement`][repyability.NonRepairable.skip_next_replacement]. + + Attributes + ---------- + resets : bool + Whether the plan resets at each replacement. + every : float + The plan's interval, in the life's time unit. + time_to_next : float + The time from the failure to the next planned replacement: + ``every`` for a plan that resets. + discount_rate : float + The continuous discount rate a unit of the life's time. + replace : float + The present-value cost of doing the next planned replacement. + On a plan that does not reset, the costs to the following planned + replacement: the replacement, the new unit's failures to then and + the failures before it. On a plan that resets, the cost of the + whole future (the first cycle, then the plan from new), which + with no discounting is endless: ``inf``. + skip : float + The present-value cost of skipping it, over the same comparison: + the failures to the following planned replacement; or, on a plan + that resets, the whole future from the longer first cycle + (``inf`` with no discounting). + npv : float + What skipping saves, ``replace - skip``: the planned cost saved, + net of the failures it adds. Positive, skipping pays. Finite + where both are endless: the first cycles' difference, each + cycle's cost less its length at the plan's cost rate. + best : str + ``"skip"`` where skipping saves, ``"replace"`` otherwise. + """ + + resets: bool + every: float + time_to_next: float + discount_rate: float + replace: float + skip: float + npv: float + best: str + + +def skip_next_replacement( + part, every, *, resets, time_to_next=None, discount_rate=0.0 +) -> SkipValue: + """The work of ``NonRepairable.skip_next_replacement`` (see there).""" + part._require_costs() + if part.model_parameterization not in ("parametric", "degradation"): + raise NotImplementedError( + "skip_next_replacement needs a life with a survival function in " + "closed form (a surpyval parametric distribution, mixture or " + "degradation process); this part's is " + f"{part.model_parameterization}." + ) + every = _time(every, "every") + if not isinstance(resets, bool): + raise TypeError( + "resets must be True (the plan resets at each replacement) or " + f"False (it keeps to fixed times); got {resets!r}." + ) + if not ( + checks.is_number(discount_rate) + and 0.0 <= float(discount_rate) < math.inf + ): + raise ValueError( + "discount_rate must be a continuous rate a unit of the life's " + f"time, at least 0; got {discount_rate!r}." + ) + rate = float(discount_rate) + if resets: + if time_to_next is not None: + raise ValueError( + "A plan that resets has its next planned replacement every " + f"({every:g}) after the failure: give no time_to_next." + ) + time_to_next = every + replace, skip, npv = _resetting(part, every, rate) + else: + if time_to_next is None: + raise ValueError( + "A plan on fixed times needs time_to_next: the time from " + "the failure to the next planned replacement." + ) + time_to_next = _time(time_to_next, "time_to_next") + if time_to_next > every: + raise ValueError( + f"time_to_next ({time_to_next:g}) must be at most every " + f"({every:g}): the next planned replacement is within one " + "interval." + ) + replace, skip = _fixed(part, every, rate, time_to_next) + npv = replace - skip + return SkipValue( + resets, + every, + time_to_next, + rate, + replace, + skip, + npv, + "skip" if npv > 0.0 else "replace", + ) + + +def _warn_discounted_away(part, rate: float, horizons) -> None: + """The check a ``RepairableRBD``'s costs make of a discount rate + (#231): warn where it discounts the costs at the life's mean (or + over ``horizons``) to almost nothing.""" + from repyability.rbd._common import _present_horizon + from repyability.rbd._model_utils import MODEL_ERRORS, model_mean + + def lives() -> list: + try: + return [float(model_mean(part.reliability))] + except MODEL_ERRORS: + return [] + + _present_horizon(np.asarray(horizons, float), rate, lives) + + +def _resetting(part, every: float, rate: float) -> tuple: + """The present-value costs of doing and of skipping the next planned + replacement on a plan that resets, and the NPV of skipping (see the + module's notes).""" + from repyability.rbd._mean_lifetime import _integrate, model_knots + + _warn_discounted_away(part, rate, [every, 2.0 * every]) + sf = part.reliability_function + + def discounted(x: np.ndarray) -> np.ndarray: + survival = np.asarray(sf(x), dtype=float).ravel() + return np.exp(-rate * x) * survival + + knots = model_knots(part.reliability) + + def length(cut: float) -> float: + inside = knots[(knots > 0.0) & (knots < cut)] + return _integrate(discounted, np.concatenate(([0.0], inside, [cut]))) + + def cost(cut: float, held: float) -> float: + kept = math.exp(-rate * cut) * float(np.ravel(sf(cut))[0]) + failed = 1.0 - kept - rate * held + return part.cu * failed + part.cp * kept + + once, twice = length(every), length(2.0 * every) + done, skipped = cost(every, once), cost(2.0 * every, twice) + if rate > 0.0: + # The whole future: the plan from new is worth A(T) / (r M(T)), + # and skipping puts it after the longer first cycle. + value = done / (rate * once) + skip = skipped + (1.0 - rate * twice) * value + return value, skip, value - skip + # Undiscounted, each whole future is endless, and the NPV is the + # first cycles' difference, each cycle's cost less its length at the + # plan's cost rate (the plan's own cycle is 0 by construction). + g = done / once + return math.inf, math.inf, (done - g * once) - (skipped - g * twice) + + +def _fixed(part, every: float, rate: float, time_to_next: float) -> tuple: + """The present-value costs of doing and of skipping the next planned + replacement on a plan on fixed times, the next ``time_to_next`` after + the failure, to the following one (see the module's notes).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + unit = RepairableRBD( + [("s", "unit"), ("unit", "t")], + { + "unit": { + "reliability": part.reliability, + "repairability": "instant", + "replace_cost": part.cu, + } + }, + ) + windows = np.array([every, time_to_next, time_to_next + every]) + interval, before, after = unit.expected_cost( + windows, discount_rate=rate + ).mean + later = math.exp(-rate * time_to_next) + return float(part.cp * later + later * interval + before), float(after) diff --git a/repyability/network.py b/repyability/network.py index 74093c29..ffeaef9a 100644 --- a/repyability/network.py +++ b/repyability/network.py @@ -27,14 +27,26 @@ The simulation draws each element's lifetime and finds, in each sample, the path that lasts longest: its lifetime is the network's. + +The importance measures beyond Birnbaum's (#315) are the diagram's +formulas (``rbd/_measures.py``) over the network's values with an element +held working or failed, from the same diagram replayed; the exact +Fussell-Vesely numerators come from the decision diagrams of each +element's minimal cut sets (``modular._critical_closures``), as a +diagram's core's do. The B-lives invert ``sf``, and the parameter +uncertainty draws the elements' models as a diagram's +(``rbd/uncertainty.py``), each draw worked out exactly. """ +import json from collections import deque from typing import ( Any, + Collection, Dict, Hashable, List, + Mapping, NamedTuple, Optional, Sequence, @@ -44,12 +56,43 @@ import numpy as np +from repyability._version import __version__ +from repyability.rbd import _measures from repyability.rbd import _montecarlo as montecarlo +from repyability.rbd._differential import check as check_differential +from repyability.rbd._differential import shares +from repyability.rbd._mean_lifetime import ( + mean_lifetime, + model_kinks, + model_knots, +) from repyability.rbd._model_utils import is_fixed_probability +from repyability.rbd._repairable_uncertainty import half_named from repyability.rbd._sampling import lifetime_sampler -from repyability.rbd.non_repairable_rbd import check_x +from repyability.rbd.modular import _critical_closures +from repyability.rbd.non_repairable_rbd import ( + NonRepairableRBD, + _times, + check_x, +) +from repyability.rbd.rbd import Pairs, _set_order +from repyability.rbd.results import UncertaintyResult +from repyability.rbd.routes import AnalysisRoute, Refused from repyability.rbd.shannon import _minimal_cut_sets, _shannon_plan +from repyability.rbd.uncertainty import ( + check_draws, + draw_models, + draw_with, + fitted_together, +) +from repyability.utils.checks import ( + no_distribution, + one_of, + real_array, + simulation_options, +) from repyability.utils.deprecation import refuse_removed_names +from repyability.utils.json_io import read_json, write_json from repyability.utils.wrappers import numpy_seed #: How the exact values are worked out: ``"bdd"`` (the default), by the @@ -71,6 +114,10 @@ # Value slots 0 and 1 of a plan: the terminals are parted, joined. _FAIL, _WORK = 0, 1 +# The values of those slots that give the probability that the terminals +# are joined, and that they are parted. +_JOINED, _PARTED = (0.0, 1.0), (1.0, 0.0) + # A decision's outcome that is no state: the terminals parted, joined. _NONE_FAIL, _NONE_WORK = -1, -2 @@ -90,7 +137,11 @@ class Network: rather than with its number of paths (see the module docstring): a grid of 100 nodes takes a second and a half. One of more than ``MAX_STATES`` (5,000,000) states refuses, pointing to the - simulation. + simulation. The importance measures are those of a + [`NonRepairableRBD`][repyability.NonRepairableRBD], over the links and + the failing nodes; [`analysis_routes`][repyability.Network.analysis_routes] + says how each analysis is worked out, and + [`to_dict`][repyability.Network.to_dict] saves the network. Parameters ---------- @@ -157,6 +208,9 @@ class Network: [['a', 'c', 'e'], ['a', 'd'], ['b', 'c', 'd'], ['b', 'e']] """ + # The kind of diagram, as the messages name it (see ``check_x``). + _KIND = "network" + def __init__( self, links: Dict[Hashable, Tuple[Hashable, Hashable, Any]], @@ -209,6 +263,9 @@ def __init__( ) self._paths: Optional[List[frozenset]] = None self._plan: Optional[_Plan] = None + # The decision diagrams of each element's minimal cut sets, and + # path sets, once built (see ``_failed_sets``). + self._closures: Dict[str, tuple] = {} # Why the exact values were refused, once found (#229): the search # that found it is not repeated. self._refused: Optional[str] = None @@ -229,6 +286,7 @@ def _model(what: str, model): from surpyval import FixedEventProbability return FixedEventProbability.from_params(float(model)) + no_distribution(model, what) if not (hasattr(model, "sf") and hasattr(model, "ff")): raise ValueError( f"{what}: the model must have sf and ff (a lifetime " @@ -287,8 +345,10 @@ def _simple_paths(self) -> List[frozenset]: return paths def path_sets(self) -> Set[frozenset]: - """The minimal path sets: for each simple path between the - terminals, the set of its links and of its nodes that can fail. + """The minimal path sets, as a set: for each simple path between + the terminals, the set of its links and of its nodes that can fail + ([`minimal_path_sets`][repyability.Network.minimal_path_sets] + lists them, smallest first). Returns ------- @@ -303,8 +363,10 @@ def path_sets(self) -> Set[frozenset]: return set(self._simple_paths()) def cut_sets(self) -> Set[frozenset]: - """The minimal cut sets: the smallest sets of links and nodes whose - failure parts the terminals. + """The minimal cut sets, as a set: the smallest sets of links and + nodes whose failure parts the terminals + ([`minimal_cut_sets`][repyability.Network.minimal_cut_sets] lists + them, smallest first). Returns ------- @@ -320,6 +382,62 @@ def cut_sets(self) -> Set[frozenset]: """ return set(_minimal_cut_sets(self._decomposition().steps())) + def minimal_path_sets(self) -> List[frozenset]: + """The minimal path sets, listed smallest first (then by name), as + a diagram's [`minimal_path_sets`][repyability.RBD.minimal_path_sets] + and a tree's list them: the sets of ``path_sets()``. + + Returns + ------- + list[frozenset] + The minimal path sets, each a frozenset of link and node names. + + Raises + ------ + NotImplementedError + As for ``path_sets``. + + Examples + -------- + >>> from repyability import Network + >>> fork = Network( + ... {"a": ("s", "x", 0.1), "b": ("x", "t", 0.1), + ... "c": ("s", "t", 0.2)}, + ... "s", "t", + ... ) + >>> [sorted(path) for path in fork.minimal_path_sets()] + [['c'], ['a', 'b']] + """ + return sorted(self.path_sets(), key=_set_order) + + def minimal_cut_sets(self) -> List[frozenset]: + """The minimal cut sets, listed smallest first (then by name), as + a diagram's [`minimal_cut_sets`][repyability.RBD.minimal_cut_sets] + and a tree's list them: the sets of ``cut_sets()``. + + Returns + ------- + list[frozenset] + The minimal cut sets, each a frozenset of link and node names. + + Raises + ------ + NotImplementedError + As for ``cut_sets``. + + Examples + -------- + >>> from repyability import Network + >>> fork = Network( + ... {"a": ("s", "x", 0.1), "b": ("x", "t", 0.1), + ... "c": ("s", "t", 0.2)}, + ... "s", "t", + ... ) + >>> [sorted(cut) for cut in fork.minimal_cut_sets()] + [['a', 'c'], ['b', 'c']] + """ + return sorted(self.cut_sets(), key=_set_order) + def _decomposition(self) -> "_Plan": """The plan of the probability that the terminals are joined (see ``shannon.py``), as a ``_Plan``: the decision diagram built from the @@ -362,33 +480,112 @@ def _decomposition(self) -> "_Plan": # Exact # ------------------------------------------------------------------ - def _probabilities(self, x: np.ndarray, names) -> Tuple[dict, dict]: + def _probabilities( + self, x: np.ndarray, names, models: Optional[Mapping] = None + ) -> Tuple[dict, dict]: """Each named element's probabilities of working and of failing at - times ``x``, from its model's ``sf`` and ``ff``.""" + times ``x``, from its model's ``sf`` and ``ff`` (the ``models`` + given by name, else its own).""" + models = self.models if models is None else models p: Dict[Hashable, np.ndarray] = {} q: Dict[Hashable, np.ndarray] = {} + # A model takes a row of times; the values are shaped as ``x`` is. + times = x.ravel() for name in names: - model = self.models[name] + model = models[name] with np.errstate(all="ignore"): - p[name] = np.asarray(model.sf(x), dtype=float).reshape(x.shape) - q[name] = np.asarray(model.ff(x), dtype=float).reshape(x.shape) + p[name] = np.asarray(model.sf(times), dtype=float).reshape( + x.shape + ) + q[name] = np.asarray(model.ff(times), dtype=float).reshape( + x.shape + ) return p, q - def _value(self, x: np.ndarray, terminals=(0.0, 1.0)) -> np.ndarray: - """The probability that the terminals are connected at times ``x``, - or with ``terminals`` ``(1.0, 0.0)`` that they are not: a sum of - products, so that a small one keeps its precision.""" + def _inputs( + self, x: np.ndarray, hold: Optional[Mapping[Hashable, bool]] = None + ) -> Tuple["_Plan", dict, dict]: + """The plan, and its elements' probabilities of working and of + failing at times ``x``, those in ``hold`` held working (True: 1 + and 0) or failed.""" plan = self._decomposition() p, q = self._probabilities(x, plan.names) + for name, works in (hold or {}).items(): + p[name] = np.full(x.shape, 1.0 if works else 0.0) + q[name] = np.full(x.shape, 0.0 if works else 1.0) + return plan, p, q + + def _value( + self, + x: np.ndarray, + terminals=_JOINED, + hold: Optional[Mapping[Hashable, bool]] = None, + models: Optional[Mapping] = None, + ) -> np.ndarray: + """The probability that the terminals are connected at times ``x``, + or with ``terminals`` ``_PARTED`` that they are not: a sum of + products, so that a small one keeps its precision. The elements + in ``hold`` are held working (True) or failed; ``models`` stand in + for the elements' own, by name (a draw's).""" + if models is None: + plan, p, q = self._inputs(x, hold) + else: + plan = self._decomposition() + p, q = self._probabilities(x, plan.names, models) return _evaluate(plan, p, q, x.shape, terminals) - def _method(self, method: str, mc_samples) -> str: - if method not in ("exact", "simulate"): - raise ValueError( - f"method must be 'exact' or 'simulate', got {method!r}." + def _birnbaum( + self, x: np.ndarray, hold: Optional[Mapping[Hashable, bool]] = None + ) -> Dict[Hashable, np.ndarray]: + """Each element's Birnbaum importance at times ``x`` (see + ``birnbaum_importance``), the elements in ``hold`` held working + (True) or failed; 0 for one in no path.""" + plan, p, q = self._inputs(x, hold) + works, up = _value_and_gradient(plan, p, q, x.shape) + fails, down = _value_and_gradient(plan, p, q, x.shape, _PARTED) + zero = np.zeros(x.shape) + # ``+ 0.0``: a negated zero (an element in no path) prints as 0.0. + return { + name: np.where( + np.asarray(fails) <= np.asarray(works), + -np.asarray(down.get(name, zero)), + np.asarray(up.get(name, zero)), + ) + * np.ones(x.shape) + + 0.0 + for name in self.models + } + + def _require_time_varying(self) -> None: + """Raise if the reliability does not vary with time: every model a + fixed probability.""" + if self.is_fixed: + raise Refused( + "The reliability does not vary with time (every model is a " + "fixed probability): the time to a reliability is undefined." + ) + + def _require_lifetimes(self, what: str) -> None: + """Raise if every element is a fixed probability: the network then + parts at the start or never, so it has no lifetimes, no mean + lifetime and no failure density over time (as a diagram of fixed + nodes has none). A fixed element among others that age is a + part that fails at the start or never: it has no density, so no + part in the failures over time, and the mean is ``inf`` where it + keeps the terminals joined for good.""" + if self.is_fixed: + raise Refused( + "The reliability does not vary with time (every model is a " + f"fixed probability): the network has no lifetimes, so no " + f"{what}." + ) + + def _method(self, method: str, mc_samples, seed, name: str) -> str: + one_of("method", method, ("exact", "simulate")) + if method == "exact": + simulation_options( + f"{name}()", {"mc_samples": mc_samples, "seed": seed} ) - if method == "exact" and mc_samples is not None: - raise ValueError("mc_samples applies only to method='simulate'.") return method @check_x @@ -426,18 +623,21 @@ def sf( ------ ValueError If ``x`` is left out of a network whose models depend on time, - or ``method`` or ``mc_samples`` is invalid. + or ``method`` is unknown, or ``mc_samples`` or ``seed`` is + given with ``method="exact"``. NotImplementedError With ``method="exact"``, if the network's decision diagram would have more than ``MAX_STATES`` states (with ``METHOD = "paths"``, if there are more than ``MAX_PATHS`` paths between the terminals). """ - if self._method(method, mc_samples) == "simulate": + if self._method(method, mc_samples, seed, "sf") == "simulate": lives = self.random( 10_000 if mc_samples is None else mc_samples, seed ) - return np.mean(lives[:, None] > x[None, :], axis=0) + return np.mean( + lives[:, None] > x.ravel()[None, :], axis=0 + ).reshape(x.shape) return self._value(x) @check_x @@ -469,12 +669,14 @@ def ff( float or numpy.ndarray The unreliability. """ - if self._method(method, mc_samples) == "simulate": + if self._method(method, mc_samples, seed, "ff") == "simulate": lives = self.random( 10_000 if mc_samples is None else mc_samples, seed ) - return np.mean(~(lives[:, None] > x[None, :]), axis=0) - return self._value(x, (1.0, 0.0)) + return np.mean( + ~(lives[:, None] > x.ravel()[None, :]), axis=0 + ).reshape(x.shape) + return self._value(x, _PARTED) @check_x def birnbaum_importance(self, x=None) -> Dict[Hashable, Any]: @@ -496,21 +698,998 @@ def birnbaum_importance(self, x=None) -> Dict[Hashable, Any]: dict Per link and failing node, by name, its importance (0 for one that is in no path). + + Raises + ------ + ValueError, NotImplementedError + As for ``sf``. + """ + return self._birnbaum(x) + + @check_x + def improvement_potential(self, x=None) -> Dict[Hashable, Any]: + """Each element's improvement potential at time/s ``x``: how much + the reliability would rise if it were made perfect. + + ``IP_i = R(i working) - R``, worked out as the Birnbaum importance + times the element's probability of failing (its model's ``ff``), + so that a small one keeps its precision; as + [`NonRepairableRBD.improvement_potential`][repyability.NonRepairableRBD.improvement_potential] + defines it. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + + Returns + ------- + dict + Per link and failing node, by name, its improvement potential: + floats for a scalar ``x``, arrays otherwise. + + Raises + ------ + ValueError, NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> round(bridge.improvement_potential()["a"], 5) + 0.01062 + """ + q = self._probabilities(x, self.models)[1] + return _measures.improvement_potential(self._birnbaum(x), q) + + @check_x + def risk_achievement_worth(self, x=None) -> Dict[Hashable, Any]: + """Each element's risk achievement worth (RAW) at time/s ``x``: the + factor by which the unreliability would grow if it were failed. + + ``RAW_i = (1 - R(i failed)) / (1 - R)``, each unreliability worked + out in its own right from the decision diagram with the element + held failed, as + [`NonRepairableRBD.risk_achievement_worth`][repyability.NonRepairableRBD.risk_achievement_worth] + defines it. It is at least 1; inf or nan, with a numpy + RuntimeWarning, where the unreliability is 0. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + + Returns + ------- + dict + Per link and failing node, by name, its RAW: floats for a + scalar ``x``, arrays otherwise. + + Raises + ------ + ValueError, NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> round(bridge.risk_achievement_worth()["a"], 4) + 5.4414 + """ + return _measures.risk_achievement_worth( + { + name: self._value(x, _PARTED, {name: False}) + for name in self.models + }, + self._value(x, _PARTED), + ) + + @check_x + def risk_reduction_worth(self, x=None) -> Dict[Hashable, Any]: + """Each element's risk reduction worth (RRW) at time/s ``x``: the + factor by which the unreliability would shrink if it were made + perfect. + + ``RRW_i = (1 - R) / (1 - R(i working))``, each unreliability worked + out in its own right from the decision diagram with the element + held working, as + [`NonRepairableRBD.risk_reduction_worth`][repyability.NonRepairableRBD.risk_reduction_worth] + defines it. It is at least 1, and inf where a perfect element makes + the network perfect (a link that alone joins the terminals). + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + + Returns + ------- + dict + Per link and failing node, by name, its RRW: floats for a + scalar ``x``, arrays otherwise. + + Raises + ------ + ValueError, NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> round(bridge.risk_reduction_worth()["a"], 4) + 1.9743 + """ + return _measures.risk_reduction_worth( + self._value(x, _PARTED), + { + name: self._value(x, _PARTED, {name: True}) + for name in self.models + }, + ) + + @check_x + def criticality_importance( + self, x=None, kind: str = "failure" + ) -> Dict[Hashable, Any]: + """Each element's criticality importance at time/s ``x``. + + With ``B_i`` the Birnbaum importance, ``R_i`` the element's + reliability and ``R`` the network's, ``kind="failure"`` (the + default) gives ``CI_i = B_i (1 - R_i) / (1 - R)``: the probability + that the element has failed and is critical, given that the + terminals are parted, the share of the network's failures it + accounts for; ``kind="success"`` gives ``CI_i = B_i R_i / R``, the + probability that it is working and critical, given that the + terminals are joined. Each is worked out from the Birnbaum + importance and the network's unreliability (or reliability) in + their own right, so a small one keeps its precision; nan where the + network cannot fail (or cannot work). As + [`NonRepairableRBD.criticality_importance`][repyability.NonRepairableRBD.criticality_importance] + defines it. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + kind : str, optional + ``"failure"`` (the default) or ``"success"``. + + Returns + ------- + dict + Per link and failing node, by name, its criticality importance: + floats for a scalar ``x``, arrays otherwise. + + Raises + ------ + ValueError + If ``kind`` is neither ``"failure"`` nor ``"success"``, or as + for ``sf``. + NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> round(bridge.criticality_importance()["a"], 4) + 0.4935 + """ + one_of("kind", kind, ("failure", "success")) + p, q = self._probabilities(x, self.models) + if kind == "success": + return _measures.criticality_importance( + self._birnbaum(x), p, self._value(x) + ) + return _measures.criticality_importance( + self._birnbaum(x), q, self._value(x, _PARTED) + ) + + @check_x + def fussell_vesely( + self, x=None, fv_type: str = "c", method: str = "exact" + ) -> Dict[Hashable, Any]: + """Each element's Fussell-Vesely importance at time/s ``x``: the + share of the network's failure probability that involves it. + + With ``fv_type="c"`` (the default), the probability that some + minimal cut set containing the element has failed (every element + in it), over the probability that the terminals are parted; with + ``fv_type="p"``, the minimal path sets take the cut sets' place. + ``method="exact"`` (the default) works out the union exactly, from + a decision diagram of each element's sets (built once), as + [`NonRepairableRBD.fussell_vesely`][repyability.NonRepairableRBD.fussell_vesely] + does; ``"rare_event"`` sums the sets' probabilities instead, the + rare-event approximation, which can exceed 1 when failures are not + rare. Each probability is worked out in its own right, so a small + one keeps its precision; nan where the network cannot fail. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + fv_type : str, optional + ``"c"`` (the default) for the minimal cut sets and ``"p"`` for + the minimal path sets. + method : str, optional + ``"exact"`` (the default) or ``"rare_event"``. + + Returns + ------- + dict + Per link and failing node, by name, its Fussell-Vesely + importance: floats for a scalar ``x``, arrays otherwise. + + Raises + ------ + ValueError + If ``fv_type`` is not "c" or "p", ``method`` is not "exact" or + "rare_event", or as for ``sf``. + NotImplementedError + As for ``sf`` (with ``method="rare_event"``, as for + ``cut_sets`` or ``path_sets``). + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> round(bridge.fussell_vesely()["a"], 4) + 0.5065 + >>> round(bridge.fussell_vesely(method="rare_event")["a"], 4) + 0.5112 + """ + if fv_type not in ("c", "p"): + raise ValueError( + "fv_type must be either 'c' (cut-set) or 'p' (path-set), " + f"fv_type={fv_type!r} was given." + ) + one_of("method", method, ("exact", "rare_event")) + p, q = self._probabilities(x, self.models) + if method == "exact": + numerators = self._failed_sets(fv_type, p, q, x.shape) + else: + sets = self.cut_sets() if fv_type == "c" else self.path_sets() + numerators = {} + for name in self.models: + total = np.zeros(x.shape) + for members in sets: + if name in members: + total = total + np.prod( + [q[m] for m in members], axis=0 + ) + numerators[name] = total + return _measures.fussell_vesely(numerators, self._value(x, _PARTED)) + + def _failed_sets( + self, fv_type: str, p: dict, q: dict, shape: tuple + ) -> Dict[Hashable, np.ndarray]: + """For each element, the probability that every element of some + minimal cut set (``fv_type`` ``"c"``) or path set (``"p"``) + containing it has failed, the numerator of its exact + Fussell-Vesely importance: a union, from one decision diagram each + of the sets through the element (see ``modular._critical_closures``: + the path sets are the dual structure's cut sets), kept once built; + 0 for an element in no such set.""" + if fv_type not in self._closures: + steps, root = self._decomposition().steps() + if fv_type == "p": + steps, root = _dual(steps, root) + self._closures[fv_type] = ( + _critical_closures((steps, root)) + if root >= 2 + else ([], [], []) + ) + variables, steps, roots = self._closures[fv_type] + values = [np.zeros(shape), np.ones(shape)] + for pivot, failed, works in steps: + values.append(q[pivot] * values[failed] + p[pivot] * values[works]) + out = {name: np.zeros(shape) for name in self.models} + for variable, root in zip(variables, roots): + out[variable] = values[root] * q[variable] + return out + + def structural_importance(self) -> Dict[Hashable, float]: + """Each element's structural importance: the fraction of the other + elements' states in which it is pivotal (the terminals joined when + it works and parted when it fails), from the structure alone. + + It is the Birnbaum importance with every element's probability at + 1/2, as + [`RBD.structural_importance`][repyability.RBD.structural_importance] + works it out, and does not depend on the models. + + Returns + ------- + dict + Per link and failing node, by name, its structural importance, + in ``[0, 1]``. + + Raises + ------ + NotImplementedError + If the network's decision diagram would have more than + ``MAX_STATES`` states. + + Examples + -------- + The bridge's middle link is pivotal in an eighth of the other + links' states, each other link in three eighths: + + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> structural = bridge.structural_importance() + >>> structural["c"], structural["a"] + (0.125, 0.375) """ plan = self._decomposition() - p, q = self._probabilities(x, plan.names) - works, up = _value_and_gradient(plan, p, q, x.shape) - fails, down = _value_and_gradient(plan, p, q, x.shape, (1.0, 0.0)) - zero = np.zeros(x.shape) - out: Dict[Hashable, Any] = {} - for name in self.models: - out[name] = np.where( - np.asarray(fails) <= np.asarray(works), - -np.asarray(down.get(name, zero)), - np.asarray(up.get(name, zero)), - ) * np.ones(x.shape) + half = {name: np.full(1, 0.5) for name in plan.names} + _, gradient = _value_and_gradient(plan, half, half, (1,)) + return { + name: float(np.ravel(gradient.get(name, 0.0))[0]) + for name in self.models + } + + def differential_importance( + self, + x=None, + *, + change: str = "uniform", + kind: str = "failure", + groups: Optional[Mapping[Hashable, Collection[Hashable]]] = None, + ) -> Dict[Hashable, Any]: + """Each element's share of the change in the reliability when they + all change together: the differential importance measure (DIM, + Borgonovo & Apostolakis, 2001), as + [`NonRepairableRBD.differential_importance`][repyability.NonRepairableRBD.differential_importance] + defines it over the components. + + ``change="uniform"`` moves every element's probability by as much, + and shares out the Birnbaum importance; ``"proportional"`` each by + the same fraction of itself, and shares out the criticality + importance of that ``kind``. The shares add up to 1, and a group's + share is the sum of its members' (``groups``). NaN where there is + nothing to share out. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + change : str, optional + ``"uniform"`` (the default) or ``"proportional"``. + kind : str, optional + With a proportional change, which of each element's + probabilities moves in proportion: ``"failure"`` (the default), + of failing, or ``"success"``, of working. + groups : dict, optional + ``{name: elements}``: each group's share, the sum of its + elements', instead of each element's. + + Returns + ------- + dict + The shares, by element or group name: floats for a scalar + ``x``, arrays otherwise. + + Raises + ------ + ValueError + For an unknown ``change`` or ``kind``, a group naming an unknown + element, or as for ``sf``. + NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> shares = bridge.differential_importance( + ... groups={"ends": ["a", "b", "d", "e"], "middle": ["c"]} + ... ) + >>> {name: round(share, 4) for name, share in shares.items()} + {'ends': 0.9633, 'middle': 0.0367} + """ + check_differential("components", change, kind) + if change == "uniform": + values = self.birnbaum_importance(x) + else: + values = self.criticality_importance(x, kind) + return shares(values, groups, scalar=x is None or np.ndim(x) == 0) + + def joint_importance(self, x=None) -> Dict[Tuple[Hashable, Hashable], Any]: + """The joint (second-order) importance of each pair of elements: + whether improving the two together is worth more than improving + each (Hong & Lie, 1993), as + [`NonRepairableRBD.joint_importance`][repyability.NonRepairableRBD.joint_importance] + defines it. + + ``JRI(i, j) = R(1_i, 1_j) - R(1_i, 0_j) - R(0_i, 1_j) + R(0_i, + 0_j)``: how much element ``j``'s Birnbaum importance rises when + element ``i`` goes from failed to working, worked out as that + difference from the decision diagram with ``i`` held. Positive, + the two are complements (links in a chain); negative, substitutes + (links side by side); 0 where neither changes what the other is + worth. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model is a fixed + probability. + + Returns + ------- + dict + ``{(i, j): JRI}`` for each pair once, its names in order as + text, and found either way round: floats for a scalar ``x``, + arrays otherwise. + + Raises + ------ + ValueError, NotImplementedError + As for ``sf``. + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> joint = bridge.joint_importance() + >>> round(joint["a", "d"], 4), round(joint["a", "b"], 4) + (0.028, -0.972) + """ + times, scalar = _times(self, x) + names = list(self.models) + up = {name: self._birnbaum(times, {name: True}) for name in names} + down = {name: self._birnbaum(times, {name: False}) for name in names} + out = Pairs() + for k, i in enumerate(names): + for j in names[k + 1 :]: # noqa: E203 + value = up[i][j] - down[i][j] + out[Pairs.oriented(i, j)] = ( + float(np.ravel(value)[0]) if scalar else value + ) return out + def barlow_proschan_importance(self, x=None) -> Dict[Hashable, Any]: + """Each element's Barlow-Proschan importance: the probability that + the network's failure is caused by the element's, given that it + fails by ``x``, or over its whole life, as + [`NonRepairableRBD.barlow_proschan_importance`][repyability.NonRepairableRBD.barlow_proschan_importance] + defines it. + + An element's failure parts the terminals when the element is + critical then, so the network has failed by ``x`` through element + ``i`` with probability ``integral_0^x B_i(t) f_i(t) dt``, its + Birnbaum importance times its failure density (its model's ``df``, + or differences of its ``ff``), integrated by adaptive quadrature + (to about ``1e-9``, relative); its share of the sum over the + elements is its importance, and the shares add up to 1. + + Parameters + ---------- + x : float or array_like, optional + The time/s by which the network has failed, by default its + whole life. + + Returns + ------- + dict + Per link and failing node, by name, its share: floats for a + scalar ``x`` or the whole life, arrays otherwise; nan where the + network cannot have failed. + + Raises + ------ + ValueError + For a negative time, or if every element is a fixed probability + (the network then has no lifetimes). A fixed element among + others has no density, so no share: it has failed from the + start, or never does. + NotImplementedError + As for ``sf``. + + Examples + -------- + Two links in a chain, with failure rates 1 and 3: the second causes + three quarters of the failures. + + >>> import surpyval as surv + >>> from repyability import Network + >>> E = surv.Exponential.from_params + >>> chain = Network( + ... {"a": (0, 1, E([1.0])), "b": (1, 2, E([3.0]))}, 0, 2 + ... ) + >>> shares = chain.barlow_proschan_importance() + >>> {name: round(share, 6) for name, share in shares.items()} + {'a': 0.25, 'b': 0.75} + """ + self._require_lifetimes("failure density over time to share out") + names = list(self.models) + + def parts(t: np.ndarray) -> np.ndarray: + importance = self._birnbaum(t) + columns = [] + for name in names: + value = importance[name] + with np.errstate(invalid="ignore"): + part = value * _measures.density(self.models[name], t) + # An element that cannot matter, with an infinite density + # (a Weibull life of shape below 1, at 0), takes no part. + columns.append(np.where(value == 0.0, 0.0, part)) + return np.column_stack(columns) + + knots = np.concatenate( + [np.empty(0)] + [model_knots(m) for m in self.models.values()] + ) + knots = np.unique(knots[np.isfinite(knots) & (knots > 0.0)]) + if x is None: + ends = np.array( + [ + _measures.life_end( + lambda t: self._value(np.asarray(t, float)), knots + ) + ] + ) + else: + ends = np.atleast_1d(real_array(x, "x")).ravel() + if np.any(~np.isfinite(ends)) or np.any(ends < 0.0): + raise ValueError( + f"Times must be finite and non-negative, got {x!r}." + ) + shares_by_end = _measures.integrated_shares(parts, knots, ends) + if x is None or np.ndim(x) == 0: + return { + name: float(shares_by_end[0, j]) + for j, name in enumerate(names) + } + return { + name: shares_by_end[:, j].reshape(np.shape(x)) + for j, name in enumerate(names) + } + + def time_to_reliability(self, target, upper_bound: Optional[float] = None): + """The time at which the reliability falls to ``target``: the + inverse of ``sf``, by root-finding, as + [`NonRepairableRBD.time_to_reliability`][repyability.NonRepairableRBD.time_to_reliability] + finds it. + + Parameters + ---------- + target : float or array_like + The reliability to solve for, in (0, 1), or several. + upper_bound : float, optional + A time at which the reliability is below ``target``; found by + doubling from 1 if None. + + Returns + ------- + float or numpy.ndarray + The time at which ``sf(t) == target``: one for each target, if + several are given. + + Raises + ------ + ValueError + If ``target`` is not in (0, 1), or exceeds the reliability at + 0; if every model is a fixed probability (the reliability does + not vary with time); or if no upper bound is found. + NotImplementedError + As for ``sf``. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Network + >>> chain = Network( + ... {"a": (0, 1, surv.Weibull.from_params([100, 2]))}, 0, 1 + ... ) + >>> round(chain.time_to_reliability(0.9), 2) + 32.46 + """ + self._require_time_varying() + return _measures.invert_reliability( + lambda t: float(self.sf(t)), target, upper_bound + ) + + def bx_life(self, x, upper_bound: Optional[float] = None): + """The Bx life: the time by which ``x`` percent of networks have + failed, ``time_to_reliability(1 - x / 100)``. + + Parameters + ---------- + x : float or array_like + The percentage failed (10 for the B10 life, not a fraction), in + (0, 100), or several. + upper_bound : float, optional + As for ``time_to_reliability``. + + Returns + ------- + float or numpy.ndarray + The time by which ``x`` percent have failed: one for each ``x``, + if several are given. + + Raises + ------ + ValueError + If ``x`` is not in (0, 100), or as for ``time_to_reliability``. + NotImplementedError + As for ``sf``. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Network + >>> chain = Network( + ... {"a": (0, 1, surv.Weibull.from_params([100, 2]))}, 0, 1 + ... ) + >>> round(chain.bx_life(10), 2) + 32.46 + """ + x = real_array(x, "x") + if not np.all((0.0 < x) & (x < 100.0)): + raise ValueError("x must be a percentage in (0, 100).") + return self.time_to_reliability(1.0 - x / 100.0, upper_bound) + + # ------------------------------------------------------------------ + # Parameter uncertainty + # ------------------------------------------------------------------ + + def sf_uncertainty( + self, + x=None, + uncertainty: Optional[Dict[Hashable, Any]] = None, + *, + n_draws: int = 1000, + seed=None, + sampling: str = "random", + ) -> UncertaintyResult: + """The reliability over plausible models of the links and nodes + (parameter uncertainty), as + [`NonRepairableRBD.sf_uncertainty`][repyability.NonRepairableRBD.sf_uncertainty] + gives a diagram's. + + Each of the ``n_draws`` draws gives every uncertain element a + plausible model, and the reliability at ``x`` is worked out + exactly for that draw. An element's uncertainty is ``"fit"`` (its + parameters drawn from the normal approximation of its surpyval + fit, through its covariance), ``{parameter name: distribution}``, + or a list of models drawn with replacement; elements given together + in a tuple share their draws, as links of one type fitted to one + set of data should. + + Parameters + ---------- + x : float or array_like, optional + The time/s; may be left out if every model, and every drawn + model, is a fixed probability. + uncertainty : dict, optional + ``{element or tuple of elements: uncertainty}`` for the + uncertain links and nodes; the others keep their models. By + default, every element whose model is a surpyval fit with a + parameter covariance is drawn as ``"fit"``, those sharing one + model object together. + n_draws : int, optional + The number of draws, by default 1000. + seed : int, optional + Seeds the draws, by default None. + sampling : str, optional + ``"random"`` (the default), or ``"sobol"``: the points of a + scrambled Sobol sequence, which cover the parameters more + evenly. + + Returns + ------- + UncertaintyResult + The reliability of every draw (``samples``), the nominal value + with the elements' own models, and summaries (``mean``, + ``interval``, ...; see + [`UncertaintyResult`][repyability.UncertaintyResult]). + + Raises + ------ + ValueError + If ``uncertainty`` is empty, names an unknown element or one + twice, or a tuple's elements have different models; if an + uncertainty cannot be drawn; if ``n_draws`` is not a positive + integer; or if ``x`` is left out for time-varying models. + NotImplementedError + As for ``sf``. + + Examples + -------- + A chain of two lines of one type, whose failure rate is known only + to lie between 0.05 and 0.15 a year, drawn once for both: + + >>> import scipy.stats as st + >>> import surpyval as surv + >>> from repyability import Network + >>> line = surv.Exponential.from_params([0.1]) + >>> chain = Network({"a": (0, 1, line), "b": (1, 2, line)}, 0, 2) + >>> rate = {"failure_rate": st.uniform(0.05, 0.1)} + >>> result = chain.sf_uncertainty( + ... 1.0, {("a", "b"): rate}, n_draws=10_000, seed=1 + ... ) + >>> round(result.nominal, 4) + 0.8187 + >>> lower, upper = result.interval(0.9) + >>> round(lower, 3), round(upper, 3) + (0.748, 0.896) + """ + drawn = self._uncertain_draws(uncertainty, n_draws, seed, sampling) + fixed = self.is_fixed and all( + is_fixed_probability(m) for ms in drawn.values() for m in ms + ) + if x is None: + if not fixed: + raise ValueError( + "x is required: an element's probability depends on " + "time." + ) + x = 1.0 + times = np.atleast_1d(real_array(x, "x")).ravel() + plan = self._decomposition() + p: Dict[Hashable, np.ndarray] = {} + q: Dict[Hashable, np.ndarray] = {} + # Each element's probabilities in every draw, a draw's times + # together: those of an element left as it is, repeated. + for name in plan.names: + models = drawn.get(name, [self.models[name]]) + rows = [ + self._probabilities(times, [name], {name: m}) for m in models + ] + p[name] = np.concatenate([row[0][name] for row in rows]) + q[name] = np.concatenate([row[1][name] for row in rows]) + if name not in drawn: + p[name], q[name] = np.tile(p[name], n_draws), np.tile( + q[name], n_draws + ) + samples = _evaluate(plan, p, q, (n_draws, len(times))) + nominal = self._value(times) + if np.ndim(x) == 0: + return UncertaintyResult( + samples=samples[:, 0], + nominal=float(nominal[0]), + n_draws=n_draws, + ) + return UncertaintyResult( + samples=samples.reshape((n_draws,) + np.shape(x)), + nominal=nominal.reshape(np.shape(x)), + n_draws=n_draws, + ) + + def mean_uncertainty( + self, + uncertainty: Optional[Dict[Hashable, Any]] = None, + *, + n_draws: int = 1000, + seed=None, + sampling: str = "random", + ) -> UncertaintyResult: + """The mean time until the terminals are parted over plausible + models of the links and nodes (parameter uncertainty), as + [`NonRepairableRBD.mean_uncertainty`][repyability.NonRepairableRBD.mean_uncertainty] + gives a diagram's: each draw's mean worked out as ``mean`` works it + out, the area under its exact reliability. + + Parameters + ---------- + uncertainty : dict, optional + As for ``sf_uncertainty``. + n_draws : int, optional + The number of draws, by default 1000. + seed : int, optional + Seeds the draws, by default None. + sampling : str, optional + ``"random"`` (the default) or ``"sobol"``, as for + ``sf_uncertainty``. + + Returns + ------- + UncertaintyResult + The mean lifetime of every draw (``samples``; ``inf`` for one in + which the terminals may never part), the exact mean with the + elements' own models (``nominal``), and summaries. + + Raises + ------ + ValueError + As for ``sf_uncertainty``, or if every element is a fixed + probability (the network then has no lifetimes). + NotImplementedError + As for ``sf``. + + Examples + -------- + >>> import scipy.stats as st + >>> import surpyval as surv + >>> from repyability import Network + >>> line = surv.Exponential.from_params([0.1]) + >>> chain = Network({"a": (0, 1, line), "b": (1, 2, line)}, 0, 2) + >>> rate = {"failure_rate": st.uniform(0.05, 0.1)} + >>> result = chain.mean_uncertainty({("a", "b"): rate}, seed=1) + >>> round(result.nominal, 6) # 1 / 0.2 + 5.0 + >>> lower, upper = result.interval(0.9) + >>> round(lower, 2), round(upper, 2) + (3.43, 9.01) + """ + self._require_lifetimes("mean lifetime") + drawn = self._uncertain_draws(uncertainty, n_draws, seed, sampling) + samples = np.empty(n_draws) + for i in range(n_draws): + samples[i] = self._mean_of( + { + name: drawn[name][i] if name in drawn else model + for name, model in self.models.items() + } + ) + return UncertaintyResult( + samples=samples, + nominal=self._mean_of(self.models), + n_draws=n_draws, + ) + + def _mean_of(self, models: Mapping) -> float: + """The mean time until the terminals are parted with ``models`` + (by name) for the elements: the exact reliability integrated over + time by quadrature (to about 1e-10).""" + return mean_lifetime( + lambda t: self._value(np.asarray(t, float), models=models), + [model_knots(m) for m in models.values()], + [model_kinks(m) for m in models.values()], + ) + + def _label(self, members: tuple) -> str: + """An uncertain element, or elements, named for a message.""" + if len(members) > 1: + return f"Elements {list(members)!r}" + kind = "Link" if members[0] in self.links else "Node" + return f"{kind} {members[0]!r}" + + def _uncertain_draws( + self, uncertainty, n_draws: int, seed, sampling: str + ) -> Dict[Hashable, list]: + """``n_draws`` plausible models for each uncertain element, after + checking ``uncertainty`` and ``n_draws`` (see ``sf_uncertainty``): + the elements given together share their models. Drawn as a + diagram's nodes' are (``uncertainty.draw_with``), so the same + sources and seed give the same models.""" + check_draws(n_draws) + sources = self._uncertainty_sources(uncertainty) + + def draw(n: int, source) -> Dict[Hashable, list]: + drawn: Dict[Hashable, list] = {} + for members, spec in sources: + models = draw_models( + self.models[members[0]], + spec, + n, + source, + self._label(members), + ) + for name in members: + drawn[name] = models + return drawn + + return draw_with(draw, n_draws, seed, sampling) + + def _uncertainty_sources(self, uncertainty) -> List[Tuple[tuple, Any]]: + """The uncertain inputs, checked (see ``sf_uncertainty``): for each + element or tuple of elements given, its elements and its + uncertainty. By default, every fitted element + (``uncertainty.fitted_together``). Given, an element drawn without + other elements holding the same model object, which are left as + they are, is warned about.""" + given = uncertainty is not None + if uncertainty is None: + uncertainty = fitted_together(self.models) + if not uncertainty: + raise ValueError( + "No link's or node's model is a surpyval fit with a " + "parameter covariance, which the uncertainty is drawn " + "from by default: give the uncertain elements, " + "uncertainty={link: 'fit'} or {link: {parameter: " + "distribution}}, for example." + ) + if not uncertainty: + raise ValueError( + "Give the uncertain elements: uncertainty={link: 'fit'}, for " + "example." + ) + sources: List[Tuple[tuple, Any]] = [] + seen: set = set() + for key, spec in uncertainty.items(): + members = ( + (key,) + if key in self.models or not isinstance(key, tuple) + else key + ) + for name in members: + if name not in self.models: + raise ValueError( + f"{name!r} in uncertainty is not a link or a failing " + "node of the network." + ) + if name in seen: + raise ValueError( + f"{self._label((name,))} is given an uncertainty " + "twice." + ) + seen.add(name) + model = self.models[members[0]] + for name in members[1:]: + if not NonRepairableRBD._same_model(self.models[name], model): + raise ValueError( + f"Elements {list(members)!r} share their " + "uncertainty, so they must have the same model." + ) + sources.append((tuple(members), spec)) + if given: + for members, _ in sources: + model = self.models[members[0]] + others = [ + name + for name in self.models + if name not in seen and self.models[name] is model + ] + if others: + half_named( + self._label(members), + others, + "model", + {tuple(members) + tuple(others): "fit"}, + ) + return sources + def mean( self, method: str = "exact", @@ -534,38 +1713,24 @@ def mean( Returns ------- float - The mean lifetime: ``inf`` if some networks never part. + The mean lifetime: ``inf`` if some networks never part (a + fixed probability that keeps the terminals joined, say, as a + diagram's ``mean`` gives it). Raises ------ ValueError - If a model is a fixed probability (it has no lifetime), or - ``method`` or ``mc_samples`` is invalid. + If every model is a fixed probability (the network then has + no lifetimes), ``method`` is unknown, or ``mc_samples`` or + ``seed`` is given with ``method="exact"``. """ - if self.is_fixed or any( - is_fixed_probability(m) for m in self.models.values() - ): - raise ValueError( - "A fixed probability has no lifetime, so the network has no " - "mean lifetime." - ) - if self._method(method, mc_samples) == "simulate": + self._require_lifetimes("mean lifetime") + if self._method(method, mc_samples, seed, "mean") == "simulate": lives = self.random( 10_000 if mc_samples is None else mc_samples, seed ) return float(np.mean(lives)) - from repyability.rbd._mean_lifetime import ( - mean_lifetime, - model_kinks, - model_knots, - ) - - models = self.models.values() - return mean_lifetime( - lambda t: self._value(np.asarray(t, float)), - [model_knots(m) for m in models], - [model_kinks(m) for m in models], - ) + return self._mean_of(self.models) # ------------------------------------------------------------------ # Simulation @@ -654,6 +1819,356 @@ def find(a: int) -> int: break return np.maximum(out, 0.0) + # ------------------------------------------------------------------ + # Saving + # ------------------------------------------------------------------ + + def to_dict(self) -> dict: + """The network as a JSON-friendly dict (see ``from_dict``), as a + diagram saves its nodes (see + [`RBD.to_dict`][repyability.RBD.to_dict]). + + Returns + ------- + dict + The ``repyability_version`` that wrote it, the ``type`` + (``"Network"``), the terminals, and the links (each its name, + the two nodes it joins and its model, serialised as a diagram's + node models are: surpyval's in surpyval's own format, so a + fit's covariance round-trips) and the failing nodes. + + Raises + ------ + NotImplementedError + If a model cannot be serialised: one that is not surpyval's + (nor a RePyability node model). + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Network + >>> chain = Network( + ... {"a": (0, 1, surv.Weibull.from_params([100, 2]))}, 0, 1 + ... ) + >>> d = chain.to_dict() + >>> d["type"], d["links"][0]["nodes"] + ('Network', [0, 1]) + """ + from repyability.rbd.serialisation import serialise_model + + return { + "repyability_version": __version__, + "type": "Network", + "source": self.source, + "target": self.target, + "links": [ + { + "link": name, + "nodes": [u, v], + "model": serialise_model(self.models[name]), + } + for name, (u, v) in self.links.items() + ], + "nodes": [ + {"node": name, "model": serialise_model(model)} + for name, model in self.nodes.items() + ], + } + + @classmethod + def from_dict(cls, d: dict) -> "Network": + """Rebuild a network from ``to_dict``'s output, by calling the + constructor again, so it is checked again. Tuple names (a grid's + ``(i, j)``), which JSON turns into lists, are turned back. + + Parameters + ---------- + d : dict + As ``to_dict`` returns (possibly after a JSON round trip). + + Returns + ------- + Network + The network. + + Raises + ------ + ValueError + If ``d`` is not a network's dict, a model kind is unknown, or + the constructor rejects the stored inputs. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Network + >>> chain = Network( + ... {"a": (0, 1, surv.Weibull.from_params([100, 2]))}, 0, 1 + ... ) + >>> Network.from_dict(chain.to_dict()).sf(50) == chain.sf(50) + True + """ + from repyability.rbd.serialisation import ( + _node_name, + deserialise_model, + ) + + if d.get("type") != "Network": + raise ValueError(f"Not a network: type {d.get('type')!r}.") + links = { + _node_name(e["link"]): ( + _node_name(e["nodes"][0]), + _node_name(e["nodes"][1]), + deserialise_model(e["model"]), + ) + for e in d["links"] + } + nodes = { + _node_name(e["node"]): deserialise_model(e["model"]) + for e in d.get("nodes") or () + } + return cls( + links, + _node_name(d["source"]), + _node_name(d["target"]), + nodes=nodes or None, + ) + + def to_json(self, fp=None, **json_kwargs) -> Optional[str]: + """The network as a JSON document (see ``to_dict``): returned, or + written to ``fp`` (as surpyval's models' ``to_json(fp)`` writes + them). + + Parameters + ---------- + fp : str, os.PathLike or file, optional + A path, or a file opened for writing, to write the document to; + by default None: it is returned. + **json_kwargs + Passed to ``json.dumps``, e.g. ``indent=2``. + + Returns + ------- + str or None + The JSON document, or None once written to ``fp``. + + Raises + ------ + NotImplementedError + If a model cannot be serialised (see ``to_dict``). + TypeError + If a name cannot be encoded as JSON. + """ + return write_json(json.dumps(self.to_dict(), **json_kwargs), fp) + + @classmethod + def from_json(cls, s) -> "Network": + """Rebuild a network from ``to_json``'s output. + + Parameters + ---------- + s : str, os.PathLike or file + The JSON document: its text, a path to a file holding it, or a + file opened for reading. + + Returns + ------- + Network + The network. + + Raises + ------ + ValueError + If ``s`` is not valid JSON, or for the reasons given in + ``from_dict``. + FileNotFoundError + If ``s`` is neither a JSON document nor the path of a file. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Network + >>> chain = Network( + ... {"a": (0, 1, surv.Weibull.from_params([100, 2]))}, 0, 1 + ... ) + >>> round(Network.from_json(chain.to_json()).sf(50), 4) + 0.7788 + """ + return cls.from_dict(json.loads(read_json(s))) + + # ------------------------------------------------------------------ + # The routes + # ------------------------------------------------------------------ + + def analysis_routes(self) -> Dict[str, AnalysisRoute]: + """How each analysis of this network is computed, found without + running it: exactly, numerically, by simulation, or not at all (as + [`NonRepairableRBD.analysis_routes`][repyability.NonRepairableRBD.analysis_routes] + reports a diagram's). + + The exact values come from the network's decision diagram, so a + network of more than ``MAX_STATES`` states refuses every analysis + but the simulation (and the paths listed), with the message the + methods raise; an element whose model has no reliability of its + own makes the analyses built on it refuse, as it does. + + Returns + ------- + dict[str, AnalysisRoute] + For each public analysis, by method name: its ``route`` + (``"exact"``, ``"numerical"``, ``"simulated"`` or + ``"refused"``), the ``reason`` and the elements that decide it + (see [`AnalysisRoute`][repyability.AnalysisRoute]). + + Examples + -------- + >>> from repyability import Network + >>> bridge = Network( + ... {"a": ("s", "x", 0.1), "b": ("s", "y", 0.1), + ... "c": ("x", "y", 0.1), "d": ("x", "t", 0.1), + ... "e": ("y", "t", 0.1)}, + ... "s", "t", + ... ) + >>> routes = bridge.analysis_routes() + >>> routes["fussell_vesely"].route, routes["random"].route + ('exact', 'simulated') + >>> routes["mean"].route # a fixed probability has no lifetime + 'refused' + """ + from repyability.rbd import routes as r + + elements = {name: r.model_route(m) for name, m in self.models.items()} + out: Dict[str, AnalysisRoute] = {} + + def give(names, route) -> None: + for name in names: + out[name] = route + + def built(route, reason): + return r.with_nodes(route, reason, elements, "reliabilities") + + give( + ("sf", "ff"), + built( + r.EXACT, + "The decision diagram over the elements' reliabilities, " + "exactly.", + ), + ) + give( + ( + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + ), + built( + r.EXACT, + "The exact reliability, and with each element held working " + "and failed, from the decision diagram.", + ), + ) + out["fussell_vesely"] = built( + r.EXACT, + "The exact probability that a minimal cut set containing each " + "element has failed, over the unreliability (method=" + "'rare_event' sums the cut sets' probabilities instead).", + ) + out["joint_importance"] = built( + r.EXACT, + "Each element's exact Birnbaum importance with the other held " + "working, less with it held failed.", + ) + out["differential_importance"] = built( + r.EXACT, + "The exact Birnbaum importance of each element (or, for a " + "proportional change, its criticality importance), shared out.", + ) + out["structural_importance"] = AnalysisRoute( + r.EXACT, "From the structure alone." + ) + give( + ("cut_sets", "minimal_cut_sets"), + AnalysisRoute(r.EXACT, "Read off the decision diagram."), + ) + paths = r.refusal(self._simple_paths) + give( + ("path_sets", "minimal_path_sets"), + ( + r.refused(paths) + if paths + else AnalysisRoute( + r.EXACT, + "Every simple path between the terminals, listed.", + ) + ), + ) + fixed = r.refusal(self._require_time_varying) + give( + ("time_to_reliability", "bx_life"), + ( + r.refused(fixed) + if fixed + else built( + r.NUMERICAL, + "The exact reliability, inverted by root-finding.", + ) + ), + ) + lifetimes = r.refusal(lambda: self._require_lifetimes("mean lifetime")) + out["mean"] = ( + r.refused(lifetimes) + if lifetimes + else built( + r.NUMERICAL, + "The exact reliability, integrated over time by adaptive " + "Gauss-Kronrod quadrature (to about 1e-10, relative).", + ) + ) + densities = r.refusal( + lambda: self._require_lifetimes( + "failure density over time to share out" + ) + ) + out["barlow_proschan_importance"] = ( + r.refused(densities) + if densities + else built( + r.NUMERICAL, + "Each element's Birnbaum importance times its failure " + "density, integrated over time by adaptive Gauss-Legendre " + "quadrature (to about 1e-9).", + ) + ) + out["random"] = AnalysisRoute( + r.SIMULATED, + "Monte-Carlo lifetimes: each element's drawn, and the " + "longest-lasting path's taken.", + ) + out["sf_uncertainty"] = built( + r.SIMULATED, + "The elements' parameters drawn from their uncertainty, and the " + "exact reliability for each draw.", + ) + out["mean_uncertainty"] = ( + r.refused(lifetimes) + if lifetimes + else built( + r.SIMULATED, + "The elements' parameters drawn from their uncertainty, and " + "the exact mean lifetime for each draw (the area under its " + "reliability).", + ) + ) + # Every exact value needs the decision diagram: without it, only + # the simulation (and the paths, listed on their own) is left. + too_large = r.refusal(self._decomposition) + if too_large: + for name in out: + if name not in ("random", "path_sets", "minimal_path_sets"): + out[name] = r.refused(too_large) + return r.AnalysisRoutes(sorted(out.items())) + def __repr__(self) -> str: return ( f"Network({len(self.links)} links, {len(self.nodes)} failing " @@ -749,6 +2264,23 @@ def steps(self) -> tuple: return steps, self.root +def _dual(steps: list, root: int) -> Tuple[list, int]: + """The plan (``shannon.py``'s format) of the dual of the structure + ``steps`` and ``root`` decide: the dual works unless every element of + some minimal path set of the structure has failed, so its minimal cut + sets are the structure's minimal path sets. Each step's branches are + swapped (the elements' states complemented) and the terminals too (the + value negated).""" + + def swapped(slot: int) -> int: + return 1 - slot if slot < 2 else slot + + return [ + (pivot, swapped(inactive), swapped(active)) + for pivot, active, inactive in steps + ], swapped(root) + + def _constant(slot: int) -> _Plan: """The plan of a constant: the terminals always parted, or joined.""" empty = np.zeros(0, dtype=np.int64) @@ -798,12 +2330,19 @@ def _plan_from_steps(steps: list, root: int) -> _Plan: _EVALUATION_SIZE = 16_000_000 -def _rows(plan: _Plan, values: dict, size: int) -> np.ndarray: - """Each of the plan's variables' ``values`` (of ``size`` elements once - broadcast), as the rows of one array.""" +def _rows(plan: _Plan, values: dict, shape: tuple) -> np.ndarray: + """Each of the plan's variables' ``values`` (of ``shape``, or as many + in a row, or one for all), as the rows of one array, a column per + evaluation.""" + size = int(np.prod(shape, dtype=np.int64)) out = np.empty((len(plan.names), size)) for i, name in enumerate(plan.names): - out[i] = np.broadcast_to(np.asarray(values[name], float), (size,)) + value = np.asarray(values[name], float) + out[i] = ( + value.reshape(-1) + if value.size == size + else np.broadcast_to(value, shape).ravel() + ) return out @@ -841,7 +2380,7 @@ def _evaluate( size = int(np.prod(shape, dtype=np.int64)) if plan.root < 2: return np.full(shape, terminals[plan.root], dtype=float) - p_rows, q_rows = _rows(plan, p, size), _rows(plan, q, size) + p_rows, q_rows = _rows(plan, p, shape), _rows(plan, q, shape) out = np.empty(size) for start, end in _chunks(plan, size): values = _forward( @@ -860,7 +2399,7 @@ def _value_and_gradient( size = int(np.prod(shape, dtype=np.int64)) if plan.root < 2: return np.full(shape, terminals[plan.root], dtype=float), {} - p_rows, q_rows = _rows(plan, p, size), _rows(plan, q, size) + p_rows, q_rows = _rows(plan, p, shape), _rows(plan, q, shape) value = np.empty(size) gradient = np.zeros((len(plan.names), size)) for start, end in _chunks(plan, size): diff --git a/repyability/non_repairable.py b/repyability/non_repairable.py index 5992d388..0c187676 100644 --- a/repyability/non_repairable.py +++ b/repyability/non_repairable.py @@ -2,7 +2,13 @@ from scipy.optimize import minimize, minimize_scalar from surpyval import ExactEventTime, NonParametric, Parametric -from repyability.maintenance import MaintenancePolicy +from repyability.maintenance import ( + CalendarPolicy, + MaintenancePolicy, + NextReplacement, + SkipValue, +) +from repyability.rbd._degradation import check_life, is_degradation from repyability.rbd._model_utils import ( distribution_name, is_exponential, @@ -11,8 +17,9 @@ never_fails, ) from repyability.rbd.helper_classes import PerfectReliability, perfect_class +from repyability.rbd.results import FailureForecast from repyability.rbd.standby_node import StandbyModel -from repyability.utils.checks import no_distribution +from repyability.utils.checks import is_number, no_distribution, real_array FAILURE = 1 REPLACE = 0 @@ -113,6 +120,14 @@ class NonRepairable: failure, used by the availability and event methods. By default None, or ``"instant"`` as a ``RepairableRBD`` spec takes it: instantaneous replacement (``ExactEventTime.from_params(0)``). + planned_cost : float, optional + What a planned (preventive) replacement costs, at least 0, by name; + as ``set_costs_planned_and_unplanned`` sets it (``cp``). By default + None: not given. + unplanned_cost : float, optional + What a replacement at a failure costs, at least 0, by name (``cu``): + with ``planned_cost``, more than it. Either may be given alone: + ``failures`` needs only this one. By default None: not given. Attributes ---------- @@ -167,7 +182,14 @@ class NonRepairable: 0.9804 """ - def __init__(self, reliability, time_to_replace=None): + def __init__( + self, + reliability, + time_to_replace=None, + *, + planned_cost=None, + unplanned_cost=None, + ): reliability = perfect_class(reliability) no_distribution(reliability, "A NonRepairable's life") no_distribution(time_to_replace, "A NonRepairable's time_to_replace") @@ -201,6 +223,12 @@ def __init__(self, reliability, time_to_replace=None): self.reliability_function, self._knots = _piecewise_linear_sf( reliability ) + elif is_degradation(reliability): + # A fitted Wiener or gamma degradation process (#271): its life + # is the time its level first reaches its threshold. + check_life(reliability, "A NonRepairable's life") + self.model_parameterization = "degradation" + self.reliability_function = reliability.sf elif isinstance(reliability, StandbyModel): # Whatever the arrangement's survival function is (a closed # form or a numerical method), its sf gives it; one with neither @@ -218,7 +246,8 @@ def __init__(self, reliability, time_to_replace=None): raise ValueError( "A NonRepairable's life must be a surpyval parametric " "distribution or MixtureModel (fitted, or built with " - "from_params), a surpyval non-parametric estimate or a " + "from_params), a surpyval Wiener or gamma degradation " + "process, a surpyval non-parametric estimate or a " f"StandbyModel; got a {type(reliability).__name__}." ) @@ -234,6 +263,126 @@ def __init__(self, reliability, time_to_replace=None): ), ) self.__next_event_type = FAILURE + self._set_costs(planned_cost, unplanned_cost) + + def _set_costs(self, planned, unplanned) -> None: + """Keep the costs given to the constructor (``cp``, ``cu``): each + a finite number at least 0, and with both, the planned less.""" + for name, cost in ( + ("planned_cost", planned), + ("unplanned_cost", unplanned), + ): + if cost is not None and not ( + is_number(cost) and 0.0 <= float(cost) < np.inf + ): + raise ValueError( + f"{name} must be a number, at least 0; got {cost!r}." + ) + if planned is not None and unplanned is not None: + self.set_costs_planned_and_unplanned( + float(planned), float(unplanned) + ) + elif planned is not None: + self.cp = float(planned) + elif unplanned is not None: + self.cu = float(unplanned) + + def __repr__(self) -> str: + """A short summary: the life model's distribution, how a + replacement takes its time, and the costs given.""" + parts = [ + distribution_name(self.reliability) + or type(self.reliability).__name__ + ] + if self.time_to_replace is not None: + replace = distribution_name(self.time_to_replace) + parts.append( + f"replaced in {replace or type(self.time_to_replace).__name__}" + ) + if hasattr(self, "cp"): + parts.append(f"planned_cost={self.cp:g}") + if hasattr(self, "cu"): + parts.append(f"unplanned_cost={self.cu:g}") + return f"NonRepairable({', '.join(parts)})" + + def failures(self, t, age: float = 0.0) -> FailureForecast: + """The failures over a window ``t`` from now, with the current unit + ``age`` old and working, each failure replaced by a new unit. + + The chance the current unit fails in the window is its life + conditioned on the age reached, ``1 - R(age + t) / R(age)``. The + expected number of failures counts its replacements' too: the + renewal count of the part as a + [`RepairableRBD`][repyability.RepairableRBD]'s component started + from that age (its ``expected_events``, exact, with no + simulation), each replacement taking its ``time_to_replace``. + + Parameters + ---------- + t : float or array_like + The window's length (or lengths), at least 0, in the models' + time unit. + age : float, optional + The current unit's age, at least 0, by default 0 (new). + + Returns + ------- + FailureForecast + The chance the current unit fails, the expected failures, and + their expected cost (with an ``unplanned_cost``). + + Examples + -------- + A bearing with a Weibull life (scale 365 days, shape 2), 224 days + old, over the next 82 days, at 400 a failure: + + >>> import surpyval as surv + >>> from repyability import NonRepairable + >>> bearing = NonRepairable( + ... surv.Weibull.from_params([365.0, 2.0]), unplanned_cost=400.0 + ... ) + >>> forecast = bearing.failures(82.0, age=224.0) + >>> round(forecast.probability, 4) + 0.2784 + >>> round(forecast.expected_failures, 4) + 0.283 + >>> round(forecast.expected_cost, 2) + 113.2 + """ + from repyability.rbd.node_state import NodeState + from repyability.rbd.repairable_rbd import RepairableRBD + + window = real_array(t, "t") + if not np.all(np.isfinite(window) & (window >= 0.0)): + raise ValueError(f"t must be at least 0, and finite; got {t!r}.") + if not (is_number(age) and 0.0 <= float(age) < np.inf): + raise ValueError(f"age must be a number, at least 0; got {age!r}.") + age = float(age) + surviving = float(np.ravel(self.reliability_function(age))[0]) + if not surviving > 0.0: + raise ValueError( + f"The life gives no chance of a unit working at age {age:g}." + ) + reached = np.asarray(self.reliability_function(age + window), float) + probability = 1.0 - reached.reshape(window.shape) / surviving + unit = RepairableRBD([("s", "unit"), ("unit", "t")], {"unit": self}) + events = unit.expected_events( + window if window.ndim else float(window), + state={"unit": NodeState(age=age)}, + ) + failures = np.asarray(events.node_failures["unit"], float) + cost = None if not hasattr(self, "cu") else failures * self.cu + + def shaped(values): + return float(values) if np.ndim(values) == 0 else values + + return FailureForecast( + shaped(window), + age, + shaped(probability), + shaped(failures), + None if cost is None else shaped(cost), + ) def set_costs_planned_and_unplanned(self, cp, cu): """Set the planned and unplanned replacement costs. @@ -267,10 +416,10 @@ def set_costs_planned_and_unplanned(self, cp, cu): self.cu = cu def _require_costs(self) -> None: - if not hasattr(self, "cp"): + if not (hasattr(self, "cp") and hasattr(self, "cu")): raise ValueError( - "costs not set: call set_costs_planned_and_unplanned" - "(cp, cu) first" + "costs not set: give planned_cost and unplanned_cost, or " + "call set_costs_planned_and_unplanned(cp, cu) first" ) def avg_replacement_time(self, t): @@ -748,6 +897,403 @@ def optimal_replacement_policy(self) -> MaintenancePolicy: rate = float(self._cost_rate(interval)) return MaintenancePolicy(interval=float(interval), cost_rate=rate) + def optimal_next_replacement( + self, age, *, now_cost=None, scheduled_outages=(), policy=None + ) -> NextReplacement: + """When to replace the part in service next: now, at one of the + scheduled outages, at a planned replacement between them, or at + its failure. + + [`optimal_replacement_policy`][repyability.NonRepairable.optimal_replacement_policy] + is the rule for a part in general; this is the next replacement of + the one in service, from its age and the outages scheduled (this + package's use case UC-12). Each choice's cost is its expected cost + until the part is replaced (at the failure price if it fails + first), less the time the part serves until then, valued at the + baseline: the long-run cost a unit of time of how the position is + run, ``policy.cost_rate``. By default that is the age + replacement's (``optimal_replacement_policy().cost_rate``); where + cheap shuts keep coming, give the calendar's policy + ([`optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy]), + whose rate is lower, so a part is kept longer. The cheapest + choice is the best. With no outages and the default baseline, + replacing now is the best from the optimal replacement age. + + A planned replacement is one later than now, at its best time. + Where no later time beats replacing at once (a part past its + optimal age), replacing at once is replacing now, which costs + ``now_cost``: the planned choice is then replacing now (time 0, + at ``now_cost``), ``now`` is True exactly when the best choice + is at time 0, and ``replace_from_age`` follows. A planned + replacement with a lead time is a scheduled outage at that time. + + Parameters + ---------- + age : float + The part's age now, at least 0, in the life's time unit. + now_cost : float, optional + What replacing it now costs, by default the planned cost. + scheduled_outages : list of (float, float), optional + The scheduled outages (shuts, turnarounds) as (time from now, + price) pairs: when each is, in the life's time unit, more than + 0, and what replacing the part at it costs, at least 0. + policy : CalendarPolicy or MaintenancePolicy, optional + The policy whose ``cost_rate`` is the baseline, by default + ``optimal_replacement_policy()``. + + Returns + ------- + NextReplacement + Every choice with its cost and break-even age (the youngest at + which replacing now costs no more), the best, and the age from + which replacing now is the best. + + Raises + ------ + ValueError + If the costs have not been set, or an argument is out of its + range. + TypeError + If ``policy`` is not a policy (its cost rate alone is not one). + NotImplementedError + For a life with no survival function in closed form (a + non-parametric estimate or a ``StandbyModel``). + + Examples + -------- + A bearing with a Weibull life (scale 365 days, shape 2), 200 days + old, $150 for a planned replacement, $400 at a failure, and + outages scheduled in 150 and 330 days, at which replacing it costs + $100: + + >>> import surpyval as surv + >>> from repyability import NonRepairable + >>> bearing = NonRepairable( + ... surv.Weibull.from_params([365.0, 2.0]), + ... planned_cost=150.0, + ... unplanned_cost=400.0, + ... ) + >>> found = bearing.optimal_next_replacement( + ... 200.0, scheduled_outages=[(150.0, 100.0), (330.0, 100.0)] + ... ) + >>> found.now, found.best.kind, found.best.time + (False, 'outage', 150.0) + >>> round(found.replace_from_age, 1) + 297.6 + + With shuts every 180 days at $100 kept coming, the calendar's + policy values a day of service at 0.990 rather than 1.117, and + replacing now beats the outage in 150 days from a younger age: + + >>> from repyability import Calendar, Opportunity + >>> shuts = bearing.optimal_calendar_policy( + ... Calendar([Opportunity("shut", 100.0, period=180.0)]) + ... ) + >>> found = bearing.optimal_next_replacement( + ... 200.0, + ... scheduled_outages=[(150.0, 100.0), (330.0, 100.0)], + ... policy=shuts, + ... ) + >>> outage = found.choices[1] + >>> round(found.cost_rate, 3), round(outage.break_even_age, 1) + (0.99, 256.1) + """ + from repyability.maintenance._next import optimal_next_replacement + + return optimal_next_replacement( + self, + age, + now_cost=now_cost, + scheduled_outages=scheduled_outages, + policy=policy, + ) + + def skip_next_replacement( + self, every, *, resets, time_to_next=None, discount_rate=0.0 + ) -> SkipValue: + """Whether to skip the next planned replacement, right after a + failure (a new unit just in): the present-value cost of doing it + against that of skipping it, and the NPV of skipping. + + The part is replaced when it fails (at its ``unplanned_cost``) and + on a plan (at its ``planned_cost``), in no time either way (this + package's use case UC-6). The plan either resets at each + replacement, planned or not (age replacement: the next planned + replacement is ``every`` after the failure), or keeps to fixed + times (block replacement: the next is ``time_to_next`` after the + failure, and the following one ``every`` later). Both costs are + exact, with no simulation; what each covers is in + [`SkipValue`][repyability.SkipValue]. + [`optimal_next_replacement`][repyability.NonRepairable.optimal_next_replacement] + is the undiscounted, long-run question of a part in service with + no plan in force. + + Parameters + ---------- + every : float + The plan's interval, more than 0, in the life's time unit. + resets : bool + Whether the plan resets at each replacement: True, the next + planned replacement is ``every`` after the last replacement, + planned or not; False, planned replacements fall on fixed + times, ``every`` apart, whatever happens. + time_to_next : float, optional + For a plan that does not reset: the time from the failure to + the next planned replacement, more than 0 and at most + ``every``. A plan that resets has its next planned replacement + ``every`` after the failure, and takes none. + discount_rate : float, optional + The continuous discount rate per unit time of the life, by + default 0 (undiscounted): for an annual rate of 8% with a life + in days, ``math.log(1.08) / 365.25``. + + Returns + ------- + SkipValue + The present-value cost of doing the replacement (``replace``) + and of skipping it (``skip``), their difference (``npv``: + positive, skipping pays), the ``best`` of the two, and what + they were worked out for. On a plan that resets each is the + whole future's, endless (``inf``) with no discounting, where + only ``npv`` is finite. A discount rate that discounts the + costs away (a rate a year given with a life in days) is + warned of. + + Raises + ------ + ValueError + If the costs have not been set, or an argument is out of its + range, or ``time_to_next`` is given with a plan that resets or + missing from one that does not. + NotImplementedError + For a life with no survival function in closed form (a + non-parametric estimate or a ``StandbyModel``). + + Examples + -------- + A bearing with a Weibull life (scale 365 days, shape 2), replaced + every 180 days for $150, and at a failure for $400; right after a + failure, at 8% a year, skipping the next planned replacement + saves: + + >>> import math + >>> import surpyval as surv + >>> from repyability import NonRepairable + >>> bearing = NonRepairable( + ... surv.Weibull.from_params([365.0, 2.0]), + ... planned_cost=150.0, + ... unplanned_cost=400.0, + ... ) + >>> a_day = math.log(1.08) / 365.25 + >>> value = bearing.skip_next_replacement( + ... 180.0, resets=True, discount_rate=a_day + ... ) + >>> round(value.npv, 2), value.best + (26.56, 'skip') + + On fixed times, it depends on how soon the next planned + replacement is: + + >>> value = bearing.skip_next_replacement( + ... 180.0, resets=False, time_to_next=30.0, discount_rate=a_day + ... ) + >>> round(value.npv, 2) + 123.08 + """ + from repyability.maintenance._skip import skip_next_replacement + + return skip_next_replacement( + self, + every, + resets=resets, + time_to_next=time_to_next, + discount_rate=discount_rate, + ) + + def optimal_calendar_policy( + self, + calendar, + *, + max_age=None, + method="auto", + tail="repeat", + mc_samples=64, + seed=None, + ) -> CalendarPolicy: + """The long-run optimal replacement policy over a calendar of + opportunities: shuts at their own prices, a planned replacement on + any day at a dearer one (or none), and a failure at the dearest. + + Age replacement + ([`optimal_replacement_policy`][repyability.NonRepairable.optimal_replacement_policy]) + has one planned price at every time, so the state is the age + alone. Here the price depends on the time, so the best policy is a + threshold age for each time, ``a*(t)``: replace if the part is at + least that old. It minimises the long-run average cost a unit of + time (opportunity-based age replacement, Dekker et al.). The + methods (this package's use case UC-13): + + - ``"dp"``: dynamic programming on a grid, its step sized from + the life (about the mean life over 1,000) and put on the + opportunities' own times. Over a periodic calendar whose cycle + has at most 20,000 steps, by relative value iteration: the cost + rate exact for the grid, which is within about the square of + the step of deciding at any moment. Over a longer one, by a + long finite horizon, the cost rate differenced. + - ``"integration"``: between opportunities in continuous time, on + quadrature nodes; only with no replacement on any day. + - ``"sampling"``: for a cycle too long to sweep (co-prime + periods), the cost rate is the mean over ``mc_samples`` sampled + times of the cycle of a window's differenced rate, with its + standard error (``cost_rate_se``); each window is an exact + solve that may take a second or more, so ``mc_samples`` sets + the cost. The rates by kind (``replacement_rates``, + ``cost_rates``) and the schedule's chances are then estimates + from a new unit run in, with no standard error, and move with + the seed; the threshold at a time is a solve over a window + from it (deterministic). + - ``"auto"``: ``"dp"``, or ``"sampling"`` for a periodic calendar + whose cycle is too long to sweep, with a warning that says so. + + The grid's step is not an argument: it is a numerical setting, + sized from the models (never in absolute time), not a choice of + when decisions are made; a calendar says that, by its + opportunities. It is a whole fraction of the periods, so a + periodic calendar keeps its period on the grid whatever its + offsets; an offset or date off the grid (a few hours on a period + of days, as dates turned into model time give) is laid at the + grid time nearest it, with a warning. + + A calendar with given dates (a plan) is extended past its last + date by ``tail``: ``"repeat"`` repeats the plan (from 0 to its + last date and its mean gap), ``"anytime"`` leaves only replacement + on any day. Both are solved, and the policy says the first time + they differ (``determined_until``); its cost rate is the repeated + plan's. + + Parameters + ---------- + calendar : Calendar + The opportunities and prices; its ``"part"`` prices are this + part's planned and unplanned costs. + max_age : float, optional + The oldest age on the grid, by default the age a unit outlives + with a chance of 1e-9. + method : str, optional + ``"auto"`` (the default), ``"dp"``, ``"integration"`` or + ``"sampling"``. + tail : str, optional + How a plan of given dates goes on after its last date: + ``"repeat"`` (the default) or ``"anytime"``. + mc_samples : int, optional + The times sampled by ``"sampling"``, by default 64; the other + methods take none. + seed : int, optional + Seeds the sampled times; the other methods draw nothing. + + Returns + ------- + CalendarPolicy + The long-run cost rate and what it is made of by kind, each + shut's threshold and chance of a replacement (``schedule()``), + a unit's next replacement (``next_replacement()``), how it + was worked out, and the life it was worked out for. + + Raises + ------ + ValueError + If a price the calendar takes from the part is not set, the + calendar allows no planned replacement at all (no + opportunities, and none on any day), or an argument is out + of its range. + NotImplementedError + For ``method="integration"`` with replacement on any day, or + ``method="sampling"`` with given dates. + + Examples + -------- + With no opportunities and a planned replacement on any day at the + planned cost, it is age replacement: + + >>> import surpyval as surv + >>> from repyability import Calendar, NonRepairable + >>> unit = NonRepairable( + ... surv.Weibull.from_params([1000, 2.5]), + ... planned_cost=1.0, + ... unplanned_cost=5.0, + ... ) + >>> policy = unit.optimal_calendar_policy(Calendar()) + >>> round(policy.cost_rate, 6) + 0.003462 + >>> round(policy.next_replacement(age=0.0).time, 1) + 493.3 + >>> round(unit.optimal_replacement_policy().interval, 1) + 493.0 + """ + from repyability.maintenance._replacement import ( + optimal_calendar_policy, + ) + + return optimal_calendar_policy( + self, + calendar, + max_age=max_age, + method=method, + tail=tail, + mc_samples=mc_samples, + seed=seed, + ) + + def calendar_policy_cost(self, calendar, rule, *, max_age=None) -> float: + """The long-run average cost a unit of time of a given replacement + rule over a calendar of opportunities: to price simple policies + against + [`optimal_calendar_policy`][repyability.NonRepairable.optimal_calendar_policy]. + + The rule gives, for each kind of replacement it uses (an + opportunity's name, or ``"anytime"`` for any day), the age from + which it replaces the part; the kinds it leaves out never replace. + Where kinds coincide, the one with the youngest age replaces (the + cheapest, at a tie). Worked out as the optimal policy's cost rate + is, on the same grid, with the decisions fixed (a plan of given + dates repeated past its last date). + + Parameters + ---------- + calendar : Calendar + The opportunities and prices. + rule : dict + The age, at least 0 (``inf`` for never), from which each kind + replaces: ``{"annual": 0.0}`` replaces at every annual shut, + ``{"anytime": 493.0}`` at age 493 on any day, ``{}`` runs to + failure. + max_age : float, optional + The oldest age on the grid, as for ``optimal_calendar_policy``. + + Returns + ------- + float + The rule's long-run cost a unit of time. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import Calendar, NonRepairable + >>> unit = NonRepairable( + ... surv.Weibull.from_params([1000, 2.5]), + ... planned_cost=1.0, + ... unplanned_cost=5.0, + ... ) + >>> rate = unit.calendar_policy_cost(Calendar(), {"anytime": 493.0}) + >>> round(rate, 6) + 0.003462 + """ + from repyability.maintenance._replacement import ( + calendar_policy_cost, + ) + + return calendar_policy_cost(self, calendar, rule, max_age=max_age) + def reset(self): """Restart the event sequence with a working, as-new unit. diff --git a/repyability/rbd/_block_replacement.py b/repyability/rbd/_block_replacement.py index 27a4bc20..74527430 100644 --- a/repyability/rbd/_block_replacement.py +++ b/repyability/rbd/_block_replacement.py @@ -32,7 +32,7 @@ import numpy as np -from ._model_utils import distribution_name, never_fails +from ._model_utils import MODEL_ERRORS, distribution_name, never_fails _GAUSS = np.polynomial.legendre.leggauss(8) @@ -125,7 +125,7 @@ def tail(self) -> float: return self.fixed try: return float(np.ravel(self.model.qf(np.array([1 - 1e-12])))[0]) - except Exception: # pragma: no cover - surpyval models have qf + except MODEL_ERRORS: # pragma: no cover - surpyval models have qf return 50.0 * float(np.ravel(self.model.mean())[0]) def cdf(self, x: np.ndarray) -> np.ndarray: diff --git a/repyability/rbd/_ccf_chain.py b/repyability/rbd/_ccf_chain.py index ab17716f..c7505e49 100644 --- a/repyability/rbd/_ccf_chain.py +++ b/repyability/rbd/_ccf_chain.py @@ -2,7 +2,8 @@ members are down together, in the long run. A group's members are identical units with a constant failure rate. Its -model (a ``BetaFactor`` or ``MGL``, see ``ccf.py``) splits that rate +model (a ``BetaFactor``, ``MGL`` or ``AlphaFactor``, see ``ccf.py``) +splits that rate between causes: each member's own, and shared ones, each a set of members that one cause fails at once, with a share of the rate (``_causes``). Each cause strikes as a Poisson process, and fails the members it names that @@ -306,7 +307,7 @@ def gth(flow: np.ndarray) -> np.ndarray: class ProofTest(NamedTuple): """A member's test: when, which member, and whether it is a full test (one that finds a failure it would otherwise miss). With ``kind``, the - end of one of its tests or repairs of a fixed length instead (#220).""" + end of one of its tests or repairs of a fixed length instead.""" time: float member: int @@ -315,7 +316,7 @@ class ProofTest(NamedTuple): class Duration(NamedTuple): - """How long a member's tests, or its repairs, take (#220): a fixed + """How long a member's tests, or its repairs, take: a fixed time (``fixed``, 0 for none), or an exponential one (at ``rate``).""" fixed: float = 0.0 @@ -614,7 +615,7 @@ def tested_at(self, v: np.ndarray, test: "ProofTest") -> np.ndarray: def timed_events(tests, timings, period: float) -> List[ProofTest]: """``tests`` (each member's, in ``(0, period]``) with the ends of the - tests and repairs of a fixed length after each (#220), each in + tests and repairs of a fixed length after each, each in ``(0, period]`` as the schedule repeats: an end past the period is the previous period's test's, at the start of this one (where, from all up at 0, it ends nothing).""" @@ -658,7 +659,7 @@ def hidden( ``period``), with ``coverage`` the chance that a test finds a cause's failures (a full test finds all), and ``counts`` copies of each member (see ``_Counted``), tested together; or, with ``timings`` that take - time (#220), one copy of each, its tests and repairs taking them (see + time, one copy of each, its tests and repairs taking them (see ``_Timed``).""" if _timed(timings): timed = _Timed(model, members, rate, coverage, timings) @@ -701,6 +702,7 @@ def __init__(self, members, combinations, settle: float, period): self.period = period self._chain: Any = None self._tests: Optional[list] = None + self._timings: Optional[Sequence[Timing]] = None self._settled: Any = None #: The chain's moves, labelled (see ``Move``), and the combination #: of the members up or down of each of its states (#199). @@ -745,7 +747,7 @@ def hidden( timings: Optional[Sequence[Timing]] = None, ) -> "OverTime": """A group whose failures are hidden (see ``hidden``): with - ``timings`` that take time (#220), its tests' and repairs' ends + ``timings`` that take time, its tests' and repairs' ends among its tests, and settling after two periods.""" chain: _Hidden if _timed(timings): @@ -755,9 +757,11 @@ def hidden( else: chain = _Hidden(model, members, rate, coverage) settle = float(period) + timings = None out = cls(members, chain.combinations, settle, float(period)) out._chain = chain out._tests = sorted(tests) + out._timings = timings # The states at the start of the period that repeats. out._settled = chain.follow(out._early(settle), np.array([settle]))[0] out.moves = chain.moves @@ -836,12 +840,61 @@ def tested(self, values: np.ndarray, tests) -> np.ndarray: out = np.vstack([self._chain.tested_at(row, test) for row in out]) return out + def jump_times(self, stop: float) -> np.ndarray: + """The times in ``(0, stop)`` at which a member's test that takes + time starts (#220), taking it off line while it works: where the + states jump down, as a node's planned outage does.""" + if self._timings is None: + return np.empty(0) + timings = self._timings + return np.array( + sorted( + time + for time, tests in self.tests(stop).items() + if time < stop + and any( + test.kind == TEST and timings[test.member].test.takes_time + for test in tests + ) + ) + ) + + def _bends(self) -> np.ndarray: + """Where its states bend or jump within a period (times in ``(0, + period]``): the tests and the ends of their fixed lengths, and + after a test of a member whose test or repair takes an + exponential time (#220), the transient it starts, at geometric + offsets at the length's scale (the test's, and the test's and + repair's together), closing in on the test as a tested unit's + breaks close in on 0.""" + assert self._tests is not None + period = self.period + times = [np.array([test.time for test in self._tests])] + for test in self._tests: + if test.kind != TEST or self._timings is None: + continue + timing = self._timings[test.member] + scales = [] + if timing.test.rate is not None: + scales.append(1.0 / timing.test.rate) + if timing.repair.rate is not None: + scales.append(sum(scales) + 1.0 / timing.repair.rate) + for scale in scales: + stop = math.ceil(math.log2(period / scale)) + offsets = scale * 2.0 ** np.arange(-3.0, stop + 1.0) + offsets = timing.test.fixed + offsets[offsets < period] + at = test.time + offsets + at -= period * np.floor((at - 1e-12 * period) / period) + times.append(at) + return np.unique(np.concatenate(times)) + def knots(self, start: float, stop: float) -> np.ndarray: - """Where its states bend or jump: the tests (every period), or the - chain's times.""" + """Where its states bend or jump: the tests and the transients + they start (every period, see ``_bends``), or the chain's + times.""" if self._tests is None: return self._chain.knots(start, stop) - times = np.array([test.time for test in self._tests]) + times = self._bends() first = int(np.floor(start / self.period)) last = int(np.ceil(stop / self.period)) every = ( @@ -896,7 +949,7 @@ class _GroupsCurve: bend.""" def __init__(self, groups: Sequence[OverTime]): - from .repairable_rbd import _settling + from ._curves import _settling self.groups = list(groups) self.settle, self.period = _settling(self.groups) @@ -907,9 +960,15 @@ def knots(self, start: float, stop: float) -> np.ndarray: breaks = knots + def jump_times(self, stop: float) -> np.ndarray: + """The times in ``(0, stop)`` at which a group's states jump + down (see ``OverTime.jump_times``).""" + parts = [group.jump_times(stop) for group in self.groups] + return np.unique(np.concatenate(parts)) if parts else np.empty(0) + class GroupsSystem: - """A system with common-cause groups over time (#158), as the analyses + """A system with common-cause groups over time, as the analyses over a window evaluate it at given times (the hook ``RepairableRBD._window_counts`` takes as ``crew``, as for repair crews' ``_chain_transient.CrewSystem``). Its nodes outside the groups @@ -950,6 +1009,14 @@ def __init__(self, rbd, groups, causes, working, broken, method: str): #: For the pieces of the integrals: the groups' knots and settling. self.curve = _GroupsCurve(self.groups) + def jump_times(self, stop: float) -> np.ndarray: + """The times in ``(0, stop)`` at which the groups' states jump + down, a member's test that takes time starting (#220): the + system's planned outages there are counted from its values + just before and at them, as a node's at its exact times (see + ``_windows._atom_groups``).""" + return self.curve.jump_times(stop) + def _split(self, values: dict, x: np.ndarray, chances=None): """The points at the times ``x`` split by the groups' combinations: each node's probability at each point, each point's probability, @@ -959,9 +1026,11 @@ def _split(self, values: dict, x: np.ndarray, chances=None): capacity over time, which takes the nodes' joint states: it refuses where the points would be too many (see ``check_split``).""" + from . import _windows + size = len(x) - probabilities = self.rbd._filled( - values, size, self.working, self.broken + probabilities = _windows._filled( + self.rbd, values, size, self.working, self.broken ) check_split( size, @@ -1001,12 +1070,14 @@ def _evaluation(self, values: dict, x: np.ndarray, chances=None): with the probability ``values[node]`` then, and each group's members in each of their combinations with its probability then (``chances[number]``, where given, in place of its chain's).""" + from . import _ccf_groups, _windows + size = len(x) p = { node: np.broadcast_to(np.asarray(v, dtype=float), (size,)) for node, v in values.items() } - p = self.rbd._filled(p, size, self.working, self.broken) + p = _windows._filled(self.rbd, p, size, self.working, self.broken) tables = [] for number, group in enumerate(self.groups): given = None if chances is None else chances[number] @@ -1018,7 +1089,7 @@ def _evaluation(self, values: dict, x: np.ndarray, chances=None): for k, member in enumerate(group.members): p[member] = table @ np.where(group.down[:, k], 0.0, 1.0) q = {node: 1.0 - value for node, value in p.items()} - return self.rbd._ccf_tabled(p, q, tables) + return _ccf_groups._ccf_tabled(self.rbd, p, q, tables) def probabilities(self, values: dict, x: np.ndarray): """The system's availability and unavailability at the times @@ -1065,7 +1136,7 @@ def given(self, values: dict, x: np.ndarray, number: int) -> np.ndarray: """The system's unavailability at the times ``x`` (rows) with the group ``number``'s members in each of their combinations (columns), the other groups' as at the times, and each node outside the groups - up with the probability ``values[node]`` (#199).""" + up with the probability ``values[node]``.""" x = np.asarray(x, dtype=float).ravel() evaluation = self._evaluation(values, x) return evaluation.combinations(number, self.groups[number].down)[1] @@ -1090,7 +1161,7 @@ def caused(self, values: dict, x: np.ndarray) -> dict: """The rate of the system's failures by each cause at the times ``x`` (see ``evaluate``): a member's own causes under its name, and each group's causes that strike more than one member under the - tuple of its members (#199).""" + tuple of its members.""" x = np.asarray(x, dtype=float).ravel() evaluation = self._evaluation(values, x) down = evaluation.system()[1] @@ -1120,14 +1191,14 @@ def evaluate(self, values: dict, x: np.ndarray): class GroupsCurve: - """A system with common-cause groups over time (#158), from its nodes + """A system with common-cause groups over time, from its nodes outside the groups (``curves``, as another RBD's) and its groups' states (``system``, a ``GroupsSystem``): its point availability, and its events counted as another RBD counts its nodes', the groups' members' failures by their causes (see ``GroupsSystem``).""" def __init__(self, rbd, curves: dict, system: GroupsSystem): - from .repairable_rbd import _settling + from ._curves import _settling self.rbd = rbd self.curves = curves @@ -1144,7 +1215,7 @@ def at(self, x: np.ndarray) -> np.ndarray: def down_at(self, x: np.ndarray) -> np.ndarray: """The system's point unavailability at the times ``x``, worked - out in its own right (#237).""" + out in its own right.""" x = np.asarray(x, dtype=float) flat = x.ravel() values = {node: curve.at(flat) for node, curve in self.curves.items()} @@ -1153,11 +1224,18 @@ def down_at(self, x: np.ndarray) -> np.ndarray: def events(self, x: np.ndarray): """The system's expected failures and planned outages before each time ``x`` (see ``RepairableRBD._window_counts``).""" + from . import _windows from ._point_availability import _events x = np.asarray(x, dtype=float) - counts = self.rbd._window_counts( - self.curves, x.ravel(), set(), set(), "p", crew=self.system + counts = _windows._window_counts( + self.rbd, + self.curves, + x.ravel(), + set(), + set(), + "p", + crew=self.system, ) return _events( counts["failures"].reshape(x.shape), @@ -1166,9 +1244,14 @@ def events(self, x: np.ndarray): def atoms(self, stop: float): """The system's failures and planned outages at exact times before - ``stop``: its nodes outside the groups' (the groups' members fail - only at random times).""" - return self.rbd._system_atoms(self.curves, stop, crew=self.system) + ``stop``: its nodes outside the groups', and the planned outages + where a group's members' tests that take time start (the groups' + members fail only at random times).""" + from . import _windows + + return _windows._system_atoms( + self.rbd, self.curves, stop, crew=self.system + ) def knots(self, start: float, stop: float) -> np.ndarray: parts = [self.system.curve.knots(start, stop)] diff --git a/repyability/rbd/_ccf_groups.py b/repyability/rbd/_ccf_groups.py new file mode 100644 index 00000000..a2d9e81e --- /dev/null +++ b/repyability/rbd/_ccf_groups.py @@ -0,0 +1,774 @@ +"""A ``RepairableRBD``'s common-cause groups: each cause's rate and timing, +the groups' chains (``_ccf_chain``) over the members' states, in the long +run and over time, and the system worked out with them, conditioned on +module by module (``_ccf_modules.Tabled``) or on every combination of the +groups' members (``_with_ccf_groups``); with the checks that refuse what +the chains cannot take. ``RepairableRBD``'s methods call these. +""" + +from functools import partial +from typing import ( + TYPE_CHECKING, + Any, + List, + Optional, + Tuple, +) + +import numpy as np + +from repyability.rbd import ( + _ccf_chain, + _ccf_modules, + _curves, + _long_run, + _requirements, +) +from repyability.rbd._common import ( + _common_period, + _constant_rate, + _fixed_length, + _safe_mean, + _squeeze_values, +) +from repyability.rbd._events import ( + _Cause, +) +from repyability.rbd.modular import GraphStructure +from repyability.utils.checks import ( + structure_method, +) + +if TYPE_CHECKING: + pass + + +def _has_ccf(rbd) -> bool: + """Whether this RBD, or one nested in it, has common-cause + groups.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + return bool(getattr(rbd, "ccf_groups", ())) or any( + isinstance(c, RepairableRBD) and _has_ccf(c) + for c in rbd.components.values() + ) + + +def _ccf_rates(rbd, group) -> Tuple[float, Optional[float]]: + """A common-cause group's members' failure rate, and their repair + rate (None for hidden failures, found by tests), for its chain + (see ``_ccf_chain``); raise if the chain does not cover them.""" + where = f"Common-cause group {list(group.members)}" + if rbd._crews_couple(): + raise NotImplementedError( + f"{where}: with limited repair crews, the crews' Markov " + "chain does not take common causes in, as yet. Estimate the " + "values by simulation, with availability() or cost(), which " + "take both." + ) + rates = set() + for member in group.members: + for kinds, what in ( + (rbd._preventive, "scheduled maintenance"), + (rbd._imperfect, "imperfect repair"), + (rbd._member_group, "a maintenance group"), + ): + if member in kinds: + raise NotImplementedError( + f"{where}: member {member!r} has {what}, which the " + "group's chain does not take in." + ) + life = _constant_rate(rbd.components[member].reliability) + if life is None: + raise NotImplementedError( + f"{where}: its chain needs exponential lives (a constant " + f"failure rate), and the life of member {member!r} is " + "not. The simulations need one too, as the rate is " + "split between the causes." + ) + rates.add(life) + tested = [member in rbd._inspection for member in group.members] + if any(tested) and not all(tested): + raise NotImplementedError( + f"{where}: some members' failures are hidden and others' " + "revealed; the group's chain takes one kind." + ) + life = rates.pop() + if all(tested): + # Tests and repairs of no time, a fixed one or an exponential + # one (#220). + _ccf_timings(rbd, group) + for member in group.members: + _requirements._require_tested_exact(rbd, member) + if len({rbd._inspection[m].coverage for m in group.members}) > 1: + raise NotImplementedError( + f"{where}: its members' tests have different coverages; " + "the group's chain takes one, which a cause's failures " + "are found by alike." + ) + return life, None + component = rbd.components[group.members[0]] + repair = _constant_rate(component.time_to_replace) + if repair is None: + raise NotImplementedError( + f"{where}: the chain of members whose failures are revealed " + "needs exponential repairs, and theirs are not. Estimate the " + "values by simulation, with availability() or cost(), which " + "take repairs of any length." + ) + return life, repair + + +def _ccf_timings(rbd, group) -> Optional[List["_ccf_chain.Timing"]]: + """How long the tests and repairs of a common-cause group's members + (whose failures are hidden) take, for its chain (#220): each a + ``_ccf_chain.Duration`` of no time, a fixed time or an exponential + one; None where none takes time. Raise for what the chain does not + take, saying what to do: another distribution, a test of an + exponential length followed by a repair of a fixed one (whose end + is then no fixed time after the test), and fixed lengths that run + into the member's next test.""" + if not all(member in rbd._inspection for member in group.members): + return None # revealed failures (see _ccf_rates) + where = f"Common-cause group {list(group.members)}" + out = [] + for member in group.members: + inspection = rbd._inspection[member] + parts = [] + for what, model in ( + ("tests", inspection.duration), + ("repairs", rbd.components[member].time_to_replace), + ): + if model is None or _safe_mean(model) == 0.0: + parts.append(_ccf_chain.Duration()) + continue + fixed = _fixed_length(model) + if fixed is not None: + parts.append(_ccf_chain.Duration(fixed=fixed)) + continue + rate = _constant_rate(model) + if rate is None: + raise NotImplementedError( + f"{where}: its chain takes tests and repairs that " + "take no time, a fixed time or an exponential one, " + f"and the {what} of member {member!r} take none of " + f"those. Estimate the values by simulation, with " + "availability() or cost(), which take any; or, where " + f"the {what} are short next to the test interval, " + "give them a fixed length." + ) + parts.append(_ccf_chain.Duration(rate=rate)) + test, repair = parts + if test.rate is not None and repair.fixed > 0.0: + raise NotImplementedError( + f"{where}: the tests of member {member!r} take an " + "exponential time, and its repairs a fixed one, which " + "its chain does not take (the repair would end no fixed " + "time after the test). Give both a fixed length, or both " + "an exponential one; or estimate the values by " + "simulation, with availability() or cost()." + ) + length = test.fixed + repair.fixed + if length >= inspection.interval: + raise NotImplementedError( + f"{where}: the test and repair of member {member!r} take " + f"{length:g} together, as long as its test interval " + f"({inspection.interval:g}) or longer, which its chain " + "does not take. Estimate the values by simulation, with " + "availability() or cost()." + ) + out.append(_ccf_chain.Timing(test, repair)) + if not any(timing.takes_time for timing in out): + return None + return out + + +def _timing_rates(rbd) -> List[float]: + """The rates at which the common-cause groups' members' tests and + repairs of an exponential length end (#220), the time scales of + their chains' transients after a test, for the integrals that + refine to them (see ``_windows._mission``).""" + out = [] + for group in rbd.ccf_groups: + for timing in _ccf_timings(rbd, group) or (): + for duration in (timing.test, timing.repair): + if duration.rate is not None: + out.append(float(duration.rate)) + return out + + +def _shared_causes(rbd) -> List[_Cause]: + """The common-cause groups' shared causes, as the simulation strikes + them (see ``_Cause``): each a Poisson process at its share of the + members' failure rate. A member's own share is its own life (see + ``_own_rate``).""" + out = [] + for group in rbd.ccf_groups: + rate = _constant_rate(rbd.components[group.members[0]].reliability) + if rate is None: + continue # refused (see _require_groups_simulated) + coins = any( + member in rbd._inspection and rbd._inspection[member].partial + for member in group.members + ) + for struck, cause in _ccf_chain.causes( + group.model, group.members, rate + ): + if len(struck) > 1: + out.append( + _Cause( + tuple(group.members[i] for i in struck), + cause, + coins, + ) + ) + return out + + +def _own_rate(rbd, node) -> Optional[float]: + """A common-cause group member's own cause's rate, at which the + simulation draws its life, the shared causes striking it too (see + ``_shared_causes``): 0 if every cause it has is shared; None for + any other node (and for a member whose life is not exponential, + which the simulation refuses, see ``_require_groups_simulated``).""" + for group in rbd.ccf_groups: + if node in group.members: + rate = _constant_rate(rbd.components[node].reliability) + if rate is None: + return None + for struck, cause in _ccf_chain.causes( + group.model, group.members, rate + ): + if struck == (group.members.index(node),): + return float(cause) + return None + + +def _require_groups_simulated(rbd, states: Optional[dict] = None): + """Raise unless the simulation takes the common-cause groups in + (#158): each member's life exponential, as its rate is split between + the causes; no member maintained on a schedule, repaired + imperfectly, in a maintenance group, or started from a state.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + for group in rbd.ccf_groups: + where = f"Common-cause group {list(group.members)}" + for member in group.members: + life = rbd.components[member].reliability + if _constant_rate(life) is None: + raise NotImplementedError( + f"{where}: its members' failure rate is split " + "between the causes, which needs exponential lives, " + f"and the life of member {member!r} is not." + ) + for kinds, what in ( + (rbd._preventive, "scheduled maintenance"), + (rbd._imperfect, "imperfect repair"), + (rbd._member_group, "a maintenance group"), + ): + if member in kinds: + raise NotImplementedError( + f"{where}: member {member!r} has {what}, which " + "the simulation of its group does not take in, " + "as yet." + ) + start = (states or {}).get(member) + if start is not None and not start.new: + raise NotImplementedError( + f"{where}: member {member!r} starts from a state, " + "which the simulation of its group does not take in, " + "as yet: its members start new." + ) + for component in rbd.components.values(): + if isinstance(component, RepairableRBD): + _require_groups_simulated(component) + + +def _require_ccf_long_run(rbd) -> None: + """Raise if a common-cause group's chain does not cover it.""" + for group in rbd.ccf_groups: + _ccf_rates(rbd, group) + + +def _calendar_period(rbd) -> float: + """The period of the schedules the long-run values average over + (see ``_long_run_grid``): of the tests and block replacements.""" + intervals = { + rbd._preventive[node].interval for node in _long_run._block_nodes(rbd) + } + for schedule in rbd._inspection.values(): + intervals |= {schedule.interval, schedule.period} + return _common_period(intervals) if intervals else 1.0 + + +def _group_states(rbd, group, times: np.ndarray, counts=None): + """A common-cause group's members' joint states, at each of + ``times`` (of ``_long_run_grid``), in the long run (see + ``_ccf_chain``). With ``counts``, each member's number of copies, + which join the group (#158, a ``BetaFactor`` model's beta holding + at any size, see ``_ccf_chain._Counted``), tested with it: the + states of the members' nodes, each down while all its copies + are.""" + life, repair = _ccf_rates(rbd, group) + counts = [1] * len(group.members) if counts is None else list(counts) + # A member with no copies counted (one made perfect, by the + # allocation's bounds) is left out: the others' states are as with + # it. + kept = [i for i, n in enumerate(counts) if n > 0] + if not kept: + return _ccf_chain.GroupStates( + (), np.zeros((1, 0), dtype=bool), np.ones((len(times), 1)) + ) + members = [group.members[i] for i in kept] + copies = [int(counts[i]) for i in kept] + if repair is not None: + states = _ccf_chain.revealed( + group.model, members, life, repair, copies + ) + return states._replace( + probabilities=np.repeat(states.probabilities, len(times), axis=0) + ) + tests, period = _group_tests(rbd, group) + place = {old: new for new, old in enumerate(kept)} + tests = [ + test._replace(member=place[test.member]) + for test in tests + if test.member in place + ] + coverage = rbd._inspection[group.members[0]].coverage + timings = _ccf_timings(rbd, group) + if timings is not None and (len(kept) < len(counts) or max(copies) > 1): + raise NotImplementedError( + f"Common-cause group {list(group.members)}: its members' " + "tests or repairs take time, and copies of them, which the " + "allocation's chains count, are not taken in, as yet. Leave " + "its members out of the nodes to allocate." + ) + return _ccf_chain.hidden( + group.model, + members, + life, + coverage, + tests, + period, + times, + copies, + timings, + ) + + +def _group_tests(rbd, group) -> Tuple[list, float]: + """A common-cause group's members' tests in one period of the + calendar (``(0, period]``, see ``_calendar_period``), and the + period.""" + period = _calendar_period(rbd) + tests = [] + for position, member in enumerate(group.members): + schedule = rbd._inspection[member] + count = int(round(period / schedule.interval)) + first = 0 if schedule.offset else 1 + for k in range(first, first + count): + # The last, at the period's end, rounded no further. + time = min(schedule.offset + k * schedule.interval, period) + tests.append( + _ccf_chain.ProofTest(time, position, schedule.is_full(time)) + ) + return tests, period + + +def _groups_over_time(rbd) -> list: + """Each common-cause group's members' joint states over time from + every member up at 0 (see ``_ccf_chain.OverTime``).""" + out = [] + for group in rbd.ccf_groups: + life, repair = _ccf_rates(rbd, group) + if repair is not None: + where = f"Common-cause group {list(group.members)}'s" + out.append( + _ccf_chain.OverTime.revealed( + group.model, + group.members, + life, + repair, + partial(_curves._uniformized, where), + ) + ) + continue + tests, period = _group_tests(rbd, group) + out.append( + _ccf_chain.OverTime.hidden( + group.model, + group.members, + life, + rbd._inspection[group.members[0]].coverage, + tests, + period, + _ccf_timings(rbd, group), + ) + ) + return out + + +def _require_groups_over_time(rbd, states: dict) -> None: + """Raise unless the common-cause groups can be followed over time + from new (see ``_groups_curve``): their chains cover them, and no + member starts from a state.""" + _require_ccf_long_run(rbd) + started = sorted( + (m for g in rbd.ccf_groups for m in g.members if m in states), + key=str, + ) + if started: + raise NotImplementedError( + f"Node(s) {started} are in a common-cause group: its " + "values over time are worked out from every member new at " + "0, and a member's current state is not taken in, as yet." + ) + + +def _groups_system(rbd, working, broken, method: str): + """The system with its common-cause groups at given times (see + ``_ccf_chain.GroupsSystem``).""" + causes = [ + _ccf_chain.causes( + group.model, group.members, _ccf_rates(rbd, group)[0] + ) + for group in rbd.ccf_groups + ] + return _ccf_chain.GroupsSystem( + rbd, + _groups_over_time(rbd), + causes, + working, + broken, + structure_method(method), + ) + + +def _groups_curve( + rbd, + horizon: float, + working_nodes=frozenset(), + broken_nodes=frozenset(), + method: str = "p", + states: Optional[dict] = None, + counts: bool = False, + stages: bool = False, +) -> "_ccf_chain.GroupsCurve": + """The system over time with common-cause groups (#158, see + ``_ccf_chain.GroupsCurve``): its nodes outside the groups as their + own curves (each from its state in ``states``) to ``horizon``, + counting their events with ``counts`` and following their stages + with ``stages``, and the groups' members' joint states from their + chains, from every member up at 0.""" + states = states or {} + working, broken = set(working_nodes), set(broken_nodes) + _require_free_members(rbd, working, broken) + _require_groups_over_time(rbd, states) + members = {m for group in rbd.ccf_groups for m in group.members} + curves = _curves._availability_curves( + rbd, + horizon, + working | broken | members, + counts=counts, + stages=stages, + state=states, + groups=True, + ) + system = _groups_system(rbd, working, broken, method) + return _ccf_chain.GroupsCurve(rbd, curves, system) + + +def _require_free_members(rbd, working_nodes, broken_nodes) -> None: + """Raise if a common-cause group's member is held working or + broken: the cause it shares would still strike the others.""" + held = set(working_nodes or ()) | set(broken_nodes or ()) + for group in rbd.ccf_groups: + caught = held & set(group.members) + if caught: + raise NotImplementedError( + f"Node(s) {sorted(caught, key=str)} are in a " + "common-cause group, whose shared causes would still " + "strike the others: they cannot be held working or " + "broken." + ) + + +def _with_ccf_groups( + rbd, + times: np.ndarray, + probabilities: dict, + failures: Optional[dict], + weights: np.ndarray, + states_by_group: Optional[list] = None, + what: str = "This analysis", +) -> Tuple[dict, Optional[dict], np.ndarray, np.ndarray]: + """The long-run points (the times of ``_long_run_grid``, and their + weights) split by the common-cause groups' joint states: each + time into one point for each combination of each group's members + up or down, weighted by its probability then, with those members + up or down for certain and the other nodes as at the time. The + long-run values are averages over these points as over the times. + Also each point's time's position in ``times``. ``states_by_group`` + gives the groups' states (see ``_group_states``), in their order, + when they are not the groups' own (an allocation's design). + + Every combination of every group's states is a point of its own, + so the points multiply with the groups: ``what`` (the analysis that + splits them) refuses, before anything is built, where they would + take more than ``_ccf_chain.SPLIT_VALUES`` values (#218). The + long-run values, the importance measures and the values over time + condition on each group within its module instead (see + ``_ccf_tabled``).""" + tables = [ + ( + _group_states(rbd, group, times) + if states_by_group is None + else states_by_group[number] + ) + for number, group in enumerate(rbd.ccf_groups) + ] + _ccf_chain.check_split( + len(times), + [len(states.down) for states in tables], + len(probabilities) * (1 if failures is None else 2), + what, + rbd.ccf_groups, + ) + index = np.arange(len(times)) + for states in tables: + combinations = len(states.down) + mass = (weights[:, None] * states.probabilities[index, :]).ravel() + keep = mass > 0.0 + weights = mass[keep] + + def spread(values, combinations=combinations, keep=keep): + points = len(keep) // combinations + values = np.broadcast_to( + np.asarray(values, dtype=float), (points,) + ) + return np.repeat(values, combinations)[keep] + + probabilities = {n: spread(v) for n, v in probabilities.items()} + if failures is not None: + failures = {n: spread(v) for n, v in failures.items()} + for k, member in enumerate(states.members): + down = np.tile(states.down[:, k], len(index))[keep] + probabilities[member] = np.where(down, 0.0, 1.0) + if failures is not None: + failures[member] = np.where(down, 1.0, 0.0) + index = np.repeat(index, combinations)[keep] + return probabilities, failures, weights, index + + +def _ccf_tabled( + rbd, p: dict, q: dict, tables: list, structure=None +) -> "_ccf_modules.Tabled": + """The system with its common-cause groups at given points (#218): + each node outside the groups up with the probability ``p[node]`` + and down with ``q[node]`` (1-d arrays of one length), and each + group's members in each combination of their states with the + probabilities ``tables`` give (each group's + ``_ccf_chain.GroupStates`` at the points). Given a group's + combination its members are independent of everything else, so + each group is conditioned on only within the smallest module + holding its members (see ``_ccf_modules.Tabled``): groups in + separate modules cost a sum, not a product. Over ``structure`` + (the dual, for Fussell-Vesely over path sets) if given, else the + decomposition.""" + structure = rbd._decomposition() if structure is None else structure + if isinstance(structure, GraphStructure): + raise NotImplementedError(structure.reason) + p, q, size = rbd._node_pairs(p, q) + return _ccf_modules.Tabled( + rbd._ccf_plan(structure), rbd.ccf_groups, p, q, size, tables + ) + + +def _ccf_long_run(rbd, working_nodes, broken_nodes) -> tuple: + """The long-run grid's times and weights (see ``_long_run_grid``), + and what ``_ccf_tabled`` takes there: the nodes' availabilities + and unavailabilities at the times (the forced nodes held at 1 or + 0), and the groups' members' joint states then, from their chains + (see ``_group_states``).""" + # The groups' chains first, as the routes report them. + _require_ccf_long_run(rbd) + _require_free_members(rbd, working_nodes, broken_nodes) + times, weights = _long_run._long_run_grid(rbd) + p = rbd._probabilities_with_overrides( + _long_run._availabilities_at(rbd, times), working_nodes, broken_nodes + ) + q = rbd._failures_with_overrides( + _long_run._unavailabilities_at(rbd, times), working_nodes, broken_nodes + ) + # The groups' chains, kept for the other long-run values (#229): + # a choice of intervals asks for the cost rate and the + # availability of each plan. + tables = rbd._cache.ccf_tables + if tables is None: + tables = rbd._cache.ccf_tables = [ + _group_states(rbd, group, times) for group in rbd.ccf_groups + ] + return times, weights, (p, q, tables) + + +def _ccf_long_run_measure( + rbd, + measure: str, + working_nodes, + broken_nodes, + kind: str = "failure", + fv_type: str = "c", + method: str = "exact", +) -> dict: + """An importance measure of every node in the long run, with the + common-cause groups (see ``_ccf_measure``).""" + working, broken = rbd._held_nodes(working_nodes, broken_nodes) + _, weights, inputs = _ccf_long_run(rbd, working, broken) + return _squeeze_values( + rbd._ccf_measure( + measure, inputs, weights, kind, fv_type, method, working | broken + ) + ) + + +def _ccf_fv_shares( + rbd, evaluation, inputs: tuple, fv_type: str, method: str +) -> dict: + """The numerators of the Fussell-Vesely importances (see + ``_fv_numerators``) with the common-cause groups, at each point: + exactly, each group conditioned on within the smallest module + holding its members (see ``_ccf_modules.Evaluation + .failed_cut_sets``); as rare events, each set's probability of + having failed summed over its groups' combinations (see + ``_ccf_modules.expected_product``).""" + size = evaluation.shape + zero = np.zeros(size) + if method == "exact": + structure = rbd._set_structure() + if structure.always_works: + failed: dict = {} + else: + if fv_type == "p": + p, q, tables = inputs + evaluation = _ccf_tabled( + rbd, p, q, tables, structure=structure.dual() + ) + failed = evaluation.failed_cut_sets() + return { + node: np.broadcast_to( + np.asarray(failed.get(node, zero), dtype=float), (size,) + ) + for node in rbd.nodes + } + outcomes = evaluation.outcomes_of_tables() + if fv_type == "c": + node_sets = rbd.get_min_cut_sets() + else: + node_sets = { + frozenset(path_set) + for path_set in rbd.get_min_path_sets(include_in_out_nodes=False) + } + out = {node: np.zeros(size) for node in rbd.nodes} + for node_set in node_sets: + value = np.broadcast_to( + np.asarray( + _ccf_modules.expected_product( + node_set, evaluation.q, outcomes + ), + dtype=float, + ), + (size,), + ) + for node in node_set: + out[node] = out[node] + value + return out + + +def _require_ccf_frequencies(rbd) -> None: + """Raise if the failure frequency with common-cause groups is not + worked out: where their long-run values are not.""" + if rbd.ccf_groups: + _require_ccf_long_run(rbd) + + +def _ccf_calendar_outages(rbd, working_nodes, broken_nodes) -> float: + """``_long_run._calendar_outages`` with common-cause groups (#293): + the groups' members' joint states just before and just after each + instant from their chains (a member's test that takes time, starting + then, is one of their jumps), the other nodes' as without groups.""" + changes = _long_run._calendar_changes(rbd, working_nodes, broken_nodes) + if changes is None: + return 0.0 + period, instants, before, after = changes + # Just after each instant, and just before it (one before 0 is at the + # period's end). + shift = 1e-9 * period + down = [] + for (p, q), times in ( + (after, instants + shift), + (before, (instants - shift) % period), + ): + tables = [_group_states(rbd, group, times) for group in rbd.ccf_groups] + down.append(_ccf_tabled(rbd, p, q, tables).system()[1]) + return float(np.sum(down[0] - down[1])) / period + + +def _ccf_outage_terms( + rbd, working_nodes, broken_nodes +) -> Tuple[List[Tuple[Any, float]], float]: + """``_outage_terms`` with common-cause groups, over the long-run + grid's times (see ``_ccf_long_run``): a node outside the groups + fails at its rate and takes the system down where it is critical + (its Birnbaum importance then, over the groups' states); each cause + strikes at its rate and takes the system down by failing the + members it names that are up (the rise in the system's + unavailability with them down).""" + _require_ccf_frequencies(rbd) + _require_free_members(rbd, working_nodes, broken_nodes) + times, weights, inputs = _ccf_long_run(rbd, working_nodes, broken_nodes) + availability = inputs[0] + evaluation = _ccf_tabled(rbd, *inputs) + R_t, Q_t = evaluation.system() + forced = set(working_nodes or ()) | set(broken_nodes or ()) + members = {m for group in rbd.ccf_groups for m in group.members} + terms: List[Tuple[Any, float]] = [] + planned = 0.0 + blocks = set(_long_run._block_nodes(rbd)) + for node in rbd.components: + if node in forced or node in members: + continue + up_1, down_1 = evaluation.system(hold={node: True}) + up_0, down_0 = evaluation.system(hold={node: False}) + importance = np.where(Q_t <= R_t, down_0 - down_1, up_1 - up_0) + if node in rbd._inspection: + node_failures: Any = _requirements._tested_intensity( + rbd, node, times, availability[node] + ) + node_planned: Any = 0.0 + elif node in blocks: + node_failures = _long_run._block_profile( + rbd, node, times, rates=True + ) + node_planned = 0.0 + else: + node_failures, _, node_planned = _long_run._node_frequencies( + rbd, node + ) + terms.append((node, float(weights @ (importance * node_failures)))) + planned += float(weights @ (importance * node_planned)) + for group in rbd.ccf_groups: + life, _ = _ccf_rates(rbd, group) + for struck, rate in _ccf_chain.causes( + group.model, group.members, life + ): + hit = {group.members[position]: False for position in struck} + rise = evaluation.system(hold=hit)[1] - Q_t + cause = ( + group.members[struck[0]] + if len(struck) == 1 + else tuple(group.members) + ) + terms.append((cause, rate * float(weights @ rise))) + planned += _ccf_calendar_outages(rbd, working_nodes, broken_nodes) + return terms, planned diff --git a/repyability/rbd/_ccf_modules.py b/repyability/rbd/_ccf_modules.py index 03168255..05c3f183 100644 --- a/repyability/rbd/_ccf_modules.py +++ b/repyability/rbd/_ccf_modules.py @@ -29,12 +29,14 @@ probabilities once (each owner's summed over its groups' outcomes), and then the system's, also with nodes held working or failed, or a group's outcome given: only the terms above them are worked out again, and the -owners among those summed over their outcomes again. ``joints`` gives -what the importance measures need of a node, and ``failed_cut_sets`` each -node's probability of failing with some minimal cut set containing it, -the numerator of its Fussell-Vesely importance (for a structure without -shocks). Every value is a sum of products, so a small probability keeps -its precision. +owners among those summed over their outcomes again. ``states`` gives +the system with one node, or a pair, in each combination of their states +(what the importance measures need, the joint importance's pairs among +them, #317), ``given`` the system given those states, and +``failed_cut_sets`` each node's probability of failing with some minimal +cut set containing it, the numerator of its Fussell-Vesely importance +(for a structure without shocks). Every value is a sum of products, so a +small probability keeps its precision. """ import math @@ -470,24 +472,45 @@ def _combine(self, i: int, R, Q) -> Tuple[Any, Any]: term[2] if term[0] == KOON else 0, ) - def _overrides(self, owner: int, fixed: dict): - """Each combination of the outcomes of the groups ``owner`` owns - (but those whose outcome ``fixed`` gives, holding their members): - its probability, and ``fixed`` with their members' probabilities - given it.""" - groups = [ - self.outcomes[g] - for g in self.c.owned[owner] - if not self.c.members[g] <= fixed.keys() - ] - for combination in product(*groups): # type: ignore[arg-type] + def _combinations(self, groups, fixed: dict, frame=None, split=()): + """Each combination of the outcomes of the groups ``groups`` + (conditioned on): its probability, ``fixed`` with their members' + probabilities given it, and the frame its values are in. With + ``split``, nodes of those groups, each combination is split by + their states too: each node up or down for certain, the chance of + that given the outcome in the probability.""" + outcomes = [self.outcomes[g] for g in groups] + ones = np.ones(self._shape(frame)) + zeros = 0.0 * ones + for combination in product(*outcomes): # type: ignore[arg-type] weight: Any = 1.0 given = dict(fixed) for outcome_weight, works, fails in combination: weight = weight * outcome_weight for member in works: given[member] = (works[member], fails[member]) - yield weight, given + for states in product((True, False), repeat=len(split)): + share, held = weight, dict(given) + for node, up in zip(split, states): + share = share * given[node][0 if up else 1] + held[node] = (ones, zeros) if up else (zeros, ones) + yield share, held, frame + + def _total(self, weight, value): + """A value at a combination's points, weighted by the combination's + probability, as it adds to the sum over the combinations.""" + return weight * value + + def _owned(self, owner: int, fixed: dict, frame=None): + """The combinations (see ``_combinations``) of the groups ``owner`` + owns, but those whose outcome ``fixed`` gives, holding their + members.""" + groups = [ + g + for g in self.c.owned[owner] + if not self.c.members[g] <= fixed.keys() + ] + return self._combinations(groups, fixed, frame) def _order(self, top: int, keys) -> List[int]: """The terms under ``top`` (with it) whose probabilities change with @@ -559,10 +582,10 @@ def _mix( """``owner``'s probabilities, summed over its groups' outcomes.""" works: Any = 0.0 fails: Any = 0.0 - for weight, given in self._overrides(owner, fixed): - r, q = self._unmixed(owner, given, hold, frame) - works = works + weight * r - fails = fails + weight * q + for weight, given, inner in self._owned(owner, fixed, frame): + r, q = self._unmixed(owner, given, hold, inner) + works = works + self._total(weight, r) + fails = fails + self._total(weight, q) return works, fails def _unmixed( @@ -620,87 +643,167 @@ def _top_terms(self, fixed: dict, hold: dict, frame=None): # -- the importance measures ------------------------------------------- - def joints(self, node) -> Tuple[Dict[str, Any], bool]: - """What the importance measures need of ``node``: the probabilities - that it works and that it has failed (``works``, ``fails``), and - that the system works and fails with it working (``up_ok``, - ``down_ok``) and with it failed (``up_bad``, ``down_bad``); and - whether those four are joint probabilities (for a group member, - whose state says something of its group's shocks: summed over - them, each weighed by its chance of the state) or given the node's - state (for one outside the groups, held).""" - g = self.group_of.get(node) - if g is None: - up_1, down_1 = self.system(hold={node: True}) - up_0, down_0 = self.system(hold={node: False}) - return ( - { - "works": self.p[node], - "fails": self.q[node], - "up_ok": up_1, - "down_ok": down_1, - "up_bad": up_0, - "down_bad": down_0, - }, - False, - ) - if g in self.causes: - return self._written_out(node, g), True - out: Dict[str, Any] = dict.fromkeys( - ("works", "fails", "up_ok", "down_ok", "up_bad", "down_bad"), 0.0 - ) - for weight, given, failing in self.outcomes[g]: # type: ignore - fixed = {m: (given[m], failing[m]) for m in given} - on, off = given[node], failing[node] - up_1, down_1 = self.system(fixed, {node: True}) - up_0, down_0 = self.system(fixed, {node: False}) - out["works"] = out["works"] + weight * on - out["fails"] = out["fails"] + weight * off - out["up_ok"] = out["up_ok"] + weight * on * up_1 - out["down_ok"] = out["down_ok"] + weight * on * down_1 - out["up_bad"] = out["up_bad"] + weight * off * up_0 - out["down_bad"] = out["down_bad"] + weight * off * down_0 - return out, True - - def _written_out(self, node, g: int) -> Dict[str, Any]: - """``joints`` for a member of a group written out as shocks: it - works when neither its own causes nor any shock striking it has - fired, and has failed through the first of them that has (its own, - then the shocks in turn), so that each probability is a sum of - products.""" - q_own, r_own, fired = self.causes[g] - strikes = [ - (Shock(g, k), fires, holds) - for k, (struck, fires, holds) in enumerate(fired) - if node in struck + def _kinds(self, nodes) -> Tuple[list, list, list, list]: + """``nodes`` sorted by how their states are given: the members of + groups conditioned on, those groups, the members of groups written + out as shocks, and the nodes outside the groups (held).""" + conditioned = [ + node + for node in nodes + if node in self.group_of and self.group_of[node] not in self.causes ] - held: Dict[Hashable, bool] = {node: True} - works: Any = r_own - for shock, _, holds in strikes: - held[shock] = True - works = works * holds - up_1, down_1 = self.system(hold=held) - up_0, down_0 = self.system(hold={node: False}) - fails: Any = q_own - up_bad: Any = q_own * up_0 - down_bad: Any = q_own * down_0 - before: Any = r_own # its own causes, and the shocks so far, held - hold: Dict[Hashable, bool] = {node: True} - for shock, fires, holds in strikes: - up, down = self.system(hold={**hold, shock: False}) - fails = fails + before * fires - up_bad = up_bad + before * fires * up - down_bad = down_bad + before * fires * down - before = before * holds - hold[shock] = True - return { - "works": works, - "fails": fails, - "up_ok": works * up_1, - "down_ok": works * down_1, - "up_bad": up_bad, - "down_bad": down_bad, + groups = sorted({self.group_of[node] for node in conditioned}) + written = [ + node + for node in nodes + if node in self.group_of and self.group_of[node] in self.causes + ] + free = [node for node in nodes if node not in self.group_of] + return conditioned, groups, written, free + + def states(self, nodes) -> Dict[tuple, Tuple[Any, Any, Any]]: + """What the importance measures need of ``nodes`` (one, or a pair + for the joint importance): for each combination of their states + (True for working, in their order), the probability that the + group members among them are in those states, and that the system + works and that it fails with them so, jointly for the members and + given the nodes outside the groups, which are held (the chance is + 1 with no member). A member's state says something of its group's + shocks, so its joint probabilities are sums over its group's + outcomes, each weighed by the chance of the state in it, the other + groups conditioned on within their modules (#219); the members of + a group written out as shock events are held through their causes + (see ``_fired``). Every term is a product of probabilities, so a + small one keeps its precision.""" + nodes = tuple(nodes) + conditioned, groups, written, free = self._kinds(nodes) + rest = written + free + out: Dict[tuple, Tuple[Any, Any, Any]] = { + combo: (0.0, 0.0, 0.0) + for combo in product((True, False), repeat=len(nodes)) } + for weight, given, frame in self._combinations( + groups, {}, None, conditioned + ): + done: Dict[tuple, list] = {} + for combo in out: + state = dict(zip(nodes, combo)) + key = tuple(state[node] for node in rest) + if key not in done: + done[key] = [ + (factor, *self.system(given, held, frame)) + for factor, held in self._fired( + [(node, state[node]) for node in written], + {node: state[node] for node in free}, + ) + ] + chance, up, down = out[combo] + for factor, r, q in done[key]: + share = factor + for node in conditioned: + share = share * given[node][0 if state[node] else 1] + chance = chance + self._total(weight, share) + up = up + self._total(weight, share * r) + down = down + self._total(weight, share * q) + out[combo] = (chance, up, down) + return out + + def _fired(self, targets, hold: dict): + """Each way the members of groups written out as shocks + (``targets``: each with the state wanted) come to be so, with + ``hold`` (the nodes held working or failed): its probability, and + ``hold`` with the members' causes held. A member works when + neither its own causes nor any shock striking it has fired, which + are held off; it has failed through the first of them that has + (its own, then the shocks in turn), with those before held off, + and the rest free, so that each probability is a product.""" + hold = dict(hold) + factor: Any = 1.0 + pending = [] + for node, up in targets: + g = self.group_of[node] + q_own, r_own, fired = self.causes[g] + causes = [(node, q_own, r_own)] + [ + (Shock(g, k), fires, holds) + for k, (struck, fires, holds) in enumerate(fired) + if node in struck + ] + if not up: + pending.append(causes) + continue + for cause, _, holds in causes: + if cause not in hold: + hold[cause] = True + factor = factor * holds + yield from self._first_fired(pending, hold, factor) + + def _first_fired(self, pending: list, hold: dict, factor): + """``_fired`` for the members still to fail, each through the + first of its causes (``pending``) not held off that fires: a cause + already fired has failed it.""" + if not pending: + yield factor, hold + return + causes, rest = pending[0], pending[1:] + if any(hold.get(cause) is False for cause, _, _ in causes): + yield from self._first_fired(rest, hold, factor) + return + before, held = factor, dict(hold) + for cause, fires, holds in causes: + if cause in held: + continue # held off: it has not fired + yield from self._first_fired( + rest, {**held, cause: False}, before * fires + ) + held[cause] = True + before = before * holds + + def held(self, nodes) -> Dict[tuple, Tuple[Any, Any]]: + """The system's probabilities of working and of failing with + ``nodes`` held in each combination of their states (as ``states`` + lays them out), the rest as they are (their groups' other members + included): what the system given the nodes' states comes to where + those states have no chance (see ``given``), as for nodes outside + the groups.""" + nodes = tuple(nodes) + _, groups, _, _ = self._kinds(nodes) + out: Dict[tuple, Tuple[Any, Any]] = {} + for combo in product((True, False), repeat=len(nodes)): + hold = dict(zip(nodes, combo)) + up: Any = 0.0 + down: Any = 0.0 + for weight, given, frame in self._combinations(groups, {}, None): + r, q = self.system(given, hold, frame) + up = up + self._total(weight, r) + down = down + self._total(weight, q) + out[combo] = (up, down) + return out + + def given(self, nodes) -> Dict[tuple, Tuple[Any, Any, Any]]: + """The system given ``nodes`` in each combination of their states + (as ``states`` lays them out): the combination's probability (1 + for nodes outside the groups, held), and the system's + probabilities of working and of failing given it, the joint + probabilities over the combination's. Where a combination has no + chance at some point (a member off line for its test, say), the + system with the nodes held so there (see ``held``), as for nodes + outside the groups.""" + nodes = tuple(nodes) + states = self.states(nodes) + held = None + out: Dict[tuple, Tuple[Any, Any, Any]] = {} + with np.errstate(divide="ignore", invalid="ignore"): + for combo, (chance, up, down) in states.items(): + if np.any(np.asarray(chance) <= 0.0): + if held is None: + held = self.held(nodes) + some = np.asarray(chance) > 0.0 + up = np.where(some, up / chance, held[combo][0]) + down = np.where(some, down / chance, held[combo][1]) + else: + up, down = up / chance, down / chance + out[combo] = (chance, up, down) + return out # -- Fussell-Vesely ---------------------------------------------------- @@ -799,9 +902,9 @@ def _fv_mix( self, owner: int, fixed: dict, frame=None ) -> Dict[Hashable, Any]: out: Dict[Hashable, Any] = {} - for weight, given in self._overrides(owner, fixed): - for node, value in self._fv_unmixed(owner, given, frame).items(): - out[node] = out.get(node, 0.0) + weight * value + for weight, given, inner in self._owned(owner, fixed, frame): + for node, value in self._fv_unmixed(owner, given, inner).items(): + out[node] = out.get(node, 0.0) + self._total(weight, value) return out def _fv_pass(self, top: int, fixed: dict, mix_top: bool, frame=None): @@ -879,21 +982,22 @@ def marginal(self, k: int) -> np.ndarray: class Tabled(Evaluation): """``Evaluation`` with each common-cause group's members' joint states - given as tables (a repairable system's, from its groups' chains, #218), + given as tables (a repairable system's, from its groups' chains), not from its model's outcomes: each combination of the members up or down, and its probability at each point (``tables``, each group's ``_ccf_chain.GroupStates``). Given its combination the members are up or down for certain, and independent of every other node, so each group is conditioned on within its owner as before (see ``Conditioned``); but where ``Evaluation`` sums an owner over its - groups' outcomes one at a time, here they are points of their own: - each point of the owner's repeated for each combination of its groups' - states with a probability there, worked out together, ``CHUNK`` at a - time, and summed back. So the memory taken stays bounded, and the time - is that of the points: a sum over the owners where the groups' states - are in separate modules, a product only where they meet. Before - anything is worked out, the most points an owner can need is checked - against ``POINTS`` (see ``_check``).""" + groups' outcomes one at a time, here they are points of their own + (``_combinations``): each point of the owner's repeated for each + combination of its groups' states with a probability there, worked + out together, ``CHUNK`` at a time, and summed back (``_total``). So + the memory taken stays bounded, and the time is that of the points: a + sum over the owners where the groups' states are in separate modules, + a product only where they meet. Before anything is worked out, the + most points an owner can need is checked against ``POINTS`` (see + ``_check``).""" def __init__(self, plan: Plan, groups, p, q, shape, tables): self.shape = shape @@ -909,7 +1013,6 @@ def __init__(self, plan: Plan, groups, p, q, shape, tables): for member in table.members } self.p, self.q = dict(p), dict(q) - self._joints: Dict[int, dict] = {} self._check() self._settle() @@ -958,20 +1061,15 @@ def _check(self) -> None: "which take the groups in." ) - # -- an owner's combinations as points of their own ---------------------- + # -- the groups' combinations as points of their own --------------------- - def _frames(self, owner: int, fixed: dict, frame): - """The combinations of ``owner``'s groups' states (but those whose - members ``fixed`` holds) as a new last axis of ``frame``, ``CHUNK`` - values at a time: for each chunk, the combinations' probabilities - at each point, ``fixed`` with the groups' members up or down in - each, and the chunk's frame. With no group left, ``fixed`` and the - frame itself.""" - groups = [ - g - for g in self.c.owned[owner] - if not set(self.tables[g].members) <= fixed.keys() - ] + def _combinations(self, groups, fixed: dict, frame=None, split=()): + """The combinations of the states of the groups ``groups`` as a + new last axis of ``frame``, ``CHUNK`` values at a time: for each + chunk, the combinations' probabilities at each point, ``fixed`` + with the groups' members up or down in each (so ``split`` asks + for nothing more), and the chunk's frame. With no group, ``fixed`` + and the frame itself, under no weight.""" if not groups: yield None, fixed, frame return @@ -1014,26 +1112,12 @@ def _frames(self, owner: int, fixed: dict, frame): ) yield weights, given, inner - def _mix(self, owner: int, fixed: dict, hold: dict, frame=None): - works: Any = 0.0 - fails: Any = 0.0 - for weights, given, inner in self._frames(owner, fixed, frame): - r, q = self._unmixed(owner, given, hold, inner) - if weights is None: - return r, q - works = works + (weights * r).sum(axis=-1) - fails = fails + (weights * q).sum(axis=-1) - return works, fails - - def _fv_mix(self, owner: int, fixed: dict, frame=None) -> dict: - out: Dict[Hashable, Any] = {} - for weights, given, inner in self._frames(owner, fixed, frame): - values = self._fv_unmixed(owner, given, inner) - if weights is None: - return values - for node, value in values.items(): - out[node] = out.get(node, 0.0) + (weights * value).sum(axis=-1) - return out + def _total(self, weight, value): + """The value summed back over the chunk's combinations (the last + axis), weighted; the value itself under no weight.""" + if weight is None: + return value + return (weight * value).sum(axis=-1) # -- what the measures need -------------------------------------------- @@ -1042,50 +1126,6 @@ def marginal(self, member) -> np.ndarray: table = self.tables[self.group_of[member]] return table.marginal(table.members.index(member)) - def joints(self, node) -> Tuple[Dict[str, Any], bool]: - """As ``Evaluation.joints``: a member's are worked out for its - whole group at once (see ``_group_joints``).""" - g = self.group_of.get(node) - if g is None: - return super().joints(node) - if g not in self._joints: - self._joints[g] = self._group_joints(g) - return self._joints[g][node], True - - def _group_joints(self, g: int) -> Dict[Hashable, Dict[str, Any]]: - """``joints`` for each member of group ``g``: the system worked out - once with the group's members in each of their combinations (a last - axis, see ``_frames``), and summed over them for each member up - and down.""" - table = self.tables[g] - count = len(table.works) - frame = (self.shape, count) - given = { - member: ( - table.works[:, k].reshape(1, count), - table.fails[:, k].reshape(1, count), - ) - for k, member in enumerate(table.members) - } - up, down = self.system(given, {}, frame) - weights = table.probabilities - - def total(value) -> np.ndarray: - return np.broadcast_to((weights * value).sum(axis=-1), self.shape) - - out: Dict[Hashable, Dict[str, Any]] = {} - for k, member in enumerate(table.members): - on, off = given[member] - out[member] = { - "works": total(on), - "fails": total(off), - "up_ok": total(on * up), - "down_ok": total(on * down), - "up_bad": total(off * up), - "down_bad": total(off * down), - } - return out - def combinations(self, g: int, down: np.ndarray) -> Tuple[Any, Any]: """The system's probabilities of working and of failing at each point (rows) with group ``g``'s members in each combination of @@ -1121,3 +1161,29 @@ def outcomes_of_tables(self) -> List[List[Outcome]]: ] for table in self.tables ] + + +def joint_importance(evaluation: Evaluation, i, j) -> np.ndarray: + """The joint (second-order) importance of nodes ``i`` and ``j`` at each + of the evaluation's points (#317): ``R(1_i, 1_j) - R(1_i, 0_j) - + R(0_i, 1_j) + R(0_i, 0_j)``, with ``R(1_i, 0_j)`` the system's + probability of working given node ``i`` working and node ``j`` + failed (see ``Evaluation.given``: a group member's state conditioned + on, two members of one group on their joint state; a node outside the + groups held); from whichever end keeps its precision, the system's + probability of failing where that is the smaller.""" + works, fails = evaluation.system() + given = evaluation.given((i, j)) + + def second(k: int) -> Any: + return ( + given[True, True][k] + - given[True, False][k] + - given[False, True][k] + + given[False, False][k] + ) + + with np.errstate(invalid="ignore"): + return np.where( + np.asarray(fails) <= np.asarray(works), -second(2), second(1) + ) diff --git a/repyability/rbd/_chain_transient.py b/repyability/rbd/_chain_transient.py index e765ec5d..5eb3d72e 100644 --- a/repyability/rbd/_chain_transient.py +++ b/repyability/rbd/_chain_transient.py @@ -318,7 +318,7 @@ class CrewSystem: each state, then for each pattern the rate of its failures by the chain's components in each state, then whether each of the chain's ``served`` components is up; with ``causes``, then for each pattern - the rate of its failures by each of them (#199). The nested RBDs have + the rate of its failures by each of them. The nested RBDs have crews of their own, so they are independent of the chain: at a time, the system is up with probability ``sum_m p(t) u_m w_m(t)``, ``w_m`` the pattern's probability from their availabilities then, and a nested @@ -336,6 +336,13 @@ def __init__( #: For the pieces of the integrals: the chain's knots and settling. self.curve = ChainCurve(chain) + def jump_times(self, stop: float) -> np.ndarray: + """The times in ``(0, stop)`` at which the chain's states jump + down at exact times: none, as its components fail and are + repaired at rates (a common-cause group's chain jumps at its + members' tests, see ``_ccf_chain.GroupsSystem.jump_times``).""" + return np.empty(0) + def evaluate(self, values: dict, x: np.ndarray): """At the times ``x``, with each nested RBD up with the probability ``values[node]`` (an array over ``x``): each nested RBD's Birnbaum @@ -414,10 +421,10 @@ class CrewCurve: RBDs, its integrals give the system's uptime and failures from 0 exactly (``integral``, ``events``); with them, its events are counted as another RBD counts its nodes', with the nested RBDs' importance - taken over the chain (#162).""" + taken over the chain.""" def __init__(self, rbd, chain: Uniformized, nested: dict): - from .repairable_rbd import _settling + from ._curves import _settling self.rbd = rbd self.chain = chain @@ -425,7 +432,7 @@ def __init__(self, rbd, chain: Uniformized, nested: dict): self.patterns = 2 ** len(nested) self.system = CrewSystem(chain, list(nested)) #: As a node with capacities of another RBD, its capacity over time - #: (``repairable_rbd._CrewCapacity``, set by ``_nested_curve``). + #: (``_repairable_capacity._CrewCapacity``, set by ``_nested_curve``). self.capacity: Any = None self.settle, self.period = _settling( [ChainCurve(chain), *nested.values()] @@ -451,10 +458,18 @@ def events(self, x: np.ndarray) -> Dict[str, np.ndarray]: """The system's expected failures before each time ``x``, and its planned outages: the chain's components have no scheduled maintenance, but its nested RBDs may.""" + from . import _windows + x = np.asarray(x, dtype=float) if self.nested: - counts = self.rbd._window_counts( - self.nested, x.ravel(), set(), set(), "p", crew=self.system + counts = _windows._window_counts( + self.rbd, + self.nested, + x.ravel(), + set(), + set(), + "p", + crew=self.system, ) return _events( counts["failures"].reshape(x.shape), @@ -468,9 +483,13 @@ def atoms(self, stop: float) -> Atoms: """The system's failures and planned outages at exact times before ``stop``: its nested RBDs', which the chain's do not share (it moves only at random times).""" + from . import _windows + if not self.nested: return Atoms.none() - return self.rbd._system_atoms(self.nested, stop, crew=self.system) + return _windows._system_atoms( + self.rbd, self.nested, stop, crew=self.system + ) def knots(self, start: float, stop: float) -> np.ndarray: parts = [self.chain.knots(start, stop)] diff --git a/repyability/rbd/_common.py b/repyability/rbd/_common.py new file mode 100644 index 00000000..f31db06b --- /dev/null +++ b/repyability/rbd/_common.py @@ -0,0 +1,411 @@ +"""Small helpers the ``RepairableRBD`` modules share: the checks of a +horizon and a discount rate, a model's mean, survival and cumulative +distribution as arrays, a constant failure rate, the common period of +inspection intervals, and the shapes results take. +""" + +import functools +import math +import numbers +import warnings +import weakref +from fractions import Fraction +from typing import ( + Any, + Callable, + Iterable, + List, + Optional, +) + +import numpy as np + +from repyability.rbd._model_utils import ( + MODEL_ERRORS, + model_mean, +) +from repyability.utils.checks import ( + number_or_nan, + real_array, +) +from repyability.utils.wrappers import outside_level + + +def _safe_mean(model) -> float: + """A model's mean (see ``model_mean``), or NaN if it has none.""" + try: + return model_mean(model) + except MODEL_ERRORS: + return float("nan") + + +#: Each model's constant rate (see ``_constant_rate``), kept while the +#: model lives: the checks and the chains ask for it many times over, on +#: every diagram built around the same model (a search over intervals +#: builds hundreds). +_CONSTANT_RATES: "weakref.WeakKeyDictionary[Any, Optional[float]]" = ( + weakref.WeakKeyDictionary() +) + + +def _constant_rate(model) -> Optional[float]: + """The failure rate of a model whose rate is constant (an exponential + life, or a Weibull of shape 1), or None.""" + try: + return _CONSTANT_RATES[model] + except (KeyError, TypeError): # not kept yet, or not a model + pass + rate = _worked_out_rate(model) + try: + _CONSTANT_RATES[model] = rate + except TypeError: # not weakly referenced (text, say) + pass + return rate + + +def _worked_out_rate(model) -> Optional[float]: + """See ``_constant_rate``: worked out from the model's hazard and + survival at a few times.""" + try: + t = model_mean(model) * np.array([0.01, 0.5, 1.0, 3.0]) + hazard = np.asarray(model.hf(t), dtype=float).ravel() + survival = np.asarray(model.sf(t), dtype=float).ravel() + except MODEL_ERRORS: + return None + rate = float(hazard[0]) + if not (np.isfinite(rate) and rate > 0.0): + return None + constant = np.allclose(hazard, rate, rtol=1e-9, atol=0.0) + exponential = np.allclose(survival, np.exp(-rate * t), rtol=1e-9) + return rate if constant and exponential else None + + +def _times_like(rbd, value) -> bool: + """Whether ``value``, given where node names go, can only be times: a + number, or numbers none of which is a node of ``rbd`` (#224).""" + if value is None or isinstance(value, (str, bytes, bool, np.bool_)): + return False + if isinstance(value, numbers.Real): + return True + if isinstance(value, np.ndarray): + if value.dtype.kind not in "iuf": + return False + items = value.ravel().tolist() + elif isinstance(value, (list, tuple)): + items = list(value) + if not items or not all( + isinstance(v, numbers.Real) and not isinstance(v, (bool, np.bool_)) + for v in items + ): + return False + else: + return False + nodes = set(rbd.nodes) + return not any(v in nodes for v in items) + + +def _times_first(target: str = "x"): + """Let a repairable diagram's measure take its times first, as a + non-repairable diagram's does (#224): a number, or numbers none of + which is a node, given where the first argument goes (node names, or + Fussell-Vesely's ``fv_type``) is ``target`` (its times ``x``, or a + window's length).""" + + def decorate(method): + @functools.wraps(method) + def wrapper(self, *args, **kwargs): + if args and _times_like(self, args[0]): + if kwargs.get(target) is not None: + raise TypeError( + f"{method.__name__}() was given times both first " + f"and as {target}=: give them once, as {target}=." + ) + kwargs[target] = args[0] + args = args[1:] + return method(self, *args, **kwargs) + + return wrapper + + return decorate + + +def _fixed_length(model) -> Optional[float]: + """The length of a time that is always the same (an + ``ExactEventTime``, say), or None.""" + try: + mean = float(model_mean(model)) + if not (np.isfinite(mean) and mean > 0.0): + return None + below, above = np.asarray( + model.ff(np.array([mean * (1.0 - 1e-9), mean * (1.0 + 1e-9)])), + dtype=float, + ).ravel() + except MODEL_ERRORS: + return None + return mean if below <= 1e-12 and above >= 1.0 - 1e-12 else None + + +def _repair_rate(model) -> Optional[float]: + """The rate of an exponential repair time, ``inf`` for an instant one + (in no time), or None.""" + if _safe_mean(model) == 0.0: + return math.inf + return _constant_rate(model) + + +def _cdf(model) -> Callable[[np.ndarray], np.ndarray]: + """A model's CDF, for arrays of times (0 where it gives none).""" + + def cdf(x: np.ndarray) -> np.ndarray: + x = np.asarray(x, dtype=float) + with np.errstate(all="ignore"): + values = np.asarray(model.ff(x), dtype=float).reshape(x.shape) + return np.nan_to_num(values, nan=0.0) + + return cdf + + +def _product(a: np.ndarray, b: np.ndarray) -> np.ndarray: + """The distribution of the sum of two independent counts, with ``a`` + and ``b`` for ``0, 1, 2, ...``, its negligible tail cut.""" + from scipy.signal import fftconvolve + + out = np.maximum(fftconvolve(a, b), 0.0) + keep = np.nonzero(out > 1e-16)[0] + return out[: keep[-1] + 1] if keep.size else np.ones(1) + + +def _fleet(probabilities: np.ndarray, fleet: int) -> np.ndarray: + """The distribution of the sum of ``fleet`` independent counts, each + with ``probabilities`` for ``0, 1, 2, ...``.""" + total, power = np.ones(1), np.asarray(probabilities, dtype=float) + while fleet: + if fleet & 1: + total = _product(total, power) + fleet >>= 1 + if fleet: + power = _product(power, power) + return total / total.sum() + + +def _summed(counts) -> np.ndarray: + """The distribution of the sum of independent counts, each given by its + probabilities for ``0, 1, 2, ...``.""" + total = np.ones(1) + for count in counts: + total = _product(total, np.asarray(count, dtype=float)) + return total / total.sum() + + +def _fractions(counts: np.ndarray) -> np.ndarray: + """The fractions of simulations with each count ``0, 1, 2, ...``.""" + counts = np.asarray(counts, dtype=np.int64) + return np.bincount(counts) / len(counts) + + +def _whole(name: str, value, least: int = 1) -> int: + """``value`` as a whole number of at least ``least``, or a + ValueError naming it.""" + if ( + isinstance(value, bool) + or not isinstance(value, (int, float, np.integer, np.floating)) + or int(value) != value + or value < least + ): + raise ValueError( + f"{name} must be a whole number, {least} or more; got {value!r}." + ) + return int(value) + + +def _curve_points(value) -> Optional[int]: + """``curve_points`` checked: None, or a whole number of at least 1.""" + if value is None: + return None + if isinstance(value, bool) or not isinstance(value, (int, np.integer)): + raise ValueError( + f"curve_points must be a whole number of grid steps, got " + f"{value!r}." + ) + if value < 1: + raise ValueError(f"curve_points must be at least 1, got {value!r}.") + return int(value) + + +def _horizon(horizon) -> float: + """``horizon`` as a finite, non-negative float, or a ValueError (text + that reads as a number too, #233).""" + value = number_or_nan(horizon) + if not (np.isfinite(value) and value >= 0.0): + raise ValueError( + f"horizon must be a finite, non-negative number, got {horizon!r}." + ) + return value + + +def _discount_rate(discount_rate) -> float: + """``discount_rate`` as a float: a ValueError unless it is a finite, + non-negative number (#184).""" + number = isinstance( + discount_rate, (int, float, np.integer, np.floating) + ) and not isinstance(discount_rate, bool) + rate = float(discount_rate) if number else float("nan") + if not (np.isfinite(rate) and rate >= 0.0): + raise ValueError( + "discount_rate must be a finite, non-negative number (a " + "continuous rate per unit time of the component models), got " + f"{discount_rate!r}." + ) + return rate + + +def _horizons(horizon, rate: float) -> np.ndarray: + """``horizon``, a number or an array of them, as an array of + non-negative floats: finite, unless costs are discounted at ``rate`` + (#231), where an endless horizon is worth ``1 / rate`` of a steady + cost rate. Else a ValueError.""" + try: + values = real_array(horizon, "horizon") + except (TypeError, ValueError): + values = np.array(np.nan) + if values.size and np.all(values >= 0.0) and np.all(values < np.inf): + return values + if values.size and np.all(values >= 0.0): + if rate > 0.0: + return values + raise ValueError( + "horizon must be finite without a discount_rate: an undiscounted " + f"cost over an endless horizon is endless; got {horizon!r}." + ) + raise ValueError( + "horizon must be a finite, non-negative number (or an array of " + f"them; inf with a discount_rate), got {horizon!r}." + ) + + +#: A discount rate times the shortest of the components' mean lives +#: beyond which their failures' costs are discounted to almost nothing +#: (``exp(-20)``, 2e-9), and times a horizon beyond which the whole time +#: scale is: a rate given per year with the models in hours, say (#231). +#: A long horizon is no mistake in itself. +_DISCOUNTED_AWAY = 20.0 + + +_HORIZON_AWAY = 1000.0 + + +def _present_horizon( + horizons: np.ndarray, + rate: float, + lives: Callable[[], List[float]] = list, +) -> np.ndarray: + """What a unit cost rate over ``[0, horizon)`` is worth at the start, + discounted continuously at ``rate`` per unit time (#184): ``(1 - + exp(-rate * horizon)) / rate``, the horizon itself undiscounted, and + ``1 / rate`` for an endless one; for each of ``horizons``. Warns where + the rate discounts the costs away (#231): times the shortest of the + components' mean ``lives`` more than ``_DISCOUNTED_AWAY``, or times a + horizon more than ``_HORIZON_AWAY``.""" + if rate == 0.0: + return horizons + known = [life for life in lives() if 0.0 < life < math.inf] + finite = horizons[np.isfinite(horizons)] + span, what = 0.0, "" + if known and rate * min(known) > _DISCOUNTED_AWAY: + span, what = min(known), "the components' shortest mean life" + elif finite.size and rate * float(finite.max()) > _HORIZON_AWAY: + span, what = float(finite.max()), "the horizon" + if what: + warnings.warn( + f"discount_rate {rate:g} discounts the costs at {what} " + f"({span:g}) by a factor of exp(-{rate * span:.3g}), to " + "almost nothing: it is a continuous rate per unit time of the " + "component models. For an annual rate i with models in hours, " + "give math.log(1 + i) / 8760 (7% a year: " + f"{math.log(1.07) / 8760:.3g} an hour).", + stacklevel=outside_level(), + ) + return -np.expm1(-rate * horizons) / rate + + +#: How closely a discounted expected cost from new is worked out (#231): +#: its integral by parts, to this share of itself. +_DISCOUNT_TOLERANCE = 1e-8 + + +#: The most rounds of halving its pieces: a jump of the expected cost (a +#: scheduled event's) is closed in on by a half each round. +_DISCOUNT_ROUNDS = 60 + + +def _discount_pieces(ends: np.ndarray, horizon: float): + """The first pieces of a discounted cost's integral (see + ``RepairableRBD._discounted_cost``): from 0 to ``horizon`` in 32, + and split at each window's end, as ``(starts, stops)``.""" + edges = np.unique( + np.concatenate(([0.0], ends, np.linspace(0.0, horizon, 33))) + ) + return edges[:-1], edges[1:] + + +def _common_period(intervals: Iterable[float]) -> float: + """The least common multiple of the inspection intervals: the period + after which the schedules repeat together.""" + intervals = sorted(set(intervals)) + if len(intervals) == 1: + return intervals[0] + fractions = [Fraction(x).limit_denominator(10**6) for x in intervals] + if any( + abs(float(f) - x) > 1e-12 * x for f, x in zip(fractions, intervals) + ): + raise NotImplementedError( + f"The inspection intervals {intervals} have no common period, " + "so the long-run values cannot be averaged over one: estimate " + "them by simulation, with availability() or cost()." + ) + numerator = math.lcm(*(f.numerator for f in fractions)) + denominator = math.gcd(*(f.denominator for f in fractions)) + return numerator / denominator + + +def _sf_values(sf, x: np.ndarray) -> np.ndarray: + """A survival function's values at ``x``, as floats in ``x``'s shape.""" + with np.errstate(divide="ignore", invalid="ignore"): + return np.asarray(sf(x), dtype=float).reshape(np.shape(x)) + + +def _matched(times: np.ndarray, at: np.ndarray, values: np.ndarray): + """``values``, given at the times ``at``, summed at each of ``times`` + (sorted) that they fall on exactly; 0 at the others.""" + index = np.searchsorted(times, at) + hit = index < len(times) + hit[hit] = times[index[hit]] == at[hit] + return np.bincount(index[hit], values[hit], len(times)) + + +def _shaped(values: np.ndarray, t, times: np.ndarray): + """``values``, one per time, as a float for a scalar ``t``, else in the + shape of ``times``.""" + values = np.asarray(values, dtype=float) + if np.ndim(t) == 0: + return float(values[0]) + return values.reshape(times.shape) + + +def _squeeze_values(d: dict) -> dict: + """Convert a dict of 1-element arrays (as produced by the base RBD + importance helpers) to a dict of plain floats. Steady-state availability + has no time dimension, so the repairable importances return floats.""" + return {k: float(np.asarray(v).reshape(-1)[0]) for k, v in d.items()} + + +def _safe_ratio(numerator, denominator): + """Ratio that is 0 when the denominator is 0. + + Several criticality/importance measures divide by a total (system uptime, + downtime, failures, restorations, or a union of intervals) that can + legitimately be 0 -- e.g. when a redundant component is forced working so + the system never fails, or a component is forced broken. With nothing to + attribute, the measure is 0 rather than a NaN/inf (or a crash). + """ + return numerator / denominator if denominator else 0.0 diff --git a/repyability/rbd/_compiled.py b/repyability/rbd/_compiled.py index 9ba50c50..50d92fe6 100644 --- a/repyability/rbd/_compiled.py +++ b/repyability/rbd/_compiled.py @@ -86,7 +86,7 @@ def unsupported( components; with numba, numba's own loop also simulates age and block replacement, inspections of hidden failures, repair crews, standby groups, nested RBDs and the - capacity of a system of up to MAX_TRACED components (#155), from new + capacity of a system of up to MAX_TRACED components, from new for maintenance, inspections and nested RBDs (a run from the components' states shifts their calendars).""" if capacity is not None: @@ -126,12 +126,14 @@ def _unsupported_level( from repyability.non_repairable import NonRepairable from repyability.rbd.repairable_rbd import RepairableRBD + from . import _crews + if rbd._too_meshed() is not None: # Its structure is the graph itself (modular.GraphStructure). return "a structure too meshed to work out" if rbd.ccf_groups: return "common-cause groups" - if rbd._crews_limited() and not numba: + if _crews._crews_limited(rbd) and not numba: return "repair crews" if rbd._standby and not numba: return "standby groups" @@ -177,12 +179,15 @@ def _unsupported_maintenance( rbd, plan: _streams.Plan, prefix: tuple = () ) -> Optional[str]: """What in a run's preventive maintenance numba's loop cannot simulate: - replacement on condition, and a maintenance time whose draws cannot be + replacement on condition or on a calendar policy, and a maintenance + time whose draws cannot be streamed. (A component whose own draws cannot be is refused for that, and every drawn cost is streamed.)""" for node, schedule in rbd._preventive.items(): if schedule.policy == "condition": return "replacement on condition" + if schedule.policy == "calendar": + return "replacement on a calendar policy" path = prefix + (node,) if ( schedule.duration is not None @@ -326,24 +331,15 @@ def _continued(total, values: np.ndarray): return float(np.cumsum(np.concatenate(([total], values)))[-1]) -def _continued_rows(totals: list, values: np.ndarray) -> list: - """``_continued`` for each column of ``values`` (one row per - simulation) and its total in ``totals``.""" - from repyability.rbd._exact import ExactSum, add_columns - - if totals and isinstance(totals[0], ExactSum): - add_columns(totals, values) - return totals - stacked = np.vstack((np.asarray(totals, dtype=float)[None, :], values)) - return np.cumsum(stacked, axis=0)[-1].tolist() - - class _System: """A run's system as arrays for the compiled loop: the components' streams and initial states, the charges, and the structure function.""" def __init__(self, rbd, plan, working, broken, method: str): - from repyability.rbd.repairable_rbd import _CATEGORIES, RepairableRBD + from repyability.rbd._tally import _CATEGORIES + from repyability.rbd.repairable_rbd import RepairableRBD + + from . import _crews nodes = list(rbd.components) index = {node: c for c, node in enumerate(nodes)} @@ -552,7 +548,7 @@ def charge_of(c, key): #: the lowest rank first). level_crews = np.array( [ - here.repair_crews if here._crews_limited() else -1 + here.repair_crews if _crews._crews_limited(here) else -1 for here in level_rbds ], np.int64, @@ -973,7 +969,7 @@ def __call__(self, start: int, stop: int) -> None: def _outputs(self, size: int) -> tuple: """The output arrays of a batch of ``size`` simulations (each simulation's row is written whole by the loop).""" - from repyability.rbd.repairable_rbd import _CATEGORIES + from repyability.rbd._tally import _CATEGORIES model = self._model out = self._out @@ -1212,7 +1208,7 @@ def _capacity_into(self, tally, starts, counts, times, masks) -> None: as ``_Tally.add`` adds each simulation's: its exact totals the same in any order.""" from repyability.rbd._exact import ExactSum - from repyability.rbd.repairable_rbd import _add_at + from repyability.rbd._time_order import _add_at kept = np.arange(times.shape[1]) < counts[:, None] visited = np.unique(np.concatenate([starts, masks[kept]])) diff --git a/repyability/rbd/_conditional.py b/repyability/rbd/_conditional.py index 64ca200f..4af1077a 100644 --- a/repyability/rbd/_conditional.py +++ b/repyability/rbd/_conditional.py @@ -514,15 +514,6 @@ def module_totals( return uptime, n * float(end) - uptime -def curve_se(curve: np.ndarray, square: np.ndarray, n: int) -> np.ndarray: - """The pointwise standard error of a conditional curve: of the mean - of the simulations' values, from their mean square.""" - if n < 2: - return np.zeros_like(curve) - variance = np.maximum(square - curve * curve, 0.0) * n / (n - 1) - return np.sqrt(variance / n) - - def steps(curve_points: Optional[int], end: float) -> np.ndarray: """The curve's times: ``curve_points`` steps (by default ``CURVE``).""" points = CURVE if curve_points is None else int(curve_points) @@ -553,7 +544,7 @@ def partial_bytes( nodes: List[Any], priced: bool, ) -> bytes: - """A module shard's partial (#189): the modules' histories in its + """A module shard's partial: the modules' histories in its simulations ``start`` to ``stop - 1``, each simulation's own cost (if ``priced``), and the costs by category and by module (in the order of ``nodes``), as the bytes of a NumPy ``.npz`` file, read without diff --git a/repyability/rbd/_costs.py b/repyability/rbd/_costs.py new file mode 100644 index 00000000..ba55a70d --- /dev/null +++ b/repyability/rbd/_costs.py @@ -0,0 +1,173 @@ +"""A ``RepairableRBD``'s long-run costs: the expected cost rate, with each +node's share and its actions (``expected_cost_rate``), the cost of buying +the components (``acquisition_cost``) and the total cost over a horizon, +discounted or not (``total_cost``). The methods of ``RepairableRBD`` of +those names call these. +""" + +from functools import partial +from typing import ( + Collection, + Hashable, + Optional, + Tuple, + Union, +) + +import numpy as np + +from repyability.rbd import _crews, _long_run, _requirements +from repyability.rbd._common import ( + _discount_rate, + _horizons, + _present_horizon, + _squeeze_values, +) +from repyability.rbd._spec import _mean_cost + + +def expected_cost_rate( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> float: + """See ``RepairableRBD.expected_cost_rate``.""" + _requirements._require_costs(rbd) + if not rbd.has_costs: + return 0.0 # an acquisition cost alone: no running cost + + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + forced = set(working_nodes) | set(broken_nodes) + setups = [group for group in rbd._maintenance.values() if group.setup_cost] + if setups: + _requirements._require_separate_setups(rbd) + + rate = 0.0 + + # Production lost while the *system* is down. + if rbd.downtime_cost_rate: + unavailability = rbd.mean_unavailability(working_nodes, broken_nodes) + rate += rbd.downtime_cost_rate * unavailability + + if not rbd.costs and not setups: + return rate + + if rbd._crews_couple(): + # Held working or broken, a node needs no crew, and the others + # have more of them: from the chain without it. + probabilities, weights = _crews._chain_probabilities( + rbd, working_nodes, broken_nodes + ) + node_availability = { + node: float(weights @ probabilities[node]) for node in rbd.nodes + } + else: + node_availability = _squeeze_values( + rbd._probabilities_with_overrides( + rbd.node_availability(), working_nodes, broken_nodes + ) + ) + for node in rbd.costs: + rate += _node_cost_rate( + rbd, node, node_availability[node], node in forced + ) + # A maintenance group's set-up, at each failure and each preventive + # replacement of a member (none is renewed early: that would have + # been refused), a forced member making none. + for group in setups: + rate += group.setup_cost * sum( + sum(_node_actions(rbd, node, node_availability[node])) + for node in group.members + if node not in forced + ) + return rate + + +def _node_cost_rate( + rbd, node, availability: float, forced: bool = False +) -> float: + """A component's own running cost per unit time, in the long run: + its corrective, preventive and inspection actions, and its own + downtime (``availability`` its long-run availability). A forced node + never changes state, so it incurs no actions.""" + node_costs = rbd.costs.get(node, {}) + rate = 0.0 + # Corrective actions, charged per failure, and preventive ones. + per_action = sum( + _mean_cost(node_costs[key]) + for key in rbd.PER_FAILURE_COST_KEYS + if key in node_costs + ) + preventive = node_costs.get("preventive_cost") + if (per_action or preventive is not None) and not forced: + failures, maintained = _node_actions(rbd, node, availability) + rate += per_action * failures + if preventive is not None: + rate += _mean_cost(preventive) * maintained + inspection = node_costs.get("inspection_cost") + if inspection is not None and not forced: + schedule = rbd._preventive.get(node) + if schedule is not None and schedule.policy == "condition": + # Inspected at each multiple of the interval at which it is + # up (one in a repair or replacement is not). + up = _long_run._block_cycle(rbd, node).before + rate += _mean_cost(inspection) * up / schedule.interval + else: + # One test per interval, but those that fall in a repair, + # which are not done. + unit = _requirements._tested_unit(rbd, node) + tests = ( + 1.0 / rbd._inspection[node].interval + if unit is None + else unit.long_run.inspections + ) + rate += _mean_cost(inspection) * tests + # Optional cost of *this component* being down, whether or not the + # system as a whole is. + downtime_cost = node_costs.get("downtime_cost", 0.0) + if downtime_cost: + rate += downtime_cost * (1.0 - availability) + return rate + + +def _node_actions(rbd, node, availability: float) -> Tuple[float, float]: + """A component's corrective and preventive actions per unit time, + in the long run (``availability`` its long-run availability).""" + if rbd._crews_couple(): + # Waiting for a crew, as while repaired, it cannot fail: it + # fails at its constant rate while it is up. + life = _crews._crew_chain_rates(rbd)[node][0] + return life * availability, 0.0 + if node in rbd._standby: + # Each of its units' failures is a repair. + return ( + _long_run._standby_long_run(rbd, node).unit_failure_frequency, + 0.0, + ) + failures, maintained, _ = _long_run._node_frequencies(rbd, node) + return failures, maintained + + +def acquisition_cost(rbd) -> float: + """See ``RepairableRBD.acquisition_cost``.""" + return float(sum(rbd.acquisition_costs.values())) + + +def total_cost( + rbd, + horizon, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + discount_rate: float, +) -> Union[float, np.ndarray]: + """See ``RepairableRBD.total_cost``.""" + _requirements._require_costs(rbd) + rate = _discount_rate(discount_rate) + present = _present_horizon( + _horizons(horizon, rate), rate, partial(_requirements._mean_lives, rbd) + ) + total = rbd.acquisition_cost + present * rbd.expected_cost_rate( + working_nodes, broken_nodes + ) + return float(total) if np.ndim(total) == 0 else total diff --git a/repyability/rbd/_crew_chain.py b/repyability/rbd/_crew_chain.py index 4df72a03..e70f3acb 100644 --- a/repyability/rbd/_crew_chain.py +++ b/repyability/rbd/_crew_chain.py @@ -51,7 +51,7 @@ class CrewChain(NamedTuple): ``_chain_transient``). ``transitions`` are its transitions one by one, ``(sources, targets, rates, components)``: each a component's failure or the end of its repair (with the jobs the crew then takes that are - done at once), labelled with that component's position (#199). + done at once), labelled with that component's position. """ nodes: Tuple[Hashable, ...] @@ -70,7 +70,7 @@ def split(self, vector: np.ndarray) -> np.ndarray: """``Q_i v`` for each component ``i`` (columns), ``Q_i`` the part of the generator made of its transitions: ``Q = sum_i Q_i``, so that ``p(t) Q_i v`` is the part of ``d/dt p(t) v`` the component's - failures and repairs make (#199).""" + failures and repairs make.""" sources, targets, rates, components = self.transitions out = np.zeros((len(self.probabilities), len(self.nodes))) vector = np.asarray(vector, dtype=float) diff --git a/repyability/rbd/_crews.py b/repyability/rbd/_crews.py new file mode 100644 index 00000000..2cdc733f --- /dev/null +++ b/repyability/rbd/_crews.py @@ -0,0 +1,254 @@ +"""A ``RepairableRBD``'s repair crews in the exact methods: whether the +crews limit or couple the components, and the crews' Markov chain +(``_crew_chain``, which copies the simulation's queue) with its rates, +state probabilities and outage terms in the long run. +``RepairableRBD``'s methods call these. +""" + +import math +from typing import ( + Any, + Dict, + List, + NoReturn, + Tuple, +) + +import numpy as np + +from repyability.rbd import ( + _ccf_groups, +) +from repyability.rbd import _crew_chain as crew_chain +from repyability.rbd import _long_run, _requirements +from repyability.rbd._common import ( + _constant_rate, + _repair_rate, +) +from repyability.rbd._events import ( + _Unit, +) + + +def _crew_served(rbd) -> list: + """The jobs the repair crews work on, by key: this RBD's own + components, not a nested RBD's (which has crews of its own), and + each unit of a standby group (``_Unit(node, unit)``).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + served: list = [] + for node, component in rbd.components.items(): + if isinstance(component, RepairableRBD): + continue + arrangement = rbd._standby.get(node) + if arrangement is None: + served.append(node) + else: + served.extend( + _Unit(node, unit) for unit in range(arrangement.units) + ) + return served + + +def _crews_limited(rbd) -> bool: + """Whether there are fewer repair crews than components that may + need one, so that a job may wait.""" + return rbd.repair_crews is not None and rbd.repair_crews < len( + _crew_served(rbd) + ) + + +def _require_unlimited_crews( + rbd, + assumes: str = "these values assume", + advice: str = "Simulate the system with availability() or cost().", +) -> None: + """Raise if a job may wait for a repair crew: components then no + longer fail and recover independently, which ``assumes`` (what + assumes it, and ``assume``): do ``advice`` instead.""" + if rbd._crews_couple(): + raise NotImplementedError( + f"With {rbd.repair_crews} repair crew(s) for " + f"{len(_crew_served(rbd))} components, a component can " + "wait for a crew, so the components no longer fail and " + f"recover independently, which {assumes}. {advice}" + ) + + +def _no_crew_chain(rbd, why: str) -> NoReturn: + """Raise that the repair crews' Markov chain does not cover this RBD, + because ``why``.""" + raise NotImplementedError( + f"With {rbd.repair_crews} repair crew(s) for " + f"{len(_crew_served(rbd))} components, a component can wait " + "for a crew, and the exact long-run values come from a Markov " + "chain of the components' states and the repair queue: " + f"{why}. Simulate the system with availability() or cost()." + ) + + +def _crew_chain_rates(rbd) -> Dict[Any, Tuple[float, float]]: + """The failure and repair rates (``inf`` for an instant repair) of + the components the repair crews work on, for their Markov chain + (see ``_crew_chain.py``). Raise if the chain does not cover them: + scheduled maintenance or inspection, a life or repair time that is + not exponential, or more states than it is solved for.""" + assert rbd.repair_crews is not None # limited crews only + for node in rbd._standby: + _no_crew_chain( + rbd, + "it has no place for a standby group, which component " + f"{node!r} is", + ) + rates: Dict[Any, Tuple[float, float]] = {} + for node in _crew_served(rbd): + component = rbd.components[node] + if node in rbd._preventive: + _no_crew_chain( + rbd, + "it has no place for scheduled maintenance, which " + f"component {node!r} has", + ) + if node in rbd._inspection: + _no_crew_chain( + rbd, + "it has no place for inspections, which component " + f"{node!r} has", + ) + if node in rbd._imperfect: + _no_crew_chain( + rbd, + "it has no place for imperfect repair, which component " + f"{node!r} has", + ) + life = _constant_rate(component.reliability) + if life is None: + _no_crew_chain( + rbd, + "it needs exponential lives (a constant failure rate), " + f"and the life of component {node!r} is not one", + ) + repair = _repair_rate(component.time_to_replace) + if repair is None: + _no_crew_chain( + rbd, + "it needs exponential repair times (or instant repair), " + f"and the repair times of component {node!r} are not", + ) + rates[node] = (life, repair) + count = crew_chain.state_count( + [rbd._priority.get(node, 0.0) for node in rates], + [math.isinf(repair) for _, repair in rates.values()], + rbd.repair_crews, + ) + if count > crew_chain.MAX_STATES: + _no_crew_chain( + rbd, + f"here it has {count:,} states, more than the " + f"{crew_chain.MAX_STATES:,} it is solved for", + ) + return rates + + +def _require_crew_chain(rbd) -> None: + """Raise if the exact long-run values with limited repair crews + cannot be computed: the Markov chain does not take common-cause + groups in (#251, see ``_ccf_rates``), does not cover the components + (see ``_crew_chain_rates``), or nested RBDs' calendars fall together + (see ``_require_calendars``).""" + _ccf_groups._require_ccf_long_run(rbd) + _crew_chain_rates(rbd) + _requirements._require_calendars(rbd) + + +def _crew_chain( + rbd, forced: frozenset = frozenset() +) -> "crew_chain.CrewChain": + """The Markov chain of the components the repair crews work on, and + its long-run distribution (see ``_crew_chain.py``), leaving out the + nodes in ``forced``: held working a node never fails, and held + broken it is never repaired, so neither needs a crew. Solved once + for each set of rates and kept.""" + _require_crew_chain(rbd) + rates = _crew_chain_rates(rbd) + assert rbd.repair_crews is not None # limited crews only + nodes = [node for node in rates if node not in forced] + priorities = [rbd._priority.get(node, 0.0) for node in nodes] + key = ( + rbd.repair_crews, + tuple(zip(nodes, (rates[node] for node in nodes), priorities)), + ) + cache = rbd._cache.kept("crew_chains") + if key not in cache: + cache[key] = crew_chain.solve( + nodes, + [rates[node][0] for node in nodes], + [rates[node][1] for node in nodes], + priorities, + rbd.repair_crews, + ) + return cache[key] + + +def _chain_probabilities( + rbd, working_nodes, broken_nodes +) -> Tuple[dict, np.ndarray]: + """With limited repair crews, every node's availability in each + state of their Markov chain (1 or 0 for the components the crews + work on and those held working or broken, and a nested RBD's own + long-run availability, as it has crews of its own), and the states' + long-run probabilities: the long-run values are then averages over + the states, as over the times of ``_long_run_grid``.""" + working_nodes = set(working_nodes or ()) + broken_nodes = set(broken_nodes or ()) + rbd._validate_node_overrides(working_nodes, broken_nodes) + chain = _crew_chain(rbd, frozenset(working_nodes | broken_nodes)) + size = len(chain.probabilities) + out: dict = { + node: chain.up[:, k].astype(float) + for k, node in enumerate(chain.nodes) + } + for node, component in rbd.components.items(): + if node in working_nodes: + out[node] = np.ones(size) + elif node in broken_nodes: + out[node] = np.zeros(size) + elif node not in out: + out[node] = np.full(size, float(component.mean_availability())) + for node in rbd.in_or_out: + out[node] = np.ones(size) + return out, chain.probabilities + + +def _chain_outage_terms( + rbd, working_nodes, broken_nodes +) -> Tuple[List[Tuple[Any, float]], float]: + """``_outage_terms`` with limited repair crews, over the + states of their Markov chain: in each, a component that is up fails + at its constant rate, and takes the system down if it is critical + there. A nested RBD enters through its own frequencies, as it has + crews of its own.""" + availability, unavailability, weights = ( + _long_run._long_run_unavailabilities(rbd, working_nodes, broken_nodes) + ) + forced = set(working_nodes or ()) | set(broken_nodes or ()) + rates = _crew_chain_rates(rbd) + birnbaum = rbd._birnbaum_importance( + availability, node_failures=unavailability + ) + terms: List[Tuple[Any, float]] = [] + planned = 0.0 + for node in rbd.components: + if node in forced: + continue + importance = np.asarray(birnbaum[node]) + if node in rates: + life = rates[node][0] + node_failures, node_planned = life * availability[node], 0.0 + else: + node_failures, _, node_planned = _long_run._node_frequencies( + rbd, node + ) + terms.append((node, float(weights @ (importance * node_failures)))) + planned += float(weights @ (importance * node_planned)) + return terms, planned diff --git a/repyability/rbd/_curves.py b/repyability/rbd/_curves.py new file mode 100644 index 00000000..b1e305d0 --- /dev/null +++ b/repyability/rbd/_curves.py @@ -0,0 +1,1784 @@ +"""A ``RepairableRBD``'s components over time: each node's availability +curve (a plain unit's renewal curve, block replacement's, a tested +unit's, a standby group's, a nested RBD's, and the repair crews' and +common-cause groups' chains where they couple the components), the +states a run or an analysis starts from, and when the curves settle to +their long run (``_settling``). The values over time (``_windows``) and +the simulations' stand-ins are worked out from these. +""" + +import functools +import math +from collections.abc import Mapping +from functools import partial +from typing import ( + Dict, + Hashable, + NoReturn, + Optional, + Tuple, +) + +import numpy as np + +from repyability.non_repairable import NonRepairable +from repyability.rbd import ( + _ccf_groups, + _chain_transient, + _crews, + _long_run, + _repairable_capacity, + _requirements, + _standby_chain, + _windows, +) +from repyability.rbd._block_replacement import ( + BlockHead, + block_availability, +) +from repyability.rbd._common import ( + _common_period, + _constant_rate, + _repair_rate, + _sf_values, +) +from repyability.rbd._condition_replacement import ( + ConditionHead, + condition_availability, +) +from repyability.rbd._hidden_life import ( + TestedLife, + TestedLifeCurve, + TestedLifeSteady, +) +from repyability.rbd._hidden_tests import TestedUnit +from repyability.rbd._long_run import ( + _TESTED_CYCLE, +) +from repyability.rbd._model_utils import ( + MODEL_ERRORS, + failure_time_scale, + is_fixed_probability, + model_mean, +) +from repyability.rbd._point_availability import ( + BlockCurve, + First, + InspectionCurve, + MinimalRepairCurve, + PartialTestCurve, + ShiftedCurve, + StartedBlockCurve, + SteadyCurve, + SystemCurve, +) +from repyability.rbd._point_availability import knots as point_knots +from repyability.rbd._point_availability import unit_curve +from repyability.rbd._sampling import stream_sampler +from repyability.rbd.node_state import NodeState +from repyability.rbd.routes import AnalysisRoute +from repyability.utils.checks import ( + structure_method, +) + + +def _draws_start(start) -> bool: + """Whether a component started from ``start`` (a ``NodeState``, or + None for new) draws what is left of its life, repair or maintenance: + when it is down, or up at an age. Up at age 0, it is new, whatever its + ``since_scheduled``.""" + return isinstance(start, NodeState) and (not start.alive or start.age > 0) + + +#: Why a simulation does not start in the long-run state. +_STATIONARY_SIMULATION = ( + "A simulation starts from given states, not from the long-run one: give " + "each component's NodeState (its age, or how long it has been down). " + "The exact methods (point_availability, mission_availability, " + "expected_failures, expected_events, expected_cost, point_capacity and " + "mission_capacity) take state='stationary'." +) + + +#: The most nested RBDs the crews' chain is followed over time with: it +#: is solved for each pattern of them up and down. +_MAX_CREW_NESTED = 10 + + +#: How a system's values over time take in its common-cause groups (#158). +_GROUPS_OVER_TIME = ( + " Each common-cause group's members' joint states come from its Markov " + "chain followed from every member up at 0 (by uniformization, or through " + "its tests' first period, after which they repeat the long run's), and " + "the system is worked out over the combinations of its members up and " + "down, the other nodes independent of them; the members' failures by " + "each cause, at its rate where it takes the system down." +) + + +#: Grid steps over a component's typical up time, for its point +#: availability (see ``_point_availability``): the error falls as the +#: square of the step, to about 4e-8 here in a mission average. +_POINT_STEPS = 1000 + + +#: The most grid points for one component's point availability. +_POINT_MAX = 2**22 + + +#: The most pieces ``mission_availability`` integrates over. +_MISSION_POINTS = 5_000_000 + + +def _settling_end(curve, long_run: Optional[float], cycle: float, end: float): + """How far to follow a component's curve that has not settled at its + long-run value ``long_run`` by ``end`` (see ``_unit_curve``). Its + distance from that value falls by about the same factor each + ``cycle`` (an up and a down time), so the largest distances in its + last two cycles say when it will be within 1e-10: the curve is followed + until that point is three quarters of the way along, as the check + wants, and a cycle more. Four times as far where they do not say (a + distance that does not fall, or too few cycles followed).""" + grow = 4.0 * end + if long_run is None or not ( + np.isfinite(cycle) and 0.0 < 2.0 * cycle < end + ): + return grow + t = curve.times + off = np.abs(curve.at(t) - long_run) + last = t[-1] + recent = float(off[t > last - cycle].max()) + before = float(off[(t > last - 2.0 * cycle) & (t <= last - cycle)].max()) + if not 0.0 < recent < before: + return grow + cycles = max(math.log(1e-10 / recent) / math.log(recent / before), 0.0) + settled = last + cycles * cycle + return min(grow, max(1.25 * end, settled / 0.75 + cycle)) + + +def _up_scale(model, age: Optional[float]) -> float: + """A typical up time for a unit with lifetime ``model`` (replaced at + ``age``, if given), to size its point-availability grid by: the smaller + of its median and the width of its middle 80%; else its typical failure + time (NaN if it has none, e.g. it never fails).""" + scale = float("nan") + qf = getattr(model, "qf", None) + if qf is not None: + try: + with np.errstate(all="ignore"): + q10, q50, q90 = np.ravel( + np.asarray(qf(np.array([0.1, 0.5, 0.9])), dtype=float) + ) + widths = [w for w in (q50, q90 - q10) if np.isfinite(w) and w > 0] + if widths: + scale = min(widths) + except MODEL_ERRORS: # a model whose qf cannot take these + pass + if not np.isfinite(scale): + try: + scale = failure_time_scale(model) + except MODEL_ERRORS: # no mean either (e.g. never fails) + scale = float("nan") + if age is not None: + scale = age if not np.isfinite(scale) else min(scale, age) + return float(scale) + + +def _settling(curves) -> Tuple[float, Optional[float]]: + """When the point availabilities ``curves`` (see + ``_point_availability``) all settle: the time after which they are + constant (a period of None) or repeat together with the period returned + (a time of inf if they never do, as far as is known).""" + curves = list(curves) + settle = max((float(curve.settle) for curve in curves), default=0.0) + periods = {curve.period for curve in curves if curve.period is not None} + if not periods: + return settle, None + try: + return settle, _common_period(periods) + except NotImplementedError: # no common period + return np.inf, None + + +def _node_over_time(rbd, node, kind: str = "availability") -> Tuple[str, str]: + """How a component's availability over time is found, for ``kind`` + (see ``_over_time``): the route, and a phrase saying how (the + message it raises, if refused).""" + from repyability.rbd import routes as r + from repyability.rbd.repairable_rbd import RepairableRBD + + component = rbd.components[node] + if isinstance(component, RepairableRBD): + if ( + kind == "capacity" + and component._crews_couple() + and component._has_capacity() + ): + return r.REFUSED, str( + r.refusal( + partial(_no_crew_window, component, None, "the capacities") + ) + ) + inner = _over_time(component, kind=kind) + if inner.route == r.REFUSED: + return r.REFUSED, inner.reason + return inner.route, f"a nested RBD's availability, {inner.route}" + if node in rbd._standby: + message = r.refusal(partial(_long_run._standby_rates, rbd, node)) + if message: + return r.REFUSED, message + return ( + r.NUMERICAL, + "its units' Markov chain, followed over time by " "uniformization", + ) + message = r.refusal(partial(_requirements._require_time_models, rbd, node)) + if not message: + message = r.refusal( + partial(_requirements._require_no_opportunities, rbd, node) + ) + if not message and node in rbd._imperfect: + message = r.refusal( + partial(_requirements._require_minimal_repair, rbd, node) + ) + if message: + return r.REFUSED, message + if _requirements._repairs_take_time(component): + return ( + r.NUMERICAL, + "minimal repair with repairs that take time: its operating " + "age followed on a grid, each failure's repair convolved in", + ) + return ( + r.EXACT, + "minimal repair in no time: up throughout, failing as often " + "as its life's cumulative hazard", + ) + if not message and node in rbd._inspection: + message = r.refusal( + partial(_requirements._require_tested_exact, rbd, node) + ) + schedule = rbd._preventive.get(node) + calendar = schedule is not None and schedule.policy != "age" + if not message and calendar: + message = r.refusal( + partial(_requirements._require_block_models, rbd, node) + ) + if message: + return r.REFUSED, message + if calendar and schedule.policy == "condition": # type: ignore + return ( + r.NUMERICAL, + "replacement on condition: followed from one inspection to " + "the next on a grid, the units kept by age", + ) + if ( + node in rbd._inspection + and _requirements._tested_kind(rbd, node) == "unit" + ): + return r.NUMERICAL, _TESTED_CYCLE + " over time" + return r.NUMERICAL, "its renewal equation, solved on a grid" + + +def _over_time( + rbd, + reason: str = ( + "Each component's renewal equation solved on a grid (to about " + "1e-7), and the system exactly at its components' availabilities " + "at each time." + ), + kind: str = "availability", +) -> "AnalysisRoute": + """How the availability over time from new is found (see + ``analysis_routes``), and what is built on it: the method's + ``reason``. ``kind`` is what is asked: the ``"availability"``, the + expected events over a ``"window"``, or the ``"capacity"``, which + with limited repair crews come from their chain (see + ``_crew_over_time``).""" + from repyability.rbd import routes as r + + if rbd._crews_couple(): + return _crew_over_time(rbd, kind) + if rbd.ccf_groups: + refusal = r.refusal( + partial( + _ccf_groups._require_groups_over_time, + rbd, + {}, + ) + ) + if refusal: + return r.refused(refusal) + reason = reason + _GROUPS_OVER_TIME + nodes = {node: _node_over_time(rbd, node, kind) for node in rbd.components} + refusals = { + n: how for n, (route, how) in nodes.items() if route == r.REFUSED + } + if refusals: + return r.refused(next(iter(refusals.values())), tuple(refusals)) + return r.with_nodes(r.NUMERICAL, reason, nodes, "availabilities") + + +def _crew_over_time(rbd, kind: str) -> "AnalysisRoute": + """How ``kind`` over time (see ``_over_time``) is found with limited + repair crews: from their chain, followed over time by + uniformization (see ``_crew_curve``), unless it refuses.""" + from repyability.rbd import routes as r + + refusal = r.refusal( + partial( + functools.partial(_require_crew_over_time, rbd), frozenset(), kind + ) + ) + if refusal: + return r.refused(refusal) + chain = ( + "The Markov chain of the components' states and the repair " + "queue, followed over time from its state at 0 by " + "uniformization (to about 1e-13): " + ) + nested = _crew_nested(rbd, frozenset()) + if kind == "window" and nested: + reason = chain + ( + "the system's expected failures are those of its " + "components, at their rates in each state, and of each " + "nested RBD, independent of the chain, at its importance " + "over the chain and the other nested RBDs' patterns, " + "integrated by quadrature." + ) + elif kind == "window": + reason = chain + ( + "the system's expected failures, and each component's " + "failures and down time, are the integrals of their rates " + "over its states." + ) + elif kind == "rate": + reason = chain + ( + "the system's rate of change is split by the component " + "whose failure or repair makes each transition (a crew " + "taking the next job belongs to the repair that freed it), " + "each part one more vector of the chain, exact" + + ( + "; a nested RBD's part is its importance over the chain " + "and the other nested RBDs' patterns times its own rate" + if nested + else "" + ) + + "." + ) + elif kind == "capacity" and nested: + reason = chain + ( + "the capacity distribution is the probability of the states " + "at each level, for each combination of the nested RBDs' " + "levels, weighted by their own distributions, and over a " + "mission its integral, by quadrature." + ) + elif kind == "capacity": + reason = chain + ( + "the capacity distribution is the probability of the states " + "at each level, and over a mission its integral." + ) + elif nested: + reason = chain + ( + "the system's availability is the probability of the states " + "it is up in, worked out for each pattern of the nested RBDs " + "up and down with their own availabilities, and the mission " + "availability its integral, by quadrature." + ) + else: + reason = chain + ( + "the system's availability is the probability of the states " + "it is up in, and the mission availability its integral, " + "exactly." + ) + inner = "availability" if kind == "rate" else kind + nodes = {node: _node_over_time(rbd, node, inner) for node in nested} + refusals = { + n: how for n, (route, how) in nodes.items() if route == r.REFUSED + } + if refusals: + return r.refused(next(iter(refusals.values())), tuple(refusals)) + return r.with_nodes(r.NUMERICAL, reason, nodes, "availabilities") + + +def _require_crew_over_time( + rbd, forced=frozenset(), kind: str = "availability" +) -> list: + """Raise what the crews' chain over time refuses, in the order the + methods check it: common-cause groups, what the chain does not + cover (see ``_require_crew_chain``), and too many nested RBDs (see + ``_crew_nested``). Return the nested RBDs, less those in + ``forced``.""" + _ccf_groups._require_ccf_long_run(rbd) + _crews._require_crew_chain(rbd) + return _crew_nested(rbd, forced) + + +def _availability_curves( + rbd, + horizon: float, + skip, + counts: bool = False, + stages: bool = False, + state: Optional[dict] = None, + groups: bool = False, +) -> dict: + """Each node's point availability over ``[0, horizon]``, from new, + as a curve (see ``_point_availability``), except the nodes in + ``skip`` (held working or failed). With ``counts``, the curves + count the nodes' expected events too, and follow them until they + have settled as well; with ``stages``, a degrading component's + curve follows its stages (see ``_capacity_models``). ``state`` + (checked by ``_states``) starts the nodes it names from their + states rather than new. With limited repair crews the nodes are not + independent: their values over time come from the crews' chain + instead (see ``_crew_curve``). Nor are the members of common-cause + groups, though each member's own curve is as without its group: + ``groups`` says the caller takes them in (see ``_groups_curve``).""" + assert not rbd._crews_couple(), "the crews' chain, not curves" + assert groups or not rbd.ccf_groups, "the groups' curve, not curves" + state = state or {} + curves: dict = {} + degrading = rbd._capacity_models() if stages else {} + memo = rbd._curve_memo + # The plain units whose curves are built, for their twins. + plain: list = [] + for node, component in rbd.components.items(): + if node in skip: + continue + start = state.get(node) + key: Optional[tuple] = (node, float(horizon), stages, start) + try: + hash(key) + except TypeError: # a nested RBD's own states, as a dict + key = None + if memo is not None and key is not None: + # A curve that counts its events serves one that need not. + found = memo.get((key, True)) + if found is None and not counts: + found = memo.get((key, False)) + if found is not None: + curves[node] = found + continue + twin = _curve_twin(rbd, node, start, node in degrading, plain) + if twin is not None: + curves[node] = curves[twin] + else: + curves[node] = _node_curve( + rbd, + node, + component, + horizon, + counts, + node in degrading, + stages, + start, + ) + if _plain_unit(rbd, node): + plain.append((node, start, node in degrading)) + if memo is not None and key is not None: + memo[(key, counts)] = curves[node] + return curves + + +def _plain_unit(rbd, node) -> bool: + """Whether a component's curve follows from its life and repair + models alone (and its state at 0): a unit with no schedule, tests, + imperfect repair or maintenance group, not a nested RBD or a + standby group. Another with the same models has the same curve.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + return not ( + isinstance(rbd.components[node], RepairableRBD) + or node in rbd._standby + or node in rbd._inspection + or node in rbd._preventive + or node in rbd._imperfect + or node in rbd._member_group + ) + + +def _curve_twin(rbd, node, start, degrading: bool, plain: list): + """A component among ``plain`` (each ``(node, start, degrading)``, + a plain unit whose curve is built) whose curve is ``node``'s: the + same life and repair models, the same state at 0 and stages followed + alike; or None. Identical components (a bank of pumps) then share + one curve.""" + if not plain or not _plain_unit(rbd, node): + return None + from repyability.rbd.non_repairable_rbd import NonRepairableRBD + + same = NonRepairableRBD._same_model + component = rbd.components[node] + for other, other_start, other_degrading in plain: + known = rbd.components[other] + if ( + other_degrading == degrading + and other_start == start + and same(known.reliability, component.reliability) + and same(known.time_to_replace, component.time_to_replace) + ): + return other + return None + + +def _node_curve( + rbd, + node, + component, + horizon: float, + counts: bool, + degrading: bool, + stages: bool, + start, +): + """A node's curve for ``_availability_curves``: a nested RBD's, a + standby group's, a tested component's or any other unit's.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if isinstance(component, RepairableRBD): + return _nested_curve(component, horizon, counts, stages, start) + if node in rbd._standby: + return _standby_curve(rbd, node, start) + if node in rbd._imperfect or node not in rbd._inspection: + # An imperfectly repaired component's goes through + # _minimal_repair_curve, which refuses one whose failures are + # found only by its tests (as _node_over_time does). + return _unit_curve(rbd, node, horizon, counts, degrading, start) + unit = _requirements._tested_unit(rbd, node) + if unit is not None: + return _tested_unit_curve(rbd, node, unit, horizon, start) + life = _requirements._tested_life(rbd, node) + if life is not None: + return _tested_life_curve(rbd, node, life, start) + rate, interval = _requirements._inspected_rate(rbd, node) + inspection = rbd._inspection[node] + if inspection.partial: + # From new (its state is not taken: see _check_state). + return PartialTestCurve( + rate, + interval, + inspection.offset, + inspection.coverage, + inspection.per_full_test, + ) + if start is None and inspection.offset: + # New at 0, first tested at the offset: on a calendar whose last + # test was interval - offset before 0, last known up at 0. + return InspectionCurve( + rate, interval, interval - inspection.offset, 0.0 + ) + if start is not None and not start.alive: + raise NotImplementedError( + f"Component {node!r} has hidden failures, repaired at once " + "when a test finds them, as the exact values need: it cannot " + "be down. Give the time since its last test as its " + "since_scheduled (it was found working), or simulate it: " + "availability(state=...)." + ) + phase = 0.0 if start is None else start.since_scheduled or 0.0 + # Last known up at its last test, or, put into service since, when + # it was. + since = ( + phase if start is None or start.stationary else min(start.age, phase) + ) + return InspectionCurve(rate, interval, phase, since) + + +def _tested_unit_curve(rbd, node, unit: TestedUnit, horizon: float, start): + """A tested component's point availability and events from 0 when + its tests or repairs take time, or its tests can miss a failure of + a life that is not exponential (see ``_hidden_tests``): new at 0, + first tested at its offset (or after an interval); in its long-run + state; up at the age its state gives, known up at its last test (or + when put into service since); or in a repair, under way for its + ``down_for``. From a state its tests are on the calendar its phase + places (``k T - phase``), as the simulation's.""" + inspection = rbd._inspection[node] + if start is None or start.new: + if inspection.offset: + return unit.curve(horizon, inspection.offset, 0) + return unit.curve(horizon, 0.0, 1) + phase = start.since_scheduled or 0.0 + if start.stationary: + return unit.curve(horizon, -phase, 1, ("stationary",)) + if not start.alive: + return unit.curve(horizon, -phase, 1, ("down", start.down_for)) + since = min(start.age, phase) + return unit.curve(horizon, -phase, 1, ("alive", start.age, since)) + + +def _tested_life_curve(rbd, node, life: TestedLife, start): + """A tested component's point availability and events from 0, for + a life other than exponential (see ``_hidden_life``): new at 0 + (first tested at its offset, or after an interval), in its long-run + state, or known up at its last test (or since put into service) at + the age its state gives.""" + inspection = rbd._inspection[node] + interval = inspection.interval + if start is None: + return TestedLifeCurve(life, inspection.offset or interval) + # Up: its repairs take no time (see _check_state). + phase = start.since_scheduled or 0.0 + if start.stationary: + return TestedLifeSteady(life, phase) + model = life.model + # Last known up at its last test (or when put into service since), + # at age ``known_age``; at 0 it is ``start.age`` old. + since = min(start.age, phase) + known_age = np.array([start.age - since]) + known_up = float(_sf_values(model.sf, known_age)[0]) + known_down = float(np.atleast_1d(model.ff(known_age))[0]) + + def initial(x): + later = start.age + np.asarray(x, dtype=float) + return _sf_values(model.sf, later) / known_up + + def initial_down(x): + later = start.age + np.asarray(x, dtype=float) + if known_down <= 0.5: + failed = np.asarray(model.ff(later), dtype=float) - known_down + else: + failed = known_up - _sf_values(model.sf, later) + return np.clip(failed / known_up, 0.0, 1.0) + + first = interval - phase if phase else interval + return TestedLifeCurve(life, first, initial, initial_down) + + +def _states(rbd, state, forced=frozenset(), simulated=False) -> dict: + """``state`` checked (see ``point_availability``): each component it + names, its ``NodeState``, and each nested RBD, its components' + states, in the same form; for ``"stationary"``, every component in + its long-run state. Nodes in ``forced`` (held working or broken) + take none. ``simulated``: for a simulation, which takes an + imperfectly repaired component's virtual age (#269).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if state is None: + return {} + if isinstance(state, str): + if state != "stationary": + raise ValueError( + "state must be a dict of NodeStates by node, or " + f"'stationary', got {state!r}." + ) + return { + node: ( + _states(component, "stationary") + if isinstance(component, RepairableRBD) + else NodeState(stationary=True) + ) + for node, component in rbd.components.items() + if node not in forced + } + if not isinstance(state, Mapping): + raise TypeError( + "state must be a dict of NodeStates by node, or " + f"'stationary', got {state!r}." + ) + out: dict = {} + for node, value in state.items(): + if node not in rbd.components: + raise _requirements._not_a_component(rbd, node, "state") + if node in forced: + raise ValueError( + f"Node {node!r} is held working or broken: give it no " + "state." + ) + component = rbd.components[node] + if isinstance(component, RepairableRBD): + if isinstance(value, NodeState): + if not value.stationary: + raise ValueError( + f"Node {node!r} is a nested RBD: give its " + "components' states, as a dict of NodeStates " + "(or 'stationary')." + ) + value = "stationary" + out[node] = _states(component, value, simulated=simulated) + continue + if value == "stationary": + value = NodeState(stationary=True) + if not isinstance(value, NodeState): + raise TypeError( + f"The state of component {node!r} must be a NodeState, " + f"got {value!r}." + ) + _check_state(rbd, node, value, simulated) + out[node] = value + return out + + +def _check_state(rbd, node, state: NodeState, simulated=False) -> None: + """Raise if a component cannot be in ``state``: a ``since_scheduled`` + off a calendar or past its interval, maintenance it does not have, an age + it cannot be up at or a repair or maintenance that is always over + sooner, a virtual age of a component not repaired imperfectly, or a + state of a kind of component whose state is not taken (an + imperfectly repaired one's, outside a simulation).""" + schedule = rbd._preventive.get(node) + inspection = rbd._inspection.get(node) + if state.virtual_age is not None and node not in rbd._imperfect: + raise ValueError( + f"Component {node!r} is not repaired imperfectly (it has no " + "'repair'), so it has no virtual age: give its age alone." + ) + if ( + schedule is not None + and schedule.level is not None + and not (state.new and state.since_scheduled is None) + ): + raise NotImplementedError( + f"Component {node!r} is replaced on condition by its " + "measured degradation level, which a state does not give: " + "leave it out (new)." + ) + if ( + inspection is not None + and inspection.partial + and (state.since_scheduled is not None or not state.new) + ): + raise NotImplementedError( + f"The tests of component {node!r} can miss a failure (a " + "coverage " + "below 1): its state (how long since its last full test, " + "and whether a failure its tests missed is waiting) is not " + "taken: leave it out (new)." + ) + interval = None + if inspection is not None: + interval = inspection.interval + elif schedule is not None and schedule.policy not in ("age", "calendar"): + interval = schedule.interval + if state.since_scheduled is not None: + if interval is None: + raise ValueError( + f"Component {node!r} is not on a calendar (block " + "replacement, replacement on condition, or tests): give " + "it no since_scheduled." + ) + if state.since_scheduled >= interval: + raise ValueError( + f"Component {node!r}: since_scheduled, " + f"{state.since_scheduled:g}, is the time since its last " + "scheduled replacement or test, " + f"less than its interval, {interval:g}." + ) + if state.maintenance and (schedule is None or schedule.duration is None): + raise ValueError( + f"Component {node!r} has no preventive maintenance that " + "takes time: it cannot be down for one." + ) + if state.new or (state.stationary and node in rbd._standby): + # A standby group in its long-run state starts its units' + # chain from its long-run distribution. + return + if node in rbd._standby: + raise NotImplementedError( + f"Component {node!r} is a standby group, whose units' " + "states are not taken as a state: leave it out (new)." + ) + imperfect = rbd._imperfect.get(node) + if imperfect is not None: + if not simulated: + raise NotImplementedError( + f"Component {node!r} is repaired imperfectly: its state " + "(its age and virtual age) is taken by the simulations " + "(availability, cost, simulate_timelines), not here: " + "leave it out (new)." + ) + if ( + imperfect.replace_after is not None + or schedule is not None + or inspection is not None + ): + raise NotImplementedError( + f"Component {node!r} is repaired imperfectly and " + "replaced after some failures, maintained or tested, " + "which would need its failures or operating time " + "since it was renewed, which a state does not give: " + "leave it out (new)." + ) + if state.stationary or ( + state.alive and not state.age and not state.virtual_age + ): + return + component = rbd.components[node] + for what, model in [ + ("reliability", component.reliability), + ("repairability", component.time_to_replace), + ]: + if is_fixed_probability(model): + raise NotImplementedError( + f"Component {node!r}: its {what} model is a probability, " + "not a distribution of times, so it cannot start part " + "way through a life or repair: leave it out (new)." + ) + if state.alive: + # A unit with hidden failures was known to be up only at its + # last test, or when put into service since. + since = 0.0 + if inspection is not None: + since = min(state.age, state.since_scheduled or 0.0) + age = state.age - since + (state.virtual_age or 0.0) + survive = _sf_values(component.reliability.sf, np.array([age])) + if not survive[0] > 0.0: + raise ValueError( + f"Component {node!r} cannot be up at age {age:g}: its " + "reliability there is 0." + ) + return + model = ( + schedule.duration # type: ignore[union-attr] + if state.maintenance + else component.time_to_replace + ) + left = _sf_values(model.sf, np.array([state.down_for])) + if not left[0] > 0.0: + raise ValueError( + f"Component {node!r} cannot have been down for " + f"{state.down_for:g}: its " + f"{'maintenance' if state.maintenance else 'repair'} is " + "always over by then." + ) + + +def _simulation_states(rbd, state, forced=frozenset()) -> dict: + """``state`` checked for a simulation (see ``availability``): as + ``_states`` checks it, none stationary (a simulation starts from + given states), and see ``_check_simulated``.""" + if state is None: + return {} + states = _states(rbd, state, forced, simulated=True) + _check_simulated(rbd, states) + return states + + +def _check_simulated(rbd, states: dict) -> None: + """Raise if a simulation cannot start from ``states`` (checked by + ``_states``): a component in its long-run state, which only the + exact methods take; one started from a state whose draws are not + streamed, being its own subclass or having a model other than a + surpyval parametric one; or more components down in a repair or + maintenance going on than repair crews to work on them.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + in_hand = 0 + served = ( + set(_crews._crew_served(rbd)) if _crews._crews_limited(rbd) else set() + ) + for node, start in states.items(): + component = rbd.components[node] + if isinstance(component, RepairableRBD): + _check_simulated(component, start) + continue + if start.stationary: + raise NotImplementedError(_STATIONARY_SIMULATION) + if start.new: + continue + if ( + type(component) is not NonRepairable + or stream_sampler(component.reliability) is None + or stream_sampler(component.time_to_replace) is None + ): + raise NotImplementedError( + f"Component {node!r} draws its events its own way (it is " + f"a {type(component).__name__}, or its models are not " + "surpyval parametric ones), so it cannot start part way " + "through a life or repair: leave it out (new)." + ) + if not start.alive and node in served: + in_hand += 1 + if served and in_hand > rbd.repair_crews: # type: ignore[operator] + raise ValueError( + f"{in_hand} components are down at the start, in repairs " + "or maintenance going on, which takes more than the " + f"{rbd.repair_crews} repair crew(s): a state does not say " + "which jobs wait." + ) + + +def _curves_at( + rbd, curves: dict, x: np.ndarray, working_nodes, broken_nodes, method +) -> np.ndarray: + """The system's point availability at the times ``x``, from its + nodes' curves (the forced nodes held at 1 or 0).""" + probabilities = {node: curve.at(x) for node, curve in curves.items()} + for node in list(rbd.in_or_out) + list(working_nodes | broken_nodes): + probabilities[node] = np.ones(len(x)) + probabilities = rbd._probabilities_with_overrides( + probabilities, working_nodes, broken_nodes + ) + return np.asarray( + rbd.system_probability(probabilities, method=method), dtype=float + ) + + +def _curves_down_at( + rbd, curves: dict, x: np.ndarray, working_nodes, broken_nodes, states +) -> np.ndarray: + """The system's point unavailability at the times ``x``, from its + nodes' curves (the forced nodes held at 1 or 0), and the system's + failing worked out in its own right, so that a small value keeps + its precision (#237): each node down with one less its + availability, or in closed form where it has one (see + ``_closed_form_down``).""" + failures = {} + for node, curve in curves.items(): + down = _closed_form_down(rbd, node, x, states) + failures[node] = 1.0 - curve.at(x) if down is None else down + probabilities = {node: 1.0 - q for node, q in failures.items()} + for node in list(rbd.in_or_out) + list(working_nodes | broken_nodes): + probabilities[node] = np.ones(len(x)) + failures[node] = np.zeros(len(x)) + probabilities = rbd._probabilities_with_overrides( + probabilities, working_nodes, broken_nodes + ) + failures = rbd._failures_with_overrides( + failures, working_nodes, broken_nodes + ) + return np.clip( + np.asarray( + rbd._system_unreliability(probabilities, failures), + dtype=float, + ), + 0.0, + 1.0, + ) + + +def _closed_form_down(rbd, node, x, states) -> Optional[np.ndarray]: + """A component's probability of being down at the times ``x``, in + closed form where it has one: new at 0, with an exponential life + and an exponential or instant repair and nothing else, + ``lambda / (lambda + mu) (1 - exp(-(lambda + mu) t))``, which keeps + its precision however small, as the curve's grid does not within + its first step (#237). Else None: its curve gives it.""" + rates = _closed_form_rates(rbd, states, (node,)).get(node) + if rates is None: + return None + life, total = _closed_form_pair(rbd, node) + if math.isinf(total): + return np.zeros(len(x)) + return -(life / total) * np.expm1(-total * np.asarray(x, float)) + + +def _closed_form_pair(rbd, node) -> Tuple[float, float]: + """A component's failure rate and the sum of its failure and + repair rates (``inf`` for an instant repair), or ``(nan, nan)`` + where they are not constant.""" + component = rbd.components.get(node) + if type(component) is not NonRepairable: + return math.nan, math.nan + life = _constant_rate(component.reliability) + repair = _repair_rate(component.time_to_replace) + if life is None or repair is None: + return math.nan, math.nan + return life, life + repair + + +def _closed_form_rates(rbd, states, nodes=None) -> Dict[Hashable, float]: + """Of ``nodes`` (by default every component), those whose + probability of being down over time has a closed form (see + ``_closed_form_down``), with the rate it settles at: the sum of + their failure and repair rates (``inf`` for an instant repair).""" + out: Dict[Hashable, float] = {} + for node in rbd.components if nodes is None else nodes: + if ( + (states or {}).get(node) is not None + or node in rbd._preventive + or node in rbd._inspection + or node in rbd._imperfect + or node in rbd._standby + ): + continue + life, total = _closed_form_pair(rbd, node) + if not math.isnan(total): + out[node] = total + return out + + +def _uniformized(what: str, generator, start, steady, vectors): + """A Markov chain followed over time from ``start`` (see + ``_chain_transient.Uniformized``), or a NotImplementedError saying + whose (``what``) cannot be.""" + try: + return _chain_transient.Uniformized(generator, start, steady, vectors) + except NotImplementedError as error: + raise NotImplementedError( + f"{what} Markov chain cannot be followed over time here: " + f"{error}. Simulate it with availability()." + ) from None + + +def _standby_curve(rbd, node, start: Optional[NodeState] = None): + """A standby group over time (see ``_chain_transient.ChainCurve``): + its units' Markov chain from every unit ready, or, for a + ``start`` that is stationary, from its long-run distribution.""" + life, repair = _long_run._standby_rates(rbd, node) + arrangement = rbd._standby[node] + group = _standby_chain.chain( + arrangement.units, + arrangement.k, + life, + repair, + arrangement.dormancy_factor, + arrangement.switching_probability, + rbd.repair_crews if _crews._crews_limited(rbd) else None, + ) + if start is not None and start.stationary: + initial = group.probabilities + else: + initial = np.zeros(len(group.states)) + initial[0] = 1.0 + chain = _uniformized( + f"Component {node!r} is a standby group, whose", + group.generator, + initial, + group.probabilities, + np.column_stack([group.up, group.failures, group.unit_failures]), + ) + return _chain_transient.ChainCurve(chain) + + +def _nested_curve( + rbd, + horizon: float, + counts: bool = False, + stages: bool = False, + start=None, +): + """This RBD as a node of another, over time: the structure function + at its own nodes' curves (a ``SystemCurve``), or, with limited + repair crews, its crews' chain (see ``_crew_curve``). ``counts``, + ``stages`` and ``start`` are as for ``_availability_curves``.""" + if rbd.ccf_groups and not rbd._crews_couple(): + return _ccf_groups._groups_curve( + rbd, horizon, states=start or {}, counts=counts, stages=stages + ) + if not rbd._crews_couple(): + inner = _availability_curves( + rbd, horizon, set(), counts, stages, start + ) + return SystemCurve(rbd, inner, *_settling(inner.values())) + curve = _crew_curve(rbd, horizon, states=start or {}, counts=counts) + if stages and rbd._has_capacity(): + curve.capacity = _repairable_capacity._crew_capacity_over( + rbd, horizon, set(), set(), start or {} + ) + return curve + + +def _crew_nested(rbd, forced) -> list: + """The nested RBDs the crews' chain is followed over time with: + those not held working or broken. Raise if there are too many.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + nested = [ + node + for node, component in rbd.components.items() + if isinstance(component, RepairableRBD) and node not in forced + ] + if len(nested) > _MAX_CREW_NESTED: + raise NotImplementedError( + f"With limited repair crews, the availability over time is " + "worked out for each pattern of the nested RBDs up and down: " + f"{len(nested)} nested RBDs are more than the " + f"{_MAX_CREW_NESTED} it takes. Simulate it with " + "availability()." + ) + return nested + + +def _crew_vectors( + rbd, chain, values: dict, working_nodes, broken_nodes, method: str +) -> Tuple[np.ndarray, dict]: + """Over the states of the crews' ``chain``, with the other nodes' + probabilities ``values`` (arrays, one entry per state): whether the + system is up, and each node's Birnbaum importance.""" + size = len(chain.probabilities) + own = { + node: chain.up[:, k].astype(float) + for k, node in enumerate(chain.nodes) + } + own.update(values) + importance, works, fails, _, _ = rbd._importances( + _windows._filled(rbd, own, size, working_nodes, broken_nodes) + ) + up = works if structure_method(method) == "p" else 1.0 - fails + return np.asarray(up, dtype=float), importance + + +def _crew_failing(rbd, chain, importance: dict) -> np.ndarray: + """The rate of the system's failures in each state of the crews' + ``chain``: a component that is up fails at its rate, and takes the + system down where it is critical (its Birnbaum ``importance`` + there, 1 or 0), as ``system_failure_frequency`` has it.""" + rates = _crews._crew_chain_rates(rbd) + failing = np.zeros(len(chain.probabilities)) + for k, node in enumerate(chain.nodes): + failing += importance[node] * chain.up[:, k] * rates[node][0] + return failing + + +def _crew_failing_each(rbd, chain, importance: dict) -> np.ndarray: + """``_crew_failing`` by component: the rate of the system's failures + by each of the chain's components (columns) in each of its states + (#199).""" + rates = _crews._crew_chain_rates(rbd) + return np.column_stack( + [ + importance[node] * chain.up[:, k] * rates[node][0] + for k, node in enumerate(chain.nodes) + ] + ) + + +def _crew_each(rbd, chain, nested: list, working, broken, method: str) -> list: + """For each pattern of the ``nested`` RBDs up and down, the rate of + the system's failures by each of the crews' chain's components in + each state (see ``_crew_failing_each``), for ``CrewSystem``'s + ``causes``.""" + size = len(chain.probabilities) + out = [] + for pattern in range(2 ** len(nested)): + _, importance = _crew_vectors( + rbd, + chain, + { + node: np.full(size, float((pattern >> j) & 1)) + for j, node in enumerate(nested) + }, + working, + broken, + method, + ) + out.append(_crew_failing_each(rbd, chain, importance)) + return out + + +def _crew_start(rbd, chain, states: dict) -> np.ndarray: + """The probabilities of the crews' chain's states at 0, from the + components' ``states`` (checked by ``_states``): every component up + (a component's age does not matter, its life being exponential), + each in its long-run state (all of them, ``"stationary"``), or as + they say: a component down is in a repair (as its repair is + exponential, however long it has taken), and none waits, so there + can be no more down than crews.""" + named = {node: states[node] for node in chain.nodes if node in states} + stationary = [node for node, s in named.items() if s.stationary] + if stationary: + if len(stationary) < len(chain.nodes): + raise NotImplementedError( + "With limited repair crews, the components' long-run " + "states are tied together by the repair queue: start " + "every component in its long-run state " + "(state='stationary'), or give each its state." + ) + return np.array(chain.probabilities, dtype=float) + down = tuple( + k + for k, node in enumerate(chain.nodes) + if node in named and not named[node].alive + ) + assert rbd.repair_crews is not None # limited crews only + if len(down) > rbd.repair_crews: + raise ValueError( + f"{len(down)} components are down at the start, in repairs " + "going on, which takes more than the " + f"{rbd.repair_crews} repair crew(s): a state does not say " + "which jobs wait." + ) + start = np.zeros(len(chain.probabilities)) + start[chain.states.index((down, ()))] = 1.0 + return start + + +def _crew_curve( + rbd, + horizon: float, + working_nodes=frozenset(), + broken_nodes=frozenset(), + method: str = "p", + states: Optional[dict] = None, + counts: bool = False, +) -> "_chain_transient.CrewCurve": + """The system over time with limited repair crews (see + ``_chain_transient.CrewCurve``): the crews' chain, without the + nodes held working or broken, from the components' ``states`` + (see ``_crew_start``), with the nested RBDs' own curves (each from + its own state in ``states``) to ``horizon``, counting their events + with ``counts``.""" + states = states or {} + working, broken = set(working_nodes), set(broken_nodes) + forced = working | broken + nested = _require_crew_over_time(rbd, frozenset(forced)) + chain = _crews._crew_chain(rbd, frozenset(forced)) + vectors = _crew_patterns(rbd, chain, nested, working, broken, method) + uniformized = _uniformized( + "The repair crews'", + chain.generator, + _crew_start(rbd, chain, states), + chain.probabilities, + np.column_stack(vectors), + ) + curves = { + node: _nested_curve( + rbd.components[node], + horizon, + counts=counts, + start=states.get(node), + ) + for node in nested + } + return _chain_transient.CrewCurve(rbd, uniformized, curves) + + +def _crew_patterns( + rbd, chain, nested: list, working, broken, method: str +) -> list: + """The crews' chain's vectors for each pattern of the ``nested`` + RBDs up and down (see ``_chain_transient.pattern_weights``): whether + the system is up in each state, for every pattern, then the rate of + its failures by the chain's components, for every pattern (see + ``_chain_transient.CrewSystem``).""" + size = len(chain.probabilities) + ups, failing = [], [] + for pattern in range(2 ** len(nested)): + up, importance = _crew_vectors( + rbd, + chain, + { + node: np.full(size, float((pattern >> j) & 1)) + for j, node in enumerate(nested) + }, + working, + broken, + method, + ) + ups.append(up) + failing.append(_crew_failing(rbd, chain, importance)) + return ups + failing + + +def _no_crew_window( + rbd, node=None, what: str = "the expected events" +) -> NoReturn: + """Raise that with limited repair crews, ``what`` over time (from + their chain) cannot take in a nested RBD (``node``), or (no + ``node``) be given to another RBD that has this one nested, as + yet.""" + where = ( + "which has no place, as yet, for those of a nested RBD (#162): " + f"node {node!r} is one" + if node is not None + else "which gives none to an RBD this one is nested in, as yet " + "(#162)" + ) + raise NotImplementedError( + f"With {rbd.repair_crews} repair crew(s) for " + f"{len(_crews._crew_served(rbd))} components, {what} over time come " + f"from the crews' Markov chain, {where}. Simulate them with " + "availability() or cost(); the availability over time and the " + "long-run values are exact." + ) + + +def _groups_window( + rbd, + ends: np.ndarray, + working_nodes, + broken_nodes, + method: str, + states: dict, + nodes: bool, + groups: Optional[dict], + causes: bool = False, +) -> Tuple[dict, dict]: + """``_window``'s curves and counts with common-cause groups (#158): + the system's up time, failures and planned outages over its groups' + joint states (see ``_ccf_chain.GroupsSystem``), and each node's own + curve, the members' as without their groups, for their own events + and down time.""" + working, broken = set(working_nodes), set(broken_nodes) + horizon = float(np.max(ends)) if len(ends) else 0.0 + system_curve = _ccf_groups._groups_curve( + rbd, horizon, working, broken, method, states, counts=True + ) + totals = _windows._window_counts( + rbd, + system_curve.curves, + ends, + working, + broken, + method, + nodes=nodes, + groups=groups, + crew=system_curve.system, + causes=causes, + ) + curves = dict(system_curve.curves) + members = [m for group in rbd.ccf_groups for m in group.members] + own = _availability_curves( + rbd, + horizon, + set(rbd.components) - set(members), + counts=True, + groups=True, + ) + curves.update(own) + return curves, totals + + +def _crew_window( + rbd, + ends: np.ndarray, + working_nodes, + broken_nodes, + method: str, + states: dict, + nodes: bool, + groups: Optional[dict], + causes: bool = False, +) -> Tuple[dict, dict]: + """``_window``'s curves and counts with limited repair crews, from + their chain over time: the system's up time and failures (the + integral of the rate of its failures in each state, as + ``system_failure_frequency`` has it), and each component's down + time and failures (at its rate while it is up), each a repair. No + component has scheduled maintenance or tests, and none fails at the + same instant as another.""" + working, broken = set(working_nodes), set(broken_nodes) + forced = working | broken + nested = _require_crew_over_time(rbd, frozenset(forced), "window") + chain = _crews._crew_chain(rbd, frozenset(forced)) + rates = _crews._crew_chain_rates(rbd) + if nested: + # The nested RBDs, independent of the chain, counted as nodes, + # with the system worked out over the chain (#162). + each = ( + _crew_each(rbd, chain, nested, working, broken, method) + if causes + else [] + ) + uniformized = _uniformized( + "The repair crews'", + chain.generator, + _crew_start(rbd, chain, states), + chain.probabilities, + np.column_stack( + _crew_patterns(rbd, chain, nested, working, broken, method) + + [chain.up.astype(float)] + + each + ), + ) + system = _chain_transient.CrewSystem( + uniformized, nested, chain.nodes, causes + ) + horizon = float(np.max(ends)) if len(ends) else 0.0 + curves = { + node: _nested_curve( + rbd.components[node], + horizon, + counts=True, + start=states.get(node), + ) + for node in nested + } + totals = _windows._window_counts( + rbd, + curves, + ends, + working, + broken, + method, + nodes=nodes, + groups=groups, + crew=system, + causes=causes, + ) + column = 2 * system.patterns + for k, node in enumerate(chain.nodes): + curves[node] = _chain_transient.CrewNodeEvents( + uniformized, column + k, rates[node][0] + ) + return curves, totals + up, importance = _crew_vectors(rbd, chain, {}, working, broken, method) + failing = _crew_failing(rbd, chain, importance) + each = [_crew_failing_each(rbd, chain, importance)] if causes else [] + uniformized = _uniformized( + "The repair crews'", + chain.generator, + _crew_start(rbd, chain, states), + chain.probabilities, + np.column_stack([up, failing, chain.up.astype(float)] + each), + ) + totals = uniformized.integrals(ends) + zero = np.zeros(len(ends)) + counts: dict = { + "uptime": np.clip(totals[:, 0], 0.0, ends), + "failures": np.maximum(totals[:, 1], 0.0), + "planned": zero, + "downtime": {}, + "overlaps": {name: zero for name in groups or {}}, + } + if causes: + base = 2 + len(chain.nodes) + counts["caused"] = { + node: np.maximum(totals[:, base + k], 0.0) + for k, node in enumerate(chain.nodes) + } + curves = {} + for k, node in enumerate(chain.nodes): + curves[node] = _chain_transient.CrewNodeEvents( + uniformized, 2 + k, rates[node][0] + ) + if nodes: + counts["downtime"][node] = np.clip( + ends - totals[:, 2 + k], 0.0, ends + ) + return curves, counts + + +#: The most grid points for a minimally repaired component's point +#: availability with repairs that take time (see +#: ``_minimal_repair_curve``): each power of its repair time summed costs +#: a few FFTs of the grid. +_MINIMAL_POINTS = 2**16 + + +def _minimal_repair_curve(rbd, node, horizon: float, counts: bool): + """A minimally repaired component's point availability and expected + failures from new over ``[0, horizon]`` (see ``MinimalRepairCurve``): + exact, as its life's cumulative hazard, with repairs in no time; with + repairs that take time, on a grid of ``_POINT_STEPS`` steps over its + typical up time (no coarser than 1024 steps over the window) to the + horizon, as the curve never settles.""" + _requirements._require_minimal_repair(rbd, node) + component = rbd.components[node] + life = component.reliability + scale = _up_scale(life, None) + if not _requirements._repairs_take_time(component): + if not (np.isfinite(scale) and scale > 0.0): + scale = horizon if horizon > 0.0 else 1.0 + return MinimalRepairCurve(life, scale, _POINT_STEPS, counts) + repair = component.time_to_replace + horizon = max(horizon, 0.0) + step = horizon / 1024.0 if horizon > 0.0 else np.inf + if np.isfinite(scale) and scale > 0.0: + step = min(step, scale / _POINT_STEPS) + if not np.isfinite(step): + step = 1.0 + n = int(np.ceil(horizon / step)) + if n > _MINIMAL_POINTS: + step = horizon / _MINIMAL_POINTS + if np.isfinite(scale) and step > scale / 250.0: + raise NotImplementedError( + f"Component {node!r} would need more than " + f"{_MINIMAL_POINTS} grid points to follow its " + f"availability to {horizon} (its typical up time is " + f"about {scale:.3g}): estimate it by simulation, " + "with availability()." + ) + n = _MINIMAL_POINTS + try: + with np.errstate(divide="ignore", invalid="ignore"): + return MinimalRepairCurve( + life, + scale, + _POINT_STEPS, + counts, + repair_sf=lambda s: _sf_values(repair.sf, s), + repair_splits=point_knots(repair), + step=step, + n=max(n, 1), + ) + except NotImplementedError as error: + raise NotImplementedError( + f"Component {node!r}: {error}. Estimate its availability " + "by simulation, with availability()." + ) from None + + +def _unit_curve( + rbd, + node, + horizon: float, + counts: bool = False, + stages: bool = False, + start: Optional[NodeState] = None, + unscheduled: bool = False, +): + """A component's point availability from new over ``[0, horizon]`` + (see ``_point_availability.unit_curve``): on a grid of + ``_POINT_STEPS`` steps over its typical up time, with its + replacement age on the grid, and only as long as it takes to settle + at its long-run availability (to 1e-10), after which the curve holds + that value. With ``counts`` it counts the component's expected + events too, until they grow at their long-run rates (to 1e-7), and + at those rates after. With ``stages``, a degrading component's + curve follows its stages too, until they have settled at their + long-run shares of its up time (to 1e-8). Under block replacement, + see ``_block_curve``. ``start`` starts it from a state rather than + new (see ``point_availability``). ``unscheduled`` leaves out its + preventive maintenance, and follows it to the horizon: the head of a + block-replacement curve, up to its first block time.""" + _requirements._require_time_models(rbd, node) + _requirements._require_no_opportunities(rbd, node) + component = rbd.components[node] + if node in rbd._imperfect: + return _minimal_repair_curve(rbd, node, horizon, counts) + schedule = None if unscheduled else rbd._preventive.get(node) + if schedule is not None and schedule.policy in ("block", "condition"): + return _block_curve(rbd, node, horizon, start) + age = None if schedule is None else float(schedule.interval) + duration = None if schedule is None else schedule.duration + life = component.reliability + up_sf = life.sf + up_splits = point_knots(life) + repair = component.time_to_replace + + def up_cdf(x): + return 1.0 - _sf_values(up_sf, x) + + def repair_sf(s): + return _sf_values(repair.sf, s) + + def duration_sf(s): + assert duration is not None + return _sf_values(duration.sf, s) + + maintenance_sf = None if duration is None else duration_sf + try: + if unscheduled: + raise ValueError("followed to the horizon") + long_run: Optional[float] = _long_run._node_availability(rbd, node) + rates: Optional[Tuple[float, float, float]] = ( + _long_run._node_frequencies(rbd, node) if counts else None + ) + except (ValueError, NotImplementedError): + long_run, rates = None, None + stage_model = None + if stages: + _requirements._require_unscheduled_stages(rbd, node) + stage_model = life + if start is not None and start.stationary: + # Long in service: in its long-run state throughout. + if long_run is None: + _long_run._node_availability(rbd, node) # raises why it has none + failures, maintained, _ = _long_run._node_frequencies(rbd, node) + return SteadyCurve( + float(long_run), # type: ignore[arg-type] + failures, + maintained, + takedown=duration is not None, + fractions=( + None if stage_model is None else stage_model.stage_fractions() + ), + ) + first = None + if _draws_start(start): + first = _first_unit( + rbd, + node, + start, # type: ignore[arg-type] + up_sf, + up_splits, + age, + stage_model, + ) + scale = _up_scale(life, age) + if schedule is not None: + up, cycle, _, _ = _long_run._maintenance_cycle(rbd, node, schedule) + else: + up = model_mean(life) + cycle = up + model_mean(repair) + horizon = max(horizon, 0.0) + step = horizon / 1024.0 if horizon > 0.0 else np.inf + if np.isfinite(scale): + step = min(step, scale / _POINT_STEPS) + if not np.isfinite(step): + step = 1.0 + if age is not None and step < age: + step = age / np.ceil(age / step) + # Units that each reach their age are maintained at nearly fixed + # times, which the curve follows until they have all but ended. + survive = 0.0 + if age is not None and duration is not None: + survive = float(_sf_values(up_sf, np.array([age]))[0]) + end = max(horizon, 16.0 * step) + settles = age is None or survive ** (horizon // age) < 1e-10 + if long_run is not None and settles and np.isfinite(cycle) and cycle > 0.0: + end = min(end, max(8.0 * cycle, 16.0 * step)) + while True: + n = int(np.ceil(end / step)) + if n > _POINT_MAX: + step = end / _POINT_MAX + if age is not None and step < age: + step = age / np.floor(age / step) + if np.isfinite(scale) and step > scale / 250.0: + raise NotImplementedError( + f"Component {node!r} would need more than " + f"{_POINT_MAX} grid points to follow its " + f"availability to {end} (its typical up time is " + f"about {scale:.3g}): estimate it by simulation, " + "with availability()." + ) + n = min(int(np.ceil(end / step)), _POINT_MAX) + try: + with np.errstate(divide="ignore", invalid="ignore"): + curve = unit_curve( + up_cdf, + up_splits, + repair_sf, + point_knots(repair), + step, + n, + age=age, + maintenance_sf=maintenance_sf, + maintenance_splits=( + () if duration is None else point_knots(duration) + ), + counts=counts, + stages=( + None + if stage_model is None + else stage_model.stage_probabilities + ), + first=first, + ) + except NotImplementedError as error: + raise NotImplementedError( + f"Component {node!r}: {error}. Estimate its availability " + "by simulation, with availability()." + ) from None + events = curve.unit_events + if events is not None and rates is not None: + events.failure_rate, events.preventive_rate = rates[:2] + if ( + curve.stages is not None + and stage_model is not None + and long_run is not None + ): + curve.stages.fractions = stage_model.stage_fractions() + if end >= horizon: + break + tail = curve.times[len(curve.times) - len(curve.times) // 4 :] + if ( + long_run is not None + and np.all(np.abs(curve.at(tail) - long_run) < 1e-10) + and (age is None or survive ** (end // age) < 1e-10) + and (events is None or events.settled()) + and (curve.stages is None or curve.stages.settled()) + ): + break + end = min(horizon, _settling_end(curve, long_run, cycle, end)) + curve.long_run = long_run + if curve.unit_events is not None and long_run is None: + # Followed to the horizon: no rates past it. + curve.unit_events.failure_rate = None + curve.unit_events.preventive_rate = None + return curve + + +def _block_curve(rbd, node, horizon: float, start: Optional[NodeState] = None): + """A component's point availability from new under block + replacement, over ``[0, horizon]`` (see + ``_block_replacement.block_availability``), or replaced on condition + (#161, see ``_condition_replacement.condition_availability``); in + its long-run state (a stationary ``start``), its settled cycle from + its phase on; from another state, its own curve up to its first + block time or inspection, and the schedule's from there (see + ``StartedBlockCurve``).""" + _requirements._refuse_level(rbd, node) + component = rbd.components[node] + schedule = rbd._preventive[node] + duration = schedule.duration + stationary = start is not None and start.stationary + knots = np.empty(0) if duration is None else point_knots(duration) + life = component.reliability + models = (life, component.time_to_replace, duration, schedule.interval) + + def followed(horizon: float, head=None): + if schedule.policy == "condition": + return condition_availability( + *models, float(schedule.threshold), horizon, node, head + ) + return block_availability(*models, horizon, node, head) + + if start is not None and not start.new and not stationary: + length = schedule.interval - (start.since_scheduled or 0.0) + head = _unit_curve( + rbd, node, length, counts=True, start=start, unscheduled=True + ) + down_sf = None + first = None + if not start.alive: + down_sf = _first_unit( + rbd, node, start, None, (), None, None + ).down_sf + else: + # The unit in service now, if unfailed by the first + # inspection, and its age there. + ages = float(start.age) + np.array([0.0, length]) + alive = _sf_values(life.sf, ages) + first = (float(alive[1] / alive[0]), float(ages[1])) + + def failures(u): + return head.events(np.asarray(u, dtype=float))["failures"] + + up = float(head.at(np.array([length]))[0]) + reached = ( + ConditionHead(length, up, failures, down_sf, first) + if schedule.policy == "condition" + else BlockHead(length, up, failures, down_sf) + ) + result = followed(max(horizon - length, 0.0), reached) + return StartedBlockCurve(head, BlockCurve(result, knots), length) + result = followed(np.inf if stationary else max(horizon, 0.0)) + curve = BlockCurve(result, knots) + if not stationary: + return curve + assert start is not None + return ShiftedCurve(curve, curve.settle + (start.since_scheduled or 0.0)) + + +def _first_unit( + rbd, node, start: NodeState, up_sf, up_splits, age, stage_model +) -> First: + """A component's unit at 0 in state ``start`` (see + ``_point_availability.First``): up at its age, with what is left of + its life (the survival function ``R(a + s) / R(a)``) and its + replacement due ``age - a`` from now; or down, with what is left of + its repair or maintenance (``G(r + s) / G(r)``, after ``r`` so + far). ``_check_state`` has checked that it can be in the state.""" + if start.alive: + a = float(start.age) + survive = float(_sf_values(up_sf, np.array([a]))[0]) + + def cdf(s): + return np.clip(1.0 - _sf_values(up_sf, a + s) / survive, 0.0, 1.0) + + splits = np.asarray(up_splits, dtype=float) - a + + def stages(s): + return stage_model.stage_probabilities(a + s) / survive + + return First( + cdf, + splits[splits > 0.0], + None if age is None else age - a, + stages=None if stage_model is None else stages, + ) + if start.maintenance: + model = rbd._preventive[node].duration + else: + model = rbd.components[node].time_to_replace + r = float(start.down_for) + left = float(_sf_values(model.sf, np.array([r]))[0]) + + def down_sf(s): + return _sf_values(model.sf, r + s) / left + + splits = point_knots(model) - r + return First(down_sf=down_sf, down_splits=splits[splits > 0.0]) diff --git a/repyability/rbd/_degradation.py b/repyability/rbd/_degradation.py new file mode 100644 index 00000000..c02d2610 --- /dev/null +++ b/repyability/rbd/_degradation.py @@ -0,0 +1,113 @@ +"""surpyval's fitted degradation processes as component lives (#271). + +A ``WienerProcess`` or ``GammaProcess`` fitted in surpyval models a unit's +degradation level ``Y(t)`` from its starting level ``y0``; the unit fails +when the level first reaches the model's ``threshold``. As a component's +life it is that first-passage time, which surpyval gives from any level: +``sf``, ``qf`` and ``mean`` take the level now as ``y0``. + +Replacement on condition by level (a spec's ``"preventive": {"policy": +"condition", "level": ...}``) inspects the level and replaces a unit found +at or past the level; between inspections it needs the level at the next +inspection given that the unit has not failed by then, which surpyval does +not give yet (SurPyval#836). :func:`level_after` works it out from the +fitted parameters, a workaround to drop once surpyval has it: + +- a Wiener process, ``Y(t) = y + mu t + sigma B(t)``: the level at ``t`` + of a path that has not reached the threshold ``L`` by then, whose + density is the free one less its reflection in ``L`` (the method of + images), ``phi(a) - exp(2 mu (L - y) / sigma^2) phi(b)``; +- a gamma process, whose increment over ``t`` is Gamma(``alpha t``, + rate ``beta``): monotone, so not having failed by ``t`` is the level + being below ``L`` at ``t``, a gamma truncated there. +""" + +import math + +import numpy as np +from scipy.special import gammainc, gammaincinv, log_ndtr, ndtr + +#: The bisection's steps for a Wiener level: enough to settle a float. +_STEPS = 200 + + +def _kinds(): + from surpyval.degradation.process_models import ( + GammaProcessModel, + WienerProcessModel, + ) + + return WienerProcessModel, GammaProcessModel + + +def is_degradation(model) -> bool: + """Whether ``model`` is a surpyval Wiener or gamma degradation process + (fitted, or built), as a component life.""" + return isinstance(model, _kinds()) + + +def check_life(model, label: str) -> None: + """Raise if a degradation process cannot be a component's life: one + fitted with stress covariates needs them, which a component does not + give it.""" + if getattr(model, "is_accelerated", False): + raise ValueError( + f"{label}: its degradation process was fitted with stress " + "covariates, which a component's life does not take; fit it " + "at the stress the component runs at." + ) + + +def level_after(model, y: float, t: float, u: float) -> float: + """The level at time ``t`` from now of a unit at level ``y`` now that + has not failed (reached ``model.threshold``) by then, from the uniform + ``u``: the inverse of its distribution function (see the module + docstring). A workaround for SurPyval#836.""" + threshold = float(model.threshold) + if t <= 0.0: + return y + wiener, _ = _kinds() + if isinstance(model, wiener): + return _wiener_level(model, y, t, u, threshold) + alpha, beta = (float(p) for p in model.params) + # P(increment < L - y) is the survival; the increment below it is a + # gamma truncated there. + survive = gammainc(alpha * t, beta * (threshold - y)) + return y + float(gammaincinv(alpha * t, u * survive)) / beta + + +def _wiener_level(model, y: float, t: float, u: float, threshold: float): + """A Wiener level (see :func:`level_after`), by bisection of its + distribution function on ``(-inf, threshold)``.""" + mu, sigma = (float(p) for p in model.params) + gap = threshold - y + spread = sigma * math.sqrt(t) + # log of exp(2 mu gap / sigma^2), the reflection's weight. + weight = 2.0 * mu * gap / sigma**2 + + def below(z: float) -> float: + """P(Y(t) <= z and no crossing by t), for z <= threshold.""" + free = ndtr((z - y - mu * t) / spread) + image = (z - 2.0 * threshold + y - mu * t) / spread + return free - math.exp(weight + float(log_ndtr(image))) + + survive = below(threshold) + target = u * survive + # A bracket: the free path's mean less a wide margin, up to L. + lo = min(y + mu * t, threshold) - 40.0 * spread + hi = threshold + for _ in range(_STEPS): + mid = 0.5 * (lo + hi) + if mid <= lo or mid >= hi: + break + if below(mid) < target: + lo = mid + else: + hi = mid + return 0.5 * (lo + hi) + + +def failure_from(model, y: float, u: float) -> float: + """The time to failure of a unit at level ``y`` now, from the uniform + ``u``: surpyval's first-passage quantile from ``y``.""" + return float(np.ravel(model.qf(np.array([u]), y0=y))[0]) diff --git a/repyability/rbd/_differential.py b/repyability/rbd/_differential.py index 64448da7..66b1c33b 100644 --- a/repyability/rbd/_differential.py +++ b/repyability/rbd/_differential.py @@ -108,3 +108,20 @@ def shares( if scalar: return {key: float(np.ravel(value)[0]) for key, value in out.items()} return out + + +def proportional_scales(levers, keys) -> Dict[Hashable, float]: + """What a proportional change moves each of ``keys`` (``(key, name)`` + of ``levers``) by, per unit of the change (see + ``_sensitivity._proportional_scale``): raise for a lever that has no + such change, a regression node's covariate.""" + scales = {(lever.key, lever.name): lever.scale for lever in levers} + unitless = [key for key in keys if scales[key] is None] + if unitless: + raise ValueError( + f"Lever(s) {unitless} are covariates, whose zero is their " + "unit's (0 degrees C is not 0 K), so a proportional change of " + "them depends on the unit they are given in: take " + "change='uniform', or the parameters of a diagram without them." + ) + return {key: float(scales[key]) for key in keys} diff --git a/repyability/rbd/_duty.py b/repyability/rbd/_duty.py new file mode 100644 index 00000000..c8866039 --- /dev/null +++ b/repyability/rbd/_duty.py @@ -0,0 +1,87 @@ +"""Duty cycles: a node whose life is in operating time, run part of the +time. + +A node that operates a fraction ``d`` of the calendar ages only then, so +its life in calendar time is its operating life over ``d``: ``R_c(t) = +R(d t)``. A surpyval parametric distribution of a time stays one under that +change of scale, with its parameters moved (``_SCALED``), so the node keeps +its model's kind and every exact method and simulation takes it as it would +the model fitted in calendar time. +""" + +import math +from typing import Callable, Dict, List + +from ._model_utils import distribution_name, is_fixed_probability + + +def _scale_first(p: List[float], d: float) -> List[float]: + """A scale parameter first (a characteristic life), shapes after.""" + return [p[0] / d, *p[1:]] + + +def _location_scale(p: List[float], d: float) -> List[float]: + """A location and a scale, both times.""" + return [p[0] / d, p[1] / d] + + +#: How each surpyval distribution's parameters move when its time is +#: divided by ``d`` (``R_c(t) = R(d t)``). +_SCALED: Dict[str, Callable[[List[float], float], List[float]]] = { + "Exponential": lambda p, d: [p[0] * d], + "Weibull": _scale_first, + "ExpoWeibull": _scale_first, + "LogLogistic": _scale_first, + "Rayleigh": _scale_first, + # The log of a time. + "LogNormal": lambda p, d: [p[0] - math.log(d), p[1]], + "Galton": lambda p, d: [p[0] - math.log(d), p[1]], + # A shape and a rate. + "Gamma": lambda p, d: [p[0], p[1] * d], + "Normal": _location_scale, + "Gauss": _location_scale, + "Gumbel": _location_scale, + "GumbelLEV": _location_scale, + "Logistic": _location_scale, + "Uniform": _location_scale, +} + + +def duty_fraction(node, value) -> float: + """A node's duty, checked: the fraction of the time it operates, in + ``(0, 1]``.""" + from repyability.utils.checks import number_or_nan + + d = number_or_nan(value) + if not 0.0 < d <= 1.0: + raise ValueError( + f"The duty of node {node!r} must be the fraction of the time it " + f"operates, in (0, 1], got {value!r}." + ) + return d + + +def on_calendar(node, model, d: float): + """``model``, a life in operating time, as the life on the calendar of + a node that operates the fraction ``d`` of the time; raise for a model + that is not a surpyval parametric distribution of a time.""" + if d == 1.0: + return model + name = distribution_name(model) + move = _SCALED.get(name) if name is not None else None + if move is None or is_fixed_probability(model): + raise ValueError( + f"Node {node!r} is given a duty, which moves a surpyval " + "parametric distribution of a time onto the calendar, but its " + f"life is {name or type(model).__name__}: give the life it has " + "on the calendar instead." + ) + import surpyval + + params = [float(p) for p in model.params] + extras = dict(getattr(model, "extras", {})) + if extras.get("gamma"): + # An offset is a time too. + extras["gamma"] = float(extras["gamma"]) / d + assert name is not None # a distribution of time (see _SCALED) + return getattr(surpyval, name).from_params(move(params, d), **extras) diff --git a/repyability/rbd/_event_loop.py b/repyability/rbd/_event_loop.py new file mode 100644 index 00000000..7cb80695 --- /dev/null +++ b/repyability/rbd/_event_loop.py @@ -0,0 +1,2495 @@ +"""A ``RepairableRBD``'s Python event loop: the components' random +streams and their stand-ins, a run's context, the queue of events and +what follows each (failures, repairs, preventive and opportunistic +maintenance, inspections, repair crews, standby groups, common causes), +one simulation (``_replicate``) and a run of them (``_run``, holding +``SIMULATIONS``), in this process, in workers or in shards. The compiled +loop (``_kernel``) must agree with this one to the last bit. +""" + +import heapq +import math +import pickle +from copy import copy +from typing import ( + TYPE_CHECKING, + Any, + Callable, + Collection, + Dict, + Hashable, + List, + NamedTuple, + Optional, + Tuple, +) + +import numpy as np + +from repyability.non_repairable import NonRepairable +from repyability.rbd import ( + _ccf_groups, + _crews, + _curves, +) +from repyability.rbd import _montecarlo as montecarlo +from repyability.rbd import _runs, _streams, _timeline_runs +from repyability.rbd._common import ( + _safe_mean, +) +from repyability.rbd._curves import ( + _draws_start, +) +from repyability.rbd._degradation import ( + failure_from, + level_after, +) +from repyability.rbd._events import ( + Event, + _Cause, + _CauseDraws, + _Crews, + _EventQueue, + _Fixed, + _Imperfect, + _Inspection, + _Preventive, + _Run, + _Standby, + _StandbyDraws, + _StandbyGroup, + _Unit, +) +from repyability.rbd._model_utils import ( + always_works, + distribution_name, + is_mixture, + lfp_p, +) +from repyability.rbd._runs import ( # noqa: E402,F401 + _UNSTREAMED, + failure_criticality_index_per_component_failures, + failure_criticality_index_per_system_failures, + restoration_criticality_index_by_component, + restoration_criticality_index_by_system, +) +from repyability.rbd._sampling import MixtureLife, stream_sampler +from repyability.rbd._tally import ( + _CATEGORIES, + _REPAIR, + _CapacityRecorder, + _Replication, + _shard_bytes, + _Tally, +) +from repyability.rbd.helper_classes import PerfectReliability +from repyability.rbd.node_state import NodeState +from repyability.utils.checks import ( + simulation_window, +) +from repyability.utils.wrappers import SIMULATIONS + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + + +class _StreamedRBD: + """Stands in for a nested :class:`RepairableRBD` component during + ``availability()``: runs the nested RBD's own simulation, with its + components drawing from their own streams through their stand-ins (see + ``RepairableRBD._streamed_components``).""" + + def __init__(self, rbd: "RepairableRBD", sources: dict): + self._rbd = rbd + self._sources = sources + + def initialize_event_queue(self, t_simulation, state=None): + """Start the nested RBD's simulation, its components from + ``state`` (checked; see ``RepairableRBD._simulation_states``).""" + _start_queue( + self._rbd, + t_simulation, + set(), + set(), + "p", + self._sources, + state or {}, + ) + + def next_event(self): + return self._rbd.next_event(sources=self._sources) + + @property + def system_state(self) -> bool: + return self._rbd._run_state.system_state + + @property + def last_change_planned(self) -> bool: + return self._rbd._run_state.last_change_planned + + +def _planned(source, status: bool) -> bool: + """Whether a nested RBD's change to ``status``, just drawn from + ``source``, starts a planned outage.""" + return not status and getattr(source, "last_change_planned", False) + + +class _StreamedComponent: + """Stands in for a :class:`NonRepairable` component during + ``availability()``: alternates failure and repair draws exactly as + ``NonRepairable.next_event`` does, but takes them from the component's + own streams (see ``_streams``). A maintained or inspected component + draws its maintenance or test times from a stream of its own too, or, + when they cannot be streamed, from ``model.random``.""" + + __slots__ = ( + "_failure", + "_repair", + "_fails_next", + "_duration", + "_model", + "_start", + "_test", + "_level", + ) + + def __init__( + self, + failure, + repair, + duration=None, + model=None, + start=None, + test=None, + level=None, + ): + self._failure = failure + self._repair = repair + self._fails_next = True + self._duration = duration + self._model = model + self._start = start + self._test = test + self._level = level + + def reset(self): + self._fails_next = True + + def maintenance_time(self) -> float: + if self._duration is not None: + return self._duration.draw() + return self._model.random(1).item() + + def level_uniform(self) -> float: + """A uniform a unit replaced on condition by its measured level + (#271) draws its level at an inspection, or its failure from it, + from: from its ``LEVEL`` stream, or numpy's global RNG.""" + if self._level is not None: + return self._level.draw() + return float(np.random.random()) + + def start_uniform(self) -> float: + """The uniform a start from a state draws what is left of the + unit's life, repair or maintenance from (see + ``RepairableRBD._started``): from its ``START`` stream, or numpy's + global RNG.""" + if self._start is not None: + return self._start.draw() + return float(np.random.random()) + + def test_uniform(self) -> float: + """The uniform that decides whether a test that can miss a failure + finds it: from its ``TEST`` stream, or numpy's global RNG.""" + if self._test is not None: + return self._test.draw() + return float(np.random.random()) + + def life_drawn(self, x: float = 0.0) -> None: + """The unit's life has been drawn (given its age, at a start from a + state; ``x`` its operating time from its last repair to the failure + drawn): its next draw is its repair.""" + self._fails_next = False + + def next_event(self): + if self._fails_next: + self._fails_next = False + return self._failure.draw(), False + self._fails_next = True + return self._repair.draw(), True + + +class _ModelDraws: + """A model's draws, one at a time, from numpy's global RNG: as a + stream gives them, and as ``NonRepairable.next_event`` draws them (one + ``random(1)`` call each).""" + + __slots__ = ("_model",) + + def __init__(self, model): + self._model = model + + def draw(self) -> float: + return self._model.random(1).item() + + +def _exponential_sampler(rate: float): + """The quantile function of an exponential time at ``rate``: ``inf`` + for a rate of 0 (a common-cause group member whose every cause is + shared never fails on its own, see ``RepairableRBD._own_rate``).""" + if rate <= 0.0: + return lambda u: np.full(np.shape(u), np.inf) + return lambda u: -np.log1p(-np.asarray(u, dtype=float)) / rate + + +class _ExponentialDraws: + """Exponential times at ``rate``, one at a time, from numpy's global RNG + (``inf`` for a rate of 0).""" + + __slots__ = ("_rate",) + + def __init__(self, rate: float): + self._rate = rate + + def draw(self) -> float: + if self._rate <= 0.0: + return math.inf + return float(np.random.exponential(1.0 / self._rate)) + + +def _level_uniform(source) -> float: + """A uniform for a unit replaced on condition by its measured level + (#271): from its stand-in's ``LEVEL`` stream, or numpy's global RNG + for a component that draws its own way.""" + if isinstance(source, _StreamedComponent): + return source.level_uniform() + return float(np.random.random()) + + +def _aged_life(model, age: float, u: float) -> float: + """The life left to a unit of lifetime ``model`` at virtual age + ``age``, from the uniform ``u``: the ``x`` with ``H(age + x) = H(age) - + log(u)`` (``H`` the cumulative hazard), as surpyval's virtual-age + renewal models draw it (``conditional_gaps``). A plain Exponential or + Weibull is worked in closed form, a mixture by ``MixtureLife.aged`` + (#295), any other model by surpyval.""" + if is_mixture(model): + return MixtureLife.of(model).aged(age, u) + dist = getattr(model, "dist", None) + name = getattr(dist, "name", None) + if ( + name in ("Exponential", "Weibull") + and lfp_p(model) == 1 + and getattr(model, "f0", None) == 0 + and not getattr(model, "gamma", 0) + ): + exposure = 0.0 - math.log(u) if u > 0.0 else math.inf + if name == "Exponential": + return exposure / float(model.params[0]) + alpha, beta = (float(p) for p in model.params) + hazard = (age / alpha) ** beta if age > 0.0 else 0.0 + if hazard == 0.0: + # As new (or too young for its hazard to register). + return max(alpha * exposure ** (1.0 / beta) - age, 0.0) + # alpha * (H(age) + exposure) ** (1 / beta) - age, without the + # cancellation. + return age * math.expm1(math.log1p(exposure / hazard) / beta) + from surpyval.recurrent.renewal.renewal_model import conditional_gaps + + return float(conditional_gaps(model, np.array([age]), np.array([u]))[0]) + + +class _ImperfectComponent(_StreamedComponent): + """Stands in for a component repaired imperfectly (a spec's + ``"repair"``; see ``RepairableRBD``) during a simulation, keeping the + unit's virtual age (Kijima): a repair after an operating time ``x`` + takes it from ``v`` to ``v + q * x`` (Kijima I) or ``q * (v + x)`` + (Kijima II), and its next life is drawn given it (``_aged_life``), + from a stream of its own. A unit as new draws from its life stream, as + any component does. At the ``replace_after``-th failure since it was + renewed it is replaced instead, as new, and a scheduled replacement + renews it too (``reset``). ``operated`` is its operating time since it + was renewed, which age replacement counts. Without streams, it draws + from its models and numpy's global RNG.""" + + __slots__ = ( + "_unit", + "_aged", + "_q", + "_second", + "_limit", + "_x", + "age", + "operated", + "count", + ) + + def __init__( + self, + unit, + imperfect: "_Imperfect", + failure=None, + repair=None, + aged=None, + duration=None, + model=None, + test=None, + ): + super().__init__(failure, repair, duration, model, test=test) + self._unit = unit + self._aged = aged + self._q = imperfect.q + self._second = imperfect.kijima == "kijima2" + self._limit = imperfect.replace_after + self._x = 0.0 + self.age = self.operated = 0.0 + self.count = 0 + + def reset(self): + self._fails_next = True + self.age = self.operated = 0.0 + self.count = 0 + + def life_drawn(self, x: float = 0.0) -> None: + # Its repair after the failure drawn takes its virtual age on by + # the operating time since its last repair (#269). + super().life_drawn(x) + self._x = x + + @property + def replacing(self) -> bool: + """Whether the unit's failure (the last drawn) is the + ``replace_after``-th since it was renewed, which replaces it.""" + return self._limit is not None and self.count + 1 >= self._limit + + def next_event(self): + if self._fails_next: + self._fails_next = False + if self.age > 0.0: + u = ( + self._aged.draw() + if self._aged is not None + else float(np.random.random()) + ) + x = _aged_life(self._unit.reliability, self.age, u) + elif self._failure is not None: + x = self._failure.draw() + else: + x = self._unit.reliability.random(1).item() + self._x = x + return x, False + self._fails_next = True + if self.replacing: + self.age = self.operated = 0.0 + self.count = 0 + else: + x = self._x + self.count += 1 + self.operated += x + self.age = ( + self._q * (self.age + x) + if self._second + else self.age + self._q * x + ) + if self._repair is not None: + return self._repair.draw(), True + return self._unit.time_to_replace.random(1).item(), True + + +def _stand_in( + component, + run, + made: dict, + path: tuple, + duration=None, + imperfect: Optional["_Imperfect"] = None, +): + """What a component draws its events from during ``availability()`` + (see ``RepairableRBD._streamed_components``): a stand-in taking its + draws from its own streams, or the component itself when they cannot be + streamed (a model other than a surpyval parametric one, or a subclass, + which may draw its events its own way), which then draws from numpy's + global RNG. ``path`` is the node's place, nested RBDs included, which + names its streams; ``duration`` its maintenance or test time model, and + ``imperfect`` its imperfect repair (see ``_ImperfectComponent``).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if type(component) is RepairableRBD: + return _StreamedRBD( + component, + _streamed_components(component, run, made, path), + ) + if type(component) is not NonRepairable: + return component + failure = run.stream(path, _streams.FAILURE) + repair = run.stream(path, _streams.REPAIR) + if imperfect is not None: + if failure is None or repair is None: + return _ImperfectComponent(component, imperfect, model=duration) + return _ImperfectComponent( + component, + imperfect, + failure, + repair, + run.stream(path, _streams.AGED), + run.stream(path, _streams.DURATION), + duration, + run.stream(path, _streams.TEST), + ) + if failure is None or repair is None: + return component + return _StreamedComponent( + failure, + repair, + run.stream(path, _streams.DURATION), + duration, + run.stream(path, _streams.START), + run.stream(path, _streams.TEST), + run.stream(path, _streams.LEVEL), + ) + + +def _uniforms(u: np.ndarray) -> np.ndarray: + """The draws of a stream of uniforms: the uniforms themselves.""" + return u + + +def _standby_draws(component, run, path: tuple, arrangement: "_Standby"): + """A standby group's draws in a run (see ``_StandbyDraws``): from + streams if every unit's lives and repairs have one.""" + units = range(arrangement.units) + lives = [run.stream(path + (u,), _streams.FAILURE) for u in units] + repairs = [run.stream(path + (u,), _streams.REPAIR) for u in units] + if any(stream is None for stream in lives + repairs): + return _StandbyDraws(component) + return _StandbyDraws( + component, lives, repairs, run.stream(path, _streams.SWITCH) + ) + + +#: Simulations per task of a parallel run of the Python engine (see +#: ``RepairableRBD``'s ``availability``). +PARALLEL_BLOCK = 250 + + +#: A parallel run's worker process's run (see ``_start_worker``). +_WORKER: Dict[str, Any] = {} + + +def _start_worker(run: bytes) -> None: + """Start a worker process of a parallel run (see ``_PythonRunner``): + unpickle the run, the system and what its simulations share, once, for + every block the worker runs.""" + rbd, args, curve_points, replacements, histories = pickle.loads(run) + _WORKER["run"] = ( + rbd, + _context(rbd, *args), + curve_points, + replacements, + histories, + ) + + +def _simulate_block(span: Tuple[int, int]) -> "_Tally": + """Simulations ``start`` to ``stop - 1`` of a parallel run, in a worker + process (see ``_start_worker``): their totals, compact.""" + rbd, context, curve_points, replacements, histories = _WORKER["run"] + tally = _Tally( + list(rbd.components), + rbd.costs, + context.t_simulation, + curve_points, + ) + if replacements: + tally.replacements = [] + if histories: + tally.histories = _timeline_runs.Records(len(rbd.components)) + for replication in range(*span): + tally.add(_replicate(rbd, context, replication)) + return tally.compact() + + +class _Context(NamedTuple): + """What a run's simulations share (see ``RepairableRBD._context``).""" + + t_simulation: float + working: set + broken: set + method: str + capacity: Optional[_CapacityRecorder] + run: _streams.Run + sources: dict + charges: tuple + downtime_cost_rates: dict + has_costs: bool + position: dict + plain: set + works: Callable + #: The components' states at the start (see ``_simulation_states``), + #: and whether the system is up at the start with every component up + #: but those held broken. + states: dict + initial_up: bool + #: Whether each simulation records its components' changes (see + #: ``simulate_timelines``). + history: bool = False + + +class _PythonRunner: + """Runs a run's simulations in Python: one after another in this + process, or, with ``jobs`` above 1, in blocks of ``PARALLEL_BLOCK`` in + that many processes. Each worker process is given the run once, when it + starts (see ``_start_worker``), and sends back each block's totals, + compact (#152), which are merged into the tally in order: the same + totals, kept exactly, as adding the simulations one by one. ``args`` + are ``RepairableRBD._context``'s.""" + + def __init__(self, rbd, tally, progress, args: tuple, jobs): + self._rbd = rbd + self._tally = tally + self._progress = progress + self._context: Optional[_Context] = None + self._executor: Any = None + if jobs is not None and jobs > 1: + run = montecarlo.dumps( + ( + rbd, + args, + tally.curve_points, + tally.replacements is not None, + tally.histories is not None, + ) + ) + self._executor = montecarlo.process_pool( + jobs, initializer=_start_worker, initargs=(run,) + ) + else: + self._context = _context(rbd, *args) + + def __call__(self, start: int, stop: int) -> None: + """Simulations ``start`` to ``stop - 1``.""" + if self._executor is None: + assert self._context is not None + for replication in range(start, stop): + self._tally.add( + _replicate(self._rbd, self._context, replication) + ) + self._progress.update() + return + spans = [ + (first, min(first + PARALLEL_BLOCK, stop)) + for first in range(start, stop, PARALLEL_BLOCK) + ] + for block in self._executor.map(_simulate_block, spans): + self._tally.merge(block) + self._progress.update(block.n) + + def close(self) -> None: + if self._executor is not None: + self._executor.shutdown() + + +class _ShardRunner: + """Runs a run's simulations as shards (see ``repyability.rbd.shards``): + each range of simulations asked for is cut into shards at the multiples + of ``step``, which ``shard_map(run_shard, shards)`` runs wherever it + sends them, and the partials it gives back, checked to be those + shards', are merged into the tally in order. ``template`` is what + every shard of the run holds (see ``RepairableRBD._shard_template``).""" + + def __init__(self, rbd, tally, progress, shard_map, template, step): + from repyability.rbd.chunks import _run_key + + self._tally = tally + self._progress = progress + self._map = shard_map + self._template = template + self._step = step + self._key = _run_key( + {**template, "fingerprint": _runs._fingerprint(rbd)} + ) + + def __call__(self, start: int, stop: int) -> None: + """Simulations ``start`` to ``stop - 1``.""" + from repyability.rbd.chunks import SimulationChunk, _run_key + from repyability.rbd.shards import run_shard + + if stop <= start: + return + step = self._step + cuts = [start, *range((start // step + 1) * step, stop, step), stop] + ranges = list(zip(cuts, cuts[1:])) + shards = [_shard_bytes(self._template, a, b) for a, b in ranges] + chunks = [] + for saved in self._map(run_shard, shards): + chunk = SimulationChunk._load(saved) + chunks.append(chunk) + self._progress.update(chunk.n_simulations) + chunks.sort(key=lambda chunk: chunk.ranges[0]) + if [chunk.ranges for chunk in chunks] != [[r] for r in ranges] or any( + _run_key(chunk.settings) != self._key for chunk in chunks + ): + raise ValueError( + "shard_map gave back other partials than those of the " + "shards it was given: it must give back run_shard's result " + "for each shard, as map(run_shard, shards) does." + ) + for chunk in chunks: + self._tally.merge(chunk._tally) + + def close(self) -> None: + pass + + +#: The per-action costs: charged at each preventive action or inspection. +_ACTION_COST_KEYS = ("preventive_cost", "inspection_cost") + + +def _validate_crews(crews) -> Optional[int]: + """``repair_crews``, validated: a whole number of crews, at least 1, or + None for as many as are needed.""" + if crews is None: + return None + if isinstance(crews, bool) or int(crews) != crews or crews < 1: + raise ValueError( + f"repair_crews must be a whole number, 1 or more, or None; got " + f"{crews!r}." + ) + return int(crews) + + +def _validate_priority(node, priority) -> float: + """A component's ``"priority"`` for a crew, validated: a finite + number.""" + if isinstance(priority, bool) or not isinstance( + priority, (int, float, np.integer, np.floating) + ): + raise ValueError( + f"Component {node!r}: priority must be a number, got " + f"{priority!r}." + ) + if not np.isfinite(priority): + raise ValueError( + f"Component {node!r}: priority must be finite, got {priority!r}." + ) + return float(priority) + + +def _failures_between(model, ages: np.ndarray) -> np.ndarray: + """The probability that a unit of ``model`` that has survived to each + of ``ages`` but the last fails before the next, ``1 - sf(next) / + sf(age)``: from its cumulative hazard where the model gives one, which + keeps a small probability precise.""" + ages = np.asarray(ages, dtype=float) + out = np.full(len(ages) - 1, np.nan) + Hf = getattr(model, "Hf", None) + if Hf is not None: + with np.errstate(all="ignore"): + hazard = np.asarray(Hf(ages), dtype=float) + out = -np.expm1(-np.maximum(np.diff(hazard), 0.0)) + missing = np.flatnonzero(np.isnan(out)) + if missing.size: + with np.errstate(all="ignore"): + survival = np.asarray(model.sf(ages), dtype=float) + now, later = survival[missing], survival[missing + 1] + fails = np.where(now > 0.0, 1.0 - later / now, 1.0) + out[missing] = np.clip(np.nan_to_num(fails, nan=1.0), 0.0, 1.0) + return out + + +def _on_duty(components: dict) -> dict: + """The components, each spec with a ``"duty"`` given its life on the + calendar (see ``_duty.on_calendar``): one object for each model and + duty, so that components sharing a model still share it.""" + from repyability.rbd._duty import duty_fraction, on_calendar + + moved: dict = {} + out = {} + for name, component in components.items(): + if isinstance(component, dict) and component.get("duty") is not None: + d = duty_fraction(name, component["duty"]) + life = component["reliability"] + key = (id(life), d) + if key not in moved: + moved[key] = on_calendar(name, life, d) + component = {**component, "reliability": moved[key]} + out[name] = component + return out + + +def _is_junction(name, component) -> bool: + """Whether ``component`` makes node ``name`` a junction, which never + fails: ``PerfectReliability`` itself, or a spec whose life it is (#175, + #182) or is a probability of failing of 0. Such a spec may give a + repair model, never used, but nothing that only a part that fails or + is maintained has.""" + if component is PerfectReliability: + return True + if not ( + isinstance(component, dict) + and always_works(component.get("reliability")) + ): + return False + life = ( + "PerfectReliability" + if component["reliability"] is PerfectReliability + else "a probability of failing of 0" + ) + other = sorted( + str(key) + for key, value in component.items() + if key not in ("reliability", "repairability") and value is not None + ) + if other: + raise ValueError( + f"Component {name!r} never fails (its reliability is {life}), " + f"so it takes no {', '.join(other)}: give it as " + "PerfectReliability alone, a junction that always works." + ) + return True + + +def _test_finds(source, coverage: float) -> bool: + """Whether a test that finds a failure with probability ``coverage`` + finds one: from the component's detection stream, or numpy's global + RNG.""" + if isinstance(source, _StreamedComponent): + return source.test_uniform() < coverage + return float(np.random.random()) < coverage + + +def _at_zero(model) -> bool: + """Whether ``model`` is a time of exactly 0 (an instant repair is).""" + return distribution_name(model) == "ExactEventTime" and not np.any( + np.ravel(model.params) + ) + + +def _never_settles(component) -> bool: + """Whether a component fails at once and is repaired at once, so that + the event loop would change its state without end.""" + return _at_zero(getattr(component, "reliability", None)) and _at_zero( + getattr(component, "time_to_replace", None) + ) + + +def _stream_specs( + rbd, + t_simulation: float, + prefix: tuple = (), + specs: Optional[dict] = None, + states: Optional[dict] = None, +) -> Tuple[dict, bool]: + """The streams this RBD's simulations draw from (see ``_streams``), + by name, nested RBDs' included; and whether every draw comes from + one. A component whose draws cannot be streamed (a model other than + a surpyval parametric one, or a subclass of the component classes) + has none, nor has a maintenance or test time that cannot: they draw + from numpy's global RNG. ``prefix`` is this RBD's place in the RBD + simulated: only that RBD's own costs are drawn. A component started + from a state in ``states`` (see ``_simulation_states``) that draws + what is left of its life or repair has a stream for that too.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + specs = {} if specs is None else specs + states = {} if states is None else states + complete = True + + def add(path: tuple, kind: int, sampler, expected: float) -> None: + rows = _streams.first_rows(expected) + specs[(path, kind)] = _streams.Spec( + path, kind, sampler, rows, _streams.block_width(rows) + ) + + for name, component in rbd.components.items(): + path = prefix + (name,) + if type(component) is RepairableRBD: + nested = _stream_specs( + component, t_simulation, path, specs, states.get(name) + ) + complete = complete and nested[1] + continue + failure = repair = None + if type(component) is NonRepairable: + # A common-cause group's member draws its own cause's life. + own = _ccf_groups._own_rate(rbd, name) + failure = ( + stream_sampler(component.reliability) + if own is None + else _exponential_sampler(own) + ) + repair = stream_sampler(component.time_to_replace) + if failure is None or repair is None: + complete = False + continue + arrangement = rbd._standby.get(name) + if arrangement is not None: + # Each unit's lives and repairs, and the group's switches. + lives, switches = _standby_draw_counts(rbd, name, t_simulation) + for unit in range(arrangement.units): + add(path + (unit,), _streams.FAILURE, failure, lives) + add(path + (unit,), _streams.REPAIR, repair, lives) + if 0.0 < arrangement.switching_probability < 1.0: + add(path, _streams.SWITCH, _uniforms, switches) + continue + expected = _expected_draws(rbd, name, t_simulation) + add(path, _streams.FAILURE, failure, expected["failure"]) + add(path, _streams.REPAIR, repair, expected["repair"]) + if _draws_start(states.get(name)): + add(path, _streams.START, _uniforms, 1.0) + if name in rbd._imperfect: + # A life after each repair, given the unit's virtual age. + add(path, _streams.AGED, _uniforms, expected["repair"]) + schedule = rbd._preventive.get(name) + if schedule is not None and schedule.level is not None: + # A level and a failure from it at each inspection (#271). + add(path, _streams.LEVEL, _uniforms, 2.0 * expected["action"]) + inspection = rbd._inspection.get(name) + if inspection is not None and inspection.partial: + # Whether a test finds each failure: one for each failure + # a test can miss. + add(path, _streams.TEST, _uniforms, expected["failure"]) + duration = _duration_model(rbd, name) + if duration is not None: + sampler = stream_sampler(duration) + if sampler is None: + complete = False + else: + add(path, _streams.DURATION, sampler, expected["action"]) + for cause in _ccf_groups._shared_causes(rbd): + # A shared common cause's strikes (#158), and at each the + # uniform its failures' tests decide by. + strikes = cause.rate * t_simulation + 1.0 + add( + prefix + cause.struck, + _streams.CAUSE, + _exponential_sampler(cause.rate), + strikes, + ) + if cause.coins: + add( + prefix + cause.struck, + _streams.CAUSE_TEST, + _uniforms, + strikes, + ) + if not prefix: + for node, node_costs in rbd.costs.items(): + expected = _expected_draws(rbd, node, t_simulation) + for key, kind in _streams.COST_KINDS.items(): + cost = node_costs.get(key) + if cost is None or isinstance(cost, float): + continue + count = expected[ + "action" if key in _ACTION_COST_KEYS else "repair" + ] + add((node,), kind, _streams.cost_sampler(cost), count) + return specs, complete + + +def _duration_model(rbd, node): + """The model of ``node``'s maintenance or test time, or None.""" + schedule = rbd._preventive.get(node) or rbd._inspection.get(node) + return None if schedule is None else schedule.duration + + +def _standby_draw_counts( + rbd, node, t_simulation: float +) -> Tuple[float, float]: + """Roughly how many lives (and repairs) a simulation draws for each + unit of a standby group, and how many switches: its operating units + fail at their rate and its spares at the dormant one (see + ``_expected_draws``).""" + arrangement = rbd._standby[node] + life = _safe_mean(rbd.components[node].reliability) + if not life > 0.0: + return float("nan"), float("nan") + active = ( + arrangement.k + + (arrangement.units - arrangement.k) * arrangement.dormancy_factor + ) + failures = t_simulation * active / life + return failures / arrangement.units + 1.0, failures + 1.0 + + +def _expected_draws(rbd, node, t_simulation: float) -> Dict[str, float]: + """Roughly how many times a simulation draws ``node``'s time to + failure (``"failure"``: one per unit put into service), its repair + time and per-failure costs (``"repair"``), and its maintenance or + test time and their costs (``"action"``). NaN where it is not known. + Only how the draws are computed depends on these (see + ``_streams``), never what they are.""" + component = rbd.components[node] + if node in rbd._standby: + # Its units' failures, each a repair charged for. + _, failures = _standby_draw_counts(rbd, node, t_simulation) + return {"failure": failures, "repair": failures, "action": 0.0} + life = repair = float("nan") + if isinstance(component, NonRepairable): + life = _safe_mean(component.reliability) + repair = _safe_mean(component.time_to_replace) + if life == np.inf: + failures = 0.0 + elif life + repair > 0.0: + failures = t_simulation / (life + repair) + else: + failures = float("nan") + # A member renewed early at its group's stops (#108) has more lives + # and actions than these: its streams then take more chunks, which + # changes only how its draws are computed. + lives, actions = failures + 1.0, 0.0 + schedule = rbd._preventive.get(node) + inspection = rbd._inspection.get(node) + if schedule is not None: + actions = t_simulation / schedule.interval + 1.0 + if schedule.policy == "age": + lives = t_simulation / np.fmin(life, schedule.interval) + 1.0 + else: + lives = failures + actions + elif inspection is not None: + actions = t_simulation / inspection.interval + 1.0 + return { + "failure": float(lives), + "repair": failures + 1.0, + "action": actions, + } + + +def _stream_plan( + rbd, + t_simulation: float, + entropy, + antithetic: bool, + states: Optional[dict] = None, +) -> Tuple[_streams.Plan, bool]: + """A run's streams, and whether every draw comes from one; the + components start from ``states``.""" + specs, complete = _stream_specs(rbd, t_simulation, states=states) + return _streams.Plan(entropy, antithetic, specs), complete + + +def _streamed_components( + rbd, + run: _streams.Run, + made: Optional[dict] = None, + prefix: tuple = (), +) -> dict[Any, Any]: + """What each component draws its events from in a run: a stand-in + replaying its draws from its own streams, nested RBDs' components + included, or the component itself when they cannot be streamed + (see ``_stand_in``). The stand-ins are called at exactly the points + the components would be. + + A component object used for several nodes has one simulation state, + so it gets one stand-in (``made``, by object). Since each node has + its own copy of a ``NonRepairable``, only a nested RBD object can be. + """ + made = {} if made is None else made + streamed: dict[Any, Any] = {} + for name, component in rbd.components.items(): + if id(component) not in made: + arrangement = rbd._standby.get(name) + made[id(component)] = ( + _stand_in( + component, + run, + made, + prefix + (name,), + _duration_model(rbd, name), + rbd._imperfect.get(name), + ) + if arrangement is None + else _standby_draws( + component, run, prefix + (name,), arrangement + ) + ) + own = _ccf_groups._own_rate(rbd, name) + if own is not None and made[id(component)] is component: + # Not streamed: its own cause's life all the same. + made[id(component)] = _own_draws(rbd, name, own) + streamed[name] = made[id(component)] + for cause in _ccf_groups._shared_causes(rbd): + path = prefix + cause.struck + streamed[cause] = _CauseDraws( + cause.rate, + run.stream(path, _streams.CAUSE), + run.stream(path, _streams.CAUSE_TEST), + ) + return streamed + + +def _own_draws(rbd, node, own: float) -> "_StreamedComponent": + """A common-cause group member's draws from numpy's global RNG: its + own cause's life (at the rate ``own``), and its repair, maintenance + and test times as it would draw them.""" + component = rbd.components[node] + return _StreamedComponent( + _ExponentialDraws(own), + _ModelDraws(component.time_to_replace), + model=_duration_model(rbd, node), + ) + + +def _keyed_charges(rbd, run: _streams.Run) -> Tuple[ + dict[Any, list[tuple[int, Any]]], + dict[Any, Any], + dict[Any, Any], +]: + """For each costed node, its ``(category, charges)`` pairs: what is + charged at each of its failures (the categories are positions in + ``_CATEGORIES``); and, for each node with a preventive-maintenance + or an inspection cost, what is charged at each preventive action or + inspection. ``charges.draw()`` gives the next amount: a fixed cost, + or the next draw from the cost's own stream (see ``_streams``).""" + + def charges(node, key: str): + cost = rbd.costs[node][key] + if isinstance(cost, float): + return _Fixed(cost) + return run.stream((node,), _streams.COST_KINDS[key]) + + failures = { + node: [ + ( + _CATEGORIES.index(key.removesuffix("_cost")), + charges(node, key), + ) + for key in rbd.PER_FAILURE_COST_KEYS + if key in node_costs + ] + for node, node_costs in rbd.costs.items() + } + preventive = { + node: charges(node, "preventive_cost") + for node, node_costs in rbd.costs.items() + if "preventive_cost" in node_costs + } + replace = _CATEGORIES.index("replace") + for node, schedule in rbd._preventive.items(): + if schedule.policy == "calendar": + preventive[node] = _CalendarCharge( + rbd, + node, + next( + ( + c + for category, c in failures[node] + if category == replace + ), + None, + ), + ) + inspection = { + node: charges(node, "inspection_cost") + for node, node_costs in rbd.costs.items() + if "inspection_cost" in node_costs + } + return failures, preventive, inspection + + +class _CalendarCharge: + """What a component on a calendar policy is charged at a planned + replacement: the price of the opportunity it falls at, kept when it + was scheduled (``_renewal``); renewed early at a stop of its + maintenance group (#349), the calendar's price for a planned + replacement then (``_stop``), or where the calendar allows none + (``inf``), its own ``replace_cost`` (``replace``, as a failure's + replacement is charged; nothing without one).""" + + __slots__ = ("rbd", "node", "replace") + + def __init__(self, rbd, node, replace): + self.rbd, self.node, self.replace = rbd, node, replace + + def draw(self) -> float: + price = self.rbd._run_state._calendar_price.get(self.node, 0.0) + if math.isinf(price): + return 0.0 if self.replace is None else self.replace.draw() + return price + + +def _context( + rbd, + t_simulation: float, + working_nodes, + broken_nodes, + method: str, + capacity: Optional[_CapacityRecorder], + entropy, + antithetic: bool, + states: Optional[dict] = None, + history: bool = False, +) -> "_Context": + """Everything a run's simulations share: the streams, what each + component draws from, the charges, the structure function, and the + components' ``states`` at the start (checked, as + ``_simulation_states`` gives them; None for new). With + ``history``, each simulation records its histories (see + ``_replicate``).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + states = {} if states is None else states + plan, complete = _stream_plan( + rbd, t_simulation, entropy, antithetic, states + ) + run = _streams.Run(plan, reseed=not complete) + has_costs = rbd.has_costs + nodes = list(rbd.components) + return _Context( + t_simulation=t_simulation, + working=set(working_nodes), + broken=set(broken_nodes), + method=method, + capacity=capacity, + run=run, + sources=_streamed_components(rbd, run), + charges=_keyed_charges(rbd, run) if has_costs else ({}, {}, {}), + downtime_cost_rates={ + node: c["downtime_cost"] + for node, c in rbd.costs.items() + if "downtime_cost" in c + }, + has_costs=has_costs, + position={node: i for i, node in enumerate(nodes)}, + # Components with no maintenance or inspection that are not + # nested RBDs: their next event is simply their next draw. + plain={ + node + for node, component in rbd.components.items() + if node not in rbd._preventive + and node not in rbd._inspection + and node not in rbd._standby + and not isinstance(component, RepairableRBD) + }, + works=rbd._decomposition().structure_function(method), + states=states, + initial_up=bool( + rbd.is_system_working( + {c: c not in broken_nodes for c in rbd.components}, method + ) + ), + history=history, + ) + + +def initialize_event_queue( + rbd, + t_simulation, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + sources: Optional[dict], + state, +): + """See ``RepairableRBD.initialize_event_queue``.""" + working_nodes = set() if working_nodes is None else set(working_nodes) + broken_nodes = set() if broken_nodes is None else set(broken_nodes) + states = _curves._simulation_states( + rbd, state, working_nodes | broken_nodes + ) + _ccf_groups._require_free_members(rbd, working_nodes, broken_nodes) + _ccf_groups._require_groups_simulated(rbd, states) + _start_queue( + rbd, t_simulation, working_nodes, broken_nodes, method, sources, states + ) + + +def _start_queue( + rbd, + t_simulation, + working_nodes: set, + broken_nodes: set, + method: str, + sources: Optional[dict], + states: dict, +) -> None: + """``initialize_event_queue``, with ``states`` checked (see + ``_simulation_states``).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + rbd._run_state = _Run() + # What the components draw their events from: themselves (an + # imperfectly repaired one through a stand-in keeping its virtual + # age, and one started from a state through one whose next draw can + # be set; kept for next_event), or stand-ins drawing from their own + # streams (see _streamed_components), which the caller passes to + # next_event too. + if sources is None: + sources = rbd._run_state._step_sources = _own_sources(rbd, states) + # The window's end, which the first events already look to (see + # _replaced_at). + rbd._run_state.t_simulation = t_simulation + # Each calendar a state shifts: by node, its since_scheduled, the + # time since its last scheduled replacement or test (see _due). + rbd._run_state._phases = { + node: start.since_scheduled + for node, start in states.items() + if isinstance(start, NodeState) + and ( + start.since_scheduled + or ( + # On a calendar from 0, not at its offset. + start.since_scheduled is not None + and node in rbd._inspection + and rbd._inspection[node].offset + ) + ) + } + + # Keep record of component status', initially they're all working + component_status: dict[Any, bool] = { + component: True for component in rbd.components.keys() + } + + for component in broken_nodes: + component_status[component] = False + + # The queue supplies failure/repair events in chronological order + event_queue = _EventQueue() + # When each working node with hidden failures is due to fail (None + # once it has failed). + rbd._run_state._pending_failure = {} + # When each working node replaced on condition is due to fail, and + # to be replaced (see _replaced_at). + rbd._run_state._in_service = {} + # Each working node replaced on condition by its measured level + # (#271): when its level was last known, and the level then. + rbd._run_state._levels = {} + rbd._run_state._calendar_price = {} + # Opportunistic maintenance (#108): when each member that can be + # renewed early was put into service as new, and its pending event; + # the pending events early renewals have cancelled, and the early + # renewals queued, by identity, and their members (see _stop). + rbd._run_state._renewed_at = {} + rbd._run_state._pending_event = {} + rbd._run_state._cancelled = {} + rbd._run_state._early = {} + rbd._run_state._renewing = set() + # Common-cause groups (#158): each member's next event, which a + # shared cause that strikes it first supersedes, and the uniform a + # strike has decided its tests by (see _strike). + rbd._run_state._ccf_members = frozenset( + m for group in rbd.ccf_groups for m in group.members + ) + rbd._run_state._ccf_pending = {} + rbd._run_state._ccf_coins = {} + # The components down at the start in a repair or maintenance going + # on, which holds a repair crew. + in_hand: list = [] + + # For each component add in the initial failure + for component_id in rbd.components.keys(): + component = rbd.components[component_id] + if component_id in working_nodes: + continue + elif component_id in broken_nodes: + continue + elif component_id in rbd._standby: + continue # started below, once the queue and crews exist + source = sources[component_id] + start = states.get(component_id) + if isinstance(component, RepairableRBD): + source.initialize_event_queue(t_simulation, state=start) + component_status[component_id] = source.system_state + t_event, event = source.next_event() + first = Event( + t_event, component_id, event, _planned(source, event) + ) + elif start is not None and not start.new: + up, first = _started(rbd, component_id, source, start) + component_status[component_id] = up + if not up and first.status: + in_hand.append(component_id) + elif component_id in rbd._preventive: + # Put into service as new at 0: it fails, or is maintained. + source.reset() + first = _renewal( + rbd, component_id, 0.0, source, rbd._preventive[component_id] + ) + elif component_id in rbd._inspection: + # Put into service as new at 0: it fails, or is inspected. + source.reset() + first = _inspected_renewal( + rbd, component_id, 0.0, source, rbd._inspection[component_id] + ) + else: + source.reset() + t_event, event = source.next_event() + first = Event(t_event, component_id, event) + + # Only consider it if it occurs within the simulation window + if first.time < t_simulation: + event_queue.put(first) + if component_id in rbd._run_state._ccf_members: + rbd._run_state._ccf_pending[component_id] = first + for cause in _ccf_groups._shared_causes(rbd): + first = Event(sources[cause].gap(), cause, False) + if first.time < t_simulation: + event_queue.put(first) + rbd._run_state._event_queue = event_queue + # The repair crews, when there are fewer than the components that + # may need one (with enough, no job ever waits). + crews = rbd.repair_crews + rbd._run_state._crews = ( + _Crews(crews, _crews._crew_served(rbd), rbd._priority) + if crews is not None and _crews._crews_limited(rbd) + else None + ) + if rbd._run_state._crews is not None: + # A repair or maintenance going on at the start holds a crew + # (_simulation_states has checked that there are enough). + for node in in_hand: + if node in rbd._run_state._crews.served: + rbd._run_state._crews.free -= 1 + rbd._run_state._crews.holding.add(node) + # Each standby group's units, which queue the group's first event. + rbd._run_state._groups = { + node: _StandbyGroup( + rbd, node, arrangement, sources[node], t_simulation + ) + for node, arrangement in rbd._standby.items() + if node not in working_nodes and node not in broken_nodes + } + rbd._run_state.last_change_planned = False + # The initial system state must reflect any forced-broken components + # (e.g. a broken component in series starts the system down), and + # any component that starts down, rather than assuming everything is + # up. + rbd._run_state.system_state = rbd.is_system_working( + component_status, method + ) + rbd._run_state.component_status = component_status + + +def _started(rbd, node, source, start: NodeState) -> Tuple[bool, Event]: + """Whether component ``node`` is up at 0, started from ``start`` + (not new), and its first event. + + One uniform, from the component's own stream (see + ``_streamed_components``), draws what is left of its repair or + maintenance given how long it has taken so far, or of its life + given its age (``_aged_life``: ``S(a + x) / S(a)`` is the chance + of more than ``x`` left). Up at age 0 it is new, and draws its life + as a new unit does. Its calendar is shifted by the state's + ``since_scheduled`` (see ``_due``); under age replacement, its + replacement is due when it reaches the age, at once if it has. A + unit with hidden failures is known to have been up only at its last + test (or when put into service, if since): its life left is drawn + from then, and if it has run out it failed unseen, and the next test + finds it. After + the first, its units are new, as from new. + """ + component = rbd.components[node] + schedule = rbd._preventive.get(node) + inspection = rbd._inspection.get(node) + source.reset() + virtual = start.virtual_age or 0.0 + if virtual: + # Repaired imperfectly (#269): at its virtual age at its last + # repair (or once its repair going on is over). + source.age = virtual + if not start.alive: + # Down: up again, as new, once its repair or maintenance is + # done. + model = ( + schedule.duration # type: ignore[union-attr] + if start.maintenance + else component.time_to_replace + ) + left = _aged_life(model, start.down_for, source.start_uniform()) + if inspection is not None: + rbd._run_state._pending_failure[node] = None + return False, Event(left, node, True, start.maintenance) + if not start.age and not virtual: + # New at 0, on a calendar at its phase. + if schedule is not None: + return True, _renewal(rbd, node, 0.0, source, schedule) + if inspection is not None: + return True, _inspected_renewal(rbd, node, 0.0, source, inspection) + delay, status = source.next_event() + return True, Event(delay, node, status) + since = 0.0 + if inspection is not None: + # Last known up at its last test, or when put into service. + since = min(start.age, start.since_scheduled or 0.0) + life = ( + _aged_life( + component.reliability, + virtual + start.age - since, + source.start_uniform(), + ) + - since + ) + source.life_drawn(start.age + life) + if inspection is not None: + if life < 0.0: + # Failed, unseen, since it was last known up. + rbd._run_state._pending_failure[node] = None + found = _finds(rbd, node, inspection, life) + return False, Event(found, node, False, inspection=True) + rbd._run_state._pending_failure[node] = life + return True, _inspected_next(rbd, node, 0.0, inspection) + if schedule is None: + return True, Event(life, node, False) + return True, _renewal( + rbd, node, 0.0, source, schedule, life=life, start=-start.age + ) + + +def _due(rbd, node, schedule, t: float) -> float: + """The next scheduled action of ``schedule`` (a ``_Preventive`` or + ``_Inspection``) after ``t``, on ``node``'s calendar: shifted, from + a state, by its phase, so that the last action fell that long + before 0.""" + phase = ( + rbd._run_state._phases.get(node) if rbd._run_state._phases else None + ) + if phase is None: + return schedule.due(t) + if isinstance(schedule, _Inspection): + # The phase places the calendar: no offset. + schedule = schedule.without_offset() + due = schedule.due(t + phase) - phase + return due if due > t else due + schedule.interval + + +def _finds(rbd, node, inspection: _Inspection, t: float) -> float: + """The test that finds a failure at ``t``, on ``node``'s calendar + (see ``_due``): the first at or after it, and not before 0.""" + phase = ( + rbd._run_state._phases.get(node) if rbd._run_state._phases else None + ) + if phase is None: + return inspection.finds(t) + inspection = inspection.without_offset() + found = inspection.finds(t + phase) - phase + if found < t or found < 0.0: + found += inspection.interval + return found + + +def _follow_up(rbd, event: Event, source) -> Event: + """The next event of ``event``'s component, drawn from ``source``. + + A component's ``next_event()`` gives the time *to* its next event, + measured from ``event``. A nested RBD's gives the time *of* its next + state change: its simulation runs on the same clock, from 0. + """ + from repyability.rbd.repairable_rbd import RepairableRBD + + node = event.component + schedule = rbd._preventive.get(node) + if schedule is not None: + return _maintained_follow_up(rbd, event, source, schedule) + inspection = rbd._inspection.get(node) + if inspection is not None: + return _inspected_follow_up(rbd, event, source, inspection) + t, status = source.next_event() + if isinstance(rbd.components[node], RepairableRBD): + return Event(t, node, status, _planned(source, status)) + return Event(event.time + t, node, status) + + +def _strike(rbd, event: Event, source) -> List[Event]: + """A shared common cause strikes (#158, see ``_Cause``): each member + it names that is up fails now, its own next event superseded (a + member down stays down), and with tests that can miss a failure, + one uniform decides for all its failures. Returns the events to + queue: the members' failures, and the cause's next strike.""" + cause: Any = event.component + t = event.time + status = rbd._run_state.component_status + out: List[Event] = [] + coin = source.coin() if cause.coins else None + for member in cause.struck: + if not status[member]: + continue + pending = rbd._run_state._ccf_pending.pop(member, None) + if pending is not None and pending.time < rbd._run_state.t_simulation: + rbd._run_state._cancelled[id(pending)] = pending + if coin is not None: + rbd._run_state._ccf_coins[member] = coin + out.append(Event(t, member, False)) + later = Event(t + source.gap(), cause, False) + if later.time < rbd._run_state.t_simulation: + out.append(later) + return out + + +def _crew_follow_up(rbd, event: Event, next_event: Event) -> Optional[Event]: + """``next_event`` as the repair crews allow (see ``_Crews``): a job + that falls due when ``event`` takes a component down waits for a + crew (None while it does), and a crew that finishes one at + ``event`` starts the next waiting job, whose end is queued here.""" + crews, node = rbd._run_state._crews, event.component + if crews is None or node not in crews.served: + return next_event + if event.status: + if node in crews.holding: + started = crews.release(node, event.time) + if started is not None: + _crew_started(rbd, started) + return next_event + if next_event.status: + return crews.request(node, event.time, next_event) + return next_event + + +def _crew_started(rbd, started: Tuple[Any, float, Event]) -> None: + """A crew has started a waiting job, ``started`` as + ``_Crews.release`` gives it: queue the end of a component's, or + tell its standby group of a unit's.""" + key, wait, ends = started + if isinstance(key, _Unit): + rbd._run_state._groups[key.node].crew_started(key.unit, ends.time) + return + ends = _started_late(rbd, key, wait, ends) + if ends.time < rbd._run_state.t_simulation: + rbd._run_state._event_queue.put(ends) + + +def _started_late(rbd, node, wait: float, ends: Event) -> Event: + """A job of ``node`` that a crew starts ``wait`` after it fell due, + and so ends with ``ends``: a component off-line for a test does not + age while it waits, so its hidden failure is ``wait`` later.""" + pending = rbd._run_state._pending_failure.get(node) + if pending is not None: + rbd._run_state._pending_failure[node] = pending + wait + return ends + + +def _maintained_follow_up( + rbd, event: Event, source, schedule: _Preventive +) -> Event: + """The next event of a component under preventive maintenance.""" + node, t = event.component, event.time + if event.inspection: + # Inspected, and kept (replacement on condition): its failure is + # still ahead of it. + return _condition_next(rbd, node, t, schedule, source) + if event.preventive: + # Replaced: the new unit has its own life ahead of it (the + # failure drawn for the old one never happens). + source.reset() + if not event.status: + # Down for maintenance: back, as new, once it is done. + if isinstance(source, _StreamedComponent): + duration = source.maintenance_time() + else: + duration = schedule.duration.random(1).item() + return Event(t + duration, node, True, True) + elif not event.status: + # Failed: back, as new, once it is repaired. + repair, status = source.next_event() + return Event(t + repair, node, status) + # Working, as new, from t. + return _renewal(rbd, node, t, source, schedule) + + +def _renewal( + rbd, + node, + t: float, + source, + schedule: _Preventive, + life: Optional[float] = None, + start: Optional[float] = None, +) -> Event: + """The first event of a unit put into service as new at ``t``: its + failure, or its preventive replacement if that is due first (a + failure at the same time comes first); replaced on condition, its + failure or its first inspection (see ``_condition_next``). Started + from a state (see ``_started``), the unit was put into service at + ``start``, before ``t``, and has ``life`` left.""" + if life is None: + life, _ = source.next_event() + if schedule.policy == "condition": + failure = t + life + if schedule.level is not None: + # As new: at the process's starting level, its failure + # drawn from there; each inspection decides (#271). + level = float(rbd.components[node].reliability.y0) + rbd._run_state._levels[node] = (t, level) + rbd._run_state._in_service[node] = (failure, math.inf) + return _condition_next(rbd, node, t, schedule, source) + rbd._run_state._in_service[node] = ( + failure, + _replaced_at( + rbd, node, t if start is None else start, failure, schedule, t + ), + ) + return _condition_next(rbd, node, t, schedule, source) + if start is None: + # Back from an imperfect repair, the unit is as old as its + # operating time since it was renewed, which age replacement + # counts. + start = t - source.operated if node in rbd._imperfect else t + if schedule.policy == "calendar": + # On the calendar policy's thresholds, from when the unit was put + # in; charged its opportunity's price when it falls. + due, price, _ = schedule.calendar._next_planned(start, t) + rbd._run_state._calendar_price[node] = price + else: + due = _due( + rbd, node, schedule, start if schedule.policy == "age" else t + ) + if due < t: + due = t # past its age at the start: replaced at once + if t + life <= due: + event = Event(t + life, node, False) + else: + # Maintenance that takes time is a planned outage; in zero time + # the unit is renewed in place, and stays up. + event = Event(due, node, schedule.duration is None, True) + if node in rbd._early_members: + # Kept, should a stop of its group renew it early (see _stop). + rbd._run_state._renewed_at[node] = start + rbd._run_state._pending_event[node] = event + return event + + +def _own_sources(rbd, states: Optional[dict] = None) -> dict: + """What the components draw their events from when no streams are + given: themselves, but for an imperfectly repaired one, a stand-in + keeping its virtual age (see ``_ImperfectComponent``), and for one + started from a state in ``states`` that draws what is left of its + life or repair, a stand-in whose next draw can be set (see + ``_started``): drawing from their models and numpy's global + RNG.""" + starting = [ + node for node, start in (states or {}).items() if _draws_start(start) + ] + if not rbd._imperfect and not starting and not rbd.ccf_groups: + return rbd.components + sources = dict(rbd.components) + for group in rbd.ccf_groups: + # Each member's own cause's life, and the shared causes (#158). + for member in group.members: + own = _ccf_groups._own_rate(rbd, member) + if own is not None: + sources[member] = _own_draws(rbd, member, own) + for cause in _ccf_groups._shared_causes(rbd): + sources[cause] = _CauseDraws(cause.rate) + for node, imperfect in rbd._imperfect.items(): + sources[node] = _ImperfectComponent( + rbd.components[node], + imperfect, + model=_duration_model(rbd, node), + ) + for node in starting: + component = rbd.components[node] + sources[node] = _StreamedComponent( + _ModelDraws(component.reliability), + _ModelDraws(component.time_to_replace), + model=_duration_model(rbd, node), + ) + return sources + + +def _stop(rbd, group, t: float, trigger=None) -> List[Event]: + """A stop of maintenance ``group`` at ``t``, opened by ``trigger`` + (a member taken down for a failure or its scheduled replacement) or + by the system going down: each other member that is working, and at + least its opportunity age, is renewed now, as its scheduled + replacement would renew it, and the failure or replacement it had + ahead of it is cancelled. A member whose own failure or replacement + is due at ``t`` keeps it: that joins the stop. A member on a + calendar policy is one like any other (#349): renewed now, it is + charged the calendar's price for a planned replacement at ``t`` + (see ``_CalendarCharge``), and its new unit's next planned + replacement counts from ``t``. Returns the renewals to queue.""" + out: List[Event] = [] + for member in rbd._maintenance[group].members: + if ( + member == trigger + or member not in rbd._early_members + or member not in rbd._run_state._renewed_at + or member in rbd._run_state._renewing + or not rbd._run_state.component_status[member] + ): + continue + schedule = rbd._preventive[member] + if t - rbd._run_state._renewed_at[member] < schedule.opportunity: + continue + pending = rbd._run_state._pending_event.get(member) + if pending is not None and pending.time <= t: + continue # due now, as part of this stop + rbd._run_state._pending_event.pop(member, None) + if pending is not None and pending.time < rbd._run_state.t_simulation: + rbd._run_state._cancelled[id(pending)] = pending + if schedule.policy == "calendar": + rbd._run_state._calendar_price[member] = ( + schedule.calendar._planned_price(t) + ) + renewal = Event(t, member, schedule.duration is None, True) + rbd._run_state._early[id(renewal)] = renewal + rbd._run_state._renewing.add(member) + out.append(renewal) + return out + + +def _replaced_at( + rbd, + node, + renewed: float, + failure: float, + schedule: _Preventive, + after: Optional[float] = None, +) -> float: + """When a unit put into service as new at ``renewed``, and due to + fail at ``failure``, is replaced on condition: at the first + inspection before its failure (and the window's end), and after + ``after`` (by default ``renewed``), at which it is more likely than + the threshold to fail before the next, given its age (the time + since ``renewed``); ``inf`` if at none.""" + end = min(failure, rbd._run_state.t_simulation) + dues = [] + due = _due(rbd, node, schedule, renewed if after is None else after) + while due < end: + dues.append(due) + due = _due(rbd, node, schedule, due) + if not dues: + return math.inf + # Its ages at those inspections, and at the one after the last. + ages = np.array(dues + [due]) - renewed + likely = _failures_between(rbd.components[node].reliability, ages) + above = np.flatnonzero(likely > schedule.threshold) + return dues[above[0]] if above.size else math.inf + + +def _condition_next( + rbd, node, t: float, schedule: _Preventive, source +) -> Event: + """The next event, from ``t``, of a working unit replaced on + condition: its failure, or the next inspection, which replaces the + unit (see ``_replaced_at``) or only checks it. A failure at an + inspection's time comes first. + + Replaced by its measured level (#271), the inspection measures it, + drawn from the unit's ``LEVEL`` stream given that it has not failed + by then (``level_after``, the Markov property: the failure drawn + from the last level is past the inspection, and nothing else of it + is kept), and replaces the unit at or past ``schedule.level``; kept, + the unit's failure is drawn afresh from the level found.""" + failure, replaced = rbd._run_state._in_service[node] + due = _due(rbd, node, schedule, t) + if failure <= due: + return Event(failure, node, False) + if schedule.level is not None: + process = rbd.components[node].reliability + known, level = rbd._run_state._levels[node] + level = level_after( + process, level, due - known, _level_uniform(source) + ) + if level >= schedule.level: + return Event(due, node, schedule.duration is None, True) + rbd._run_state._levels[node] = (due, level) + left = failure_from(process, level, _level_uniform(source)) + rbd._run_state._in_service[node] = (due + left, math.inf) + return Event(due, node, True, inspection=True) + if due >= replaced: + # Replaced, as under block replacement. + return Event(due, node, schedule.duration is None, True) + return Event(due, node, True, inspection=True) + + +def _inspected_follow_up( + rbd, event: Event, source, inspection: _Inspection +) -> Event: + """The next event of a component whose failures are hidden.""" + node, t = event.component, event.time + if not event.inspection: + if event.status: + # Repaired: as new, from t. + return _inspected_renewal(rbd, node, t, source, inspection) + # Failed, unseen: found by the first inspection at or after t, + # unless that one can miss it and does (a shared common cause + # has decided for all its failures, see _strike). + rbd._run_state._pending_failure[node] = None + found = _finds(rbd, node, inspection, t) + coin = ( + rbd._run_state._ccf_coins.pop(node, None) + if rbd._run_state._ccf_coins + else None + ) + if not inspection.is_full(found) and not ( + coin < inspection.coverage + if coin is not None + else _test_finds(source, inspection.coverage) + ): + return Event(found, node, False, inspection=True, missed=True) + return Event(found, node, False, inspection=True) + if event.missed: + # A test that missed the failure: the next test misses it too, + # unless it is a full test. + due = _due(rbd, node, inspection, t) + return Event( + due, + node, + False, + inspection=True, + missed=not inspection.is_full(due), + ) + if event.status: + # Tested in zero time, or back from a test: working, with its + # failure still ahead of it. + return _inspected_next(rbd, node, t, inspection) + # A test that takes time, of a failed unit (repaired once the test is + # done) or of a working one (off-line, and not ageing, until then). + duration = 0.0 + if inspection.duration is not None: + if isinstance(source, _StreamedComponent): + duration = source.maintenance_time() + else: + duration = inspection.duration.random(1).item() + failure = rbd._run_state._pending_failure[node] + if failure is None: + repair, status = source.next_event() + return Event(t + duration + repair, node, status) + rbd._run_state._pending_failure[node] = failure + duration + return Event(t + duration, node, True, inspection=True) + + +def _inspected_renewal( + rbd, node, t: float, source, inspection: _Inspection +) -> Event: + """The first event of a unit with hidden failures put into service + as new at ``t``.""" + life, _ = source.next_event() + rbd._run_state._pending_failure[node] = t + life + return _inspected_next(rbd, node, t, inspection) + + +def _inspected_next(rbd, node, t: float, inspection: _Inspection) -> Event: + """The next event of a working unit with hidden failures, at ``t``: + its failure, or the next inspection if that comes first (a failure + at the same time comes first, and is found by it). A test that takes + time takes the unit off-line; one in zero time leaves it up.""" + failure = rbd._run_state._pending_failure[node] + due = _due(rbd, node, inspection, t) + if failure <= due: # type: ignore[operator] + return Event(failure, node, False) # type: ignore[arg-type] + return Event(due, node, inspection.duration is None, inspection=True) + + +def next_event(rbd, method, sources: Optional[dict]): + """See ``RepairableRBD.next_event``.""" + run = rbd.__dict__.get("_run_state") + if run is None or not hasattr(run, "_event_queue"): + raise ValueError("Need to initialize the event queue") + # The components' draws come from the same sources the queue was + # initialised with (see initialize_event_queue). + if sources is None: + sources = rbd._run_state._step_sources + if sources is None: + sources = rbd.components + new_system_state = copy(rbd._run_state.system_state) + + # Use a while loop to find the next time/event at which the system + # status changes. + while new_system_state == rbd._run_state.system_state: + if rbd._run_state._event_queue.qsize() == 0: + del rbd._run_state._event_queue + rbd._run_state.last_change_planned = False + return rbd._run_state.t_simulation, rbd._run_state.system_state + + event = rbd._run_state._event_queue.get() + if ( + rbd._run_state._cancelled + and rbd._run_state._cancelled.get(id(event)) is event + ): + del rbd._run_state._cancelled[id(event)] + continue # superseded by an early renewal, or a strike + if event.component.__class__ is _Cause: + # A shared common cause strikes (#158, see _strike). + for follow in _strike(rbd, event, sources[event.component]): + rbd._run_state._event_queue.put(follow) + continue + renewing_early = False + if ( + rbd._run_state._early + and rbd._run_state._early.get(id(event)) is event + ): + del rbd._run_state._early[id(event)] + rbd._run_state._renewing.discard(event.component) + renewing_early = True + group = rbd._run_state._groups.get(event.component) + if group is not None: + # A standby group's own event (see _StandbyGroup). + if not group.holds(event): + continue # superseded by an earlier one + now, _ = group.advance(event.time) + if now == rbd._run_state.component_status[event.component]: + continue + rbd._run_state.component_status[event.component] = now + if now != rbd._run_state.system_state: + new_system_state = rbd.is_system_working( + rbd._run_state.component_status, method + ) + continue # the group has queued its next event + was = rbd._run_state.component_status[event.component] + rbd._run_state.component_status[event.component] = event.status + node = event.component + if node in rbd._member_group and not renewing_early: + # A failure, or a scheduled replacement starting (or done in + # place), opens a stop of its maintenance group. + if (not event.status and not event.inspection) or ( + event.preventive and event.status and was + ): + for renewal in _stop( + rbd, rbd._member_group[node], event.time, node + ): + rbd._run_state._event_queue.put(renewal) + # Only a change against the system's state can change it (the + # structure is coherent; see _replicate). + if event.status != rbd._run_state.system_state: + new_system_state = rbd.is_system_working( + rbd._run_state.component_status, method + ) + if rbd._run_state.system_state and not new_system_state: + for name, spec in rbd._maintenance.items(): + if spec.system_down: + for renewal in _stop(rbd, name, event.time): + rbd._run_state._event_queue.put(renewal) + + follow = _follow_up(rbd, event, sources[event.component]) + next_event: Optional[Event] = ( + follow + if rbd._run_state._crews is None + else _crew_follow_up(rbd, event, follow) + ) + # But only queue up the event if it occurs before the end + # of the simulation + if ( + next_event is not None + and next_event.time < rbd._run_state.t_simulation + ): + rbd._run_state._event_queue.put(next_event) + if node in rbd._run_state._ccf_members: + rbd._run_state._ccf_pending[node] = next_event + + rbd._run_state.system_state = new_system_state + # A system taken down by maintenance or a test is a planned outage. + rbd._run_state.last_change_planned = ( + event.preventive or event.inspection + ) and not new_system_state + + return event.time, rbd._run_state.system_state + + +def _run(rbd, *args, **kwargs) -> "_Tally": + """A run of simulations (see ``_run_alone``), one at a time in the + process (``SIMULATIONS``, #216): the event loop keeps its state on + the diagram, and draws that cannot be streamed come from numpy's + global RNG. Not a ``sharded`` run, whose shards each run (and take + it) where they are sent: a ``shard_map`` on threads would wait for + it.""" + if kwargs.get("sharded") is not None: + return _run_alone(rbd, *args, **kwargs) + with SIMULATIONS: + return _run_alone(rbd, *args, **kwargs) + + +def _run_alone( + rbd, + t_simulation: float, + working_nodes, + broken_nodes, + method: str, + N: int, + verbose: bool, + seed, + antithetic: bool = False, + stop: Optional[Callable[["_Tally"], int]] = None, + capacity: Optional[_CapacityRecorder] = None, + jobs: Optional[int] = None, + engine: str = "auto", + entropy: Any = None, + common: bool = False, + replacements: bool = False, + first: int = 0, + states: Optional[dict] = None, + curve_points: Optional[int] = None, + sharded: Optional[tuple] = None, + histories: bool = False, +) -> "_Tally": + """Run ``N`` simulations, from simulation ``first`` (then more, + while ``stop`` asks for them), and return their totals: each from + the components' ``states`` (checked; see ``_simulation_states``), + new by default. + + Every draw comes from a stream of its own (see ``_streams``), seeded + from the run's ``entropy``: the ``seed``, or without one a number + drawn from numpy's global RNG. Draws that cannot be streamed come + from the global RNG, seeded afresh for each simulation. So each + simulation is the same whichever ``engine`` runs it, in however many + processes or threads (``jobs``), and in a run to a tolerance as in a + run of its final size. The global RNG is left as it was, except for + the number drawn for the entropy. ``compare`` passes both systems + the same ``entropy``, and ``common``: that, like + ``antithetic``, needs every draw to come from a stream. With + ``capacity``, each simulation also follows the system's capacity; + with ``replacements``, the tally keeps each simulation's + replacements of each component, which only the Python engine + counts; with ``curve_points``, it counts the changes of state on a + grid (see ``availability``). With ``sharded``, ``(shard_map, + template, step)``, the simulations run as shards (see + ``_ShardRunner``), on the engine each shard's worker chooses. With + ``histories``, the tally keeps each simulation's histories in place + of every total (see ``simulate_timelines``), on the Python engine + or numba's. + """ + _ccf_groups._require_free_members(rbd, working_nodes, broken_nodes) + _ccf_groups._require_groups_simulated(rbd, states) + t_simulation = simulation_window(t_simulation) + from tqdm import tqdm + + state = np.random.get_state() + after = state + tally = _Tally(list(rbd.components), rbd.costs, t_simulation, curve_points) + if replacements: + # Only the Python engine counts them. + tally.replacements = [] + engine = "python" + if histories: + tally.histories = _timeline_runs.Records(len(rbd.components)) + runner: Any = None + try: + if entropy is None: + entropy = _streams.entropy_of(seed) + if seed is None: + after = np.random.get_state() + plan, complete = _stream_plan( + rbd, t_simulation, entropy, antithetic, states + ) + if (antithetic or common) and not complete: + raise NotImplementedError(_UNSTREAMED) + engine = _runs._simulation_engine( + rbd, + engine, + plan, + capacity, + N, + here=sharded is None, + states=states, + ) + progress = tqdm( + total=N, disable=not verbose, desc="Running simulations" + ) + if sharded is not None: + runner = _ShardRunner(rbd, tally, progress, *sharded) + elif engine == "numba": + from repyability.rbd import _compiled + + runner = _compiled.Runner( + rbd, + plan, + tally, + progress, + working_nodes, + broken_nodes, + method, + jobs, + capacity=capacity, + ) + elif engine != "python": + from repyability.rbd import engines + + runner = engines.registered()[engine].runner( + rbd, + plan, + tally, + progress, + working_nodes, + broken_nodes, + method, + jobs, + ) + else: + runner = _PythonRunner( + rbd, + tally, + progress, + ( + t_simulation, + working_nodes, + broken_nodes, + method, + capacity, + entropy, + antithetic, + states, + histories, + ), + jobs, + ) + start, goal = first, first + N + while True: + runner(start, goal) + more = 0 if stop is None else stop(tally) + if not more: + break + start, goal = goal, goal + more + progress.total = goal + progress.refresh() + progress.close() + finally: + if runner is not None: + runner.close() + if sharded is None: + # A sharded run draws nothing here, and another thread's + # run may have the global RNG now. + np.random.set_state(after) + _forget_run(rbd) + return tally + + +def _forget_run(rbd) -> None: + """Clean up the interim variables of a simulation, this RBD's and + its nested RBDs': left behind, a nested standby group's draws (whose + samplers cannot be pickled) would keep the RBD from going to the + processes of a later parallel run.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + rbd.__dict__.pop("_run_state", None) + for component in rbd.components.values(): + if isinstance(component, RepairableRBD): + _forget_run(component) + + +def _replicate(rbd, ctx: "_Context", replication: int) -> _Replication: + """Simulation ``replication`` of ``[0, t_simulation]``. + + Besides the counts, it follows each component's up time and its + overlaps with the system's as it goes: when a component changes + state, the time since its last change is added to its up time (if + it was up), and the system's up (or down) time over the same span + to the time both were up (or down), as the difference of the + system's up (or down) time so far at either end. + """ + ctx.run.begin(replication) + t_simulation = ctx.t_simulation + sources = ctx.sources + failure_charges, preventive_charges, inspection_charges = ctx.charges + _start_queue( + rbd, + t_simulation, + ctx.working, + ctx.broken, + ctx.method, + sources, + ctx.states, + ) + status = rbd._run_state.component_status + index, plain, works = ctx.position, ctx.plain, ctx.works + inspected = rbd._inspection + crews = rbd._run_state._crews + groups = rbd._run_state._groups + n = len(index) + # Each component's failures, the system failures they caused, its + # restorations and the system restorations they caused. + failed, caused_down = [0] * n, [0] * n + restored, caused_up = [0] * n, [0] * n + # Each component's replacements: the spares it used. + replaced = [0] * n + # Imperfect repair (#109): a failure repaired, not replaced, uses no + # spare and is charged only its repair. + imperfect = rbd._imperfect + # Opportunistic maintenance (#108): each member's early renewals. + maintenance, member_group = rbd._maintenance, rbd._member_group + cancelled, early = rbd._run_state._cancelled, rbd._run_state._early + opportunistic = [0] * n + failures = restorations = planned = 0 + changes: list = [] + deltas: list = [] + if ctx.initial_up and not rbd._run_state.system_state: + # Down at the start, from the components' states: the curve of + # the availability over time counts it as a change at 0. + changes.append(0.0) + deltas.append(-1) + by_category = [0.0] * len(_CATEGORIES) + by_node = dict.fromkeys(rbd.costs, 0.0) + rep_cost = 0.0 + + def pay(node, category: int, charges) -> float: + """The next charge of ``charges`` (nothing without any).""" + if charges is None: + return 0.0 + charge = charges.draw() + by_category[category] += charge + by_node[node] += charge + return charge + + trace = None if ctx.capacity is None else ctx.capacity.trace(status) + # With ctx.history (see simulate_timelines): each component's state + # at 0, the times of its changes, and which of them (their places + # among its changes) were planned changes down, maintenance or a + # test off line; and the system's changes: their times, the + # component whose change made each, which of its changes that was, + # and whether it was planned. + history = ctx.history + at_start = [status[node] for node in index] if history else None + times_of: list = [[] for _ in range(n)] if history else [] + planned_of: list = [[] for _ in range(n)] if history else [] + system_record: tuple = ([], [], [], []) if history else () + # The system's state, and its up and down time until ``since``, its + # last change; each component's last change, and the system's up and + # down time until then. + up = rbd._run_state.system_state + started_up = up + system_up = system_down = since = 0.0 + last, up_at, down_at = [0.0] * n, [0.0] * n, [0.0] * n + node_up, both_up, both_down = [0.0] * n, [0.0] * n, [0.0] * n + + # The queue's heap, worked directly (see _EventQueue). + heap = rbd._run_state._event_queue._heap + push, pop = heapq.heappush, heapq.heappop + + # When each group last stopped: the work started at one instant is + # one stop, with one set-up. + stopped: Dict[Hashable, float] = {} + # Common-cause groups' members, whose next events a shared cause can + # supersede (see _strike). + ccf_members, ccf_pending = ( + rbd._run_state._ccf_members, + rbd._run_state._ccf_pending, + ) + + def corrective(node, c: int) -> float: + """Charge a failure's corrective action, and count the spare a + replacement uses: an imperfect repair (see _ImperfectComponent) + uses none, and is charged only as a repair.""" + charge = 0.0 + renewed = not imperfect or node not in imperfect + renewed = renewed or sources[node].replacing + if renewed: + replaced[c] += 1 + for category, charges in failure_charges.get(node, ()): + if renewed or category == _REPAIR: + charge += pay(node, category, charges) + return charge + + def stop(group, t: float, trigger=None) -> float: + """Open a stop of maintenance ``group`` (see ``_stop``): queue + its early renewals, and return its set-up cost, charged once + per stop, if a member opened it or it renews any.""" + renewals = _stop(rbd, group, t, trigger) + for renewal in renewals: + push(heap, (renewal.time, renewal)) + if (trigger is None and not renewals) or stopped.get(group) == t: + return 0.0 + stopped[group] = t + setup = maintenance[group].setup_cost + if setup: + by_category[6] += setup + return setup + + # No event at or after the end of the window is queued, so the + # simulation runs until the queue is empty. + while heap: + event = pop(heap)[1] + if cancelled and cancelled.get(id(event)) is event: + del cancelled[id(event)] + continue # superseded by an early renewal, or a strike + node = event.component + if node.__class__ is _Cause: + # A shared common cause strikes (#158). + for follow in _strike(rbd, event, sources[node]): + push(heap, (follow.time, follow)) + continue + renewing_early = False + if early and early.get(id(event)) is event: + del early[id(event)] + rbd._run_state._renewing.discard(node) + opportunistic[index[node]] += 1 + renewing_early = True + grouped = False + if groups and node in groups: + # A standby group's own event (see _StandbyGroup): a unit + # failure, charged as a repair, or a repair's end. + group = groups[node] + if not group.holds(event): + continue # superseded by an earlier one + now, broken = group.advance(event.time) + if broken: + replaced[index[node]] += 1 + for category, charges in failure_charges.get(node, ()): + rep_cost += pay(node, category, charges) + if now == status[node]: + continue + event, grouped = Event(event.time, node, now), True + if (event.preventive or event.inspection) and ( + event.status == status[node] + ): + # No change of state: maintenance or a test in zero time of a + # working unit (renewed in place, or found working), or a + # test that finds a hidden failure, whose repair starts now. + if event.preventive: + replaced[index[node]] += 1 + rep_cost += pay(node, 2, preventive_charges.get(node)) + # Replaced on condition: at an inspection, charged too. + rep_cost += pay(node, 3, inspection_charges.get(node)) + if node in member_group and not renewing_early: + # Renewed in place on schedule: a stop of its group. + rep_cost += stop(member_group[node], event.time, node) + else: + rep_cost += pay(node, 3, inspection_charges.get(node)) + if not event.status and not event.missed: + rep_cost += corrective(node, index[node]) + follow = _follow_up(rbd, event, sources[node]) + next_event: Optional[Event] = ( + follow + if crews is None + else _crew_follow_up(rbd, event, follow) + ) + if next_event is not None and next_event.time < t_simulation: + push(heap, (next_event.time, next_event)) + if ccf_members and node in ccf_members: + ccf_pending[node] = next_event + continue + t = event.time + c = index[node] + if up: + system_up_t, system_down_t = ( + system_up + (t - since), + system_down, + ) + else: + system_up_t, system_down_t = system_up, system_down + (t - since) + if status[node]: + node_up[c] += t - last[c] + both_up[c] += system_up_t - up_at[c] + else: + both_down[c] += system_down_t - down_at[c] + last[c], up_at[c], down_at[c] = t, system_up_t, system_down_t + status[node] = event.status + if trace is not None: + trace.change(t, node, event.status) + if history: + times_of[c].append(t) + if not event.status and (event.preventive or event.inspection): + planned_of[c].append(len(times_of[c]) - 1) + if event.status: + restored[c] += 1 + elif event.preventive: + # A planned outage: charged the preventive cost, and the + # inspection's if replaced on condition (a nested RBD's own + # costs are not counted). + replaced[c] += 1 + rep_cost += pay(node, 2, preventive_charges.get(node)) + rep_cost += pay(node, 3, inspection_charges.get(node)) + if node in member_group and not renewing_early: + rep_cost += stop(member_group[node], t, node) + elif event.inspection: + # A test that takes a working unit off-line. + rep_cost += pay(node, 3, inspection_charges.get(node)) + else: + failed[c] += 1 + # Repair and replace are charged per corrective action, at + # the failure that triggers it (for a hidden failure, when an + # inspection finds it; for a standby group, at each unit's). + if node not in inspected and not grouped: + if imperfect and node in imperfect: + rep_cost += corrective(node, c) + else: + replaced[c] += 1 + for category, charges in failure_charges.get(node, ()): + rep_cost += pay(node, category, charges) + if node in member_group: + rep_cost += stop(member_group[node], t, node) + + # The structure is coherent: a restoration can't take the + # system down, nor a failure bring it up, so only a change + # against the system's state needs the structure function. + if event.status != up and works(status) != up: + system_up, system_down, since = system_up_t, system_down_t, t + up = not up + changes.append(t) + if history: + system_record[0].append(t) + system_record[1].append(c) + system_record[2].append(len(times_of[c]) - 1) + system_record[3].append( + not up and bool(event.preventive or event.inspection) + ) + if up: + deltas.append(1) + restorations += 1 + caused_up[c] += 1 + else: + deltas.append(-1) + if event.preventive or event.inspection: + planned += 1 + else: + failures += 1 + caused_down[c] += 1 + for name, spec in maintenance.items(): + if spec.system_down: + # Any system outage is a stop of this group. + rep_cost += stop(name, t) + + if grouped: + continue # the group has queued its next event + # The component's next event: its next failure if it has just + # been restored, or its restoration if it has just gone down; + # queued only if it falls inside the window. + if node in plain: + delay, next_status = sources[node].next_event() + next_event = Event(t + delay, node, next_status) + else: + next_event = _follow_up(rbd, event, sources[node]) + if crews is not None: + # A job waits for a repair crew (see _Crews). + next_event = _crew_follow_up(rbd, event, next_event) + if next_event is not None and next_event.time < t_simulation: + push(heap, (next_event.time, next_event)) + if ccf_members and node in ccf_members: + ccf_pending[node] = next_event + rbd._run_state.system_state = up + + # Close the system's time, and each component's, at the end of the + # window. + if up: + system_up += t_simulation - since + else: + system_down += t_simulation - since + for c, node in enumerate(index): + if status[node]: + node_up[c] += t_simulation - last[c] + both_up[c] += system_up - up_at[c] + else: + both_down[c] += system_down - down_at[c] + + rates = ctx.downtime_cost_rates + if rates: + for c, node in enumerate(index): + rate = rates.get(node) + if rate is not None: + charge = rate * (t_simulation - node_up[c]) + rep_cost += charge + by_category[4] += charge + by_node[node] += charge + if ctx.has_costs: + charge = rbd.downtime_cost_rate * (t_simulation - system_up) + rep_cost += charge + by_category[5] += charge + + rec = _Replication() + rec.uptime = system_up + rec.node_up, rec.both_up, rec.both_down = node_up, both_up, both_down + rec.counts = (failed, caused_down, restored, caused_up) + rec.failures, rec.restorations = failures, restorations + rec.planned = planned + rec.changes, rec.deltas = changes, deltas + rec.cost = rep_cost if ctx.has_costs else None + rec.by_category, rec.by_node = by_category, by_node + rec.capacity_changes = rec.capacity_time = rec.delivered = None + rec.replacements = replaced + rec.opportunistic = opportunistic + rec.history = ( + (at_start, times_of, planned_of, started_up, system_record) + if history + else None + ) + if trace is not None: + trace.finish(t_simulation, rec) + return rec diff --git a/repyability/rbd/_events.py b/repyability/rbd/_events.py new file mode 100644 index 00000000..5a2fa518 --- /dev/null +++ b/repyability/rbd/_events.py @@ -0,0 +1,681 @@ +"""The records and state of a ``RepairableRBD`` simulation's event loop. + +The events on its queue (``Event``, ``_EventQueue``), a component's +maintenance and inspection schedules (``_Preventive``, ``_Inspection``, +``_MaintenanceGroup``), the repair crews' queue (``_Crews``), a standby +group's units (``_StandbyGroup``) and a common-cause group's causes +(``_Cause``). The loop itself is ``RepairableRBD._replicate``; the crews' +and standby groups' Markov chains (``_crew_chain``, ``_standby_chain``) +copy these rules, and the compiled engine (``_kernel``) the loop's. +""" + +import heapq +import math +from collections import deque +from dataclasses import dataclass, field +from typing import ( + TYPE_CHECKING, + Any, + Dict, + Hashable, + NamedTuple, + Optional, + Tuple, +) + +import numpy as np + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + + +# The kinds of a standby group's own events. +_UNIT_FAILS, _SPARE_FAILS, _UNIT_REPAIRED = 0, 1, 2 + + +class _EventQueue: + """The simulation's event queue. + + ``queue.PriorityQueue`` is this same heap (``heappush``/``heappop``) plus + thread locking on every operation, which a single-threaded simulation + only pays for; events come out in exactly the same order. + + The heap holds ``(time, event)`` pairs, so that times compare as floats + rather than through the events' own ordering. Two events at the same time + compare equal either way, so ties come out as they would with the events + alone. (A ``(time, counter, event)`` entry would release ties in the + order they were queued instead, which changes the results.) + """ + + __slots__ = ("_heap",) + + def __init__(self): + self._heap: list = [] + + def put(self, event: "Event") -> None: + heapq.heappush(self._heap, (event.time, event)) + + def get(self) -> "Event": + return heapq.heappop(self._heap)[1] + + def empty(self) -> bool: + return not self._heap + + def qsize(self) -> int: + return len(self._heap) + + +@dataclass(frozen=True) +class _Unit: + """A unit of a standby group: its repair is a job for the repair crews + under this key (a dataclass, so equal to no node name).""" + + node: Any + unit: int + + +@dataclass(frozen=True) +class _Cause: + """A shared cause of a common-cause group, as the simulation strikes + it (#158; a dataclass, so equal to no node name): the members it fails + (``struck``, those up when it strikes), its rate, and whether a test + that can miss its failures decides for all of them at once + (``coins``).""" + + struck: tuple + rate: float + coins: bool + + +class _CauseDraws: + """What a shared common cause draws in a simulation (#158): the times + between its strikes, exponential at its rate, and at each strike the + uniform that decides whether the tests that can miss its failures find + them (found, or missed, alike); from its own streams, or numpy's global + RNG.""" + + __slots__ = ("_rate", "_gaps", "_coins") + + def __init__(self, rate: float, gaps=None, coins=None): + self._rate = rate + self._gaps = gaps + self._coins = coins + + def gap(self) -> float: + if self._gaps is not None: + return self._gaps.draw() + return float(np.random.exponential(1.0 / self._rate)) + + def coin(self) -> float: + if self._coins is not None: + return self._coins.draw() + return float(np.random.random()) + + +class _StandbyDraws: + """What a standby group's units draw during a simulation: each unit's + lives and repair times from streams of its own, and the group's switches + from one (see ``_streams``); or, when they cannot be streamed, from the + unit models' ``random`` and numpy's global RNG.""" + + __slots__ = ("_models", "_lives", "_repairs", "_switches") + + def __init__(self, models, lives=None, repairs=None, switches=None): + self._models = models + self._lives = lives + self._repairs = repairs + self._switches = switches + + def life(self, unit: int) -> float: + if self._lives is None: + return self._models.reliability.random(1).item() + return self._lives[unit].draw() + + def repair(self, unit: int) -> float: + if self._repairs is None: + return self._models.time_to_replace.random(1).item() + return self._repairs[unit].draw() + + def switch(self) -> float: + if self._switches is None: + return float(np.random.random()) + return self._switches.draw() + + +class _StandbyGroup: + """A standby group's units during one simulation (a component spec's + ``"standby"``; see ``RepairableRBD``). + + Each unit put into service as new draws a life, used up at rate 1 while + it operates and at ``dormancy_factor`` while it waits as a spare (the + cumulative-exposure model of ``StandbyModel``): a spare that uses it up + fails in standby, and is found at once. A failed unit's repair is a job + for the RBD's repair crews, under the key ``_Unit(node, unit)`` (see + ``_Crews``), and its repair time is drawn when it fails. When an + operating unit fails and a spare is waiting, the one that has waited + longest is switched in, which works with ``switching_probability``; a + failed switch leaves the position empty until a repaired unit fills it, + and the spare waits on. A repaired unit fills an empty position, or + joins the spares. The group is up while ``k`` units operate. + + The group keeps its units' events in a queue of its own and has one + event in the RBD's queue, ``entry``: its next. The RBD passes that + event to ``advance``, which returns the group's state after it. + """ + + __slots__ = ( + "_rbd", + "_node", + "_k", + "_dormancy", + "_switching", + "_draws", + "_end", + "_operating", + "_spares", + "_life", + "_since", + "_version", + "_events", + "_order", + "entry", + "up", + ) + + def __init__( + self, + rbd: "RepairableRBD", + node, + arrangement: "_Standby", + draws, + t_simulation: float, + ): + self._rbd = rbd + self._node = node + self._k = arrangement.k + self._dormancy = arrangement.dormancy_factor + self._switching = arrangement.switching_probability + self._draws = ( + draws if isinstance(draws, _StandbyDraws) else _StandbyDraws(draws) + ) + self._end = t_simulation + self._operating: set = set() + self._spares: deque = deque() + # Each unit's life left, as of ``_since``; and a count that a + # queued event of the unit's must match to be current. + self._life = [0.0] * arrangement.units + self._since = [0.0] * arrangement.units + self._version = [0] * arrangement.units + self._events: list = [] + self._order = 0 + self.entry: Optional["Event"] = None + self.up = True + for unit in range(arrangement.units): + self._life[unit] = self._draws.life(unit) + if unit < self._k: + self._operate(unit, 0.0) + else: + self._wait(unit, 0.0) + self._arm() + + def _queue(self, time: float, kind: int, unit: int) -> None: + heapq.heappush( + self._events, + (time, self._order, kind, unit, self._version[unit]), + ) + self._order += 1 + + def _operate(self, unit: int, t: float) -> None: + self._operating.add(unit) + self._since[unit] = t + self._queue(t + self._life[unit], _UNIT_FAILS, unit) + + def _wait(self, unit: int, t: float) -> None: + self._spares.append(unit) + self._since[unit] = t + if self._dormancy > 0.0: + self._queue( + t + self._life[unit] / self._dormancy, _SPARE_FAILS, unit + ) + + def _repair(self, unit: int, t: float) -> None: + """Unit ``unit`` has failed at ``t``: its repair, drawn now, is a + job for a crew.""" + self._version[unit] += 1 + ends = t + self._draws.repair(unit) + crews = self._rbd._run_state._crews + if crews is None or crews.request( + _Unit(self._node, unit), t, Event(ends, None, True) + ): + self._queue(ends, _UNIT_REPAIRED, unit) + + def crew_started(self, unit: int, ends: float) -> None: + """A crew has started unit ``unit``'s repair late: it ends at + ``ends``.""" + self._queue(ends, _UNIT_REPAIRED, unit) + if self.entry is None or ends < self.entry.time: + self._arm() + + def _arm(self) -> None: + """Queue the group's next event in the RBD's queue (an earlier one + already there is left, and ignored).""" + events, version = self._events, self._version + while events and events[0][4] != version[events[0][3]]: + heapq.heappop(events) + if not events or events[0][0] >= self._end: + self.entry = None + return + self.entry = Event(events[0][0], self._node, self.up) + self._rbd._run_state._event_queue.put(self.entry) + + def holds(self, event: "Event") -> bool: + """Whether ``event``, from the RBD's queue, is the group's current + one (not superseded by an earlier).""" + return event is self.entry + + def advance(self, t: float) -> Tuple[bool, int]: + """Take the group's event at ``t`` (its ``entry``): whether it is up + after it, and how many of its units failed (each is repaired).""" + from . import _event_loop + + self.entry = None + events, version = self._events, self._version + while events[0][4] != version[events[0][3]]: + heapq.heappop(events) + _, _, kind, unit, _ = heapq.heappop(events) + failures = 0 + if kind == _UNIT_REPAIRED: + crews = self._rbd._run_state._crews + if crews is not None: + started = crews.release(_Unit(self._node, unit), t) + if started is not None: + _event_loop._crew_started(self._rbd, started) + self._life[unit] = self._draws.life(unit) + if len(self._operating) < self._k: + self._operate(unit, t) + else: + self._wait(unit, t) + elif kind == _SPARE_FAILS: + self._spares.remove(unit) + failures = 1 + self._repair(unit, t) + else: + self._operating.discard(unit) + failures = 1 + self._repair(unit, t) + if self._spares: + spare = self._spares.popleft() + switching = self._switching + if switching >= 1.0 or ( + switching > 0.0 and self._draws.switch() < switching + ): + # It has used up its life at the dormant rate so far (all + # of it, at most: a spare due to fail in standby now). + self._version[spare] += 1 + used = self._dormancy * (t - self._since[spare]) + self._life[spare] = max(self._life[spare] - used, 0.0) + self._operate(spare, t) + else: + self._spares.appendleft(spare) + self.up = len(self._operating) == self._k + self._arm() + return self.up, failures + + +class _Crews: + """The repair crews of one simulation (``RepairableRBD``'s + ``repair_crews``): how many are free, the components they are working + on, and the jobs waiting for one. + + A job is the work that brings a component back up: a repair or + replacement, maintenance that takes time, or a test that takes time. + The component is down from when the job fell due. A job that finds no + crew free waits, and the next crew to finish starts the waiting job of + the highest priority, and of those the one that fell due first. The + job's length was drawn when it fell due, so waiting shifts the + component's restoration without changing its draws. + """ + + __slots__ = ("free", "served", "holding", "waiting", "_order", "_rank") + + def __init__(self, crews: int, served, priority: dict): + self.free = crews + self.served = frozenset(served) + self.holding: set = set() + self.waiting: list = [] + self._order = 0 + # Higher priority first; a standby group's units have the group's. + self._rank = { + key: -priority.get( + key.node if isinstance(key, _Unit) else key, 0.0 + ) + for key in served + } + + def request(self, node, due: float, done: "Event") -> Optional["Event"]: + """``node``'s job, due at ``due`` and ending with ``done`` if a crew + starts it at once: ``done``, or None while the job waits.""" + if self.free: + self.free -= 1 + self.holding.add(node) + return done + entry = (self._rank[node], due, self._order, node, done) + heapq.heappush(self.waiting, entry) + self._order += 1 + return None + + def release(self, node, t: float) -> Optional[Tuple[Any, float, "Event"]]: + """``node``'s job is done at ``t``: its crew starts the next waiting + job, if any. That job's component, how long it waited, and the event + that now ends it; or None.""" + self.holding.discard(node) + if not self.waiting: + self.free += 1 + return None + _, due, _, waiting, done = heapq.heappop(self.waiting) + self.holding.add(waiting) + wait = t - due + ends = Event( + done.time + wait, + waiting, + done.status, + done.preventive, + done.inspection, + ) + return waiting, wait, ends + + +class _Fixed: + """A cost charged at the same amount every time.""" + + __slots__ = ("amount",) + + def __init__(self, amount: float): + self.amount = amount + + def draw(self) -> float: + return self.amount + + +class _Run: + """A simulation in progress on a diagram: what ``initialize_event_queue`` + starts and ``next_event`` advances. The diagram holds it as + ``_run_state`` while it runs (its public ``system_state``, + ``component_status``, ``t_simulation`` and ``last_change_planned`` + read it), a run starts a new one, and its end forgets it + (``_event_loop._forget_run``), so that nothing of a run is left on the + diagram to be shared, pickled or read by the next.""" + + __slots__ = ( + "_event_queue", + "system_state", + "t_simulation", + "component_status", + "last_change_planned", + "_pending_failure", + "_in_service", + "_levels", + "_renewed_at", + "_pending_event", + "_cancelled", + "_early", + "_renewing", + "_crews", + "_groups", + "_step_sources", + "_phases", + "_ccf_members", + "_ccf_pending", + "_ccf_coins", + "_calendar_price", + ) + + #: The events to come, in time order. + _event_queue: "_EventQueue" + #: Whether the system works, and each component. + system_state: bool + component_status: Dict[Hashable, bool] + #: The window's end. + t_simulation: float + #: Whether the last change of the system's state was planned. + last_change_planned: bool + #: When each working node with hidden failures is due to fail (None + #: once it has failed). + _pending_failure: Dict[Hashable, Optional[float]] + #: When each working node replaced on condition is due to fail, and to + #: be replaced (see ``_event_loop._replaced_at``). + _in_service: dict + #: Each working node replaced on condition by its measured level: when + #: its level was last known, and the level then. + _levels: dict + #: Opportunistic maintenance: when each member that can be renewed + #: early was put into service as new, and its pending event; the + #: pending events early renewals have cancelled, and the early + #: renewals queued, by identity, and their members (see + #: ``_event_loop._stop``). + _renewed_at: dict + _pending_event: dict + _cancelled: dict + _early: dict + _renewing: set + #: The repair crews, when there are fewer than the components that + #: may need one (None otherwise). + _crews: Optional["_Crews"] + #: Each standby group's units. + _groups: dict + #: What the components draw their events from, when the diagram + #: started the run itself (None when the caller passed them). + _step_sources: Optional[dict] + #: Each calendar a state shifts: by node, the time since its + #: last scheduled replacement or test. + _phases: dict + #: Common-cause groups: their members, each member's next event, + #: and the uniform a strike has decided its tests by. + _ccf_members: frozenset + _ccf_pending: dict + _ccf_coins: dict + #: Each component on a calendar policy: the price of its next planned + #: replacement (the opportunity's), charged when it falls; of its + #: early renewal at a stop of its maintenance group, once queued + #: (``inf``: the calendar allows none then, see + #: ``_event_loop._CalendarCharge``). + _calendar_price: dict + + def __init__(self) -> None: + self._step_sources = None + + +@dataclass(order=True, slots=True) +class Event: + """A component's change of state in the availability simulation. + + Events compare by ``time`` alone (the other fields take no part in + comparisons), so the simulation's event queue releases them in time + order. + + Attributes + ---------- + time : float + When the change happens, on the simulation clock. + component : Hashable + The node whose state changes. + status : bool + The node's state after the event: False for a failure, True for a + restoration. + preventive : bool + True for scheduled preventive maintenance, by default False. With + ``status`` False it starts a planned outage (maintenance that takes + time, or a nested RBD's planned outage); with ``status`` True it + ends one or, if the node is up, renews it in place (maintenance in + zero time). + inspection : bool + True for an inspection of a node whose failures are hidden, by + default False. With ``status`` True it is a test in zero time of a + working node, or the end of a test that took time; with ``status`` + False it starts such a test (a planned outage) or, if the node has + failed, finds the failure, and the repair starts. + missed : bool + True for an inspection of a failed node that does not find the + failure (a test whose coverage is below 1), by default False: the + node stays down until a full test finds it. + + Examples + -------- + >>> from repyability.rbd._events import Event + >>> Event(12.0, "pump", True) > Event(10.0, "valve", False) + True + """ + + time: float + component: Hashable = field(compare=False) + status: bool = field(compare=False) + preventive: bool = field(default=False, compare=False) + inspection: bool = field(default=False, compare=False) + missed: bool = field(default=False, compare=False) + + +class _Preventive(NamedTuple): + """A node's scheduled preventive maintenance: a replacement every + ``interval`` under the ``"age"`` or ``"block"`` policy, or under + ``"condition"`` an inspection at every multiple of ``interval`` that + replaces the unit if it is more likely than ``threshold`` to fail + before the next (or, with a ``level``, if its measured degradation + level is at or past it), or under ``"calendar"`` at the first time + its ``calendar`` policy's threshold is reached (``interval`` is then + ``inf``); a replacement takes a time drawn from ``duration`` (None: + no time).""" + + interval: float + policy: str + duration: Any + threshold: float = 0.0 + #: Under ``"age"`` or ``"calendar"``, the age from which the unit is + #: renewed early at a stop of its maintenance group (#108, #349); + #: ``inf`` for never. + opportunity: float = math.inf + #: Under ``"condition"`` with a degradation process for a life (#271): + #: the measured level at or past which an inspection replaces the + #: unit, in place of the age-based ``threshold``; None for none. + level: Optional[float] = None + #: Under ``"calendar"`` (#319): the ``CalendarPolicy`` whose threshold + #: at each time replaces the unit, at its calendar's price there. + calendar: Any = None + + def due(self, renewed: float) -> float: + """When the next preventive action (under ``"condition"``, the next + inspection) falls, for a unit put into service as new at + ``renewed``, or inspected then.""" + if self.policy == "age": + return renewed + self.interval + if self.policy == "calendar": + return self.calendar._next_planned(renewed, renewed)[0] + # Block, or inspections: the next multiple of the interval after + # ``renewed`` (a unit renewed on the schedule is not renewed again + # there). + due = float(np.floor(renewed / self.interval) + 1.0) * self.interval + return due if due > renewed else due + self.interval + + +class _Imperfect(NamedTuple): + """A component's imperfect repair (#109): ``kijima`` (``"kijima1"`` or + ``"kijima2"``) and its restoration factor ``q`` (0 renews, 1 is minimal + repair), and the failure at which it is replaced instead, counting + from its last renewal (None for never).""" + + kijima: str + q: float + replace_after: Optional[int] + + +class _MaintenanceGroup(NamedTuple): + """A maintenance group (#108): its members, the set-up cost charged + once at each of its stops, and whether every system outage is a stop + too.""" + + members: Tuple[Hashable, ...] + setup_cost: float + system_down: bool + + +class _Inspection(NamedTuple): + """A node's periodic inspection, which is what finds its (hidden) + failures: at ``offset`` and every ``interval`` after it (at every + multiple of ``interval`` with no offset), taking a time drawn from + ``duration`` (None: no time). A test finds a failure with probability + ``coverage``; the tests at ``offset`` and every ``full_test`` after it + (a whole multiple of ``interval``) find every failure, and with a + ``coverage`` of 1 every test does.""" + + interval: float + duration: Any + offset: float = 0.0 + coverage: float = 1.0 + full_test: Optional[float] = None + + @property + def partial(self) -> bool: + """Whether its tests can miss a failure (a coverage below 1).""" + return self.coverage < 1.0 + + @property + def period(self) -> float: + """The time after which its tests repeat: its full tests' interval + when its tests can miss a failure, else its interval.""" + if self.partial and self.full_test is not None: + return self.full_test + return self.interval + + @property + def per_full_test(self) -> int: + """Its tests from one full test to the next.""" + return int(round(self.period / self.interval)) + + def due(self, t: float) -> float: + """The first inspection after ``t``.""" + if not self.offset: + k = np.floor(t / self.interval) + 1.0 + due = float(k * self.interval) + return due if due > t else float((k + 1.0) * self.interval) + k = np.floor((t - self.offset) / self.interval) + 1.0 + due = float(self.offset + k * self.interval) + if due > t: + return due + return float(self.offset + (k + 1.0) * self.interval) + + def finds(self, t: float) -> float: + """The inspection that finds a failure at ``t``: the first at or + after it.""" + if not self.offset: + k = np.ceil(t / self.interval) + due = float(k * self.interval) + return due if due >= t else float((k + 1.0) * self.interval) + k = np.ceil((t - self.offset) / self.interval) + due = float(self.offset + k * self.interval) + if due >= t: + return due + return float(self.offset + (k + 1.0) * self.interval) + + def is_full(self, t: float) -> bool: + """Whether the test at ``t`` (one of its tests) finds every + failure.""" + if not self.partial: + return True + k = int(round((t - self.offset) / self.interval)) + return k % self.per_full_test == 0 + + def without_offset(self) -> "_Inspection": + """The schedule on a calendar from 0 (a start from a state, whose + ``since_scheduled`` places it).""" + return self._replace(offset=0.0) if self.offset else self + + +class _Standby(NamedTuple): + """A standby group (a component spec's ``"standby"``): ``units`` + identical units, ``k`` of which must operate, the rest waiting as + spares that age at ``dormancy_factor`` of the operating rate, switched + in with ``switching_probability``.""" + + units: int + k: int + dormancy_factor: float + switching_probability: float diff --git a/repyability/rbd/_hidden_tests.py b/repyability/rbd/_hidden_tests.py index bf93dc67..00175c60 100644 --- a/repyability/rbd/_hidden_tests.py +++ b/repyability/rbd/_hidden_tests.py @@ -254,11 +254,11 @@ def __init__(self, setup: "_Setup", left, right): masses[:, :-1] += left masses[:, 1:] += right # The intervals it can fall in. - total = masses.sum(axis=1) - last = np.flatnonzero(total > 0.0) + in_each = masses.sum(axis=1) + last = np.flatnonzero(in_each > 0.0) count = int(last[-1]) + 1 if last.size else 1 self.masses = masses[:count] - share = total[:count] + share = in_each[:count] self._pending = np.concatenate([np.cumsum(share[::-1])[::-1], [0.0]]) #: Its CDF at the long grid's points. self.P = np.concatenate([[0.0], np.cumsum((left + right).ravel())])[ @@ -1565,6 +1565,7 @@ def head_failures(x): before = float(_interpolated(failures0, [0], [frame], h)[0]) def head_up(x): + assert first_restart is not None s = frame + np.asarray(x, dtype=float) index = np.zeros(len(s), dtype=int) up = first_restart.cdf(0, s) diff --git a/repyability/rbd/_importance_time.py b/repyability/rbd/_importance_time.py index 518b4f8b..efbd8059 100644 --- a/repyability/rbd/_importance_time.py +++ b/repyability/rbd/_importance_time.py @@ -26,7 +26,11 @@ the system's availability so held, at each time or over the window. The criticality and Fussell-Vesely measures are averages over the chain's states: each is ``p(t) v`` for a vector ``v`` over the states (a node's -being down and critical in each, say), from the chain over time. +being down and critical in each, say), from the chain over time. A nested +RBD has crews of its own, so it is independent of the chain, and enters +by its own curve: the vectors are taken for each pattern of the nested +RBDs up and down, and averaged with the patterns' probabilities at each +time (#312, as ``point_availability`` has it). """ import math @@ -44,6 +48,9 @@ RRW = "rrw" CRITICALITY = "criticality" FUSSELL_VESELY = "fussell_vesely" +#: The joint importance of each pair, with common-cause groups (#317): +#: without them, each node's Birnbaum importance with the other held. +JOINT = "joint" #: The measures the crews' chain holds each node for (the others are #: averages over its states). @@ -65,22 +72,22 @@ def measure( """``spec``'s measure of each node of ``rbd`` at the times ``x``, or over the windows ``[0, window)``: floats for a single time or window, else arrays in its shape.""" - from .repairable_rbd import _check_times + from repyability.utils.checks import nonnegative_times + + from . import _curves if x is not None and window is not None: raise ValueError( "Give the times x or a window's length, not both: x evaluates " "the measure at each time, window averages it over [0, window)." ) - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) - rbd._validate_node_overrides(working, broken) - states = rbd._states(state, working | broken) + working, broken = rbd._held_nodes(working_nodes, broken_nodes) + states = _curves._states(rbd, state, working | broken) if x is not None: - times = _check_times(x) + times = nonnegative_times(x) values = _at(rbd, spec, times.ravel(), working, broken, states, state) return _shaped(values, x, times.shape) - ends = _check_times(window) + ends = nonnegative_times(window) if np.any(ends <= 0.0): raise ValueError( f"A window must be a positive length of time, got {window!r}." @@ -128,11 +135,14 @@ def _base(rbd, spec: Spec, p: dict, q: dict, weights) -> dict: return RBD._fussell_vesely(rbd, p, spec.fv_type, spec.method, weights, q) -def _grouped(rbd, spec: Spec, groups: list, p: dict, q: dict, x, weights): +def _grouped( + rbd, spec: Spec, groups: list, p: dict, q: dict, x, weights, held +): """The measure with common-cause groups (their chains over time, ``groups``, see ``_ccf_chain.OverTime``) at the times ``x``: averaged with ``weights``, or at each time without (see - ``RepairableRBD._ccf_measure``).""" + ``RepairableRBD._ccf_measure``); ``held`` are the nodes held up or + down.""" from . import _ccf_chain tables = [ @@ -156,15 +166,18 @@ def _grouped(rbd, spec: Spec, groups: list, p: dict, q: dict, x, weights): spec.kind, spec.fv_type, spec.method, + held, ) def _independent(rbd, curves: dict, x, working, broken) -> Tuple[dict, dict]: """The nodes' probabilities of working and of failing at the times ``x``, from their ``curves`` (the forced nodes held at 1 or 0).""" + from . import _windows + size = len(x) p = {node: np.asarray(c.at(x), dtype=float) for node, c in curves.items()} - p = rbd._filled(p, size, working, broken) + p = _windows._filled(rbd, p, size, working, broken) q = rbd._failures_with_overrides( {node: 1.0 - value for node, value in p.items()}, working, broken ) @@ -176,6 +189,8 @@ def _independent(rbd, curves: dict, x, working, broken) -> Tuple[dict, dict]: def _at(rbd, spec, times, working, broken, states, state) -> dict: """The measure of every node at each of ``times``, as arrays.""" + from . import _ccf_groups, _curves + if rbd._crews_couple(): if spec.name in _HELD: return _held(rbd, spec, working, broken, state, times=times) @@ -183,14 +198,18 @@ def _at(rbd, spec, times, working, broken, states, state) -> dict: horizon = float(times.max()) if times.size else 0.0 forced = working | broken if rbd.ccf_groups: - grouped = rbd._groups_curve(horizon, working, broken, "p", states) + grouped = _ccf_groups._groups_curve( + rbd, horizon, working, broken, "p", states + ) p, q = _independent(rbd, grouped.curves, times, working, broken) - values = _grouped(rbd, spec, grouped.system.groups, p, q, times, None) + values = _grouped( + rbd, spec, grouped.system.groups, p, q, times, None, forced + ) return { node: np.asarray(value, dtype=float) for node, value in values.items() } - curves = rbd._availability_curves(horizon, forced, state=states) + curves = _curves._availability_curves(rbd, horizon, forced, state=states) p, q = _independent(rbd, curves, times, working, broken) return { node: np.asarray(value, dtype=float) @@ -216,7 +235,7 @@ def window_points( constant or a period they are not followed: the constant takes the rest of the window at one point, a period's points its number of repeats.""" - from .repairable_rbd import _settling + from ._curves import _settling settle, period = _settling(curves) reach = min(end, settle if period is None else settle + period) @@ -268,7 +287,8 @@ def estimate(a, b): def _over(rbd, spec, end, working, broken, states, state) -> dict: """The measure of every node over the window ``[0, end)``.""" - from .repairable_rbd import _MISSION_POINTS + from . import _ccf_groups, _curves + from ._curves import _MISSION_POINTS if rbd._crews_couple(): if spec.name in _HELD: @@ -276,7 +296,9 @@ def _over(rbd, spec, end, working, broken, states, state) -> dict: return _chain_measure(rbd, spec, working, broken, states, end=end) forced = working | broken if rbd.ccf_groups: - grouped = rbd._groups_curve(end, working, broken, "p", states) + grouped = _ccf_groups._groups_curve( + rbd, end, working, broken, "p", states + ) def system(x): return {"up": grouped.at(x)} @@ -284,9 +306,9 @@ def system(x): x, weights = window_points([grouped], end, system, _MISSION_POINTS) p, q = _independent(rbd, grouped.curves, x, working, broken) return _grouped( - rbd, spec, grouped.system.groups, p, q, x, weights / end + rbd, spec, grouped.system.groups, p, q, x, weights / end, forced ) - curves = rbd._availability_curves(end, forced, state=states) + curves = _curves._availability_curves(rbd, end, forced, state=states) def integrands(x): # The system's availability and each node's Birnbaum importance: @@ -348,44 +370,84 @@ def _chain_measure(rbd, spec, working, broken, states, times=None, end=None): measure of every node, at the ``times`` or over ``[0, end)``: its numerator and denominator are averages over the crews' chain's states (as in the long run), so each is ``p(t) v`` for a vector ``v`` over - the states, followed over time from the components' ``states``.""" + the states, followed over time from the components' ``states``. A + nested RBD has crews of its own, so it is independent of the chain + (as ``_chain_transient.CrewCurve`` has it): the vectors are taken for + each pattern of the nested RBDs up and down, ``v_m``, and the average + is ``sum_m p(t) v_m w_m(t)``, ``w_m(t)`` the pattern's probability + from their own curves then (see ``_chain_transient.pattern_weights``; + with no nested RBD, the one pattern has weight 1). Over a window, the + integral is the chain's own where no pattern's weight changes, and by + quadrature otherwise.""" + from . import _chain_transient, _crews, _curves, _windows + from ._curves import _MISSION_POINTS + forced = frozenset(working | broken) - nested = rbd._require_crew_over_time(forced) - if nested: - raise NotImplementedError( - f"With {rbd.repair_crews} repair crew(s), the criticality and " - "Fussell-Vesely measures over time are averages over the " - "crews' chain's states, which take no nested RBD's own curve " - f"in, as yet: node {nested[0]!r} is one. Their long-run values " - "are worked out, and the Birnbaum importance, improvement " - "potential and risk worths over time." - ) - p, _ = rbd._chain_probabilities(working, broken) - q = {node: 1.0 - value for node, value in p.items()} - importance, works, fails, p, q = rbd._importances(p, q) - size = len(works) - if spec.name == CRITICALITY: - member, system = (p, works) if spec.kind == "success" else (q, fails) - numerators = { - node: importance[node] * member[node] for node in rbd.nodes - } - else: - numerators = rbd._fv_numerators(p, q, size, spec.fv_type, spec.method) - system = fails + nested = _curves._require_crew_over_time(rbd, forced) + chain = _crews._crew_chain(rbd, forced) + size = len(chain.probabilities) nodes: List = list(rbd.nodes) - columns = np.column_stack([system] + [numerators[n] for n in nodes]) - chain = rbd._crew_chain(forced) - followed = rbd._uniformized( + columns = [] + for pattern in range(2 ** len(nested)): + own = { + node: chain.up[:, k].astype(float) + for k, node in enumerate(chain.nodes) + } + for j, node in enumerate(nested): + own[node] = np.full(size, float((pattern >> j) & 1)) + importance, works, fails, p, q = rbd._importances( + _windows._filled(rbd, own, size, working, broken) + ) + if spec.name == CRITICALITY: + member, system = ( + (p, works) if spec.kind == "success" else (q, fails) + ) + numerators = { + node: importance[node] * member[node] for node in nodes + } + else: + numerators = rbd._fv_numerators( + p, q, size, spec.fv_type, spec.method + ) + system = fails + columns += [system] + [numerators[node] for node in nodes] + followed = _curves._uniformized( "The repair crews'", chain.generator, - rbd._crew_start(chain, states), + _curves._crew_start(rbd, chain, states), chain.probabilities, - columns, + np.column_stack(columns), ) + horizon = end if times is None else float(times.max(initial=0.0)) + curves = [ + _curves._nested_curve( + rbd.components[node], horizon, start=states.get(node) + ) + for node in nested + ] + + def patterned(x: np.ndarray) -> np.ndarray: + """The averages over the chain's states and the nested RBDs' + patterns at the times ``x`` (rows), for the system and each + node's numerator (columns).""" + values = followed.values(x).reshape(len(x), 2 ** len(curves), -1) + weights = _chain_transient.pattern_weights( + [curve.at(x) for curve in curves] + ) + return np.einsum("tm,tmk->tk", weights, values) + if times is not None: - values = followed.values(times) - else: + values = patterned(times) + elif not curves: values = followed.integrals(np.array([end])) / end + else: + x, weights = window_points( + [_chain_transient.ChainCurve(followed), *curves], + end, + lambda x: dict(enumerate(patterned(x).T)), + _MISSION_POINTS, + ) + values = (weights @ patterned(x))[None] / end total = values[:, 0] out: Dict = {} with np.errstate(divide="ignore", invalid="ignore"): diff --git a/repyability/rbd/_intervals.py b/repyability/rbd/_intervals.py new file mode 100644 index 00000000..53d481ab --- /dev/null +++ b/repyability/rbd/_intervals.py @@ -0,0 +1,885 @@ +"""A ``RepairableRBD``'s maintenance intervals: the age-replacement and +proof-test intervals that meet a target at the lowest cost +(``optimal_replacement_intervals``, ``optimal_inspection_intervals``), +chosen from the allowed calendar every combination at a time or, past +``_MAX_COMBINATIONS``, by a local search, and the diagram with given +intervals (``with_intervals``). The methods of ``RepairableRBD`` of those +names call these. +""" + +import itertools +import math +from typing import ( + TYPE_CHECKING, + Callable, + Collection, + Hashable, + Optional, + Tuple, +) + +import numpy as np +from scipy.optimize import minimize + +from repyability.rbd import _crews, _requirements +from repyability.rbd._model_utils import ( + failure_time_scale, +) +from repyability.rbd.results import ( + MaintenancePlan, +) +from repyability.rbd.routes import Refused + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + + +#: What the interval choices say when a component can wait for a crew +#: (#184): see ``_require_interval_crews``. +_INTERVAL_CREWS = ( + "the exact long-run values the intervals are chosen by assume (the " + "Markov chain of the repair queue has no place for scheduled " + "maintenance or tests)", + "Choose them as if every repair started at once, with " + "assume_unlimited_crews=True, then simulate the plan with the crews: " + "with_intervals(plan).cost() or .availability(), or compare() it with " + "the schedules you have.", +) + + +def _objective(cost: float, availability: float, max_cost_rate) -> float: + """What an interval choice minimises: the cost rate, or the + unavailability when the cost rate is capped.""" + return 1.0 - availability if max_cost_rate is not None else cost + + +def _meets(cost, availability, min_availability, max_cost_rate) -> bool: + """Whether long-run values meet an interval choice's target.""" + if min_availability is not None and availability < min_availability: + return False + if max_cost_rate is not None and cost > max_cost_rate: + return False + return True + + +def _choose_intervals( + nodes: list, + evaluate, + starts: list, + bounds: Tuple[np.ndarray, np.ndarray], + min_availability: Optional[float], + max_cost_rate: Optional[float], +) -> dict: + """The intervals (by node) that minimise the cost rate, or, with a cost + cap, the unavailability, subject to the target: SLSQP over the + logarithms of the intervals from each of ``starts``, keeping the best + that meets the target. ``evaluate(intervals)`` gives their exact + ``(cost rate, availability)``. A ValueError if no intervals in + ``bounds`` meet the target, giving the best they can do.""" + low, high = bounds + box = list(zip(low, high)) + options = {"ftol": 1e-10, "maxiter": 200, "eps": 1e-6} + cache: dict = {} + + def point(x) -> tuple: + """``x`` rounded: the intervals evaluated, and returned.""" + return tuple(np.round(np.asarray(x, dtype=float), 12)) + + def values(x) -> Tuple[float, float]: + key = point(x) + if key not in cache: + cache[key] = evaluate( + {node: math.exp(xi) for node, xi in zip(nodes, key)} + ) + return cache[key] + + # The objective and the constraint, scaled to be about 1. + cost0, availability0 = values(high) + down0 = max(1.0 - availability0, 1e-300) + rate0 = max(abs(cost0), 1e-300) + + def unavailable(x): + return (1.0 - values(x)[1]) / down0 + + def cost(x): + return values(x)[0] / rate0 + + if max_cost_rate is not None: + objective = unavailable + + def slack(x): + return (max_cost_rate * (1.0 - 1e-12) - values(x)[0]) / ( + max_cost_rate + ) + + else: + objective = cost + + def slack(x): + assert min_availability is not None + return (values(x)[1] - min_availability - 1e-12) / ( + 1.0 - min_availability + ) + + constrained = min_availability is not None or max_cost_rate is not None + if constrained and not any( + _meets(*values(start), min_availability, max_cost_rate) + for start in starts + ): + # Can the target be met at all? The most available intervals, or + # the cheapest, from the starts nearest to it first. + extreme = unavailable if min_availability is not None else cost + found = [] + for start in sorted(starts, key=extreme): + reach = minimize( + extreme, start, method="SLSQP", bounds=box, options=options + ).x + found.append(reach) + if _meets(*values(reach), min_availability, max_cost_rate): + break + else: + best_cost, best_availability = values( + min(found + list(starts), key=extreme) + ) + if min_availability is not None: + raise ValueError( + f"min_availability {min_availability} cannot be met: the " + f"most any intervals give is {best_availability:.6g}." + ) + raise ValueError( + f"max_cost_rate {max_cost_rate} cannot be met: the least any " + f"intervals cost is {best_cost:.6g} per unit time." + ) + starts = [reach] + list(starts) + best, best_value = None, math.inf + for start in starts: + found = minimize( + objective, + start, + method="SLSQP", + bounds=box, + constraints=( + [{"type": "ineq", "fun": slack}] if constrained else [] + ), + options=options, + ) + for x in (found.x, start): + rate, availability = values(x) + if not _meets(rate, availability, min_availability, max_cost_rate): + continue + value = _objective(rate, availability, max_cost_rate) + if value < best_value: + best, best_value = np.asarray(x, dtype=float), value + assert best is not None # the start that meets the target is kept + return {node: math.exp(xi) for node, xi in zip(nodes, point(best))} + + +#: Allowed intervals are chosen by trying every combination, up to this many; +#: by a local search beyond. +_MAX_COMBINATIONS = 2000 + + +#: Plans whose costs are within this share of each other, or whose +#: availabilities are within this of each other, are as good: which is +#: chosen does not hang on the last bits of their arithmetic (#229), as it +#: did for identical members' intervals in turn, and of plans that cost the +#: same the most available is chosen however their costs' last bits fall. +_AS_GOOD = 1e-10 + + +def _lower(a: tuple, b: tuple) -> bool: + """Whether merit ``a`` is lower than merit ``b``: their ``(value, + size)`` pairs compared in turn, values within ``_AS_GOOD`` of their + size taken as equal.""" + for (x, size_x), (y, size_y) in zip(a, b): + if x == y: + continue + if abs(x - y) > _AS_GOOD * max(size_x, size_y) or math.isinf(x - y): + return x < y + return False + + +def _divides(interval: float, full: float) -> bool: + """Whether ``interval`` divides ``full`` a whole number of times.""" + count = full / interval + return round(count) >= 1 and abs(count - round(count)) <= 1e-9 * count + + +def _require_chosen(per_node, chosen: list, name: str) -> None: + """Raise if a dict of per-node options names a node not chosen + (#222): a typo would otherwise be dropped.""" + if isinstance(per_node, dict): + others = [node for node in per_node if node not in chosen] + if others: + raise ValueError( + f"{name} names {others}, which are not among the components " + f"whose intervals are chosen: {chosen}." + ) + + +def _choose_divisor( + node, + full: float, + bounds: Tuple[float, float], + evaluate, + min_availability: Optional[float], + max_cost_rate: Optional[float], +) -> dict: + """The interval of a component whose tests can miss a failure (#221), + from those that divide its full tests' interval ``full`` (``full / + k``) within ``bounds``, as ``_choose_from`` would choose from them all: + from some 50 spread evenly in their logarithms, then every one between + the best's neighbours among those.""" + low, high = bounds + first = max(1, math.ceil(full / high)) + last = max(first, math.floor(full / low)) + + def best_of(ks) -> int: + options = {node: tuple(full / k for k in ks)} + chosen = _choose_from( + [node], options, evaluate, min_availability, max_cost_rate + ) + return int(round(full / chosen[node])) + + if last - first < 64: + return {node: full / best_of(range(first, last + 1))} + spread = sorted({int(k) for k in np.round(np.geomspace(first, last, 50))}) + k = best_of(spread) + i = spread.index(k) + around = range( + spread[max(i - 1, 0)], spread[min(i + 1, len(spread) - 1)] + 1 + ) + return {node: full / best_of(around)} + + +def _choose_from( + nodes: list, + options: dict, + evaluate, + min_availability: Optional[float], + max_cost_rate: Optional[float], +) -> dict: + """The intervals, each from its node's ``options``, that minimise the + cost rate, or with a cost cap the unavailability, subject to the target: + every combination when there are at most 2000, a local search (one + interval changed at a time, from several starts) otherwise. Of plans + that cost the same, the most available is chosen (and of those as + available, the cheapest): tests' offsets change no cost. Of plans as + good (``_AS_GOOD``), the first tried is kept. A ValueError if none + meets the target, giving the best any does.""" + + def merit(intervals: dict) -> tuple: + """(How far from the target, the objective, the other one), each + with the size its differences are measured against: lower is + better.""" + cost, availability = evaluate(intervals) + priced = abs(cost) + if min_availability is not None: + short = (max(0.0, min_availability - availability), 1.0) + elif max_cost_rate is not None: + short = (max(0.0, cost - max_cost_rate), priced) + else: + short = (0.0, 1.0) + down = (1.0 - availability, 1.0) + if max_cost_rate is None: + return short, (cost, priced), down + return short, down, (cost, priced) + + count = math.prod(len(options[node]) for node in nodes) + best: Optional[dict] = None + best_value: tuple = () + if count <= _MAX_COMBINATIONS: + for combination in itertools.product( + *(options[node] for node in nodes) + ): + candidate = dict(zip(nodes, combination)) + candidate_value = merit(candidate) + if best is None or _lower(candidate_value, best_value): + best, best_value = candidate, candidate_value + else: + starts = [ + {node: options[node][0] for node in nodes}, + {node: options[node][-1] for node in nodes}, + {node: options[node][len(options[node]) // 2] for node in nodes}, + ] + for current in starts: + value = merit(current) + improved = True + while improved: + improved = False + for node in nodes: + for option in options[node]: + candidate = {**current, node: option} + candidate_value = merit(candidate) + if _lower(candidate_value, value): + current, value = candidate, candidate_value + improved = True + if best is None or _lower(value, best_value): + best, best_value = current, value + assert best is not None + cost, availability = evaluate(best) + if not _meets(cost, availability, min_availability, max_cost_rate): + if min_availability is not None: + raise ValueError( + f"min_availability {min_availability} cannot be met: the " + f"most the allowed intervals give is {availability:.6g}." + ) + raise ValueError( + f"max_cost_rate {max_cost_rate} cannot be met: the least the " + f"allowed intervals cost is {cost:.6g} per unit time." + ) + return best + + +def _with_intervals( + rbd, preventive=None, inspection=None, offsets=None +) -> "RepairableRBD": + """This RBD with the intervals of some of its preventive or + inspection schedules changed, for the exact long-run values: a + shallow copy, sharing everything else, including the renewal cycles + already worked out (kept by interval). ``offsets`` gives some + inspected components' first tests as shares of their intervals.""" + # The renewal cycles, kept by interval, are shared with the plan; + # what its own schedules decide is worked out again. + for name in ("age_cycles", "block_cycles", "tested_units"): + rbd._cache.kept(name) + plan = rbd._shallow_copy("ccf_tables") + if preventive: + plan._preventive = dict(rbd._preventive) + for node, interval in preventive.items(): + plan._preventive[node] = rbd._preventive[node]._replace( + interval=float(interval) + ) + if inspection: + plan._inspection = dict(rbd._inspection) + for node, interval in inspection.items(): + schedule = rbd._inspection[node] + # An offset keeps its share of the interval (a pair tested + # half an interval apart stays so). + share = schedule.offset / schedule.interval + plan._inspection[node] = schedule._replace( + interval=float(interval), offset=share * float(interval) + ) + if offsets: + plan._inspection = dict(plan._inspection) + for node, share in offsets.items(): + schedule = plan._inspection[node] + plan._inspection[node] = schedule._replace( + offset=float(share) * schedule.interval + ) + return plan + + +def _interval_targets(rbd, min_availability, max_cost_rate): + """The checked targets of an interval choice.""" + if min_availability is not None and max_cost_rate is not None: + raise ValueError( + "Give at most one of min_availability and max_cost_rate." + ) + if not rbd.has_costs: + raise Refused( + "Nothing is priced, so no interval costs more than another: " + "give the components costs (or a downtime_cost_rate)." + ) + if min_availability is not None: + if not 0.0 < float(min_availability) < 1.0: + raise ValueError( + "min_availability must be a number in (0, 1), got " + f"{min_availability!r}." + ) + return float(min_availability), None + if max_cost_rate is not None: + if not 0.0 < float(max_cost_rate) < math.inf: + raise ValueError( + "max_cost_rate must be a positive number, got " + f"{max_cost_rate!r}." + ) + return None, float(max_cost_rate) + return None, None + + +def optimal_replacement_intervals( + rbd, + nodes: Optional[Collection[Hashable]], + *, + allowed, + min_availability: Optional[float], + max_cost_rate: Optional[float], + assume_unlimited_crews: bool, +) -> MaintenancePlan: + """See ``RepairableRBD.optimal_replacement_intervals``.""" + chosen = _maintained(rbd, nodes) + min_availability, max_cost_rate = _interval_targets( + rbd, min_availability, max_cost_rate + ) + if rbd._crews_couple(): + if not assume_unlimited_crews: + _require_interval_crews(rbd) + return _unlimited_crews(rbd).optimal_replacement_intervals( + chosen, + allowed=allowed, + min_availability=min_availability, + max_cost_rate=max_cost_rate, + ) + + def evaluate(intervals: dict) -> Tuple[float, float]: + plan = _with_intervals(rbd, preventive=intervals) + return plan.expected_cost_rate(), plan.mean_availability() + + if allowed is not None: + best = _choose_from( + chosen, + _allowed_intervals(allowed, chosen, never=True), + evaluate, + min_availability, + max_cost_rate, + ) + else: + best = _searched_replacements( + rbd, chosen, evaluate, min_availability, max_cost_rate + ) + cost, availability = evaluate(best) + return MaintenancePlan( + {node: float(best[node]) for node in chosen}, cost, availability + ) + + +def _searched_replacements( + rbd, + chosen: list, + evaluate: Callable[[dict], Tuple[float, float]], + min_availability: Optional[float], + max_cost_rate: Optional[float], +) -> dict: + """The age-replacement intervals of ``chosen`` that + ``optimal_replacement_intervals`` gives without ``allowed``: by a + gradient search over their logarithms, each from a thousandth to a + thousand times its component's mean life, one at the top of its + range compared with never replacing it.""" + scale = {} + for node in chosen: + life = failure_time_scale(rbd.components[node].reliability) + scale[node] = life if np.isfinite(life) and life > 0.0 else 1.0 + low = np.array([math.log(1e-3 * scale[n]) for n in chosen]) + high = np.array([math.log(1e3 * scale[n]) for n in chosen]) + current = [ + math.log(min(rbd._preventive[n].interval, 1e3 * scale[n])) + for n in chosen + ] + starts = [ + np.clip(current, low, high), + np.array([math.log(scale[n]) for n in chosen]), + np.array([math.log(0.3 * scale[n]) for n in chosen]), + high.copy(), + ] + best = _choose_intervals( + chosen, + evaluate, + starts, + (low, high), + min_availability, + max_cost_rate, + ) + # An interval at the top of its range may be better never used. + for i, node in enumerate(chosen): + if best[node] >= 0.99 * math.exp(high[i]): + never = {**best, node: math.inf} + cost, availability = evaluate(never) + if _meets( + cost, availability, min_availability, max_cost_rate + ) and _objective(cost, availability, max_cost_rate) <= _objective( + *evaluate(best), max_cost_rate + ): + best = never + return best + + +def optimal_inspection_intervals( + rbd, + nodes: Optional[Collection[Hashable]], + *, + allowed, + min_availability: Optional[float], + max_cost_rate: Optional[float], + assume_unlimited_crews: bool, + offset_shares, +) -> MaintenancePlan: + """See ``RepairableRBD.optimal_inspection_intervals``.""" + chosen = _inspected(rbd, nodes) + min_availability, max_cost_rate = _interval_targets( + rbd, min_availability, max_cost_rate + ) + if rbd._crews_couple(): + if not assume_unlimited_crews: + _require_interval_crews(rbd) + return _unlimited_crews(rbd).optimal_inspection_intervals( + chosen, + allowed=allowed, + min_availability=min_availability, + max_cost_rate=max_cost_rate, + offset_shares=offset_shares, + ) + rates = {node: _requirements._tested_scale(rbd, node) for node in chosen} + + def evaluate(intervals: dict) -> Tuple[float, float]: + plan = _with_intervals(rbd, inspection=intervals) + return plan.expected_cost_rate(), plan.mean_availability() + + if offset_shares is not None: + if allowed is None: + raise ValueError( + "offset_shares are chosen with intervals from allowed: " + "give allowed (when a lone component is tested changes " + "nothing in the long run)." + ) + return _choose_tests( + rbd, + chosen, + _dividing(rbd, _allowed_intervals(allowed, chosen)), + _allowed_shares(offset_shares, chosen), + min_availability, + max_cost_rate, + ) + if allowed is None and rbd._inspection[chosen[0]].partial: + _requirements._require_one_inspected(rbd) + (node,) = chosen + rate = rates[node] + best = _choose_divisor( + node, + # Its full tests' interval (see _Inspection.period). + rbd._inspection[node].period, + (1e-4 / rate, 10.0 / rate), + evaluate, + min_availability, + max_cost_rate, + ) + elif allowed is None: + _requirements._require_one_inspected(rbd) + (node,) = chosen + rate = rates[node] + low = np.array([math.log(1e-4 / rate)]) + high = np.array([math.log(10.0 / rate)]) + current = math.log(rbd._inspection[node].interval) + starts = [ + np.clip([current], low, high), + np.array([math.log(0.01 / rate)]), + np.array([math.log(0.1 / rate)]), + np.array([math.log(1.0 / rate)]), + ] + best = _choose_intervals( + chosen, + evaluate, + starts, + (low, high), + min_availability, + max_cost_rate, + ) + else: + best = _choose_from( + chosen, + _dividing(rbd, _allowed_intervals(allowed, chosen)), + evaluate, + min_availability, + max_cost_rate, + ) + cost, availability = evaluate(best) + # The offsets the plan's tests keep: each its share of the interval. + offsets_kept = { + node: float(best[node]) + * rbd._inspection[node].offset + / rbd._inspection[node].interval + for node in chosen + } + return MaintenancePlan( + {node: float(best[node]) for node in chosen}, + cost, + availability, + offsets=offsets_kept, + ) + + +def _dividing(rbd, options: dict) -> dict: + """``options``, the intervals allowed each chosen component, + checked to divide its full tests' interval where its tests can + miss a failure (#221): every full test is a whole number of tests + on, so an interval that does not divide it cannot be its.""" + for node, intervals in options.items(): + schedule = rbd._inspection[node] + if not schedule.partial: + continue + full = schedule.period + bad = [i for i in intervals if not _divides(i, full)] + if bad: + some = ", ".join(f"{full / k:g}" for k in range(1, 5)) + raise ValueError( + f"The tests of component {node!r} can miss a failure, so " + f"its full tests' interval, {full:g}, must be a whole " + "number of its test intervals: " + f"{', '.join(map(repr, bad))} " + f"do{'es' if len(bad) == 1 else ''} not divide it. Allow " + f"intervals that do ({some}, ...)." + ) + return options + + +def _require_interval_crews(rbd) -> None: + """Raise if a component can wait for a repair crew, for the interval + choices (see ``_INTERVAL_CREWS``).""" + _crews._require_unlimited_crews(rbd, *_INTERVAL_CREWS) + + +def _unlimited_crews(rbd) -> "RepairableRBD": + """This RBD with as many repair crews as jobs, built as it was.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + return RepairableRBD(**{**rbd._init_args, "repair_crews": None}) + + +def with_intervals(rbd, intervals, offsets) -> "RepairableRBD": + """See ``RepairableRBD.with_intervals``.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if isinstance(intervals, MaintenancePlan): + if offsets is None: + offsets = intervals.offsets + intervals = intervals.intervals + offsets = dict(offsets or {}) + components = dict(rbd._init_args["components"]) + for node in list(dict(intervals)) + list(offsets): + _requirements._require_component(rbd, node, "with_intervals") + for node, interval in dict(intervals).items(): + spec = components.get(node) + # A component has one schedule at most (see the constructor). + kinds = [ + kind + for kind in ("preventive", "inspection") + if isinstance(spec, dict) and spec.get(kind) is not None + ] + if not kinds: + raise ValueError( + f"Node {node!r} has no maintenance or test schedule to " + "give an interval." + ) + (kind,) = kinds + assert isinstance(spec, dict) + if node in offsets and kind != "inspection": + raise ValueError( + f"Node {node!r} has an offset but no test schedule: an " + "offset is the time of a component's first test." + ) + old = dict(spec[kind]) + interval = float(interval) + new = {**old, "interval": interval} + if kind == "inspection": + if node in offsets: + new["offset"] = float(offsets[node]) + elif old.get("offset"): + share = float(old["offset"]) / float(old["interval"]) + new["offset"] = share * interval + spec = {**spec, kind: new} + if kind == "preventive" and math.isinf(interval): + if old.get("policy", "age") != "age": + raise ValueError( + f"Node {node!r}: only age replacement can be left to " + "failures (an interval of inf)." + ) + spec = {k: v for k, v in spec.items() if k != "preventive"} + components[node] = spec + unknown = set(offsets) - set(dict(intervals)) + for node in unknown: + spec = components.get(node) + if not (isinstance(spec, dict) and spec.get("inspection")): + raise ValueError( + f"Node {node!r} has an offset but no test schedule." + ) + components[node] = { + **spec, + "inspection": { + **spec["inspection"], + "offset": float(offsets[node]), + }, + } + return RepairableRBD(**{**rbd._init_args, "components": components}) + + +def _choose_tests( + rbd, + chosen: list, + intervals: dict, + shares: dict, + min_availability: Optional[float], + max_cost_rate: Optional[float], +) -> MaintenancePlan: + """``optimal_inspection_intervals`` with the offsets chosen too: each + node's interval from ``intervals`` and its first test's share of it + from ``shares``, searched together as ``_choose_from`` searches + intervals.""" + if set(chosen) == set(rbd._inspection): + # Shifting every test by one time changes nothing in the long + # run: the first node's tests are kept from 0. + shares = {**shares, chosen[0]: (0.0,)} + options = {("interval", n): intervals[n] for n in chosen} + options.update({("offset", n): shares[n] for n in chosen}) + + def evaluate(choice: dict) -> Tuple[float, float]: + plan = _with_intervals( + rbd, + inspection={n: choice[("interval", n)] for n in chosen}, + offsets={n: choice[("offset", n)] for n in chosen}, + ) + return plan.expected_cost_rate(), plan.mean_availability() + + best = _choose_from( + list(options), options, evaluate, min_availability, max_cost_rate + ) + cost, availability = evaluate(best) + return MaintenancePlan( + {n: float(best[("interval", n)]) for n in chosen}, + cost, + availability, + offsets={ + n: float(best[("offset", n)] * best[("interval", n)]) + for n in chosen + }, + ) + + +def _allowed_shares(offsets, chosen: list) -> dict: + """``offsets`` (see ``optimal_inspection_intervals``) as a sorted + tuple of shares of the interval per chosen node.""" + if isinstance(offsets, str): + if offsets != "stagger": + raise ValueError( + "offset_shares must be shares of the interval in [0, " + "1), a dict of them per node, or 'stagger', got " + f"{offsets!r}." + ) + n = len(chosen) + return {node: tuple(k / n for k in range(n)) for node in chosen} + per_node = ( + offsets + if isinstance(offsets, dict) + else {node: offsets for node in chosen} + ) + _require_chosen(per_node, chosen, "offset_shares") + out = {} + for node in chosen: + if node not in per_node: + raise ValueError( + f"offset_shares gives no share of the interval for node " + f"{node!r}: give one, in [0, 1), for each of {chosen}." + ) + given = per_node[node] + if isinstance(given, (str, bytes)) or not isinstance( + given, Collection + ): + given = [given] + values = [] + for value in given: + number = isinstance( + value, (int, float, np.integer, np.floating) + ) and not isinstance(value, bool) + share = float(value) if number else float("nan") + if not 0.0 <= share < 1.0: + raise ValueError( + "offset_shares are shares of the interval, in [0, " + f"1), got {value!r} for node {node!r}." + ) + values.append(share) + if not values: + raise ValueError( + f"offset_shares gives no share of the interval for node " + f"{node!r}." + ) + out[node] = tuple(sorted(set(values))) + return out + + +def _inspected(rbd, nodes) -> list: + """The components with hidden failures named by ``nodes`` (all of + them by default), checked.""" + if nodes is None: + if not rbd._inspection: + raise Refused( + "No component has hidden failures: give the components " + "an 'inspection' schedule to have its interval chosen." + ) + chosen = list(rbd._inspection) + else: + chosen = list(nodes) + if not chosen: + raise ValueError("nodes is empty.") + for node in chosen: + _requirements._require_component(rbd, node, "nodes") + if node not in rbd._inspection: + raise ValueError( + f"Node {node!r} has no hidden failures: give it an " + "'inspection' schedule to have its interval chosen." + ) + return chosen + + +def _allowed_intervals(allowed, chosen: list, never: bool = False) -> dict: + """``allowed`` as a sorted tuple of intervals per chosen node; with + ``never``, ``inf`` among them (never replacing a component).""" + per_node = ( + allowed + if isinstance(allowed, dict) + else {node: allowed for node in chosen} + ) + _require_chosen(per_node, chosen, "allowed") + options = {} + for node in chosen: + if node not in per_node: + raise ValueError(f"allowed gives no intervals for node {node!r}.") + values = [] + for value in per_node[node]: + try: + interval = float(value) + except (TypeError, ValueError): + interval = float("nan") + if not (0.0 < interval < math.inf or (never and interval > 0.0)): + kind = ( + "positive numbers (inf for never)" + if never + else "positive, finite numbers" + ) + raise ValueError( + f"allowed intervals must be {kind}, got {value!r} " + f"for node {node!r}." + ) + values.append(interval) + if not values: + raise ValueError(f"allowed gives no intervals for node {node!r}.") + options[node] = tuple(sorted(set(values))) + return options + + +def _maintained(rbd, nodes) -> list: + """The components under age replacement named by ``nodes`` (all of + them by default), checked.""" + if nodes is None: + chosen = [ + node + for node, schedule in rbd._preventive.items() + if schedule.policy == "age" + ] + if not chosen: + raise Refused( + "No component is under age replacement: give the " + "components a 'preventive' schedule to have its " + "interval chosen." + ) + return chosen + chosen = list(nodes) + if not chosen: + raise ValueError("nodes is empty.") + for node in chosen: + schedule = rbd._preventive.get(node) + if schedule is None or schedule.policy != "age": + raise ValueError( + f"Node {node!r} is not a component under age " + "replacement: give it a 'preventive' schedule with " + "'policy': 'age' to have its interval chosen." + ) + return chosen diff --git a/repyability/rbd/_long_run.py b/repyability/rbd/_long_run.py new file mode 100644 index 00000000..0c1c415f --- /dev/null +++ b/repyability/rbd/_long_run.py @@ -0,0 +1,1154 @@ +"""A ``RepairableRBD``'s long-run values: each component's availability +and failure frequency in the long run, exact or numerical (renewal +cycles, block replacement's calendar grid, tested units' cycles), and +the system's (``mean_availability``, ``system_failure_frequency``, +``mean_time_between_failures``, ``mean_up_time``, ``mean_down_time``, +``node_availability``). The methods of ``RepairableRBD`` of those names +call these. +""" + +import functools +import math +from functools import partial +from typing import ( + Any, + Collection, + Dict, + Hashable, + List, + NoReturn, + Optional, + Tuple, +) + +import numpy as np + +from repyability.non_repairable import NonRepairable +from repyability.rbd import ( + _ccf_groups, + _crews, + _requirements, + _standby_chain, +) +from repyability.rbd._block_replacement import ( + block_cycle, +) +from repyability.rbd._common import ( + _common_period, + _constant_rate, +) +from repyability.rbd._condition_replacement import ( + condition_cycle, +) +from repyability.rbd._events import ( + _Inspection, + _Preventive, +) +from repyability.rbd._model_utils import ( + model_mean, +) +from repyability.rbd.routes import AnalysisRoute +from repyability.utils.checks import ( + structure_method, +) + +#: How the values of a component with hidden failures whose tests or +#: repairs take time, or whose tests can miss a failure of a life that is +#: not exponential, are found (#159). +_TESTED_CYCLE = ( + "hidden failures with tests or repairs that take time, or tests that " + "miss: the cycle from one test that finds a failure to the next, " + "followed test by test on a grid" +) + + +def _failed_by(component: NonRepairable, age: float) -> float: + """The probability that a component's unit fails before ``age``: from + its model's own ``ff``, which keeps a small one's precision.""" + return float(np.ravel(component.reliability.ff(age))[0]) + + +def _log_kept(rate: float, inspection: "_Inspection") -> float: + """``log(rho)``, ``rho = 1 - (1 - c) (1 - exp(-rate * interval))``: the + chance that a unit with hidden failures, up after a test, is up after + the next one or had its failure found by it (a coverage ``c``).""" + missed = (1.0 - inspection.coverage) * -math.expm1( + -rate * inspection.interval + ) + return math.log1p(-missed) + + +def _tests_in(inspection: "_Inspection", period: float) -> list: + """The times of an inspection's tests in ``[0, period)`` that are not + multiples of its interval (those of a schedule with an offset), on a + calendar that repeats every ``period``.""" + if not inspection.offset: + return [] + count = int(round(period / inspection.interval)) + times = inspection.offset + inspection.interval * np.arange(count) + return list(np.where(times >= period, times - period, times)) + + +def mean_unavailability( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, +) -> float: + """See ``RepairableRBD.mean_unavailability``.""" + method = structure_method(method) + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + if rbd.ccf_groups: + # Each group conditioned on within its module (#218). + _, weights, inputs = _ccf_groups._ccf_long_run( + rbd, working_nodes, broken_nodes + ) + return float( + weights @ _ccf_groups._ccf_tabled(rbd, *inputs).system()[1] + ) + availability, unavailability, weights = _long_run_unavailabilities( + rbd, working_nodes, broken_nodes + ) + system = rbd._system_unreliability(availability, unavailability) + return float(weights @ system) + + +def _node_long_run(rbd, node) -> Tuple[str, str]: + """How a component's long-run values are found: the route, and a + phrase saying how (the message it raises, if refused).""" + from repyability.rbd import routes as r + from repyability.rbd.repairable_rbd import RepairableRBD + + component = rbd.components[node] + if isinstance(component, RepairableRBD): + inner = _long_run_route(component) + if inner.route == r.REFUSED: + return r.REFUSED, inner.reason + return ( + inner.route, + f"a nested RBD's long-run values, {inner.route}", + ) + for check in ( + partial(_requirements._require_perfect_repair, rbd), + partial(_requirements._require_times, rbd), + ): + message = r.refusal(partial(check, node)) + if message: + return r.REFUSED, message + if node in rbd._standby: + message = r.refusal( + partial(functools.partial(_standby_rates, rbd), node) + ) + if message: + return r.REFUSED, message + return r.EXACT, "a standby group's Markov chain" + if node in rbd._inspection: + message = r.refusal( + partial(_requirements._require_tested_exact, rbd, node) + ) + if message: + return r.REFUSED, message + kind = _requirements._tested_kind(rbd, node) + if kind == "unit": + return r.NUMERICAL, _TESTED_CYCLE + if kind == "life": + return ( + r.NUMERICAL, + "hidden failures, renewed at the tests, summed over the " + "test intervals", + ) + return r.EXACT, "hidden failures at a constant rate" + schedule = rbd._preventive.get(node) + if schedule is None: + message = r.refusal(component.mean_availability) + if message: + return r.REFUSED, message + life, how = r.mean_route(component.reliability) + if life == r.EXACT: + return r.EXACT, "its mean life and mean repair time" + return life, f"its mean life, {how}" + message = r.refusal( + partial(_requirements._require_no_opportunities, rbd, node) + ) + if message: + return r.REFUSED, message + life, how = r.model_route(component.reliability) + if schedule.policy in ("block", "condition"): + message = r.refusal( + partial(_requirements._require_block_models, rbd, node) + ) + if message: + return r.REFUSED, message + if schedule.policy == "condition": + return ( + r.NUMERICAL, + "replacement on condition: its renewal cycle, followed " + "from one inspection to the next on a grid", + ) + return ( + r.NUMERICAL, + "block replacement: its renewal cycle, solved on a grid", + ) + if life == r.SIMULATED: + return life, f"age replacement of a life from {how}" + return r.NUMERICAL, "age replacement: its mean up time, by quadrature" + + +def _long_run_nodes(rbd) -> Dict[Any, Tuple[str, str]]: + """Each component's long-run route (see ``_node_long_run``).""" + return {node: _node_long_run(rbd, node) for node in rbd.components} + + +def _long_run_route(rbd, groups: bool = True) -> "AnalysisRoute": + """How the exact long-run values are found (see + ``analysis_routes``); without ``groups``, the components' own, as + ``node_availability`` gives them (the common-cause groups change + only which are down together).""" + from repyability.rbd import routes as r + + chains = ( + r.refusal(partial(_ccf_groups._require_ccf_long_run, rbd)) + if groups + else None + ) + if chains: + # (Before the crews: the groups' chains do not take them in.) + return r.refused(chains) + if rbd._crews_couple(): + return _crew_chain_route(rbd) + calendars = r.refusal(partial(_requirements._require_calendars, rbd)) + if calendars: + return r.refused(calendars) + nodes = _long_run_nodes(rbd) + refusals = { + n: how for n, (route, how) in nodes.items() if route == r.REFUSED + } + if refusals: + return r.refused(next(iter(refusals.values())), tuple(refusals)) + return r.with_nodes( + r.EXACT, + "The structure function over the components' long-run " + "availabilities, exactly" + + ( + ", and over each common-cause group's members' joint states " + "(a Markov chain)." + if rbd.ccf_groups and groups + else "." + ), + nodes, + "long-run values", + ) + + +def _crew_chain_route(rbd) -> "AnalysisRoute": + """How the exact long-run values are found with limited repair + crews: from their Markov chain, and nested RBDs' own values.""" + from repyability.rbd import routes as r + from repyability.rbd.repairable_rbd import RepairableRBD + + chain = r.refusal(partial(_crews._require_crew_chain, rbd)) + if chain: + return r.refused(chain) + nested = { + node: _node_long_run(rbd, node) + for node, component in rbd.components.items() + if isinstance(component, RepairableRBD) + } + refusals = { + n: how for n, (route, how) in nested.items() if route == r.REFUSED + } + if refusals: + return r.refused(next(iter(refusals.values())), tuple(refusals)) + return r.with_nodes( + r.EXACT, + "The long-run distribution of the Markov chain of the " + "components' states and the repair queue, solved exactly, and " + "the system's values averaged over its states.", + nested, + "long-run values", + ) + + +def mean_availability( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, +) -> float: + """See ``RepairableRBD.mean_availability``.""" + # Good reference on the Availability of a system + # https://www.diva-portal.org/smash/get/diva2:986067/FULLTEXT01.pdf + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + + if rbd.ccf_groups: + # Each group conditioned on within its module (#218). + method = structure_method(method) + _, weights, inputs = _ccf_groups._ccf_long_run( + rbd, working_nodes, broken_nodes + ) + up, down = _ccf_groups._ccf_tabled(rbd, *inputs).system() + return float(weights @ (up if method == "p" else 1.0 - down)) + # Over one period of the inspection schedules, if any (see + # _long_run_grid), or over the states of the repair crews' Markov + # chain; otherwise at one point, as the node availabilities are + # constant. + availability, weights = _long_run_probabilities( + rbd, working_nodes, broken_nodes + ) + system = rbd.system_probability(availability, method=method) + return float(weights @ system) + + +def node_availability(rbd) -> dict[Hashable, float]: + """See ``RepairableRBD.node_availability``.""" + return {node: _node_availability(rbd, node) for node in rbd.components} + + +def _node_availability(rbd, node) -> float: + """A component's long-run availability (see ``node_availability``): + with limited repair crews, its long-run probability of being up in + their Markov chain, or a nested RBD's own, as it has crews of its + own.""" + _requirements._require_perfect_repair(rbd, node) + _requirements._require_times(rbd, node) + if rbd._crews_couple(): + chain = _crews._crew_chain(rbd) + if node in chain.nodes: + return chain.availability(node) + return float(rbd.components[node].mean_availability()) + if node in rbd._standby: + return _standby_long_run(rbd, node).availability + if node in rbd._inspection: + unit = _requirements._tested_unit(rbd, node) + if unit is not None: + return unit.long_run.availability + life = _requirements._tested_life(rbd, node) + if life is not None: + return life.availability + rate, interval = _requirements._inspected_rate(rbd, node) + inspection = rbd._inspection[node] + if inspection.partial: + # Up with probability rho ** k * exp(-rate * u) (see + # _tested_profile): averaged over a full test's cycle, + # (1 - rho ** m) / ((1 - coverage) * rate * full_test). + kept = _log_kept(rate, inspection) + return float( + -np.expm1(inspection.per_full_test * kept) + / ((1.0 - inspection.coverage) * rate * inspection.period) + ) + return float(-np.expm1(-rate * interval) / (rate * interval)) + schedule = rbd._preventive.get(node) + if schedule is None: + component = rbd.components[node] + return float(np.atleast_1d(component.mean_availability())[0]) + up, cycle, _, _ = _maintenance_cycle(rbd, node, schedule) + return min(1.0, up / cycle) + + +def _node_unavailability(rbd, node) -> float: + """A component's long-run unavailability, as ``_node_availability`` + gives its availability, but worked out in its own right (from a + standby group's down states, or the mean down time over a cycle), + so that a small one keeps its precision. For a component whose + value is constant over the long-run grid, and while the crews do + not couple the components (see ``_long_run_unavailabilities``).""" + _requirements._require_perfect_repair(rbd, node) + _requirements._require_times(rbd, node) + if node in rbd._standby: + return _standby_long_run(rbd, node).unavailability + component = rbd.components[node] + schedule = rbd._preventive.get(node) + if schedule is None: + return float(np.atleast_1d(component.mean_unavailability())[0]) + up, cycle, fails, survives = _maintenance_cycle(rbd, node, schedule) + if schedule.policy == "block": + return max(0.0, (cycle - up) / cycle) + # Down for a repair after a failure before the age (its probability + # from the model's own ff), else for the maintenance. + maintenance = ( + 0.0 if schedule.duration is None else model_mean(schedule.duration) + ) + down = ( + fails * model_mean(component.time_to_replace) + survives * maintenance + ) + return min(1.0, down / cycle) + + +def _block_nodes(rbd) -> list: + """The components renewed on a calendar, whose long-run values vary + with it: under block replacement, or replaced on condition at + inspections (#145).""" + return [ + node + for node, schedule in rbd._preventive.items() + if schedule.policy in ("block", "condition") + ] + + +def _block_cycle(rbd, node): + """The renewal cycle of a component under block replacement (see + ``_block_replacement``) or replaced on condition (see + ``_condition_replacement``), computed once and kept.""" + _requirements._refuse_level(rbd, node) + component = rbd.components[node] + schedule = rbd._preventive[node] + cache = rbd._cache.kept("block_cycles") + key = ( + node, + id(component.reliability), + id(component.time_to_replace), + float(schedule.interval), + id(schedule.duration), + schedule.policy, + schedule.threshold, + ) + if key not in cache: + if schedule.policy == "condition": + cache[key] = condition_cycle( + component.reliability, + component.time_to_replace, + schedule.duration, + schedule.interval, + float(schedule.threshold), + node, + ) + else: + cache[key] = block_cycle( + component.reliability, + component.time_to_replace, + schedule.duration, + schedule.interval, + node, + ) + return cache[key] + + +def _has_calendar(rbd) -> bool: + """Whether a component here, or in a nested RBD, is inspected or + replaced on a calendar: its long-run availability then varies with + the time of the schedule.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + return ( + bool(rbd._inspection) + or bool(_block_nodes(rbd)) + or any( + isinstance(c, RepairableRBD) and _has_calendar(c) + for c in rbd.components.values() + ) + ) + + +def _unit_nodes(rbd) -> list: + """The components with hidden failures whose values are worked out + on a grid (see ``_tested_kind``).""" + return [ + node + for node in rbd._inspection + if _requirements._tested_kind(rbd, node) == "unit" + ] + + +def _calendar_grid( + rbd, blocks: list, units: Optional[list] = None +) -> Tuple[np.ndarray, np.ndarray]: + """``_long_run_grid`` with components under block replacement, or + tested on a grid (``units``): the middles of cells over one common + period of the block and inspection intervals. The cells' edges are + those of every block-replaced component's profile (its long-run + availability over its interval, cell by cell), of every such tested + component's (its grid's points across each test interval, and where + its test's and its repair's CDFs bend), the block and inspection + times, and enough points in between for an inspected component's + availability to vary little across a cell; so each cell lies in one + cell of every profile, and the mean over the cells is as exact as + the profiles.""" + intervals = {rbd._preventive[node].interval for node in blocks} + intervals |= {rbd._inspection[node].interval for node in rbd._inspection} + schedules = list(rbd._inspection.values()) + period = _common_period(intervals | {s.period for s in schedules}) + + def too_long() -> NoReturn: + raise NotImplementedError( + f"The block-replacement and inspection intervals " + f"{sorted(intervals)} repeat together only after too long a " + "time to average over finely enough: estimate the long-run " + "values by simulation, with availability() or cost()." + ) + + # The grid's size, before it is built: a common period of many + # repeats of the profiles (near-equal intervals, say) would not fit + # in memory. + size = 0 + for node in blocks: + phase = _block_cycle(rbd, node).phase + size += int(round(period / phase[-1])) * len(phase) + for node in units or (): + unit = _requirements._tested_unit(rbd, node).long_run # type: ignore + size += int(round(period / unit.period)) * len(unit.edges()) + for node in rbd._inspection: + interval = rbd._inspection[node].interval + per_interval = np.ceil( + 256.0 * _requirements._tested_rate(rbd, node) * interval + ) + size += int(per_interval) * int(round(period / interval)) + if size > 4_000_000: + too_long() + pieces = [np.array([0.0, period])] + for interval in intervals: + pieces.append(interval * np.arange(int(round(period / interval)))) + for schedule in schedules: + pieces.append(np.array(sorted(_tests_in(schedule, period)))) + for node in blocks: + phase = _block_cycle(rbd, node).phase + repeats = int(round(period / phase[-1])) + pieces.append( + ( + phase[-1] * np.arange(repeats)[:, None] + phase[None, :-1] + ).ravel() + ) + for node in units or (): + tested = _requirements._tested_unit(rbd, node) + long_run = tested.long_run # type: ignore + repeats = int(round(period / long_run.period)) + edges = ( + rbd._inspection[node].offset + + long_run.period * np.arange(repeats)[:, None] + + long_run.edges()[None, :] + ).ravel() + pieces.append(edges - period * np.floor(edges / period)) + for node in rbd._inspection: + rate = _requirements._tested_rate(rbd, node) + interval = rbd._inspection[node].interval + per_interval = int(np.ceil(256.0 * rate * interval)) + pieces.append( + np.linspace( + 0.0, + period, + 1 + per_interval * int(round(period / interval)), + ) + ) + edges = np.concatenate(pieces) + if len(edges) > 4_000_000: + too_long() + # Edges closer than rounding are one. + edges = np.unique(np.round(edges / period, 12)) * period + middle, width = 0.5 * (edges[1:] + edges[:-1]), np.diff(edges) + if not units: + return middle, width / period + # A tested component's test and repair times have exact CDFs in its + # profile, which may bend sharply within a cell (a test of hours, + # say): two Gauss-Legendre points a cell, exact for its linear grid + # values, and to the cell's fourth power for the rest. + side = 0.5 * width / np.sqrt(3.0) + times = np.stack([middle - side, middle + side], axis=1).ravel() + return times, np.repeat(0.5 * width / period, 2) + + +def _block_profile(rbd, node, times: np.ndarray, rates: bool = False): + """A block-replaced component's long-run availability (or failure + intensity) at each of ``times``: the value of its profile's cell + (over a block interval) that the time falls in.""" + cycle = _block_cycle(rbd, node) + interval = float(cycle.phase[-1]) + values = cycle.failure_rate if rates else cycle.availability + phase = times - interval * np.floor(times / interval) + cell = np.searchsorted(cycle.phase, phase, side="right") - 1 + return values[np.clip(cell, 0, len(values) - 1)] + + +def _calendar_outages(rbd, working_nodes, broken_nodes) -> float: + """The system's planned outages per unit time, in the long run, + at exact times on the calendar: replacements at block times that + take time, and tests that take time, each taking a unit working + then off line. At each such time, the probability that the system + is up just before and down just after, which (as they only take + units down) is the fall in the system availability: the rise in its + unavailability, a difference of small values in a reliable system, + not of values near 1. With common-cause groups, see + ``_ccf_groups._ccf_calendar_outages``.""" + changes = _calendar_changes(rbd, working_nodes, broken_nodes) + if changes is None: + return 0.0 + period, _, before, after = changes + rise = rbd._system_unreliability(*after) - rbd._system_unreliability( + *before + ) + return float(np.sum(rise)) / period + + +def _calendar_changes(rbd, working_nodes, broken_nodes): + """Where the system's planned outages at exact times on the calendar + start (see ``_calendar_outages``): None if none do, else the common + period, the instants in it, and the nodes' availabilities and + unavailabilities just before and just after each (the forced nodes + held at 1 or 0). The nodes due at an instant take their values then + from their own models; the others are at the instant, before its + outages start. Units due at the same time go down together; an + instant inspection due then comes first.""" + blocks = [ + node + for node in _block_nodes(rbd) + if rbd._preventive[node].duration is not None + ] + units = [ + node + for node in _unit_nodes(rbd) + if _requirements._tested_unit(rbd, node).setup.timed # type: ignore + ] + if not blocks and not units: + return None + intervals = {rbd._preventive[node].interval for node in blocks} + intervals |= {rbd._inspection[node].period for node in rbd._inspection} + period = _common_period(intervals) + # At each instant (as a share of the period), the nodes due then, + # with their values up and down just before and just after. + due: dict = {} + for node in blocks: + cycle = _block_cycle(rbd, node) + values = ( + (cycle.before, 1.0 - cycle.before), + (cycle.after, 1.0 - cycle.after), + ) + interval = rbd._preventive[node].interval + for k in range(int(round(period / interval))): + instant = round(k * interval / period, 12) + due.setdefault(instant, []).append((node, values)) + for node in units: + inspection = rbd._inspection[node] + tested = _requirements._tested_unit(rbd, node) + long_run = tested.long_run # type: ignore + for k in range(int(round(period / inspection.interval))): + time = inspection.offset + k * inspection.interval + time -= period * math.floor(time / period) + due.setdefault(round(time / period, 12), []).append( + (node, long_run.around(k % long_run.per)) + ) + instants = np.array(sorted(due)) * period + # Just after each instant, before its outages start (an + # inspection due then is done). + just_after = instants + 1e-9 * period + up, down = _availabilities_at(rbd, just_after), ( + _unavailabilities_at(rbd, just_after) + ) + before, after = dict(up), dict(up) + before_down, after_down = dict(down), dict(down) + for column, key in enumerate(sorted(due)): + for node, values in due[key]: + for works, fails, (value, failed) in zip( + (before, after), (before_down, after_down), values + ): + works[node] = np.array(works[node], dtype=float) + fails[node] = np.array(fails[node], dtype=float) + works[node][column] = value + fails[node][column] = failed + return ( + period, + instants, + ( + rbd._probabilities_with_overrides( + before, working_nodes, broken_nodes + ), + rbd._failures_with_overrides( + before_down, working_nodes, broken_nodes + ), + ), + ( + rbd._probabilities_with_overrides( + after, working_nodes, broken_nodes + ), + rbd._failures_with_overrides( + after_down, working_nodes, broken_nodes + ), + ), + ) + + +def _long_run_grid(rbd) -> Tuple[np.ndarray, np.ndarray]: + """The times, and their weights (which sum to 1), that the exact + long-run values average over. + + Components with hidden failures are up with a probability that falls + between inspections and is restored at each one, and components + inspected at the same times are down together, so the system's + long-run values are averages over one period of the inspection + schedules: by Gauss-Legendre quadrature between consecutive + inspections, on pieces short enough (a failure rate times their + length at most 1) for it to be exact to rounding. With no such + components they are constant: one time, of weight 1. A nested RBD + with hidden failures enters through its own long-run values, which + is exact only if nothing else varies with the inspections. With a + component whose profile is on a grid (block-replaced, or tested + with tests or repairs that take time: see ``_calendar_grid``), the + middles of the grids' cells. + """ + _crews._require_unlimited_crews(rbd) + _requirements._require_calendars(rbd) + blocks = _block_nodes(rbd) + units = _unit_nodes(rbd) + if blocks or units: + return _calendar_grid(rbd, blocks, units) + if not rbd._inspection: + return np.zeros(1), np.ones(1) + rates = { + node: ( + _requirements._tested_rate(rbd, node), + rbd._inspection[node].interval, + ) + for node in rbd._inspection + } + intervals = {interval for _, interval in rates.values()} + schedules = list(rbd._inspection.values()) + period = _common_period(intervals | {s.period for s in schedules}) + breaks = {0.0, period} + for interval in intervals: + count = int(round(period / interval)) + breaks.update(k * interval for k in range(1, count)) + for schedule in schedules: + breaks.update(_tests_in(schedule, period)) + fastest = max(rate for rate, _ in rates.values()) + # The tests of components whose lives are not exponential: a unit + # renewed there may not age smoothly from it (a Weibull life's + # survival, say), so the pieces after it close in on it. + graded: set = set() + for node, schedule in rbd._inspection.items(): + life = _requirements._tested_life(rbd, node) + if life is None: + continue + count = int(round(period / schedule.interval)) + starts = schedule.offset + schedule.interval * np.arange(count) + starts = np.where(starts >= period, starts - period, starts) + graded.update(round(float(s) / period, 12) for s in starts) + # Where its life bends within an interval (a threshold, say). + bends = (starts[:, None] + life.bends[None, :]).ravel() + breaks.update(float(b) for b in np.mod(bends, period)) + edges = np.array(sorted(breaks)) + points, weights = np.polynomial.legendre.leggauss(16) + times, masses = [], [] + for a, b in zip(edges[:-1], edges[1:]): + pieces = np.linspace(a, b, max(1, math.ceil((b - a) * fastest)) + 1) + if round(float(a) / period, 12) in graded: + width = pieces[1] - pieces[0] + pieces = np.concatenate( + [ + pieces[:1], + a + width * 2.0 ** -np.arange(48.0, 0.0, -1.0), + pieces[1:], + ] + ) + for lo, hi in zip(pieces[:-1], pieces[1:]): + half = 0.5 * (hi - lo) + times.append(lo + half * (points + 1.0)) + masses.append(half * weights) + return np.concatenate(times), np.concatenate(masses) / period + + +def _tested_profile( + rbd, node, times: np.ndarray +) -> Tuple[np.ndarray, np.ndarray]: + """A component with hidden failures: its probabilities of being up + and down at each of ``times``, in the long run. + + Tested at ``offset`` and every ``interval`` after it, it is up with + probability ``exp(-rate * u)``, ``u`` the time since its last test. + A test that can miss a failure (a ``coverage`` ``c`` below 1) finds + what it can: up just after the ``k``-th test since a full one with + probability ``rho ** k``, where ``rho = 1 - (1 - c) * (1 - + exp(-rate * interval))`` keeps the units that were up or whose + failure it found; so up with probability ``rho ** k * exp(-rate * + u)``. Down is worked out in its own right, by ``expm1``. Any other + life's profile is its ``TestedLife``'s (#144), and with tests or + repairs that take time, or tests that can miss a failure of a life + that is not exponential, its ``TestedLongRun``'s (#159).""" + unit = _requirements._tested_unit(rbd, node) + if unit is not None: + return unit.long_run.profile( + _requirements._unit_phase(rbd, node, times) + ) + life = _requirements._tested_life(rbd, node) + if life is not None: + return life.profile(_requirements._tested_phase(rbd, node, times)) + rate, interval = _requirements._inspected_rate(rbd, node) + inspection = rbd._inspection[node] + position = times - inspection.offset if inspection.offset else times + if not inspection.partial: + since = position - interval * np.floor(position / interval) + return np.exp(-rate * since), -np.expm1(-rate * since) + cycle = inspection.period + within = position - cycle * np.floor(position / cycle) + tests = np.clip( + np.floor(within / interval), 0, inspection.per_full_test - 1 + ) + exponent = tests * _log_kept(rate, inspection) - rate * ( + within - tests * interval + ) + return np.exp(exponent), -np.expm1(exponent) + + +def _availabilities_at(rbd, times: np.ndarray) -> dict: + """Every node's availability at each of ``times`` (see + ``_long_run_grid``): ``exp(-lambda * u)``, ``u`` the time since the + last inspection, for a component with hidden failures; its constant + long-run availability for any other.""" + out: dict = {} + blocks = set(_block_nodes(rbd)) + for node in rbd.components: + if node in rbd._inspection: + out[node] = _tested_profile(rbd, node, times)[0] + elif node in blocks: + out[node] = _block_profile(rbd, node, times) + else: + out[node] = np.full(len(times), _node_availability(rbd, node)) + for node in rbd.in_or_out: + out[node] = np.ones(len(times)) + return out + + +def _unavailabilities_at(rbd, times: np.ndarray) -> dict: + """Every node's unavailability at each of ``times``, as + ``_availabilities_at`` gives their availabilities, each worked out + in its own right so that a small one keeps its precision: + ``1 - exp(-lambda * u)`` by ``expm1`` at a time ``u`` since a test, + for a component with hidden failures, and ``_node_unavailability`` + for one whose value is constant. + (A block-replaced component's profile is numerical, to about 1e-7, + so one less it loses nothing.)""" + out: dict = {} + blocks = set(_block_nodes(rbd)) + for node in rbd.components: + if node in rbd._inspection: + out[node] = _tested_profile(rbd, node, times)[1] + elif node in blocks: + out[node] = 1.0 - _block_profile(rbd, node, times) + else: + out[node] = np.full(len(times), _node_unavailability(rbd, node)) + for node in rbd.in_or_out: + out[node] = np.zeros(len(times)) + return out + + +def _long_run_unavailabilities( + rbd, working_nodes, broken_nodes +) -> Tuple[dict, dict, np.ndarray]: + """``_long_run_probabilities``, with every node's unavailability + too (at each time of ``_long_run_grid``, or in each state of the + repair crews' Markov chain), each worked out in its own right, so + that the system's unavailability keeps a small one's precision.""" + probabilities, failures, weights, _ = _long_run_points( + rbd, working_nodes, broken_nodes + ) + return probabilities, failures, weights + + +def _long_run_points( + rbd, working_nodes, broken_nodes +) -> Tuple[dict, dict, np.ndarray, Optional[np.ndarray]]: + """``_long_run_unavailabilities``, and each point's time's position + in ``_long_run_grid``, or None in the states of the repair crews' + chain.""" + if rbd._crews_couple(): + probabilities, weights = _crews._chain_probabilities( + rbd, working_nodes, broken_nodes + ) + forced = set(working_nodes or ()) | set(broken_nodes or ()) + chain = _crews._crew_chain(rbd, frozenset(forced)) + # In each state a node in the chain, or held working or broken, + # is up or down for certain: one less it is exact. + failures = {node: 1.0 - value for node, value in probabilities.items()} + for node, component in rbd.components.items(): + if node not in forced and node not in chain.nodes: + failures[node] = np.full( + len(weights), float(component.mean_unavailability()) + ) + return probabilities, failures, weights, None + times, weights = _long_run_grid(rbd) + probabilities = rbd._probabilities_with_overrides( + _availabilities_at(rbd, times), working_nodes, broken_nodes + ) + failures = rbd._failures_with_overrides( + _unavailabilities_at(rbd, times), working_nodes, broken_nodes + ) + # (Common-cause groups are conditioned on module by module, see + # _ccf_tabled, by every method that would come here with them.) + assert not rbd.ccf_groups + return probabilities, failures, weights, np.arange(len(times)) + + +def _long_run_probabilities( + rbd, working_nodes, broken_nodes, what: str = "This analysis" +) -> Tuple[dict, np.ndarray]: + """The node availabilities the long-run values are evaluated at + (with the forced nodes held at 1 or 0), over the times of + ``_long_run_grid``, and those times' weights; or, with limited + repair crews, over the states of their Markov chain (see + ``_chain_probabilities``). Each value is then a ratio of averages + of system quantities. With common-cause groups, the times are split + by every combination of their states (see ``_with_ccf_groups``, + which ``what`` refuses where they are too many): for the capacity + distribution, which takes the nodes' joint states.""" + if rbd.ccf_groups: + # The groups' chains refuse the crews (see _ccf_rates). + _ccf_groups._require_ccf_long_run(rbd) + if rbd._crews_couple(): + return _crews._chain_probabilities(rbd, working_nodes, broken_nodes) + times, weights = _long_run_grid(rbd) + probabilities = rbd._probabilities_with_overrides( + _availabilities_at(rbd, times), working_nodes, broken_nodes + ) + if rbd.ccf_groups: + _ccf_groups._require_free_members(rbd, working_nodes, broken_nodes) + probabilities, _, weights, _ = _ccf_groups._with_ccf_groups( + rbd, times, probabilities, None, weights, what=what + ) + return probabilities, weights + + +def _standby_rates(rbd, node) -> Tuple[float, float]: + """A standby group's units' failure and repair rates, for its exact + long-run values; raise if their lives and repair times are not + exponential.""" + component = rbd.components[node] + life = _constant_rate(component.reliability) + repair = _constant_rate(component.time_to_replace) + if life is None or repair is None: + raise NotImplementedError( + f"Component {node!r} is a standby group, whose exact " + "long-run values come from a Markov chain of its units, " + "which needs their lives and repair times exponential: " + "simulate it with availability() or cost()." + ) + return life, repair + + +def _standby_long_run(rbd, node) -> "_standby_chain.StandbyLongRun": + """A standby group's long-run values, from its Markov chain (see + ``_standby_chain``). The crews do not tie it to other nodes here (see + ``_crews_couple``): its units' repairs wait only for each other, + with ``repair_crews`` crews when they are the crews' only jobs.""" + life, repair = _standby_rates(rbd, node) + arrangement = rbd._standby[node] + return _standby_chain.long_run( + arrangement.units, + arrangement.k, + life, + repair, + arrangement.dormancy_factor, + arrangement.switching_probability, + rbd.repair_crews if _crews._crews_limited(rbd) else None, + ) + + +def _node_frequencies(rbd, node) -> Tuple[float, float, float]: + """A component's long-run failures, preventive replacements and + planned outages, per unit time: recursive for a nested + RepairableRBD (whose own preventive replacements are not this RBD's + to count), ``1 / (MTTF + MTTR)`` failures for a NonRepairable, and + from its renewal cycle for one under age replacement.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + _requirements._require_perfect_repair(rbd, node) + _requirements._require_times(rbd, node) + component = rbd.components[node] + if isinstance(component, RepairableRBD): + failures, planned = _outage_frequencies(component) + return failures, 0.0, planned + if node in rbd._standby: + return _standby_long_run(rbd, node).failure_frequency, 0.0, 0.0 + if node in rbd._inspection: + # At most one failure per inspection interval: the unit, down + # from its failure, is renewed at the inspection that finds it. + unit = _requirements._tested_unit(rbd, node) + if unit is not None: + # Its tests that take it off line, working, are planned + # outages. + long_run = unit.long_run + return long_run.failures, 0.0, long_run.planned + life = _requirements._tested_life(rbd, node) + if life is not None: + return life.failures, 0.0, 0.0 + rate, interval = _requirements._inspected_rate(rbd, node) + if rbd._inspection[node].partial: + # It fails at its constant rate while it is up. + return rate * _node_availability(rbd, node), 0.0, 0.0 + return float(-np.expm1(-rate * interval) / interval), 0.0, 0.0 + schedule = rbd._preventive.get(node) + if schedule is None: + return component.failure_frequency(), 0.0, 0.0 + _, cycle, failures, maintenances = _maintenance_cycle(rbd, node, schedule) + maintained = maintenances / cycle + planned = 0.0 if schedule.duration is None else maintained + return failures / cycle, maintained, planned + + +def _maintenance_cycle( + rbd, node, schedule: _Preventive +) -> Tuple[float, float, float, float]: + """A component's renewal cycle under preventive maintenance: its mean + up time, mean length, mean number of failures and mean number of + preventive replacements. + + Under age replacement a cycle ends at a failure or at a preventive + replacement, whichever comes first. Under block replacement it runs + from one block time at which the unit is up (and replaced) to the + next, with any failures and repairs in between (see + ``_block_replacement``); replaced on condition, from one inspection + that replaces the unit to the next (see ``_condition_replacement``). + It is computed once and kept.""" + _requirements._require_no_opportunities(rbd, node) + _requirements._require_perfect_repair(rbd, node) + component = rbd.components[node] + if schedule.policy == "block": + block = _block_cycle(rbd, node) + return block.up, block.length, block.failures, 1.0 + if schedule.policy == "condition": + kept = _block_cycle(rbd, node) + return kept.up, kept.length, kept.failures, kept.replaced + # Kept by interval, as a search over intervals revisits them. + cache = rbd._cache.kept("age_cycles") + key = ( + node, + id(component.reliability), + id(component.time_to_replace), + float(schedule.interval), + id(schedule.duration), + ) + if key in cache: + return cache[key] + up = float(component.avg_replacement_time(schedule.interval)) + survives = float( + np.ravel(component.reliability_function(schedule.interval))[0] + ) + fails = _failed_by(component, schedule.interval) + maintenance = ( + 0.0 if schedule.duration is None else model_mean(schedule.duration) + ) + cycle = ( + up + + fails * model_mean(component.time_to_replace) + + survives * maintenance + ) + cache[key] = (up, cycle, fails, survives) + return cache[key] + + +def system_failure_frequency( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> float: + """See ``RepairableRBD.system_failure_frequency``.""" + return _outage_frequencies(rbd, working_nodes, broken_nodes)[0] + + +def _outage_frequencies( + rbd, + working_nodes: Optional[Collection[Hashable]] = None, + broken_nodes: Optional[Collection[Hashable]] = None, +) -> Tuple[float, float]: + """The system's long-run failures and planned outages per unit time: + ``sum_i I_B(i) * omega_i`` over the nodes' failures, and the same + over their planned outages (a node's outage takes the system down + when the node is critical, which it is with probability + ``I_B(i)``): the sum of ``_outage_terms``.""" + terms, planned = _outage_terms(rbd, working_nodes, broken_nodes) + failures = 0.0 + for _, term in terms: + failures += term + return failures, planned + + +def _outage_terms( + rbd, + working_nodes: Optional[Collection[Hashable]] = None, + broken_nodes: Optional[Collection[Hashable]] = None, +) -> Tuple[List[Tuple[Any, float]], float]: + """The system's long-run failures per unit time, as the terms of + their sum, each with what causes it (a node, or a common-cause + group's members together, for its causes that strike more than one + of them), and its planned outages per unit time (see + ``_outage_frequencies``). With limited repair crews, see + ``_chain_outage_terms``; with common-cause groups, + ``_ccf_outage_terms``.""" + if rbd.ccf_groups: + return _ccf_groups._ccf_outage_terms(rbd, working_nodes, broken_nodes) + if rbd._crews_couple(): + return _crews._chain_outage_terms(rbd, working_nodes, broken_nodes) + times, weights = _long_run_grid(rbd) + availability = rbd._probabilities_with_overrides( + _availabilities_at(rbd, times), working_nodes, broken_nodes + ) + unavailability = rbd._failures_with_overrides( + _unavailabilities_at(rbd, times), working_nodes, broken_nodes + ) + forced = (set() if working_nodes is None else set(working_nodes)) | ( + set() if broken_nodes is None else set(broken_nodes) + ) + # Each node's Birnbaum importance, from the nodes' unavailabilities + # as well as their availabilities, so that a small one (a node + # backed up by reliable redundancy) keeps its precision. + birnbaum = rbd._birnbaum_importance( + availability, node_failures=unavailability + ) + terms: List[Tuple[Any, float]] = [] + planned = 0.0 + blocks = set(_block_nodes(rbd)) + for node in rbd.components: + if node in forced: + # A forced node never changes state, so it contributes no + # system failures. + continue + importance = np.asarray(birnbaum[node]) + if node in rbd._inspection: + # It fails at its constant rate whenever it is up (any other + # life: at its profile's rate). + node_failures: Any = _requirements._tested_intensity( + rbd, node, times, availability[node] + ) + node_planned: Any = 0.0 + elif node in blocks: + # Its failure intensity varies over its block interval; its + # replacements fall at block times (counted below). + node_failures = _block_profile(rbd, node, times, rates=True) + node_planned = 0.0 + else: + node_failures, _, node_planned = _node_frequencies(rbd, node) + terms.append((node, float(weights @ (importance * node_failures)))) + planned += float(weights @ (importance * node_planned)) + planned += _calendar_outages(rbd, working_nodes, broken_nodes) + return terms, planned + + +def mean_time_between_failures( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> float: + """See ``RepairableRBD.mean_time_between_failures``.""" + omega = rbd.system_failure_frequency(working_nodes, broken_nodes) + return 1.0 / omega if omega > 0.0 else float("inf") + + +def mean_up_time( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> float: + """See ``RepairableRBD.mean_up_time``.""" + availability = rbd.mean_availability(working_nodes, broken_nodes) + omega = sum(_outage_frequencies(rbd, working_nodes, broken_nodes)) + if omega > 0.0: + return float(availability) / omega + return float("inf") if availability > 0.0 else 0.0 + + +def mean_down_time( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> float: + """See ``RepairableRBD.mean_down_time``.""" + unavailability = rbd.mean_unavailability(working_nodes, broken_nodes) + omega = sum(_outage_frequencies(rbd, working_nodes, broken_nodes)) + if omega > 0.0: + return unavailability / omega + return 0.0 if unavailability <= 0.0 else float("inf") diff --git a/repyability/rbd/_mean_lifetime.py b/repyability/rbd/_mean_lifetime.py index 67160dfa..fdfa65c1 100644 --- a/repyability/rbd/_mean_lifetime.py +++ b/repyability/rbd/_mean_lifetime.py @@ -28,6 +28,7 @@ from repyability.utils.wrappers import outside_level +from ._model_utils import MODEL_ERRORS from ._point_availability import knots as quantile_knots #: The Gauss-Kronrod (7, 15) rule on [-1, 1] (QUADPACK's): its 15 points, @@ -307,9 +308,10 @@ def _collect( inner = [model.model, getattr(model, "_sf_model", None)] elif isinstance(model, RegressionNode): try: - kinks.append(_thinned(model._survival_grid()[0])) - except Exception: # a model whose grid cannot be built + hints.append(model._knots()) + except MODEL_ERRORS: # a model whose quantiles cannot be taken pass + kinks.append(model._kinks()) return else: quantiles = quantile_knots(model) diff --git a/repyability/rbd/_measures.py b/repyability/rbd/_measures.py new file mode 100644 index 00000000..318b176b --- /dev/null +++ b/repyability/rbd/_measures.py @@ -0,0 +1,370 @@ +"""The measures and lives that are functions of a system's values (#315). + +Each importance measure is a function of the system's probabilities of +working and of failing, with an element held working or failed, and the +elements' own. A diagram's (``rbd.py``), a fault tree's +(``fault_tree.py``) and a network's (``network.py``) differ in how those +values are worked out (a decomposition, the top event conditioned, a +decision diagram replayed with an element held), not in the measures. The +ratios are here, as a network takes them; a diagram's are worked out on its +decomposition's arrays in ``rbd.py``. With them, what a measure over time +or a life needs of the system's reliability alone, which a diagram and a +network share: an element's failure density, the Barlow-Proschan shares +(each element's part of the system's failure density, integrated over +time) and the inversion of the reliability for a B-life. +""" + +from typing import ( + Any, + Callable, + Collection, + Dict, + Hashable, + Iterable, + Mapping, + Optional, + Tuple, + Union, +) + +import numpy as np +from scipy.optimize import brentq + +from repyability.rbd._mean_lifetime import model_knots +from repyability.rbd._model_utils import ( + MODEL_ERRORS, + failure_time_scale, + is_fixed_probability, +) + +#: A time past every finite life: what a system's reliability is there, it +#: never loses. +LATE = 1e300 + + +def improvement_potential( + birnbaum: Mapping, q: Mapping +) -> Dict[Hashable, Any]: + """Each element's improvement potential, ``R(i working) - R``: its + Birnbaum importance times its probability of failing ``q``, so that a + small one keeps its precision.""" + return {name: value * q[name] for name, value in birnbaum.items()} + + +def risk_achievement_worth( + fails_held: Mapping, fails: Any +) -> Dict[Hashable, Any]: + """Each element's risk achievement worth: the system's probability of + failing with the element held failed, over its probability of failing + as it is (inf, or nan at 0/0, where that is 0: a system that cannot + fail, no cause for a warning).""" + with np.errstate(divide="ignore", invalid="ignore"): + return {name: value / fails for name, value in fails_held.items()} + + +def risk_reduction_worth( + fails: Any, fails_held: Mapping +) -> Dict[Hashable, Any]: + """Each element's risk reduction worth: the system's probability of + failing as it is over that with the element held working. An element + whose working ends the risk is worth infinitely much, which is no + cause for a warning.""" + with np.errstate(divide="ignore", invalid="ignore"): + return {name: fails / value for name, value in fails_held.items()} + + +def criticality_importance( + birnbaum: Mapping, member: Mapping, system: Any +) -> Dict[Hashable, Any]: + """Each element's criticality importance: its Birnbaum importance times + its own probability (``member``: of failing, for the failure-oriented + form, or of working, for the success-oriented one) over the system's + (of failing, or of working); nan where the system's is 0.""" + with np.errstate(divide="ignore", invalid="ignore"): + return { + name: np.where(system > 0, value * member[name] / system, np.nan) + for name, value in birnbaum.items() + } + + +def fussell_vesely(numerators: Mapping, fails: Any) -> Dict[Hashable, Any]: + """Each element's Fussell-Vesely importance: the probability that a + minimal cut set containing it has failed (or the sum of the sets' + probabilities, the rare-event approximation), over the system's + probability of failing; nan where that is 0.""" + with np.errstate(divide="ignore", invalid="ignore"): + return {name: value / fails for name, value in numerators.items()} + + +def model_scale(model) -> float: + """A typical failure time of an element's model (see + ``failure_time_scale``), or NaN where it has none: a fixed probability, + or a model whose mean cannot be worked out.""" + if is_fixed_probability(model): + return float("nan") + try: + return float(failure_time_scale(model)) + except MODEL_ERRORS: + return float("nan") + + +def difference_steps(x: np.ndarray, dx: float, scale) -> np.ndarray: + """The steps of a finite difference at the times ``x``: ``dx`` of each + time, and at 0, where there is none to be relative to, ``dx`` of + ``scale()``, a typical failure time (1 if there is none), so that the + step does not depend on the unit of time.""" + step = dx * np.abs(x) + if np.any(step == 0.0): + typical = scale() + typical = typical if np.isfinite(typical) and typical > 0 else 1.0 + step = np.where(step == 0.0, dx * typical, step) + return step + + +def density(model, x: np.ndarray) -> np.ndarray: + """An element's failure density at the times ``x``: its model's own + (``df``) where it has one, else by central differences of its + probability of failing (one-sided near 0).""" + try: + with np.errstate(all="ignore"): + return np.asarray(model.df(x), dtype=float).reshape(x.shape) + except MODEL_ERRORS: + pass + step = difference_steps(x, 1e-6, lambda: model_scale(model)) + hi, lo = x + step, np.maximum(x - step, 0.0) + if hasattr(model, "ff"): + change = np.asarray(model.ff(hi), dtype=float) - np.asarray( + model.ff(lo), dtype=float + ) + else: + change = np.asarray(model.sf(lo), dtype=float) - np.asarray( + model.sf(hi), dtype=float + ) + return np.clip(change.reshape(x.shape) / (hi - lo), 0.0, None) + + +def filled_edges(knots: np.ndarray, start: float, end: float) -> np.ndarray: + """The pieces of an integral from ``start`` to ``end``: at the + ``knots`` within, and at least eight to a decade from the first.""" + inside = knots[(knots > start) & (knots < end)] + low = max(start, float(inside[0]) if inside.size else end * 1e-6) + if low <= 0.0: + low = end * 1e-6 + decades = max(np.log10(end / low), 1.0) + filler = np.geomspace(low, end, int(np.ceil(decades * 8)) + 1) + return np.unique(np.concatenate([[start, end], inside, filler])) + + +def integrated_shares( + parts: Callable[[np.ndarray], np.ndarray], + knots: np.ndarray, + ends: np.ndarray, +) -> np.ndarray: + """Each column's share of the integral of ``parts`` (``parts(t)``: a + row per time, a column per element, each the density with which that + element's failure fails the system) from 0 to each of ``ends``: the + Barlow-Proschan importances by those times, a row each; nan where the + integrals' total is 0. The integrals are summed in order of the ends, + by adaptive quadrature on pieces that start at the ``knots`` (see + ``_rates.integral``).""" + from repyability.rbd import _rates + + order = np.argsort(ends) + totals = np.zeros((len(ends), int(np.shape(parts(np.ones(1)))[1]))) + running = np.zeros(totals.shape[1]) + start = 0.0 + for k in order: + end = float(ends[k]) + if end > start: + running = running + _rates.integral( + parts, filled_edges(knots, start, end) + ) + start = end + totals[k] = running + whole = totals.sum(axis=1) + with np.errstate(divide="ignore", invalid="ignore"): + return np.where(whole[:, None] > 0.0, totals / whole[:, None], np.nan) + + +def life_end( + sf: Callable[[np.ndarray], np.ndarray], knots: np.ndarray +) -> float: + """A time by which what is left of a system's failures is negligible: + its reliability ``sf`` (of an array of times) within ``1e-14`` of + what it never loses, found by doubling from the last of the + ``knots``.""" + never = float(np.ravel(sf(np.array([LATE])))[0]) + start = float(knots[-1]) if knots.size else 1.0 + tail = start * 2.0 ** np.arange(int(np.log2(LATE / start)) + 1) + values = np.ravel(sf(tail)) + ended = np.flatnonzero(values - never <= 1e-14) + return float(tail[ended[0]] if ended.size else tail[-1]) + + +def invert_reliability( + sf: Callable[[float], float], target, upper_bound: Optional[float] = None +): + """Solve ``sf(t) == target`` for ``t >= 0``: the time a system's + reliability falls to ``target``; for each target, as an array, if + ``target`` is one. + + ``sf`` maps a time to the system's reliability, non-increasing, so the + solution is unique: the crossing is bracketed between 0 and + ``upper_bound``, found by doubling from 1 unless given, and found with + Brent's method. Raises ``ValueError`` if ``target`` is not in (0, 1), + exceeds the reliability at 0 or what the reliability never falls below + (a part of the system that never fails: it is then never reached), or + cannot be bracketed below ``upper_bound``. + """ + if np.ndim(target) > 0: + targets = np.asarray(target, dtype=float) + return np.array( + [ + invert_reliability(sf, float(t), upper_bound) + for t in targets.ravel() + ] + ).reshape(targets.shape) + if not 0.0 < target < 1.0: + raise ValueError("target reliability must be in (0, 1).") + r0 = float(sf(0.0)) + if target > r0: + raise ValueError( + f"target reliability {target} exceeds the system reliability " + f"at t=0 ({r0:.6g}); it is never reached." + ) + + def f(t): + return float(sf(t)) - target + + hi = upper_bound + if hi is None: + hi = 1.0 + for _ in range(1000): + if f(hi) < 0.0: + break + hi *= 2.0 + else: + # Doubled past every finite time: the reliability never falls + # that far (a part of the system never fails). + floor = float(sf(LATE)) + raise ValueError( + f"target reliability {target} is never reached: the system " + f"reliability never falls below {floor:.6g} (a part of the " + "system never fails)." + ) + return float(brentq(f, 0.0, hi)) + + +def cumulative_hazard(sf, ff) -> np.ndarray: + """The cumulative hazard ``-ln R`` from a system's probabilities of + working ``sf`` and of failing ``ff`` (see ``NonRepairableRBD.Hf``): + ``-log1p(-F)`` while it is more likely to work than not, so that a + small one keeps its precision, and ``+inf`` where it cannot work.""" + sf, ff = np.asarray(sf, dtype=float), np.asarray(ff, dtype=float) + with np.errstate(divide="ignore"): + return np.where(ff < 0.5, -np.log1p(-np.minimum(ff, 0.5)), -np.log(sf)) + + +def density_by_differences(at, x: np.ndarray, h: np.ndarray) -> np.ndarray: + """A failure density at the times ``x`` by central differences of + step ``h`` of ``at(t)``, the probabilities of working and of failing + at the times ``t`` (see ``NonRepairableRBD.df``): the lower point + clipped at 0 (the difference is one-sided near 0), the change taken of + whichever of the two is the smaller (it keeps its precision), and a + negative result (rounding) clipped to 0.""" + x_hi, x_lo = x + h, np.maximum(x - h, 0.0) + sf_lo, ff_lo = at(x_lo) + sf_hi, ff_hi = at(x_hi) + change = np.where(ff_hi <= 0.5, ff_hi - ff_lo, sf_lo - sf_hi) + return np.clip(change / (x_hi - x_lo), 0.0, None) + + +def rate_parts( + importance: Mapping[Any, Any], + density_of: Callable[[Any, np.ndarray], np.ndarray], + x: np.ndarray, + groups=(), + inputs: Optional[Callable[[np.ndarray], Tuple[dict, dict]]] = None, + fails: Optional[Callable[[dict, dict], Any]] = None, + scale: Optional[Callable[[], float]] = None, + held: Collection[Any] = (), +) -> Dict[Any, np.ndarray]: + """Each element's part in the system reliability's rate of change at + the times ``x`` (see ``NonRepairableRBD.reliability_rate``): its + Birnbaum ``importance`` times ``density_of(element, x)``, negated; 0 + for an element ``held``, and for one that cannot matter with an + infinite density (a Weibull life of shape below 1, at 0). And each of + the common-cause ``groups``' part, under the tuple of its members: the + system's change with its members' lives moved on and the rest held, + by central differences of ``fails(p, q)``, the system's probability + of failing at the elements' probabilities of working and of failing + that ``inputs(t)`` gives, the step at 0 sized by ``scale()``, a + typical failure time.""" + members = {m for group in groups for m in group.members} + parts: Dict[Any, np.ndarray] = {} + for node, value in importance.items(): + if node in members: + continue + value = np.asarray(value, dtype=float).reshape(x.shape) + if node in held: + parts[node] = np.zeros(x.shape) + continue + with np.errstate(invalid="ignore"): + part = -value * density_of(node, x) + parts[node] = np.where(value == 0.0, 0.0, part) + if groups: + assert inputs is not None and fails is not None and scale is not None + step = difference_steps(x, 1e-6, scale) + hi, lo = x + step, np.maximum(x - step, 0.0) + p, q = inputs(x) + ends = [inputs(t) for t in (hi, lo)] + for group in groups: + downs = [] + for moved_p, moved_q in ends: + given, failing = dict(p), dict(q) + for member in group.members: + given[member] = moved_p[member] + failing[member] = moved_q[member] + downs.append( + np.asarray(fails(given, failing), dtype=float).reshape( + x.shape + ) + ) + parts[tuple(group.members)] = -(downs[0] - downs[1]) / (hi - lo) + return parts + + +def barlow_proschan( + parts: Callable[[np.ndarray], Dict[Any, np.ndarray]], + models: Iterable[Any], + x, + end: Callable[[np.ndarray], float], +) -> Dict[Any, Union[float, np.ndarray]]: + """Each element's (and common-cause group's) Barlow-Proschan + importance (see ``NonRepairableRBD.barlow_proschan_importance``): its + part of the system's failure density, ``parts(t)`` (see + ``rate_parts``), negated and integrated up to each time of ``x``, or + over the whole life (``x`` None), which ends at ``end(knots)``, over + the sum of the parts (``integrated_shares``); the integrals split at + the ``models``' knots (``model_knots``).""" + keys = list(parts(np.ones(1))) + + def columns(t: np.ndarray) -> np.ndarray: + found = parts(t) + return -np.column_stack([found[key] for key in keys]) + + knots = np.concatenate([np.empty(0)] + [model_knots(m) for m in models]) + knots = np.unique(knots[np.isfinite(knots) & (knots > 0.0)]) + if x is None: + ends = np.array([end(knots)]) + else: + ends = np.atleast_1d(np.asarray(x, dtype=float)).ravel() + if np.any(~np.isfinite(ends)) or np.any(ends < 0.0): + raise ValueError( + f"Times must be finite and non-negative, got {x!r}." + ) + shares = integrated_shares(columns, knots, ends) + if x is None or np.ndim(x) == 0: + return {key: float(shares[0, j]) for j, key in enumerate(keys)} + shape = np.shape(x) + return {key: shares[:, j].reshape(shape) for j, key in enumerate(keys)} diff --git a/repyability/rbd/_model_utils.py b/repyability/rbd/_model_utils.py index 703395a0..4f2b84b7 100644 --- a/repyability/rbd/_model_utils.py +++ b/repyability/rbd/_model_utils.py @@ -9,10 +9,29 @@ with a compatibility test) and gives the call sites intention-revealing names. """ -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, NamedTuple, Optional import numpy as np +#: What a model raises when it cannot work out what it is asked (a quantile, +#: a survival, a mean) at the values given: surpyval's models, and anything +#: else given as one, fail in these ways. Code that probes what a model can +#: do catches them, and lets anything else (a bug) through. +MODEL_ERRORS = ( + ArithmeticError, + AttributeError, + LookupError, + NotImplementedError, + RuntimeError, + TypeError, + ValueError, +) + +#: What saving a model that cannot be saved raises (``serialise_model``, +#: a model's ``to_dict`` and ``json.dumps``). +SAVE_ERRORS = (AttributeError, NotImplementedError, TypeError, ValueError) + + # surpyval distribution names whose event probability does not vary with time, # i.e. ``sf(t)`` is constant. These behave as fixed per-demand probabilities # rather than lifetime distributions. @@ -76,6 +95,22 @@ def never_fails(model) -> float: return max(0.0, 1.0 - float(p)) +def always_works(model) -> bool: + """Whether ``model`` never fails at all: ``PerfectReliability``, or a + fixed probability of failing of 0 (as a ``node_availability`` of 1 + gives). A node with such a model is a junction.""" + from .helper_classes import PerfectReliability + + if model is PerfectReliability or isinstance(model, PerfectReliability): + return True + if is_fixed_probability(model): + try: + return float(np.ravel(model.ff(1.0))[0]) == 0.0 + except MODEL_ERRORS: + return False + return False + + def is_exponential(model) -> bool: """True if ``model`` is a plain surpyval Exponential lifetime: no offset, every unit fails and none is dead on arrival. Only then is it @@ -114,20 +149,51 @@ def failure_time_scale(model) -> float: return float("nan") -def parametric_spec(model): - """Return ``(surpyval_class, params, parameter_names, extras)`` for a - parametric node model, or ``None`` when it has no reconstructable - distribution parameters (a ``StandbyModel``, ``RepeatedNode``, nested - RBD, a repeated node's source name, the perfect-reliability helpers, or - a fitted non-parametric model). ``extras`` are its offset, - limited-failure-population and zero-inflation parameters (surpyval's - ``model.extras``): ``surpyval_class.from_params(params, **extras)`` - rebuilds it. - - Used by parameter-sensitivity analysis to rebuild a distribution with a - perturbed parameter. It is faithful for exactly the models - ``serialisation`` round-trips (surpyval parametric distributions), since it - goes through the same ``dist name`` + ``from_params`` reconstruction. +#: The shares a model may have after its distribution's parameters, in the +#: order of surpyval's ``covariance()`` (a limited failure population's +#: ``lfp_p``, then a zero-inflated one's ``f0``), each with the flag that +#: says a fit estimated it and the value that means it has none. +_SHARES = (("lfp", "lfp_p", 1.0), ("zi", "f0", 0.0)) + + +class ParametricSpec(NamedTuple): + """A parametric model's parameters (see ``parametric_spec``).""" + + cls: Any + params: List[float] + names: List[str] + extras: Dict[str, Any] + bounds: List[tuple] + + def build(self, values) -> Any: + """The model with its parameters at ``values``, in the order of + ``params``.""" + shares = {share for _, share, _ in _SHARES} + own = [float(v) for n, v in zip(self.names, values) if n not in shares] + given = { + n: float(v) for n, v in zip(self.names, values) if n in shares + } + return self.cls.from_params(own, **{**self.extras, **given}) + + +def parametric_spec(model) -> Optional[ParametricSpec]: + """The parameters of a parametric node model, or ``None`` when it has no + reconstructable distribution parameters (a ``StandbyModel``, + ``RepeatedNode``, nested RBD, a repeated node's source name, the + perfect-reliability helpers, or a fitted non-parametric model). + + ``params`` and ``names`` are its distribution's, then the share that + ever fails (``lfp_p``, a limited failure population) and the share dead + on arrival (``f0``, zero inflation) where it has them: the order of + surpyval's ``covariance()``. ``extras`` are surpyval's ``model.extras`` + (the offset kept as it is), ``bounds`` each parameter's range as its + distribution bounds it (None where it does not), and ``build(values)`` + the model with its parameters at ``values``. + + Used by parameter sensitivity and uncertainty to rebuild a model with + moved or drawn parameters. It is faithful for exactly the models + ``serialisation`` round-trips (surpyval parametric distributions), since + it goes through the same ``dist name`` + ``from_params`` reconstruction. """ import surpyval @@ -142,7 +208,60 @@ def parametric_spec(model): names = getattr(dist, "parameter_names", None) if not names or len(list(names)) != len(params): names = [f"param{i}" for i in range(len(params))] - return cls, params, list(names), dict(getattr(model, "extras", {})) + names = list(names) + bounds = list(getattr(cls, "bounds", None) or []) + if len(bounds) != len(params): + bounds = [(None, None)] * len(params) + extras = dict(getattr(model, "extras", {})) + for flag, share, none in _SHARES: + value = extras.get(share, none) + if getattr(model, flag, False) or value != none: + params.append(float(value)) + names.append(share) + bounds.append((0, 1)) + return ParametricSpec(cls, params, names, extras, bounds) + + +class CovariateSpec(NamedTuple): + """A ``RegressionNode``'s covariates as levers, with the + interface of a ``ParametricSpec``: ``params`` the covariates, ``names`` + ``"covariate."`` (the model's feature names, or their places), + ``bounds`` none, and ``build(values)`` the node at those covariates.""" + + node: Any + params: List[float] + names: List[str] + bounds: List[tuple] + + def build(self, values) -> Any: + """The node at the covariates ``values``.""" + from repyability.rbd.regression_node import RegressionNode + + covariates = [float(v) for v in values] + return RegressionNode(self.node.model, covariates=covariates) + + +def covariate_spec(model) -> Optional[CovariateSpec]: + """The covariates of a ``RegressionNode`` at fixed covariates, as + levers, or None for any other model (one along a schedule has no one + value to move).""" + from repyability.rbd.regression_node import RegressionNode + + if not isinstance(model, RegressionNode) or model.covariates is None: + return None + params = [float(z) for z in model.covariates] + features = getattr(model.model, "feature_names", None) + if not features or len(list(features)) != len(params): + features = [str(i) for i in range(len(params))] + names = [f"covariate.{name}" for name in features] + return CovariateSpec(model, params, names, [(None, None)] * len(params)) + + +def lever_spec(model): + """What parameter sensitivity moves in a node's model: its + parameters (``parametric_spec``), or a regression node's covariates + (``covariate_spec``); None for a model with neither.""" + return parametric_spec(model) or covariate_spec(model) def nonparametric(model) -> bool: @@ -183,7 +302,7 @@ def nonparametric_nodes(models: Dict[Any, Any]) -> List[Any]: def refuse_nonparametric(models) -> None: """Refuse the nodes whose models (or component specs) hold a surpyval - non-parametric fit (#149).""" + non-parametric fit.""" nodes = nonparametric_nodes(models) if nodes: raise ValueError( diff --git a/repyability/rbd/_montecarlo.py b/repyability/rbd/_montecarlo.py index 58db8f34..8edfe083 100644 --- a/repyability/rbd/_montecarlo.py +++ b/repyability/rbd/_montecarlo.py @@ -21,15 +21,22 @@ from repyability.utils.wrappers import outside_level -def check_count(n, antithetic: bool, name: str) -> None: +def check_count(n, antithetic: bool, name: str, least: int = 1) -> None: """A ValueError unless ``n`` is a positive integer (and even, for - antithetic pairs).""" + antithetic pairs), of at least ``least`` independent samples (pairs, + with ``antithetic``): two for a standard error.""" if isinstance(n, bool) or not isinstance(n, (int, np.integer)) or n < 1: raise ValueError(f"{name} must be a positive integer, got {n!r}.") if antithetic and n % 2: raise ValueError( f"Antithetic samples come in pairs: {name} must be even, got {n}." ) + fewest = least * (2 if antithetic else 1) + if n < fewest: + raise ValueError( + f"{name} must be at least {fewest} (a standard error needs two " + f"independent samples), got {n}." + ) def check_confidence(confidence) -> None: @@ -125,7 +132,7 @@ def more_samples( infinite (or NaN) value stops the run at once: the mean is infinite (or undefined) however many more are taken. With ``unjudged``, why the values do not show the error yet (a run of modules that have not - changed state, #215): the run goes on, to the limit.""" + changed state): the run goes on, to the limit.""" if not np.all(np.isfinite(values)): return 0 if ( diff --git a/repyability/rbd/_point_availability.py b/repyability/rbd/_point_availability.py index 9235ac87..5ce57a7e 100644 --- a/repyability/rbd/_point_availability.py +++ b/repyability/rbd/_point_availability.py @@ -53,8 +53,9 @@ A unit under block replacement is followed interval by interval instead (``_block_replacement.block_availability``, a ``BlockCurve``), a unit with hidden failures by its closed form (an ``InspectionCurve``), and a -unit minimally repaired in no time by its life's cumulative hazard (a -``MinimalRepairCurve``). Every +unit minimally repaired by its life's cumulative hazard, exactly in no +time and on a grid of its operating age where its repairs take time (a +``MinimalRepairCurve``, by ``minimal_repair_lattice``). Every curve is constant (``period`` None) or repeats with ``period`` after ``settle``, which is what lets a mission average over many years integrate one period and repeat it. @@ -84,6 +85,8 @@ import numpy as np +from ._model_utils import MODEL_ERRORS + _GL_X, _GL_W = np.polynomial.legendre.leggauss(4) #: Fine-grid steps per standard deviation of the time a preventive @@ -118,7 +121,7 @@ def knots(model) -> np.ndarray: try: with np.errstate(all="ignore"): q = np.asarray(qf(_KNOT_PROBABILITIES), dtype=float).ravel() - except Exception: # a model whose qf cannot take these + except MODEL_ERRORS: # a model whose qf cannot take these return np.empty(0) return q[np.isfinite(q)] @@ -828,8 +831,13 @@ def __init__( mean, variance = _moments( sf, np.linspace(0.0, self.top, 4097), self.splits ) - if variance <= (1e-9 * self.top) ** 2: - self.fixed = mean + # A fixed time, to rounding: the variance is a difference of two + # numbers about ``top ** 2``, so it is uncertain by a few units of + # rounding in them (numpy 2.0 sums them to 1e-15 at a top of 5). + if variance <= (1e-6 * self.top) ** 2: + # A time known exactly is its atom's, not the mean's rounding. + atoms = _atoms(lambda s: 1.0 - sf(s), self.splits, self.top) + self.fixed = atoms[0][0] if len(atoms) == 1 else mean return from scipy.signal import fftconvolve @@ -1829,63 +1837,219 @@ def stages_at(self, x: np.ndarray) -> np.ndarray: return self.fractions[:, None] * self.at(np.atleast_1d(x))[None, :] +def cumulative_hazard(life, x: np.ndarray) -> np.ndarray: + """``life``'s cumulative hazard at each time ``x`` (from its survival + function where the model gives no hazard, or a NaN one), never below + 0.""" + x = np.asarray(x, dtype=float) + with np.errstate(divide="ignore", invalid="ignore", over="ignore"): + hazard = np.asarray(life.Hf(x), dtype=float).reshape(x.shape) + missing = np.isnan(hazard) + if missing.any(): + survival = np.asarray(life.sf(x[missing]), dtype=float) + hazard[missing] = -np.log(np.clip(survival, 0.0, 1.0)) + return np.maximum(hazard, 0.0) + + +#: The most powers of its repair time a minimally repaired unit's curve +#: sums (see ``minimal_repair_lattice``): about as many failures as the +#: unit can have in the window. +_MINIMAL_TERMS = 20_000 +#: The powers' sum is cut once what is left of it is below this. +_NEGLIGIBLE = 1e-17 + + +def minimal_repair_lattice( + hazard: np.ndarray, repair: np.ndarray +) -> Tuple[np.ndarray, np.ndarray]: + """A minimally repaired unit's point availability, and its expected + failures before each point, on a grid ``k h`` from new (see + ``MinimalRepairCurve``): ``hazard`` is its life's cumulative hazard + ``H`` at the grid's points and ``repair`` its repair time as a measure + on them (see ``_lattice``). + + The unit's operating age ``a`` is the time it has been up, and it + fails at its life's hazard at that age: its failures by operating age + ``a`` are Poisson with mean ``H(a)``, and each is followed by a repair + time ``S``, independent of them all. Reaching operating age ``a`` + takes calendar time ``a + D(a)``, ``D(a)`` the repairs so far, a + compound Poisson sum. The unit is up at calendar time ``t`` at the + operating age ``a`` with ``a + D(a) = t``, and has had its ``(k + + 1)``-th failure by ``t`` if the ``k + 1``-th point of its failures' + Poisson process in operating time, plus the ``k`` repairs before it, + comes by ``t``, so that, with ``N(a)`` the failures by operating age + ``a`` and ``G`` the repair time's distribution, + + ``A(t) = sum_k integral P(N(t - s) = k) G^{*k}(ds)``, and + ``E[N(t)] = sum_k integral P(N(t - s) > k) G^{*k}(ds)``, + + ``G^{*k}`` the distribution of ``k`` repairs' total. Each term is a + smooth function of the operating age integrated against a measure, + which the grid keeps the mass and mean of in each cell (``_lattice``, + the powers by FFT convolution): the error falls as the square of the + step, as a renewal unit's. The sum is cut once the failures past + ``k`` are negligible (the Poisson tail at the grid's end, for a + hazard finite throughout) or ``k`` repairs no longer fit in the + window. + With repairs in no time every ``G^{*k}`` is the atom at 0, and the + sums are 1 and ``H(t)``: the closed form ``MinimalRepairCurve`` takes + as it is. + """ + from scipy import fft + from scipy.special import gammaln, pdtrc + + size = len(hazard) + length = fft.next_fast_len(2 * size - 1, real=True) + finite = np.isfinite(hazard) + most = float(hazard[finite].max()) if finite.any() else 0.0 + with np.errstate(divide="ignore"): + log_hazard = np.log(hazard) + spectrum = fft.rfft(repair, length) + # G^{*k}: the atom at 0 first. + power = np.zeros(size) + power[0] = 1.0 + up = np.zeros(len(spectrum), dtype=complex) + failed = np.zeros(len(spectrum), dtype=complex) + # P(N(a) > k) at each grid age, less each P(N(a) = k) in turn: a life + # whose hazard is infinite there has failed more than any k times. + more = np.ones(size) + k = 0 + while True: + with np.errstate(invalid="ignore"): + count = np.where( + finite, + np.exp( + (k * log_hazard if k else 0.0) - hazard - gammaln(k + 1.0) + ), + 0.0, + ) + more = np.maximum(more - count, 0.0) + transformed = fft.rfft(power, length) + up += fft.rfft(count, length) * transformed + failed += fft.rfft(more, length) * transformed + if ( + finite.all() and pdtrc(k, most) < _NEGLIGIBLE + ) or power.sum() < _NEGLIGIBLE: + break + k += 1 + if k > _MINIMAL_TERMS: + raise NotImplementedError( + f"minimally repaired, it may fail more than {_MINIMAL_TERMS} " + "times in the window, more than its curve follows" + ) + power = fft.irfft(spectrum * transformed, length)[:size] + available = fft.irfft(up, length)[:size] + failures = fft.irfft(failed, length)[:size] + # New at 0 (its life works there), to rounding. + available[0], failures[0] = 1.0, 0.0 + return np.clip(available, 0.0, 1.0), np.maximum(failures, 0.0) + + class MinimalRepairCurve: - """A unit minimally repaired in no time (Kijima's models with ``q = 1`` - and an instant repair): up throughout, and its failures a - non-homogeneous Poisson process whose intensity is its life's hazard at - its age, so that it fails ``H(x)`` times before ``x`` on average, ``H`` - the life's cumulative hazard, exactly. - - Its counts never settle (with a wearing-out life they grow ever faster), - so it is followed to the horizon; for the integrals of its counts it - declares a grid of ``steps`` steps over its first ``scale`` past the - life's ``offset``, whose step doubles each time that span does - (``grids``): a piece then spans the same share of the time it starts at - however far on it is, as many pieces for each doubling of the window. - The offset, before which the unit cannot fail, is a break. ``counts`` - False, only its availability is asked for, which is 1 from the start. + """A unit minimally repaired (Kijima's models with ``q = 1``): its + failures are a non-homogeneous Poisson process in its operating time + whose intensity is its life's hazard at its age, the time it has been + up, which its repairs leave as it is. + + Repaired in no time it is up throughout, and fails ``H(x)`` times + before ``x`` on average, ``H`` the life's cumulative hazard, exactly. + Its counts never settle (with a wearing-out life they grow ever + faster), so it is followed to the horizon; for the integrals of its + counts it declares a grid of ``steps`` steps over its first ``scale`` + past the life's ``offset``, whose step doubles each time that span + does (``grids``): a piece then spans the same share of the time it + starts at however far on it is, as many pieces for each doubling of + the window. The offset, before which the unit cannot fail, is a break. + ``counts`` False, only its availability is asked for, which is 1 from + the start. + + With repairs that take time (``repair_sf`` their survival function, + ``repair_splits`` its quantiles) its availability and its expected + failures are worked out on the grid ``k step``, ``k`` up to ``n``, by + ``minimal_repair_lattice``, and are linear between the grid's points; + the curve then never settles, and is followed to the horizon. """ period = None - def __init__(self, life, scale: float, steps: int, counts: bool): + def __init__( + self, + life, + scale: float, + steps: int, + counts: bool, + repair_sf: Optional[Callable] = None, + repair_splits=(), + step: Optional[float] = None, + n: Optional[int] = None, + ): self.life = life self.offset = max(float(getattr(life, "gamma", 0.0) or 0.0), 0.0) self.scale = float(scale) self.steps = int(steps) self.counts = counts - self.settle = np.inf if counts else 0.0 + self.times: Optional[np.ndarray] = None + self.up: Optional[np.ndarray] = None + self.failures: Optional[np.ndarray] = None + if repair_sf is not None: + assert step is not None and n is not None + self.times = float(step) * np.arange(int(n) + 1) + repair = _lattice( + *_cells( + lambda s: 1.0 - repair_sf(s), self.times, repair_splits + ) + ) + self.up, self.failures = minimal_repair_lattice( + cumulative_hazard(life, self.times), repair + ) + self.settle = np.inf if counts or self.times is not None else 0.0 def at(self, x: np.ndarray) -> np.ndarray: - return np.ones(np.shape(x)) + if self.times is None: + return np.ones(np.shape(x)) + assert self.up is not None # set with times + return np.interp(x, self.times, self.up) def knots(self, start: float, stop: float) -> np.ndarray: - return self.breaks(start, stop) + """The times in ``[start, stop]`` between which the curve is + smooth: its breaks, and its grid's points, if it has one.""" + breaks = self.breaks(start, stop) + if self.times is None: + return breaks + times = np.concatenate([self.times, breaks]) + return times[(times >= start) & (times <= stop)] def breaks(self, start: float, stop: float) -> np.ndarray: + """The offset, and the grid's ends, if it has one (see + ``curve_breaks``).""" times = np.array([self.offset]) if self.offset > 0.0 else np.empty(0) + if self.times is not None: + times = np.concatenate([times, self.times[[0, -1]]]) return times[(times >= start) & (times <= stop)] def grids(self) -> list: - """Its grids (see ``curve_grids``): to the offset plus ``scale`` - doubled ``k`` times, a step of that time over ``steps``.""" + """Its grids (see ``curve_grids``): its grid, to its end, with + repairs that take time; else, for its counts, to the offset plus + ``scale`` doubled ``k`` times, a step of that time over + ``steps``.""" + if self.times is not None: + if len(self.times) < 2: + return [] + return [(float(self.times[1]), float(self.times[-1]))] if not self.counts: return [] ends = self.offset + self.scale * 2.0 ** np.arange(64) return [(float(end) / self.steps, float(end)) for end in ends] def events(self, x: np.ndarray) -> Dict[str, np.ndarray]: - """The unit's expected failures before each time ``x``: its life's - cumulative hazard there (from its survival function where the - model gives no hazard, or a NaN one), each a repair.""" + """The unit's expected failures before each time ``x``, each a + repair: its life's cumulative hazard there with repairs in no + time, its grid's values otherwise.""" x = np.asarray(x, dtype=float) - with np.errstate(divide="ignore", invalid="ignore", over="ignore"): - hazard = np.asarray(self.life.Hf(x), dtype=float).reshape(x.shape) - missing = np.isnan(hazard) - if missing.any(): - survival = np.asarray(self.life.sf(x[missing]), dtype=float) - hazard[missing] = -np.log(np.clip(survival, 0.0, 1.0)) - return _events(np.maximum(hazard, 0.0)) + if self.times is None: + return _events(cumulative_hazard(self.life, x)) + assert self.failures is not None # set with times + return _events(np.interp(x, self.times, self.failures)) def atoms(self, stop: float) -> Atoms: return Atoms.none() @@ -2088,7 +2252,9 @@ def __init__(self, rbd, curves: dict, settle: float, period): self.period = period def at(self, x: np.ndarray) -> np.ndarray: - return self.rbd._curves_at(self.curves, x, set(), set(), "p") + from . import _curves + + return _curves._curves_at(self.rbd, self.curves, x, set(), set(), "p") def derivative(self, x: np.ndarray, scale: float) -> np.ndarray: """Its rate of change at each ``x``: each of its nodes' (see @@ -2097,9 +2263,11 @@ def derivative(self, x: np.ndarray, scale: float) -> np.ndarray: ``_rates``).""" from repyability.rbd._rates import derivative + from . import _windows + values = {node: c.at(x) for node, c in self.curves.items()} importance = self.rbd._importances( - self.rbd._filled(values, len(x), set(), set()) + _windows._filled(self.rbd, values, len(x), set(), set()) )[0] out = np.zeros(len(x)) for node, curve in self.curves.items(): @@ -2108,17 +2276,31 @@ def derivative(self, x: np.ndarray, scale: float) -> np.ndarray: def events(self, x: np.ndarray) -> Dict[str, np.ndarray]: """The nested RBD's expected failures and planned outages before - each time ``x`` (see ``RepairableRBD._window_counts``): its own - maintenance is not this RBD's to count.""" - counts = self.rbd._window_counts( - self.curves, np.asarray(x, dtype=float), set(), set(), "p" + each time ``x`` (see ``RepairableRBD._window_counts``), as the + simulation counts them for the node: its planned outages are its + preventive actions, its failures its corrective ones, and it has + no tests (its own components' maintenance is not this RBD's to + count).""" + from . import _windows + + counts = _windows._window_counts( + self.rbd, + self.curves, + np.asarray(x, dtype=float), + set(), + set(), + "p", + ) + return _events( + counts["failures"], counts["planned"], preventive=counts["planned"] ) - return _events(counts["failures"], counts["planned"]) def atoms(self, stop: float) -> Atoms: """The nested RBD's failures and planned outages at exact times before ``stop`` (see ``RepairableRBD._atom_groups``).""" - return self.rbd._system_atoms(self.curves, stop) + from . import _windows + + return _windows._system_atoms(self.rbd, self.curves, stop) def knots(self, start: float, stop: float) -> np.ndarray: parts = [curve.knots(start, stop) for curve in self.curves.values()] diff --git a/repyability/rbd/_processes.py b/repyability/rbd/_processes.py new file mode 100644 index 00000000..12bf4f9c --- /dev/null +++ b/repyability/rbd/_processes.py @@ -0,0 +1,84 @@ +"""surpyval's fitted repairable-unit models as components (#269). + +surpyval fits a repairable unit's failure history: a Poisson process +(``CrowAMSAA``, ``Duane``, ``HPP``) or a generalised renewal process +(``GeneralizedRenewal``, Kijima I or II). Given as a component's +``"reliability"``, each is a life and a repair RePyability already +simulates and works out: + +- a Poisson process is minimal repair (Kijima ``q = 1``) of the life whose + cumulative hazard is the process's cumulative intensity: Crow-AMSAA's + ``(t / alpha) ** beta`` is a Weibull(``alpha``, ``beta``), Duane's + ``b * t ** alpha`` a Weibull(``b ** (-1 / alpha)``, ``alpha``), and a + homogeneous process's ``lambda * t`` an Exponential(``lambda``), which + any repair renews; +- a generalised renewal process is its life distribution repaired by its + Kijima model with its restoration factor ``q`` (0 renews, 1 is minimal, + as a spec's ``"repair"`` takes it). + +The other renewal models (ARA, ARI, G1) are refused: their repairs are not +Kijima's, and drawing their next failure needs surpyval (SurPyval#833). +""" + +from typing import Any, Optional, Tuple + +#: Kijima's model, by surpyval's ``kijima_type``. +_KIJIMA = {"i": "kijima1", "ii": "kijima2"} + + +def _translated(model) -> Optional[Tuple[Any, Optional[dict], str]]: + """``(life, repair, name)`` for a fitted process, or None for any other + model.""" + import surpyval as surv + + fitter = getattr(model, "dist", None) + if fitter is surv.HPP: + (rate,) = (float(p) for p in model.params) + return surv.Exponential.from_params([rate]), None, "HPP" + if fitter is surv.CrowAMSAA: + alpha, beta = (float(p) for p in model.params) + life = surv.Weibull.from_params([alpha, beta]) + return life, {"model": "kijima1", "q": 1.0}, "CrowAMSAA" + if fitter is surv.Duane: + alpha, b = (float(p) for p in model.params) + life = surv.Weibull.from_params([b ** (-1.0 / alpha), alpha]) + return life, {"model": "kijima1", "q": 1.0}, "Duane" + kind = str(getattr(model, "kind", "")) + if not (hasattr(model, "restoration") and kind.endswith("Renewal")): + return None + kijima = _KIJIMA.get(str(getattr(model, "kijima_type", ""))) + if kind != "Generalized Renewal" or not kijima: + raise ValueError( + f"its reliability is a surpyval {model.kind} model, whose " + "repairs are not Kijima's; RePyability takes a " + "GeneralizedRenewal (Kijima I or II), CrowAMSAA, Duane or HPP " + "fit (drawing the others' failures needs SurPyval#833)." + ) + repair = {"model": kijima, "q": float(model.restoration)} + return model.model, repair, "GeneralizedRenewal" + + +def as_spec(name, spec): + """``spec`` with a fitted process given as its ``"reliability"`` + written as the life and ``"repair"`` it is (see the module docstring); + any other spec as it is.""" + if not isinstance(spec, dict): + return spec + try: + translated = _translated(spec.get("reliability")) + except ValueError as error: + raise ValueError(f"Component {name!r}: {error}") from None + if translated is None: + return spec + life, repair, kind = translated + if "repair" in spec: + raise ValueError( + f"Component {name!r}: its reliability, a fitted {kind}, brings " + "its own repair (a Poisson process is minimal repair, a " + "generalised renewal process its Kijima model and q): give no " + "'repair'." + ) + out = {**spec, "reliability": life} + if repair is not None: + out["repair"] = repair + return out diff --git a/repyability/rbd/_repairable_allocation.py b/repyability/rbd/_repairable_allocation.py new file mode 100644 index 00000000..067e23f5 --- /dev/null +++ b/repyability/rbd/_repairable_allocation.py @@ -0,0 +1,976 @@ +"""A ``RepairableRBD``'s allocations: the redundancy that meets a target +or a budget at the lowest cost (``allocate_redundancy``, trains of nodes +copied whole), and the component availabilities, or mean lives and repair +times, that meet a system availability target (``availability_allocation``, +``mttf_mttr_allocation``). The methods of ``RepairableRBD`` of those names +call these. +""" + +import math +import warnings +from collections.abc import Mapping +from dataclasses import dataclass +from functools import partial +from typing import ( + TYPE_CHECKING, + Any, + Collection, + Dict, + Hashable, + List, + NamedTuple, + Optional, + Tuple, + Union, +) + +import numpy as np +from scipy.optimize import OptimizeResult, brentq, minimize +from scipy.special import expit, logit, logsumexp, softmax + +from repyability.rbd import ( + _ccf_groups, + _costs, + _crews, + _long_run, + _requirements, +) +from repyability.rbd._common import ( + _discount_rate, + _horizons, + _present_horizon, +) +from repyability.rbd._model_utils import ( + model_mean, +) +from repyability.rbd.redundancy_allocation import ( + lowest_total_cost, + redundancy_caps, +) +from repyability.rbd.results import ( + AvailabilityAllocation, + TotalCostAllocation, +) +from repyability.rbd.routes import Refused +from repyability.utils.checks import ( + number_or_nan, + one_of, +) +from repyability.utils.wrappers import outside_level + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + +_ALLOCATION_CREWS = ( + "the allocations assume", + "Compare designs by their long-run values, or simulate them with " + "compare().", +) + + +class _Train(NamedTuple): + """A train ``allocate_redundancy`` may copy (see its ``trains``): its + name, its nodes in order, the node its last one feeds, where its + copies join, and how many of its inputs that node needs.""" + + name: Hashable + members: Tuple[Hashable, ...] + exit: Hashable + k: int + + +@dataclass(frozen=True) +class _TrainCopy: + """A node of a copy of a train, drawn by ``allocate_redundancy``: the + train's ``index``-th copy of its ``node``. Equal only to itself, so + that no node of the diagram can share its name.""" + + train: Hashable + index: int + node: Hashable + + +def _allocation_target(target) -> float: + """An availability allocation's target, checked to be in [0, 1].""" + value = number_or_nan(target) + if not 0.0 <= value <= 1.0: + raise ValueError(f"target must be a number in [0, 1], got {target!r}.") + return value + + +def _require_groups_allocated(rbd, chosen, drawn) -> dict: + """Raise unless the redundancy allocation takes in the common-cause + groups (#158): their chains cover them, a member given copies is in + a ``BetaFactor`` group (its copies join the group, and an MGL + model's letters are for its group's size), and no member is in a + train. Returns each member given copies' group.""" + from repyability.rbd.ccf import BetaFactor + + _ccf_groups._require_ccf_long_run(rbd) + group_of = {m: g for g in rbd.ccf_groups for m in g.members} + for node in chosen: + group = group_of.get(node) + if ( + group is not None + and _ccf_groups._ccf_timings(rbd, group) is not None + ): + raise NotImplementedError( + f"Node {node!r} is in a common-cause group whose " + "members' tests or repairs take time: its copies would " + "join the group, which its chain does not take in, as " + "yet. Leave the group's members out of nodes, or " + "estimate the design by simulation, with cost()." + ) + if group is not None and not isinstance(group.model, BetaFactor): + raise NotImplementedError( + f"Node {node!r} is in a common-cause group whose model, " + f"{group.model!r}, is for a group of " + f"{len(group.members)}: its copies would join the group, " + "and an MGL model's letters (an AlphaFactor model's " + "alpha factors) for a larger group are not known. Give " + "the group a BetaFactor model, whose beta " + "holds at any size, or leave its members out of nodes." + ) + for train in drawn: + inside = [m for m in train.members if m in group_of] + if inside: + raise NotImplementedError( + f"Train {train.name!r} holds {inside}, in a common-cause " + "group, whose copies the allocation of trains does not " + "take in, as yet: give the members copies as nodes." + ) + return {node: group_of[node] for node in chosen if node in group_of} + + +def allocate_redundancy( + rbd, + horizon: float, + *, + nodes: Optional[Collection[Hashable]], + trains: Optional[Mapping], + min_availability: Optional[float], + max_units: Union[int, Dict[Hashable, int], None], + method: str, + discount_rate: float, +) -> TotalCostAllocation: + """See ``RepairableRBD.allocate_redundancy``.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + rate = _discount_rate(discount_rate) + if np.ndim(horizon) != 0: + raise ValueError( + f"horizon must be a number for allocate_redundancy, got " + f"{horizon!r}." + ) + horizon = float(_horizons(horizon, rate)) + # The running costs count over the horizon's present value. + present = float( + _present_horizon( + np.array(horizon), rate, partial(_requirements._mean_lives, rbd) + ) + ) + one_of("method", method, ("exact", "greedy")) + if nodes is None and trains is not None: + chosen: list = [] + elif nodes is None: + chosen = [n for n in rbd.components if n in rbd.acquisition_costs] + if not chosen: + raise ValueError( + "No component has an acquisition_cost, so there is " + "nothing to buy copies of: give the components one, or " + "name the nodes that may be given copies with `nodes`." + ) + else: + chosen = list(dict.fromkeys(nodes)) + if not chosen: + raise ValueError("nodes must name at least one component.") + for node in chosen: + _requirements._require_component(rbd, node, "nodes") + if isinstance(rbd.components[node], RepairableRBD): + raise ValueError( + f"Node {node!r} is a nested RepairableRBD, which " + "cannot be given copies here: its costs are not " + "this RBD's." + ) + drawn = _allocation_trains(rbd, trains, chosen) + items = chosen + [train.name for train in drawn] + caps = redundancy_caps( + items, max_units, named="nodes or trains" if drawn else "nodes" + ) + max_unavailability = None + if min_availability is not None: + try: + target = float(min_availability) + except (TypeError, ValueError): + target = float("nan") + if not 0.0 < target < 1.0: + raise ValueError( + "min_availability must be a number in (0, 1), got " + f"{min_availability!r}." + ) + max_unavailability = 1.0 - target + + # Adding copies adds jobs for the crews: the search assumes none waits. + _crews._require_unlimited_crews(rbd, *_ALLOCATION_CREWS) + grouped = _require_groups_allocated(rbd, chosen, drawn) + # Every node's availability (and unavailability) over the times the + # long-run values average over, and each component's own cost. + times, weights = _long_run._long_run_grid(rbd) + up = { + node: np.atleast_1d(np.asarray(a, dtype=float)) + for node, a in _long_run._availabilities_at(rbd, times).items() + } + down = { + node: np.atleast_1d(np.asarray(u, dtype=float)) + for node, u in _long_run._unavailabilities_at(rbd, times).items() + } + copy_cost = {} + for node in rbd.components: + rate = _costs._node_cost_rate( + rbd, node, _long_run._node_availability(rbd, node) + ) + copy_cost[node] = ( + rbd.acquisition_costs.get(node, 0.0), + rate, + rbd.acquisition_costs.get(node, 0.0) + present * rate, + ) + # A train's copy costs its nodes'. + train_cost = [ + math.fsum(copy_cost[node][2] for node in train.members) + for train in drawn + ] + item_cost = [copy_cost[node][2] for node in chosen] + train_cost + for index, (name, cost, cap) in enumerate(zip(items, item_cost, caps)): + if cost <= 0.0 and cap == math.inf: + what = "train " if index >= len(chosen) else "" + raise ValueError( + f"A copy of {what}{name!r} costs nothing over the horizon " + "(it has no acquisition or running cost), so copies " + "could be added without end: give it an " + "acquisition_cost or a max_units." + ) + size = len(times) + structures: Dict[tuple, Any] = {(): rbd._decomposition()} + + def structure(extra: tuple): + # The diagram with extra[i] copies of train i drawn alongside it. + if not any(extra): + return structures[()] + if extra not in structures: + structures[extra] = rbd._decompose_graph( + _with_train_copies(rbd, drawn, extra) + ) + return structures[extra] + + # With common-cause groups (#158), a member's copies join its group: + # each design's groups' states, kept. + designs: Dict[tuple, list] = {} + + def group_states(copies: dict) -> list: + key = tuple( + copies.get(m, 1) for g in rbd.ccf_groups for m in g.members + ) + if key not in designs: + designs[key] = [ + _ccf_groups._group_states( + rbd, + group, + times, + [ + ( + 0 + if copies.get(m, 1) == math.inf + else copies.get(m, 1) + ) + for m in group.members + ], + ) + for group in rbd.ccf_groups + ] + return designs[key] + + def unavailability(counts) -> float: + p, q = dict(up), dict(down) + joining: dict = {} + for node, n in zip(chosen, counts): + if node in grouped: + joining[node] = n + if n == math.inf: + p[node], q[node] = np.ones(size), np.zeros(size) + elif n != 1: + q[node] = down[node] ** n + p[node] = 1.0 - q[node] + extra = [] + for train, n in zip(drawn, counts[len(chosen) :]): + # Made perfect (the search's bound), it is as many perfect + # copies as its vote needs. + copies = train.k if n == math.inf else int(n) - 1 + extra.append(copies) + for index in range(1, copies + 1): + for node in train.members: + name = _TrainCopy(train.name, index, node) + if n == math.inf: + p[name], q[name] = np.ones(size), np.zeros(size) + else: + p[name], q[name] = up[node], down[node] + points, at = size, weights + if rbd.ccf_groups: + p, split, at, _ = _ccf_groups._with_ccf_groups( + rbd, + times, + p, + q, + weights, + group_states(joining), + "The redundancy allocation", + ) + assert split is not None + q, points = split, len(at) + _, fails = structure(tuple(extra)).probabilities( + p, q, shape=points, works=False, fails=True + ) + fails = np.broadcast_to(np.asarray(fails, dtype=float), (points,)) + return float(at @ fails) + + copies_down: Dict[tuple, np.ndarray] = {} + + def all_down(node, k: int) -> np.ndarray: + # The chance that all k copies of a common-cause group member + # are down, at each time: from its group's chain of them alone, + # as the others' copies do not change their law. + if (node, k) not in copies_down: + group = grouped[node] + counts = [k if m == node else 0 for m in group.members] + states = _ccf_groups._group_states(rbd, group, times, counts) + copies_down[(node, k)] = states.probabilities[:, -1] + return copies_down[(node, k)] + + def gain(node): + # The most the k + 1-th copy of the node can lower the system + # unavailability: the node's own fall in unavailability (a + # common-cause group member's, all its copies down, from its + # group's chain). + if node in grouped: + return lambda k: float( + weights @ (all_down(node, k) - all_down(node, k + 1)) + ) + return lambda k: float(weights @ (down[node] ** k * up[node])) + + def in_series(node) -> bool: + # Down alone, it takes the system down: it lies on every path. + alone = {n: np.ones(1) for n in rbd.nodes} + alone[node] = np.zeros(1) + return float(rbd.system_probability(alone)[0]) == 0.0 + + def train_gain(train): + # The n + 1-th train changes the system only if it brings its + # vote to k working inputs, so at most k - 1 of the n there + # work: it lowers the unavailability by at most its own + # availability times the chance of that. + from scipy.stats import binom + + own = -np.expm1( + np.sum([np.log1p(-down[node]) for node in train.members], 0) + ) + return lambda n: float( + weights + @ ((1.0 - own) * binom.sf(n - train.k, n, np.minimum(own, 1))) + ) + + series = None + varying = set(rbd._inspection) | set(_long_run._block_nodes(rbd)) + if not drawn and all( + node not in varying and node not in grouped and in_series(node) + for node in chosen + ): + # Each such node's availability is constant, and the system's is + # theirs times the rest's: the exact search is then a dynamic + # program over the nodes. + series = [(lambda n, q=float(down[node][0]): q**n) for node in chosen] + counts, _, u = lowest_total_cost( + unavailability, + item_cost, + present * rbd.downtime_cost_rate, + [gain(node) for node in chosen] + + [train_gain(train) for train in drawn], + caps, + max_unavailability, + method, + series, + ) + units = {name: int(n) for name, n in zip(items, counts)} + # Every component once, its copies as many times more, and each + # train's copies their nodes' costs as many times. + extra = { + node: units[train.name] - 1 + for train in drawn + for node in train.members + } + acquisition = math.fsum( + (units.get(node, 1) + extra.get(node, 0)) * copy_cost[node][0] + for node in rbd.components + ) + cost_rate = ( + math.fsum( + (units.get(node, 1) + extra.get(node, 0)) * copy_cost[node][1] + for node in rbd.components + ) + + rbd.downtime_cost_rate * u + ) + return TotalCostAllocation( + units=units, + total_cost=acquisition + present * cost_rate, + acquisition_cost=acquisition, + cost_rate=cost_rate, + availability=1.0 - u, + horizon=horizon, + method=method, + discount_rate=float(discount_rate), + trains=( + {train.name: list(train.members) for train in drawn} + if drawn + else None + ), + ) + + +def _allocation_trains(rbd, trains, chosen) -> List[_Train]: + """The trains ``allocate_redundancy`` may copy (see its ``trains``), + checked: each a chain of components in series ending at a node that + feeds one node, a node in one train at most and not in ``chosen`` + (the nodes copied alone), and named apart from the nodes.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if trains is None: + return [] + if not isinstance(trains, Mapping) or not trains: + raise ValueError( + "trains must be a non-empty dict {name: [nodes in series]}, " + f"got {trains!r}." + ) + graph = rbd.G + out: List[_Train] = [] + seen: Dict[Hashable, Hashable] = {} + for name, members in trains.items(): + if name in graph.nodes: + raise ValueError( + f"Train {name!r} is named as a node: name a train apart " + "from the nodes, as units holds both." + ) + if isinstance(members, (str, bytes)) or not isinstance( + members, Collection + ): + raise ValueError( + f"Train {name!r} must be a list of nodes, got " f"{members!r}." + ) + members = list(members) + if not members: + raise ValueError(f"Train {name!r} has no nodes.") + for node in members: + _requirements._require_component(rbd, node, f"train {name!r}") + if isinstance(rbd.components[node], RepairableRBD): + raise ValueError( + f"Node {node!r} of train {name!r} is a nested " + "RepairableRBD, which cannot be given copies here: " + "its costs are not this RBD's." + ) + if node in seen: + raise ValueError( + f"Node {node!r} is in trains {seen[node]!r} and " + f"{name!r}: a node is in one train at most." + ) + if node in chosen: + raise ValueError( + f"Node {node!r} is in nodes and in train {name!r}: " + "copy it on its own or with its train." + ) + seen[node] = name + for a, b in zip(members, members[1:]): + if list(graph.successors(a)) != [b] or list( + graph.predecessors(b) + ) != [a]: + raise ValueError( + f"Train {name!r} must be a chain of nodes in series: " + f"{a!r} feeding {b!r} alone, and {b!r} fed by {a!r} " + "alone." + ) + exits = list(graph.successors(members[-1])) + if len(exits) != 1: + raise ValueError( + f"Train {name!r} must end at a node feeding one node, " + f"which its copies join; {members[-1]!r} feeds " + f"{len(exits)}." + ) + out.append( + _Train( + name, + tuple(members), + exits[0], + int(graph.nodes[exits[0]]["k"]), + ) + ) + return out + + +def _with_train_copies(rbd, trains: List[_Train], extra: tuple): + """The diagram's graph with ``extra[i]`` copies of ``trains[i]`` + drawn alongside it: each fed as its first node is, feeding the node + its last feeds, whose ``k`` is kept (see ``allocate_redundancy``).""" + graph = rbd.G.copy() + for train, copies in zip(trains, extra): + first = list(rbd.G.predecessors(train.members[0])) + for index in range(1, copies + 1): + names = [_TrainCopy(train.name, index, n) for n in train.members] + for name, node in zip(names, train.members): + graph.add_node(name, **rbd.G.nodes[node]) + graph.add_edges_from((u, names[0]) for u in first) + graph.add_edges_from(zip(names, names[1:])) + graph.add_edge(names[-1], train.exit) + return graph + + +def availability_allocation( + rbd, + target: float, + method: str, + *, + fixed: Optional[Collection[Hashable]], + weights: Optional[Dict], + max_availabilities: Optional[Dict], + feasibility: Optional[Dict], +) -> AvailabilityAllocation: + """See ``RepairableRBD.availability_allocation``.""" + methods = ("cost_based", "improvement", "minimum_effort", "equal") + if method not in methods: + raise ValueError( + "method must be one of " + f"{', '.join(repr(m) for m in methods)}, got {method!r}." + ) + options = { + "weights": (weights, "improvement"), + "max_availabilities": (max_availabilities, "cost_based"), + "feasibility": (feasibility, "cost_based"), + } + for name, (value, owner) in options.items(): + if value is not None and method != owner: + raise ValueError(f"{name} applies to method={owner!r} only.") + target = _allocation_target(target) + current, free, held = _allocatable(rbd, fixed) + for name, (value, _) in options.items(): + _allocation_option(rbd, name, value, held) + view = _allocation_view(rbd, held) + start = {node: current[node] for node in rbd.nodes} + if method == "cost_based": + maxima = {node: current[node] for node in held} + maxima.update(max_availabilities or {}) + allocated = view.cost_based_allocation( + target, + start, + max_probabilities=maxima, + feasibility=feasibility, + ) + elif method == "improvement": + allocated = view.improvement_allocation( + target, start, fixed=list(held), weights=weights + ) + elif method == "equal": + # From 0.5 for every component allocated, scaled alike, so they + # all end up the same. + allocated = view.improvement_allocation( + target, + { + node: current[node] if node in held else 0.5 + for node in rbd.nodes + }, + fixed=list(held), + ) + else: + allocated = _minimum_effort_held(view, target, current, held) + if view is not rbd and method != "minimum_effort": + rbd.res = view.res + mttf, mttr = {}, {} + for node, (up, down) in free.items(): + a = allocated[node] + mttf[node] = down * a / (1.0 - a) if a < 1.0 else math.inf + mttr[node] = up * (1.0 - a) / a if a > 0.0 else math.inf + return _allocation_result(rbd, view, allocated, mttf, mttr) + + +def mttf_mttr_allocation( + rbd, + target: float, + *, + levers: str, + fixed: Optional[Collection[Hashable]], + max_mttf: Optional[Dict], + min_mttr: Optional[Dict], + mttf_feasibility: Optional[Dict], + mttr_feasibility: Optional[Dict], +) -> AvailabilityAllocation: + """See ``RepairableRBD.mttf_mttr_allocation``.""" + if levers not in ("both", "mttf", "mttr"): + raise ValueError( + f"levers must be 'both', 'mttf' or 'mttr', got {levers!r}." + ) + target = _allocation_target(target) + current, free, held = _allocatable(rbd, fixed) + most = _allocation_option(rbd, "max_mttf", max_mttf, held) + least = _allocation_option(rbd, "min_mttr", min_mttr, held) + ease = ( + _allocation_option(rbd, "mttf_feasibility", mttf_feasibility, held), + _allocation_option(rbd, "mttr_feasibility", mttr_feasibility, held), + ) + names = ("mttf_feasibility", "mttr_feasibility") + # Each component's failure rate and repair time, as (start, floor): + # each falls from its start towards its floor as its lever is used, + # to floor + (start - floor) * exp(-v) for the lever's v >= 0. + span: Dict[Hashable, Tuple[Tuple[float, float], ...]] = {} + variables: List[Tuple[Hashable, int, float]] = [] + for node, (mttf, mttr) in free.items(): + top = number_or_nan(most.get(node, math.inf)) + if not top >= mttf * (1.0 - 1e-12): + raise ValueError( + f"max_mttf[{node!r}] must be at least the component's " + f"MTTF, {mttf:.6g}, got {most[node]!r}." + ) + bottom = number_or_nan(least.get(node, 0.0)) + if not 0.0 <= bottom <= mttr * (1.0 + 1e-12): + raise ValueError( + f"min_mttr[{node!r}] must be between 0 and the " + f"component's MTTR, {mttr:.6g}, got {least[node]!r}." + ) + span[node] = ( + (1.0 / mttf, 1.0 / max(top, mttf)), + (mttr, min(bottom, mttr)), + ) + for lever, name in enumerate(names): + f = number_or_nan(ease[lever].get(node, 0.5)) + if not 0.0 <= f < 1.0: + raise ValueError( + f"{name}[{node!r}] must be in [0, 1), got " + f"{ease[lever][node]!r}." + ) + start, floor = span[node][lever] + if levers in ("both", ("mttf", "mttr")[lever]) and (start > floor): + variables.append((node, lever, 1.0 - f)) + if not variables: + raise ValueError( + "No MTTF or MTTR can change: every lever is held (by levers, " + "max_mttf or min_mttr)." + ) + steepness = np.array([s for _, _, s in variables]) + size = len(variables) + + def design(v) -> Dict[Hashable, List[float]]: + """Each component's failure rate and repair time.""" + values = { + node: [ends[0][0], ends[1][0]] for node, ends in span.items() + } + for (node, lever, _), x in zip(variables, v): + if x > 0.0: + start, floor = span[node][lever] + gap = (start - floor) * math.exp(-x) + values[node][lever] = floor + gap + return values + + def probabilities(v) -> Tuple[Dict, Dict]: + p = {node: current[node] for node in rbd.nodes} + q = {node: 1.0 - current[node] for node in rbd.nodes} + for node, (rate, repair) in design(v).items(): + odds = rate * repair # of being down + p[node] = 1.0 / (1.0 + odds) + q[node] = odds / (1.0 + odds) + return p, q + + def result(v) -> AvailabilityAllocation: + values = design(v) + mttf = {} + for node, (rate, _) in values.items(): + if rate == span[node][0][0]: + mttf[node] = free[node][0] # unchanged, exactly + else: + mttf[node] = 1.0 / rate if rate > 0.0 else math.inf + return _allocation_result( + rbd, + view, + probabilities(v)[0], + mttf, + {node: repair for node, (_, repair) in values.items()}, + ) + + view = _allocation_view(rbd, held) + goal = float(logit(target)) + now = view._log_odds(*probabilities(np.zeros(size)))[0] + if goal <= now: + rbd.res = OptimizeResult( + x=np.zeros(size), + fun=float(np.log(size)), + success=True, + message="The target is already met.", + ) + return result(np.zeros(size)) + best = view._log_odds(*probabilities(np.full(size, np.inf)))[0] + # The limits are only approached (at an ever-growing cost), so a + # target at them, to within rounding, cannot be met either. + if goal >= best - 1e-9: + raise ValueError( + f"target {target} cannot be reached: from the current " + f"availability ({expit(now):.6g}) the system can only " + f"approach {expit(best):.6g}, with every MTTF and MTTR that " + "may change at its limit." + ) + + def shortfall(v: np.ndarray) -> float: + return view._log_odds(*probabilities(v))[0] - goal + + def shortfall_gradient(v: np.ndarray) -> np.ndarray: + p, q = probabilities(v) + derivative = view._log_odds(p, q)[1] + values = design(v) + gradient = np.empty(size) + for i, ((node, lever, _), x) in enumerate(zip(variables, v)): + start, floor = span[node][lever] + # The lever lowers the odds of being down, rate * repair, at + # the other quantity times its own fall; p = 1 / (1 + odds). + other = values[node][1 - lever] + fall = other * (start - floor) * math.exp(-x) + gradient[i] = derivative[node] * p[node] ** 2 * fall + return gradient + + def log_total_cost(v: np.ndarray) -> Tuple[float, np.ndarray]: + costs = steepness * np.expm1(v) + return float(logsumexp(costs)), softmax(costs) * steepness * np.exp(v) + + def common_shift(start: np.ndarray, along: np.ndarray) -> np.ndarray: + """The start moved up by the least common v, on the levers + ``along`` marks, meeting the target (it exists, as the target is + below what the limits allow).""" + high = 1.0 + while shortfall(start + high * along) < 0.0 and high < 1e6: + high *= 2.0 + return start + along * brentq( + lambda d: shortfall(start + d * along), 0.0, high, xtol=1e-14 + ) + + every = np.ones(size) + res = minimize( + log_total_cost, + common_shift(np.zeros(size), every), + jac=True, + method="SLSQP", + bounds=[(0.0, None)] * size, + constraints=[ + {"type": "ineq", "fun": shortfall, "jac": shortfall_gradient} + ], + options={"ftol": 1e-12, "maxiter": 1000}, + ) + # A lever the solver left within a hair of its bound is unused. + v = np.where(res.x < 1e-9, 0.0, res.x) + if shortfall(v) < 0.0: + # Close the solver's last sliver of constraint tolerance, with + # the levers in use. + used = (v > 0.0).astype(float) + v = common_shift(v, used if used.any() else every) + res.x = v + res.fun = log_total_cost(v)[0] + rbd.res = res + if not res.success: + warnings.warn( + "the cost minimisation stopped before converging " + f"({res.message}); the design meets the target but may not " + "be the cheapest.", + stacklevel=outside_level(), + ) + return result(v) + + +def _allocatable( + rbd, fixed +) -> Tuple[Dict[Hashable, float], Dict[Hashable, Tuple[float, float]], set]: + """For an availability allocation: every component's long-run + availability; the MTTF and MTTR of those allocated one (with + corrective repair alone: they fail and take time to repair, with no + preventive or inspection schedule, are not in a common-cause group, + whose members' MTTF and MTTR are the group's, and are not in + ``fixed``); and the set of the others, which keep theirs.""" + _crews._require_unlimited_crews(rbd, *_ALLOCATION_CREWS) + _ccf_groups._require_ccf_long_run(rbd) + held = set() + if fixed is not None: + held = set(fixed) + unknown = [node for node in held if node not in rbd.components] + if unknown: + raise ValueError( + f"fixed names {sorted(unknown, key=str)}, which are not " + "components of this RBD." + ) + current = { + node: _long_run._node_availability(rbd, node) + for node in rbd.components + } + free = _allocated_levers(rbd, held) + for node, (mttf, mttr) in free.items(): + current[node] = mttf / (mttf + mttr) + return current, free, held + + +def _allocated_levers(rbd, held: set) -> Dict[Hashable, Tuple[float, float]]: + """The MTTF and MTTR of each component an availability allocation + allocates one: those with corrective repair alone (they fail and + take time to repair, with no preventive or inspection schedule), + not nested, not in a common-cause group (whose members' MTTF and + MTTR are the group's) and not in ``held``, which gains the others. + Raises if there are none: ``analysis_routes`` asks it, as it follows + from the diagram.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + members = {m for group in rbd.ccf_groups for m in group.members} + free: Dict[Hashable, Tuple[float, float]] = {} + for node, component in rbd.components.items(): + if ( + node in held + or isinstance(component, RepairableRBD) + or node in rbd._preventive + or node in rbd._inspection + or node in rbd._standby + or node in members + ): + held.add(node) + continue + mttf = model_mean(component.reliability) + mttr = model_mean(component.time_to_replace) + if not (0.0 < mttf < math.inf and 0.0 < mttr < math.inf): + # Never down, down for good in the end, or up for good. + held.add(node) + continue + free[node] = (mttf, mttr) + if not free: + raise Refused( + "No component can be allocated an availability: only " + "components with corrective repair alone (that fail and take " + "time to repair, with no preventive or inspection schedule), " + "not in fixed, can." + ) + return free + + +def _allocation_option(rbd, name: str, mapping, held: set) -> dict: + """A per-component option of an availability allocation, checked to + name only components allocated an availability.""" + values = rbd._node_overrides(name, mapping) + kept = [node for node in values if node in held] + if kept: + raise ValueError( + f"{name} names {sorted(kept, key=str)}, which keep their " + "availability: only components with corrective repair alone, " + "not in fixed, are allocated one." + ) + return values + + +def _allocation_view(rbd, held: set) -> "RepairableRBD": + """This RBD as an availability allocation scores it. The components + allocated an availability have a constant one, so when every held + component does too the system's availability is the structure + function of theirs, and the RBD itself will do. A held component + that is inspected or block-replaced is up with a probability that + varies over its schedule, together with the others on the calendar: + the system's availability is then averaged over the long-run grid, + as ``mean_availability`` averages it, by a view of the RBD whose + ``_allocation_probability`` and ``_log_odds`` do so. With + common-cause groups, whose members are held, its points are split + by the groups' joint states (see ``_with_ccf_groups``).""" + times, weights = _long_run._long_run_grid(rbd) + if len(times) == 1 and not rbd.ccf_groups: + return rbd + profiles = _long_run._availabilities_at(rbd, times) + if rbd.ccf_groups: + profiles, _, weights, _ = _ccf_groups._with_ccf_groups( + rbd, + times, + profiles, + None, + weights, + what="The availability allocation", + ) + view = rbd._shallow_copy("ccf_tables") + view.__dict__["_allocation_calendar"] = ( + weights, + {node: profiles[node] for node in held}, + [node for node in rbd.nodes if node not in held], + ) + return view + + +def _calendar_arrays(rbd, p: Dict, q: Optional[Dict]) -> Tuple[Dict, Dict]: + """Node probabilities ``p`` and their complements ``q`` (by default + ``1 - p``) over the allocation calendar's grid, with the held + components' own availabilities there.""" + weights, profiles, _ = rbd.__dict__["_allocation_calendar"] + size = len(weights) + works = {node: np.full(size, float(p[node])) for node in p} + fails = { + node: np.full( + size, 1.0 - float(p[node]) if q is None else float(q[node]) + ) + for node in p + } + for node, profile in profiles.items(): + works[node] = profile + fails[node] = 1.0 - profile + for node in rbd.in_or_out: + works[node] = np.ones(size) + fails[node] = np.zeros(size) + return works, fails + + +def _calendar_means(rbd, works: Dict, fails: Dict) -> Tuple[float, float]: + """The system's probabilities of working and of failing over the + allocation calendar's grid, averaged over it.""" + weights = rbd.__dict__["_allocation_calendar"][0] + size = len(weights) + up, down = rbd._decomposition().probabilities(works, fails, shape=size) + return ( + float(weights @ np.broadcast_to(up, (size,))), + float(weights @ np.broadcast_to(down, (size,))), + ) + + +def _minimum_effort_held(rbd, target: float, current: Dict, held: set): + """Albert's minimum-effort allocation to the components not + ``held``: in series, the held components' availabilities (averaged + over their calendars) are a factor of the system's, and the others + make up the rest.""" + rbd._require_series() + limit = rbd._allocation_probability( + {node: current[node] if node in held else 1.0 for node in rbd.nodes} + ) + if target > limit: + raise ValueError( + f"target {target} cannot be reached: the components that " + "keep their availability hold the system to at most " + f"{limit:.6g}." + ) + raised = rbd.minimum_effort_allocation( + target / limit if limit > 0.0 else 0.0, + {node: 1.0 if node in held else current[node] for node in rbd.nodes}, + ) + return { + node: current[node] if node in held else raised[node] + for node in rbd.nodes + } + + +def _allocation_result( + rbd, view: "RepairableRBD", allocated: Dict, mttf: Dict, mttr: Dict +) -> AvailabilityAllocation: + """An availability allocation's result, scored as ``view`` scores + it.""" + return AvailabilityAllocation( + availability={node: float(allocated[node]) for node in rbd.components}, + mttf=mttf, + mttr=mttr, + system_availability=view._allocation_probability( + {node: allocated[node] for node in rbd.nodes} + ), + ) diff --git a/repyability/rbd/_repairable_capacity.py b/repyability/rbd/_repairable_capacity.py new file mode 100644 index 00000000..8eb49cc6 --- /dev/null +++ b/repyability/rbd/_repairable_capacity.py @@ -0,0 +1,646 @@ +"""A ``RepairableRBD``'s capacity: its distribution in the long run +(``capacity_distribution``), at points in time (``point_capacity``) and +over a mission (``mission_capacity``), from each component's chance of +working (``rbd.capacity``'s algorithm on the diagram's structure), with +repair crews' and common-cause groups' chains where they couple the +components. The methods of ``RepairableRBD`` of those names call these. +""" + +import itertools +from functools import partial +from typing import ( + Collection, + Dict, + Hashable, + Optional, + Tuple, +) + +import numpy as np + +from repyability.rbd import ( + _ccf_chain, + _ccf_groups, + _chain_transient, + _crews, + _curves, + _long_run, + _quadrature, + _requirements, + _windows, +) +from repyability.rbd import capacity as _capacity +from repyability.rbd.results import ( + CapacityDistribution, +) +from repyability.utils.checks import ( + nonnegative_times, +) + +#: The most patterns of nested RBDs' capacity levels the capacity over time +#: with limited repair crews follows the crews' chain for (#162). +_MAX_CREW_LEVELS = 4096 + + +class _CrewCapacity: + """The capacity over time of a system whose components wait for repair + crews, around nested RBDs (#162). The nested RBDs have crews of their + own, so they are independent of the chain and of each other: at a time + the system is at a level with probability ``sum_c w_c(t) p(t) v_c``, + over each combination ``c`` of the nested RBDs' levels (up or down, or + their own capacities), ``w_c(t)`` its probability from their own + curves, and ``v_c`` whether the system is at the level in each of the + chain's states with the nested RBDs at ``c``. The chain is followed + over time (see ``_chain_transient.Uniformized``) for each combination + as it first has a probability, and kept.""" + + def __init__(self, rbd, chain, start, nested: dict, working, broken): + self.rbd, self.chain, self.start = rbd, chain, start + self.nested = nested + size = len(chain.probabilities) + own = { + node: chain.up[:, k].astype(float) + for k, node in enumerate(chain.nodes) + } + # The nested RBDs' places, which each combination of their levels + # takes (see ``_chain_for``). + own.update({node: np.ones(size) for node in nested}) + self.arrays, _ = rbd._node_arrays( + _windows._filled(rbd, own, size, working, broken) + ) + self.size = size + self.followed: Dict[tuple, tuple] = {} + timing = _curves._uniformized( + "The repair crews'", + chain.generator, + start, + chain.probabilities, + np.ones((size, 1)), + ) + #: For the pieces of the integrals: the chain's and the nested + #: RBDs' curves. + self.curves = [_chain_transient.ChainCurve(timing), *nested.values()] + + def _chain_for(self, levels: tuple): + """The system's levels with the nested RBDs at ``levels``, and the + chain followed over time with whether it is at each, in each + state.""" + if levels not in self.followed: + own = { + node: (np.array([level]), np.ones((1, self.size))) + for node, level in zip(self.nested, levels) + } + values, rows = self.rbd._capacity_arrays( + self.arrays, self.size, own + ) + self.followed[levels] = ( + values, + _curves._uniformized( + "The repair crews'", + self.chain.generator, + self.start, + self.chain.probabilities, + rows.T, + ), + ) + return self.followed[levels] + + def _distributions(self, x: np.ndarray) -> list: + """Each nested RBD's capacity distribution at the times ``x``: its + own, if it has capacities, else up (at its capacity here) or down, + with its point availability.""" + out = [] + models = self.rbd._capacity_models() + for node, curve in self.nested.items(): + if node in models: + out.append(_capacity_at(self.rbd.components[node], curve, x)) + else: + up = np.clip(np.asarray(curve.at(x), dtype=float), 0.0, 1.0) + out.append( + _capacity.node_distribution( + self.rbd.capacity.get(node, np.inf), up, 1.0 - up + ) + ) + return out + + def rows(self, x) -> Tuple[np.ndarray, np.ndarray]: + """The system's capacity levels and their probabilities at the times + ``x`` (one row per level).""" + x = np.asarray(x, dtype=float).ravel() + distributions = self._distributions(x) + count = int(np.prod([len(levels) for levels, _ in distributions])) + if count > _MAX_CREW_LEVELS: + raise NotImplementedError( + f"With {self.rbd.repair_crews} repair crew(s), the capacity " + "over time is worked out for each combination of the nested " + f"RBDs' levels: {count} are more than the {_MAX_CREW_LEVELS} " + "it takes. Simulate it with availability(demand=...)." + ) + totals: Dict[float, np.ndarray] = {} + for choice in itertools.product( + *[range(len(levels)) for levels, _ in distributions] + ): + weight = np.ones(len(x)) + for (_, chances), i in zip(distributions, choice): + weight = weight * chances[i] + if not weight.any(): + continue + key = tuple( + float(levels[i]) + for (levels, _), i in zip(distributions, choice) + ) + values, chain = self._chain_for(key) + at = np.clip(chain.values(x), 0.0, 1.0) + for level, column in zip(values, at.T): + level = float(level) + totals[level] = totals.get(level, 0.0) + weight * column + levels = np.array(sorted(totals), dtype=float) + rows = np.array([totals[level] for level in levels]).reshape( + len(levels), len(x) + ) + return levels, rows + + +def _capacity_refusal(rbd) -> Optional[Tuple[str, tuple]]: + """What ``capacity_distribution`` refuses beyond the long-run + values, in the order it checks: a degrading component on a + schedule, here or in a nested RBD, then no capacity at all. The + message and the nodes, or None.""" + from repyability.rbd import routes as r + from repyability.rbd.repairable_rbd import RepairableRBD + + for node, model in rbd._capacity_models().items(): + if isinstance(model, RepairableRBD): + inner = _capacity_refusal(model) + if inner: + return inner[0], (node,) + continue + message = r.refusal( + partial(_requirements._require_unscheduled_stages, rbd, node) + ) + if message: + return message, (node,) + message = r.refusal(rbd._require_capacity) + return (message, ()) if message else None + + +def _crew_capacity( + rbd, working_nodes, broken_nodes, states: dict +) -> Tuple[np.ndarray, "_chain_transient.Uniformized"]: + """The capacity over time with limited repair crews and no nested + RBDs: its levels, and the crews' chain followed over time with, as + its vectors, whether the system is at each level in each state (as + ``capacity_distribution`` averages them in the long run).""" + working, broken = set(working_nodes), set(broken_nodes) + forced = working | broken + _curves._require_crew_over_time(rbd, frozenset(forced), "capacity") + chain = _crews._crew_chain(rbd, frozenset(forced)) + size = len(chain.probabilities) + own = { + node: chain.up[:, k].astype(float) + for k, node in enumerate(chain.nodes) + } + arrays, _ = rbd._node_arrays( + _windows._filled(rbd, own, size, working, broken) + ) + levels, rows = rbd._capacity_arrays(arrays, size, {}) + uniformized = _curves._uniformized( + "The repair crews'", + chain.generator, + _curves._crew_start(rbd, chain, states), + chain.probabilities, + rows.T, + ) + return levels, uniformized + + +def _crew_capacity_over( + rbd, horizon: float, working_nodes, broken_nodes, states: dict +) -> "_CrewCapacity": + """The capacity over time with limited repair crews around nested + RBDs (#162): see ``_CrewCapacity``.""" + working, broken = set(working_nodes), set(broken_nodes) + forced = working | broken + nested = _curves._require_crew_over_time(rbd, frozenset(forced)) + chain = _crews._crew_chain(rbd, frozenset(forced)) + curves = { + node: _curves._nested_curve( + rbd.components[node], + horizon, + stages=node in rbd._capacity_models(), + start=states.get(node), + ) + for node in nested + } + return _CrewCapacity( + rbd, + chain, + _curves._crew_start(rbd, chain, states), + curves, + working, + broken, + ) + + +def _capacity_at(rbd, curve, x: np.ndarray) -> Tuple[np.ndarray, np.ndarray]: + """This RBD's capacity distribution at the times ``x``, as a node of + another, from its curve over time (see ``_nested_curve``, with + ``stages``): from its nodes' curves, or with limited repair crews + from their chain (#162).""" + capacity = getattr(curve, "capacity", None) + if capacity is not None: + return capacity.rows(x) + if isinstance(curve, _ccf_chain.GroupsCurve): + return _groups_capacity_rows( + rbd, curve.curves, curve.system, x, set(), set() + ) + return _capacity_rows(rbd, curve.curves, x, set(), set()) + + +def capacity_distribution( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], +) -> CapacityDistribution: + """See ``RepairableRBD.capacity_distribution``.""" + probabilities, weights = _long_run._long_run_probabilities( + rbd, working_nodes, broken_nodes, "The capacity distribution" + ) + working_nodes = set(working_nodes or ()) + broken_nodes = set(broken_nodes or ()) + arrays, size = rbd._node_arrays(probabilities) + own = {} + for node in rbd._capacity_models(): + levels, shares = _long_run_capacity(rbd, node) + own[node] = rbd._forced( + (levels, np.repeat(shares[:, None], size, axis=1)), + node, + working_nodes, + broken_nodes, + ) + levels, rows = rbd._capacity_arrays(arrays, size, own) + return CapacityDistribution(levels, rows @ weights) + + +def _require_capacity_models(rbd) -> None: + """Raise what ``_capacity_refusal`` reports, in the same order: a + degrading component on a schedule, here or in a nested RBD, then no + capacity at all.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + for node, model in rbd._capacity_models().items(): + if isinstance(model, RepairableRBD): + _require_capacity_models(model) + else: + _requirements._require_unscheduled_stages(rbd, node) + rbd._require_capacity() + + +def _capacity_curves( + rbd, horizon: float, working_nodes, broken_nodes, state=None +) -> dict: + """The nodes' curves for the capacity over time (see + ``point_capacity``): a degrading component's following its stages, + and a node that takes its capacity from its model kept even when + held working (its levels are then those it is up at).""" + held = set(working_nodes) - set(rbd._capacity_models()) + return _curves._availability_curves( + rbd, + horizon, + held | set(broken_nodes), + stages=True, + state=_curves._states( + rbd, state, set(working_nodes) | set(broken_nodes) + ), + ) + + +def _capacity_rows( + rbd, curves: dict, x: np.ndarray, working_nodes, broken_nodes +) -> Tuple[np.ndarray, np.ndarray]: + """The distribution of the system's capacity at each of the times + ``x``, from its nodes' curves (see ``_capacity_curves``): its + levels, and one row of probabilities per level, one column per + time. A node with a capacity carries it with its point availability + there; a degrading component is in each stage with the probability + its curve gives, and a nested RBD with capacities brings its own + distribution.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + size = len(x) + values = {node: curve.at(x) for node, curve in curves.items()} + arrays, _ = rbd._node_arrays( + _windows._filled(rbd, values, size, working_nodes, broken_nodes) + ) + own = {} + for node, model in rbd._capacity_models().items(): + if node in broken_nodes: + own[node] = (np.zeros(1), np.ones((1, size))) + continue + curve = curves[node] + if isinstance(model, RepairableRBD): + distribution = _capacity_at(model, curve, x) + else: + distribution = _capacity.merged( + np.array((0.0,) + model.capacities), + np.vstack([1.0 - values[node], curve.stages_at(x)]), + ) + own[node] = rbd._forced( + distribution, node, working_nodes, broken_nodes + ) + return rbd._capacity_arrays(arrays, size, own) + + +def _groups_capacity( + rbd, horizon: float, working_nodes, broken_nodes, state +) -> tuple: + """With common-cause groups (#158): the curves of the nodes outside + them for the capacity over time (see ``_capacity_curves``), and the + groups' system (see ``_ccf_chain.GroupsSystem``).""" + working, broken = set(working_nodes), set(broken_nodes) + states = _curves._states(rbd, state, working | broken) + _ccf_groups._require_free_members(rbd, working, broken) + _ccf_groups._require_groups_over_time(rbd, states) + members = {m for group in rbd.ccf_groups for m in group.members} + held = working - set(rbd._capacity_models()) + curves = _curves._availability_curves( + rbd, + horizon, + held | broken | members, + stages=True, + state=states, + groups=True, + ) + return curves, _ccf_groups._groups_system(rbd, working, broken, "p") + + +def _groups_capacity_rows( + rbd, curves: dict, system, x: np.ndarray, working_nodes, broken_nodes +) -> Tuple[np.ndarray, np.ndarray]: + """``_capacity_rows`` with common-cause groups: worked out at the + points the groups' combinations split each time into (see + ``_ccf_chain.GroupsSystem``), and averaged back onto the times with + the combinations' probabilities.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + size = len(x) + values = {node: curve.at(x) for node, curve in curves.items()} + probabilities, weights, index, _ = system._split(values, x) + points = len(index) + arrays, _ = rbd._node_arrays(probabilities) + own = {} + for node, model in rbd._capacity_models().items(): + if node in broken_nodes: + own[node] = (np.zeros(1), np.ones((1, points))) + continue + curve = curves[node] + if isinstance(model, RepairableRBD): + levels, rows = _capacity_at(model, curve, x) + else: + levels, rows = _capacity.merged( + np.array((0.0,) + model.capacities), + np.vstack([1.0 - values[node], curve.stages_at(x)]), + ) + own[node] = rbd._forced( + (levels, np.asarray(rows)[:, index]), + node, + working_nodes, + broken_nodes, + ) + levels, rows = rbd._capacity_arrays(arrays, points, own) + out = np.array( + [np.bincount(index, weights * row, size) for row in rows] + ).reshape(len(levels), size) + return levels, np.clip(out, 0.0, 1.0) + + +def point_capacity( + rbd, + x, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + state, +) -> CapacityDistribution: + """See ``RepairableRBD.point_capacity``.""" + times = nonnegative_times(x) + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + _require_capacity_models(rbd) + ends = times.ravel() + horizon = float(ends.max()) if ends.size else 0.0 + forced = set(working_nodes) | set(broken_nodes) + if rbd._crews_couple() and _curves._crew_nested(rbd, forced): + capacity = _crew_capacity_over( + rbd, + horizon, + working_nodes, + broken_nodes, + _curves._states(rbd, state, forced), + ) + levels, rows = capacity.rows(ends) + elif rbd._crews_couple(): + levels, chain = _crew_capacity( + rbd, + working_nodes, + broken_nodes, + _curves._states(rbd, state, forced), + ) + rows = np.clip(chain.values(ends).T, 0.0, 1.0) + elif rbd.ccf_groups: + curves, system = _groups_capacity( + rbd, horizon, working_nodes, broken_nodes, state + ) + levels, rows = _groups_capacity_rows( + rbd, curves, system, ends, working_nodes, broken_nodes + ) + else: + curves = _capacity_curves( + rbd, horizon, working_nodes, broken_nodes, state + ) + levels, rows = _capacity_rows( + rbd, curves, ends, working_nodes, broken_nodes + ) + return CapacityDistribution( + levels, rows[:, 0] if np.ndim(x) == 0 else rows + ) + + +def mission_capacity( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + state, +) -> CapacityDistribution: + """See ``RepairableRBD.mission_capacity``.""" + windows = nonnegative_times(t) + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + _require_capacity_models(rbd) + ends = windows.ravel() + horizon = float(ends.max()) if ends.size else 0.0 + forced = set(working_nodes) | set(broken_nodes) + if rbd._crews_couple() and _curves._crew_nested(rbd, forced): + capacity = _crew_capacity_over( + rbd, + horizon, + working_nodes, + broken_nodes, + _curves._states(rbd, state, forced), + ) + levels, rows = _mission_rows( + rbd, capacity.curves, capacity.rows, ends, horizon + ) + return CapacityDistribution( + levels, rows[:, 0] if np.ndim(t) == 0 else rows + ) + if rbd._crews_couple(): + levels, chain = _crew_capacity( + rbd, + working_nodes, + broken_nodes, + _curves._states(rbd, state, forced), + ) + positive = ends > 0.0 + rows = chain.values(ends).T + rows[:, positive] = chain.integrals(ends[positive]).T / ends[positive] + rows = np.clip(rows, 0.0, 1.0) + keep = np.any(rows != 0.0, axis=1) + levels, rows = levels[keep], rows[keep] + return CapacityDistribution( + levels, rows[:, 0] if np.ndim(t) == 0 else rows + ) + if rbd.ccf_groups: + grouped, system = _groups_capacity( + rbd, horizon, working_nodes, broken_nodes, state + ) + levels, rows = _mission_rows( + rbd, + [*grouped.values(), system.curve], + lambda x: _groups_capacity_rows( + rbd, grouped, system, x, working_nodes, broken_nodes + ), + ends, + horizon, + ) + return CapacityDistribution( + levels, rows[:, 0] if np.ndim(t) == 0 else rows + ) + curves = _capacity_curves(rbd, horizon, working_nodes, broken_nodes, state) + levels, rows = _mission_rows( + rbd, + list(curves.values()), + lambda x: _capacity_rows(rbd, curves, x, working_nodes, broken_nodes), + ends, + horizon, + ) + return CapacityDistribution( + levels, rows[:, 0] if np.ndim(t) == 0 else rows + ) + + +def _mission_rows( + rbd, followed: list, rows_at, ends: np.ndarray, horizon: float +) -> Tuple[np.ndarray, np.ndarray]: + """The capacity distribution averaged over each window ``[0, end)`` + (see ``mission_capacity``): ``rows_at(x)`` its levels and their + probabilities at the times ``x``, integrated on pieces cut where + the ``followed`` curves bend, and extended exactly past the time + they have settled. Returns the levels and one row per level.""" + from repyability.rbd._curves import _MISSION_POINTS, _settling + + settle, period = _settling(followed) + reach = min(horizon, settle if period is None else settle + period) + beyond = ends > reach + fixed = [ends[~beyond]] + cycles = rest = np.empty(0) + if period is not None and beyond.any(): + cycles = np.floor((ends[beyond] - settle) / period) + rest = np.clip(ends[beyond] - settle - cycles * period, 0.0, period) + fixed += [np.array([settle]), settle + rest] + + def estimate(a: np.ndarray, b: np.ndarray) -> dict: + # The probability of each capacity level, integrated. + x, half = _quadrature.points(a, b) + levels, rows = rows_at(x) + return { + float(level): _quadrature.summed(row, half) + for level, row in zip(levels, rows) + } + + try: + edges, finest = _quadrature.pieces( + followed, np.concatenate(fixed), reach, _MISSION_POINTS + ) + edges, integrals = _quadrature.refined( + estimate, edges, finest, _MISSION_POINTS + ) + except _quadrature.TooMany as error: + raise NotImplementedError( + f"Integrating the capacity over [0, {reach}] takes " + f"{error.count} pieces (the components' curves bend that " + "often), more than the limit: estimate it by simulation, " + "with availability(demand=...)." + ) from None + # The levels, as ``estimate`` keyed them. + running: Dict[float, np.ndarray] = { + float(level): _quadrature.running(values, len(edges) - 1) + for level, values in integrals.items() + if isinstance(level, float) + } + settled: Dict[float, float] = {} + if period is None and beyond.any(): + levels, rows = rows_at(np.array([horizon])) + settled = {float(v): float(r) for v, r in zip(levels, rows[:, 0])} + inside = np.searchsorted(edges, ends[~beyond]) + positive = ends > 0.0 + at_zero: Dict[float, float] = {} + if not positive.all(): + levels, rows = rows_at(np.zeros(1)) + at_zero = {float(v): float(r) for v, r in zip(levels, rows[:, 0])} + averages = {} + for level in sorted(set(running) | set(settled) | set(at_zero)): + values = running.get(level, np.zeros(len(edges))) + totals = np.empty(len(ends)) + totals[~beyond] = values[inside] + if beyond.any(): + if period is None: + totals[beyond] = values[-1] + ( + ends[beyond] - reach + ) * settled.get(level, 0.0) + else: + base = values[np.searchsorted(edges, settle)] + partial = values[np.searchsorted(edges, settle + rest)] + totals[beyond] = ( + base + cycles * (values[-1] - base) + (partial - base) + ) + average = np.empty(len(ends)) + average[positive] = totals[positive] / ends[positive] + average[~positive] = at_zero.get(level, 0.0) + averages[level] = average + levels = np.array(sorted(averages), dtype=float) + rows = np.array([averages[level] for level in levels]).reshape( + len(levels), len(ends) + ) + keep = np.any(rows != 0.0, axis=1) + return levels[keep], rows[keep] + + +def _long_run_capacity(rbd, node) -> Tuple[np.ndarray, np.ndarray]: + """The long-run distribution of the capacity of a node whose model + has one (see ``_capacity_models``): a nested RBD's own; for a + degrading component, down a fraction ``1 - A`` of the time and in + each stage the rest of it in proportion to the stage's share of its + working time.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + component = rbd.components[node] + if isinstance(component, RepairableRBD): + distribution = component.capacity_distribution() + return distribution.levels, distribution.probabilities + _requirements._require_unscheduled_stages(rbd, node) + stages = component.reliability + up = _long_run._node_availability(rbd, node) + shares = np.concatenate([[1.0 - up], up * stages.stage_fractions()]) + levels, rows = _capacity.merged( + np.array((0.0,) + stages.capacities), shares[:, None] + ) + return levels, rows[:, 0] diff --git a/repyability/rbd/_repairable_importance.py b/repyability/rbd/_repairable_importance.py new file mode 100644 index 00000000..f30e0afd --- /dev/null +++ b/repyability/rbd/_repairable_importance.py @@ -0,0 +1,896 @@ +"""A ``RepairableRBD``'s importance measures: Birnbaum's and the measures +built on it (improvement potential, risk achievement and reduction +worth, criticality, Fussell-Vesely), in the long run or at a time, and +the measures of change (``differential_importance``, ``joint_importance``, +``availability_rate``, ``barlow_proschan_importance``), with repair crews' +and common-cause groups' chains where they couple the components. The +methods of ``RepairableRBD`` of those names call these. +""" + +import math +from collections.abc import Mapping +from typing import ( + Any, + Collection, + Dict, + Hashable, + List, + Optional, + Tuple, +) + +import numpy as np + +from repyability.rbd import ( + _ccf_chain, + _ccf_groups, + _chain_transient, + _crews, + _curves, + _importance_time, + _long_run, + _rates, + _sensitivity, + _windows, +) +from repyability.rbd._common import ( + _squeeze_values, +) +from repyability.rbd.rbd import Pairs +from repyability.rbd.results import ( + RateBreakdown, +) +from repyability.utils.checks import ( + nonnegative_times, + one_of, +) + +#: The most that may jump at one instant with a hidden common-cause +#: group's tests among them (the Shapley value takes every subset). +_MAX_PLAYERS = 12 + + +def _importance_probabilities( + rbd, working_nodes, broken_nodes +) -> Tuple[dict, dict, np.ndarray, Optional[np.ndarray]]: + """``_long_run_points`` for the importance measures: with limited + repair crews the states of the crews' Markov chain, in each of + which every node the crews work on is up or down for certain + (#146). (With common-cause groups, see ``_ccf_long_run_measure``.)""" + return _long_run._long_run_points(rbd, working_nodes, broken_nodes) + + +def _importance_over_time( + rbd, + spec: "_importance_time.Spec", + working_nodes, + broken_nodes, + x, + window, + state, +) -> dict: + """An importance measure at the times ``x`` or over the window + ``[0, window)`` (#191, see ``_importance_time``), from new or from + the components' ``state``.""" + if x is None and window is None: + raise ValueError( + "state is where the components start from, for a measure at " + "times (x) or over a window (window): give one of them. The " + "long-run measure does not depend on it." + ) + if spec.name == _importance_time.FUSSELL_VESELY: + if spec.fv_type not in ("c", "p"): + raise ValueError( + "fv_type must be either 'c' (cut-set) or 'p' (path-set), " + f"fv_type={spec.fv_type!r} was given." + ) + one_of("method", spec.method, ("exact", "rare_event")) + if spec.name == _importance_time.CRITICALITY and spec.kind not in ( + "failure", + "success", + ): + raise ValueError( + f"kind must be 'failure' or 'success', got {spec.kind!r}." + ) + return _importance_time.measure( + rbd, + spec, + working_nodes, + broken_nodes, + x=x, + window=window, + state=state, + ) + + +def _crew_held_importance( + rbd, measure: str, working_nodes, broken_nodes +) -> dict: + """With limited repair crews (#146), an importance measure of each + node from the system's long-run unavailability with the node held + working, ``U(1_i)``, and held failed, ``U(0_i)``: the crews' Markov + chain solved without it, as held it needs no crew. ``"birnbaum"`` + is ``U(0_i) - U(1_i)``, ``"improvement"`` ``U - U(1_i)``, ``"raw"`` + ``U(0_i) / U`` and ``"rrw"`` ``U / U(1_i)``. A node already held + is held the other way for its own measure.""" + working, broken = rbd._held_nodes(working_nodes, broken_nodes) + base = np.float64(rbd.mean_unavailability(working, broken)) + out: dict = {} + for node in rbd.nodes: + up = np.float64( + rbd.mean_unavailability(working | {node}, broken - {node}) + ) + down = np.float64( + rbd.mean_unavailability(working - {node}, broken | {node}) + ) + with np.errstate(divide="ignore", invalid="ignore"): + out[node] = { + "birnbaum": down - up, + "improvement": base - up, + "raw": down / base, + "rrw": base / up, + }[measure] + return _squeeze_values(out) + + +def _failure_importance( + rbd, + measure, + name, + of_probabilities, + working, + broken, + x, + window, + state, +) -> dict[Any, Any]: + """A measure of the nodes' importance to the system's failure + (Birnbaum, improvement potential, RAW, RRW): over time where ``x``, + ``window`` or ``state`` is given, through the common-cause groups' + or the coupled crews' chains where there are any, and otherwise + from the nodes' long-run availabilities (``of_probabilities``, the + structure's measure of them).""" + if x is not None or window is not None or state is not None: + return _importance_over_time( + rbd, + _importance_time.Spec(measure, "failure"), + working, + broken, + x, + window, + state, + ) + if rbd.ccf_groups: + return _ccf_groups._ccf_long_run_measure(rbd, name, working, broken) + if rbd._crews_couple(): + return _crew_held_importance(rbd, name, working, broken) + node_probabilities, node_failures, weights, _ = _importance_probabilities( + rbd, working, broken + ) + return _squeeze_values( + of_probabilities(node_probabilities, weights, node_failures) + ) + + +def birnbaum_importance( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.birnbaum_importance``.""" + return _failure_importance( + rbd, + _importance_time.BIRNBAUM, + "birnbaum", + rbd._birnbaum_importance, + working_nodes, + broken_nodes, + x, + window, + state, + ) + + +def improvement_potential( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.improvement_potential``.""" + return _failure_importance( + rbd, + _importance_time.IMPROVEMENT, + "improvement", + rbd._improvement_potential, + working_nodes, + broken_nodes, + x, + window, + state, + ) + + +def risk_achievement_worth( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.risk_achievement_worth``.""" + return _failure_importance( + rbd, + _importance_time.RAW, + "raw", + rbd._risk_achievement_worth, + working_nodes, + broken_nodes, + x, + window, + state, + ) + + +def risk_reduction_worth( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.risk_reduction_worth``.""" + return _failure_importance( + rbd, + _importance_time.RRW, + "rrw", + rbd._risk_reduction_worth, + working_nodes, + broken_nodes, + x, + window, + state, + ) + + +def criticality_importance( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + kind: str, + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.criticality_importance``.""" + if x is not None or window is not None or state is not None: + return _importance_over_time( + rbd, + _importance_time.Spec(_importance_time.CRITICALITY, kind), + working_nodes, + broken_nodes, + x, + window, + state, + ) + if rbd.ccf_groups: + return _ccf_groups._ccf_long_run_measure( + rbd, "criticality", working_nodes, broken_nodes, kind=kind + ) + node_probabilities, node_failures, weights, _ = _importance_probabilities( + rbd, working_nodes, broken_nodes + ) + return _squeeze_values( + rbd._criticality_importance( + node_probabilities, + kind, + weights=weights, + node_failures=node_failures, + ) + ) + + +def fussell_vesely( + rbd, + fv_type: str, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + *, + x, + window, + state, +) -> dict[Any, Any]: + """See ``RepairableRBD.fussell_vesely``.""" + if x is not None or window is not None or state is not None: + return _importance_over_time( + rbd, + _importance_time.Spec( + _importance_time.FUSSELL_VESELY, + "failure", + fv_type=fv_type, + method=method, + ), + working_nodes, + broken_nodes, + x, + window, + state, + ) + if rbd.ccf_groups: + return _ccf_groups._ccf_long_run_measure( + rbd, + "fussell_vesely", + working_nodes, + broken_nodes, + fv_type=fv_type, + method=method, + ) + node_probabilities, node_failures, weights, _ = _importance_probabilities( + rbd, working_nodes, broken_nodes + ) + return _squeeze_values( + rbd._fussell_vesely( + node_probabilities, + fv_type, + method, + weights=weights, + node_failures=node_failures, + ) + ) + + +def differential_importance( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, + over: str, + change: str, + kind: str, + improving: bool, + groups: Optional[Mapping[Hashable, Collection[Hashable]]], + rel_step: Optional[float], +) -> dict: + """See ``RepairableRBD.differential_importance``.""" + from ._differential import ( + check, + flattened, + proportional_scales, + shares, + ) + + check(over, change, kind) + held = set(working_nodes or ()) | set(broken_nodes or ()) + timed = { + key: value + for key, value in (("x", x), ("window", window), ("state", state)) + if value is not None + } + if over == "components": + if change == "uniform": + values = rbd.birnbaum_importance( + working_nodes, broken_nodes, **timed + ) + else: + values = rbd.criticality_importance( + working_nodes, broken_nodes, kind, **timed + ) + contributions = { + node: (0.0 * np.asarray(v) if node in held else v) + for node, v in values.items() + } + else: + derivatives = flattened( + rbd.parameter_sensitivity( + working_nodes, + broken_nodes, + x=x, + window=window, + state=state, + rel_step=rel_step, + ) + ) + levers = [ + lever for lever in _sensitivity._levers(rbd) if not lever.discrete + ] + continuous = {(lever.key, lever.name) for lever in levers} + kept = [key for key in derivatives if key in continuous] + scale = ( + proportional_scales(levers, kept) + if change == "proportional" + else dict.fromkeys(kept, 1.0) + ) + contributions = { + key: np.asarray(derivatives[key], dtype=float) * scale[key] + for key in kept + } + scalar = np.ndim(x) == 0 and np.ndim(window) == 0 + return shares(contributions, groups, scalar, improving) + + +def _independent_rates( + rbd, flat, horizon: float, scale: float, working, broken, states +) -> Tuple[dict, np.ndarray, dict]: + """``availability_rate``'s parts with independent components: each + component's Birnbaum importance times its own rate, the times the + components jump, and each one's part in each jump (see + ``_rates``).""" + # A little past the last time, for the differences there. + curves = _curves._availability_curves( + rbd, 1.01 * scale, working | broken, state=states + ) + + def importances(values: dict) -> dict: + size = len(next(iter(values.values()))) if values else len(flat) + return rbd._importances( + _windows._filled(rbd, values, size, working, broken) + )[0] + + importance = importances( + {node: curve.at(flat) for node, curve in curves.items()} + ) + parts = { + node: importance[node] * _rates.derivative(curves[node], flat, scale) + for node in curves + } + jump_times, before, after = _rates.jumps(curves, horizon) + return ( + parts, + jump_times, + _rates.split_jumps(importances, before, after), + ) + + +def _crew_rates( + rbd, flat, horizon: float, scale: float, working, broken, states +) -> Tuple[dict, np.ndarray, dict]: + """``availability_rate``'s parts with limited repair crews (#199). + The system is up with probability ``sum_m p(t) u_m w_m(t)`` (see + ``_chain_transient.CrewCurve``), so its rate is ``sum_m p(t) Q u_m + w_m(t)`` plus ``sum_m p(t) u_m w_m'(t)``. Split by the component + whose failures and repairs make each transition, ``Q = sum_i Q_i`` + (``_crew_chain.CrewChain.split``), the first is the chain's + components' parts, each ``p(t) Q_i u_m`` one more vector of the + crews' uniformized chain, exact; the second, the nested RBDs', each + one's Birnbaum importance over the chain times its own rate, as for + independent components, and their jumps are split likewise.""" + nested = _curves._require_crew_over_time(rbd, frozenset(working | broken)) + chain = _crews._crew_chain(rbd, frozenset(working | broken)) + size, count = len(chain.probabilities), len(chain.nodes) + patterns = 2 ** len(nested) + ups = [ + _curves._crew_vectors( + rbd, + chain, + { + node: np.full(size, float((m >> j) & 1)) + for j, node in enumerate(nested) + }, + working, + broken, + "p", + )[0] + for m in range(patterns) + ] + uniformized = _curves._uniformized( + "The repair crews'", + chain.generator, + _curves._crew_start(rbd, chain, states), + chain.probabilities, + np.column_stack(ups + [chain.split(up) for up in ups]), + ) + curves = { + node: _curves._nested_curve( + rbd.components[node], 1.01 * scale, start=states.get(node) + ) + for node in nested + } + + def importances(values: dict, times: np.ndarray) -> dict: + columns = uniformized.values(times)[:, :patterns] + return dict( + zip( + nested, + _chain_transient.pattern_importances( + columns, [values[node] for node in nested] + ), + ) + ) + + columns = uniformized.values(flat) + available = { + node: np.asarray(curve.at(flat), dtype=float) + for node, curve in curves.items() + } + weights = _chain_transient.pattern_weights( + [available[node] for node in nested] + ) + parts: Dict[Hashable, np.ndarray] = {} + for k, node in enumerate(chain.nodes): + own = columns[:, patterns + k + count * np.arange(patterns)] + parts[node] = np.sum(own * weights, axis=1) + importance = importances(available, flat) + for node in nested: + parts[node] = importance[node] * _rates.derivative( + curves[node], flat, scale + ) + jump_times, before, after = _rates.jumps(curves, horizon) + + def along(values: dict) -> dict: + # Each jump's time, for each point of the path between the + # values either side of it. + points = len(next(iter(values.values()))) // len(jump_times) + return importances(values, np.repeat(jump_times, points)) + + return parts, jump_times, _rates.split_jumps(along, before, after) + + +def _groups_rates( + rbd, flat, horizon: float, scale: float, working, broken, states +) -> Tuple[dict, np.ndarray, dict]: + """``availability_rate``'s parts with common-cause groups (#199). + The nodes outside the groups are independent of the groups and of + each other, so each one's part is its Birnbaum importance (over the + groups' joint states, see ``_ccf_chain.GroupsSystem``) times its own + rate. A group's members' states move by its chain, ``p_g'(t) = + p_g(t) Q_g``, so its part is ``p_g(t) Q_g a_g(t)``, ``a_g(t)`` the + system's availability with the group in each of its states then. + Split by whose each move is (``_ccf_chain.Move``), ``Q_g`` gives + each member the part of its own cause and its repairs, and the + group, under the tuple of its members, that of its causes that + strike more than one. The jumps are split by ``_groups_jumps``.""" + _ccf_groups._require_free_members(rbd, working, broken) + _ccf_groups._require_groups_over_time(rbd, states) + curve = _ccf_groups._groups_curve( + rbd, 1.01 * scale, working, broken, "p", states + ) + curves, system = curve.curves, curve.system + values = { + node: np.asarray(c.at(flat), dtype=float) for node, c in curves.items() + } + importance = system.importance(values, flat) + parts: Dict[Any, np.ndarray] = { + node: importance[node] * _rates.derivative(c, flat, scale) + for node, c in curves.items() + } + for number, group in enumerate(system.groups): + # The system's unavailability with the group in each of its + # combinations, and the group's chain's states. + down = system.given(values, flat, number) + chances = group.raw(flat) + for label, begin, end, rate in group.moves: + key = ( + group.members[label] + if label != _ccf_chain.SHARED + else tuple(group.members) + ) + fall = ( + down[:, group.combination[end]] + - down[:, group.combination[begin]] + ) + part = -np.sum(chances[:, begin] * rate * fall, axis=1) + parts[key] = parts.get(key, 0.0) + part + jump_times, split = _groups_jumps(rbd, curves, system, horizon) + return parts, jump_times, split + + +def _groups_jumps( + rbd, curves: dict, system, horizon: float +) -> Tuple[np.ndarray, dict]: + """The times in ``(0, horizon]`` at which the system with + common-cause groups jumps (a scheduled event of a node outside the + groups, a hidden group's tests), and each one's part in each jump. + Where only nodes outside the groups jump, independent and the + system multilinear in them, ``_rates.split_jumps`` splits it. Where + a test is among what jumps, the members' states are no longer + independent, and the jump is split by the Shapley value of what + jumps then (the nodes, and each test, under its member's name), + the system worked out exactly with each subset of them done: for + independent nodes alone, the same split.""" + out_times, before, after = _rates.jumps(curves, horizon) + tests = [group.tests(horizon) for group in system.groups] + tested = sorted({t for by in tests for t in by}) + times = np.unique(np.concatenate([out_times, np.array(tested)])) + split: Dict[Any, np.ndarray] = {} + + def part(key) -> np.ndarray: + if key not in split: + split[key] = np.zeros(len(times)) + return split[key] + + alone = np.array([t not in set(tested) for t in out_times], bool) + if alone.any(): + at = out_times[alone] + + def importances(values: dict) -> dict: + points = len(next(iter(values.values()))) // len(at) + return system.importance(values, np.repeat(at, points)) + + shares = _rates.split_jumps( + importances, + {node: v[alone] for node, v in before.items()}, + {node: v[alone] for node, v in after.items()}, + ) + rows = np.searchsorted(times, at) + for node, values in shares.items(): + part(node)[rows] += values + for time in tested: + row = int(np.searchsorted(times, time)) + just = np.nextafter(time, -np.inf) + low = { + node: float(np.ravel(c.at(np.array([just])))[0]) + for node, c in curves.items() + } + high = { + node: float(np.ravel(c.at(np.array([time])))[0]) + for node, c in curves.items() + } + players: List[Tuple[Any, Any]] = [ + (node, None) + for node in curves + if abs(high[node] - low[node]) > _rates.JUMP + ] + for number, by in enumerate(tests): + for test in by.get(time, []): + players.append((number, test)) + for (who, test), value in zip( + players, _shapley(rbd, system, players, low, high, time) + ): + key = ( + who + if test is None + else system.groups[who].members[test.member] + ) + part(key)[row] += value + return times, split + + +def _shapley( + rbd, system, players: list, low: dict, high: dict, time: float +) -> np.ndarray: + """Each player's Shapley value in the system's jump at ``time`` + (see ``_groups_jumps``): the system's availability with each + subset of the players done, a node at its value after the jump + (``high``) rather than before (``low``), and a test applied to its + group's states just before the time.""" + count = len(players) + if count > _MAX_PLAYERS: + raise NotImplementedError( + f"{count} nodes and tests change at time {time:g}, with a " + "common-cause group's tests among them: their jump is split " + f"for at most {_MAX_PLAYERS}, as yet. Simulate it with " + "availability()." + ) + subsets = np.arange(2**count) + values = {} + for node in low: + if (node, None) in players: + j = players.index((node, None)) + values[node] = np.where( + (subsets >> j) & 1, high[node], low[node] + ).astype(float) + else: + values[node] = np.full(len(subsets), high[node]) + chances = [] + just = np.array([np.nextafter(time, -np.inf)]) + for number, group in enumerate(system.groups): + mine = [ + (j, test) + for j, (who, test) in enumerate(players) + if test is not None and who == number + ] + if not mine: + chances.append( + np.repeat( + group.probabilities(np.array([time])), len(subsets), 0 + ) + ) + continue + start = group.raw(just) + table = np.empty((len(subsets), len(group.down))) + for subset in subsets: + done = [test for j, test in mine if (subset >> j) & 1] + table[subset] = group.grouped(group.tested(start, done))[0] + chances.append(table) + worth = system.availability(values, chances) + out = np.zeros(count) + weight = [ + math.factorial(k) + * math.factorial(count - k - 1) + / math.factorial(count) + for k in range(count) + ] + sizes = np.array([bin(int(s)).count("1") for s in subsets]) + for j in range(count): + without = subsets[(subsets >> j) & 1 == 0] + out[j] = sum( + weight[sizes[s]] * (worth[s | (1 << j)] - worth[s]) + for s in without + ) + return out + + +def availability_rate( + rbd, + x, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + state, +) -> RateBreakdown: + """See ``RepairableRBD.availability_rate``.""" + times = nonnegative_times(x) + working, broken = rbd._held_nodes(working_nodes, broken_nodes) + forced = working | broken + states = _curves._states(rbd, state, forced) + flat = times.ravel() + horizon = float(flat.max()) if flat.size else 0.0 + scale = horizon if horizon > 0.0 else 1.0 + if rbd._crews_couple(): + parts, jump_times, split = _crew_rates( + rbd, flat, horizon, scale, working, broken, states + ) + elif rbd.ccf_groups: + parts, jump_times, split = _groups_rates( + rbd, flat, horizon, scale, working, broken, states + ) + else: + parts, jump_times, split = _independent_rates( + rbd, flat, horizon, scale, working, broken, states + ) + keys = list(rbd.components) + [ + key for key in {**parts, **split} if key not in rbd.components + ] + node_rate: Dict[Hashable, np.ndarray] = {} + rate = np.zeros(len(flat)) + node_jumps: Dict[Hashable, np.ndarray] = {} + jumps = np.zeros(len(jump_times)) + for key in keys: + part = np.asarray(parts.get(key, np.zeros(len(flat))), dtype=float) + node_rate[key] = part + rate = rate + part + part = np.asarray( + split.get(key, np.zeros(len(jump_times))), dtype=float + ) + node_jumps[key] = part + jumps = jumps + part + + def shaped(values: np.ndarray): + if np.ndim(x) == 0: + return float(values[0]) + return values.reshape(times.shape) + + return RateBreakdown( + x=shaped(flat), + rate=shaped(rate), + node_rate={node: shaped(v) for node, v in node_rate.items()}, + jump_times=jump_times, + jumps=jumps, + node_jumps=node_jumps, + ) + + +def barlow_proschan_importance( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + window, + state, +) -> dict: + """See ``RepairableRBD.barlow_proschan_importance``.""" + if window is None: + if state is not None: + raise ValueError( + "state is where the components start from, for the " + "shares over a window: give its length (window). The " + "long-run shares do not depend on it." + ) + terms, _ = _long_run._outage_terms(rbd, working_nodes, broken_nodes) + parts: Dict[Any, Any] = {node: 0.0 for node in rbd.components} + for cause, term in terms: + parts[cause] = parts.get(cause, 0.0) + term + scalar = True + else: + ends = nonnegative_times(window) + if np.any(ends <= 0.0): + raise ValueError( + "A window must be a positive length of time, got " + f"{window!r}." + ) + windows, flat, _, counts, _, _ = _windows._window( + rbd, + window, + working_nodes, + broken_nodes, + "p", + state=state, + causes=True, + ) + caused = counts["caused"] + parts = { + key: np.asarray(caused.get(key, np.zeros(len(flat))), dtype=float) + for key in [*rbd.components, *caused] + } + scalar = np.ndim(window) == 0 + total: Any = 0.0 + for value in parts.values(): + total = total + np.asarray(value, dtype=float) + with np.errstate(divide="ignore", invalid="ignore"): + shares = { + key: np.where( + total > 0.0, np.asarray(value, dtype=float) / total, np.nan + ) + for key, value in parts.items() + } + if scalar: + return { + key: float(np.ravel(value)[0]) for key, value in shares.items() + } + return { + key: np.asarray(value).reshape(windows.shape) + for key, value in shares.items() + } + + +def joint_importance( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, +) -> dict: + """See ``RepairableRBD.joint_importance``.""" + timed = { + key: value + for key, value in (("x", x), ("window", window), ("state", state)) + if value is not None + } + if rbd.ccf_groups: + # Each pair conditioned on its states through the groups' chains + # (#317, see ``RepairableRBD._ccf_measure``), as the other measures + # are: refused where their chains do not cover the groups. + _ccf_groups._require_ccf_long_run(rbd) + if timed: + return Pairs( + _importance_over_time( + rbd, + _importance_time.Spec(_importance_time.JOINT), + working_nodes, + broken_nodes, + x, + window, + state, + ) + ) + return Pairs( + _ccf_groups._ccf_long_run_measure( + rbd, _importance_time.JOINT, working_nodes, broken_nodes + ) + ) + with rbd._sharing_curves(): + return rbd._joint_pairs( + lambda working, broken: rbd.birnbaum_importance( + sorted(working, key=str), sorted(broken, key=str), **timed + ), + working_nodes, + broken_nodes, + ) diff --git a/repyability/rbd/_repairable_routes.py b/repyability/rbd/_repairable_routes.py new file mode 100644 index 00000000..ce7db961 --- /dev/null +++ b/repyability/rbd/_repairable_routes.py @@ -0,0 +1,725 @@ +"""A ``RepairableRBD``'s analysis routes: for each public analysis, +without running anything, whether it is exact, numerical, simulated or +refused, with the methods' own reasons (``routes``). The method +``RepairableRBD.analysis_routes`` calls this; ``test_analysis_routes`` +checks it against the methods on every catalogue diagram. +""" + +import dataclasses +from functools import partial +from typing import ( + Dict, + Hashable, + Tuple, +) + +from repyability.rbd import ( + _ccf_groups, + _crews, + _curves, + _event_loop, + _intervals, + _long_run, + _repairable_allocation, + _repairable_capacity, + _repairable_uncertainty, + _requirements, + _runs, + _spares, + _timeline_runs, +) +from repyability.rbd._runs import ( # noqa: E402,F401 + _UNSTREAMED, + failure_criticality_index_per_component_failures, + failure_criticality_index_per_system_failures, + restoration_criticality_index_by_component, + restoration_criticality_index_by_system, +) +from repyability.rbd.routes import AnalysisRoute + + +def analysis_routes(rbd) -> Dict[str, "AnalysisRoute"]: + """See ``RepairableRBD.analysis_routes``.""" + from repyability.rbd import routes as r + + out: Dict[str, r.AnalysisRoute] = {} + + def give(names, route) -> None: + for name in names: + out[name] = route + + long_run = _long_run._long_run_route(rbd) + + long_run_nodes = _long_run._long_run_nodes(rbd) + + def from_long_run(route, reason): + """A method of ``route``, explained by ``reason``, built on the + long-run values: refused as they are, and numerical or simulated + as the components' are.""" + if long_run.route == r.REFUSED: + return long_run + return r.with_nodes(route, reason, long_run_nodes, "long-run values") + + give(("mean_availability", "mean_unavailability"), long_run) + out["node_availability"] = ( + _long_run._long_run_route(rbd, groups=False) + if rbd.ccf_groups + else long_run + ) + frequencies = ( + None + if long_run.route == r.REFUSED + else r.refusal(partial(_ccf_groups._require_ccf_frequencies, rbd)) + ) + give( + ( + "system_failure_frequency", + "mean_time_between_failures", + "mean_up_time", + "mean_down_time", + ), + r.refused(frequencies) if frequencies else long_run, + ) + give( + ("barlow_proschan_importance",), + ( + r.refused(frequencies) + if frequencies + else ( + long_run + if long_run.route == r.REFUSED + else dataclasses.replace( + long_run, + reason="Each component's share of the system's " + "long-run failure frequency (the Birnbaum/Vesely " + "formula's terms). " + long_run.reason, + ) + ) + ), + ) + # The allocations assume independent components, which limited + # repair crews make them not; the common-cause groups' chains take + # theirs in. + allocation = r.refusal( + partial( + _crews._require_unlimited_crews, + rbd, + *_repairable_allocation._ALLOCATION_CREWS, + ) + ) or r.refusal(partial(_ccf_groups._require_ccf_long_run, rbd)) + conditioned = long_run + if rbd.ccf_groups and long_run.route != r.REFUSED: + conditioned = dataclasses.replace( + long_run, + reason=long_run.reason + + " A common-cause group's member is conditioned on its " + "state at each time, as the shared causes tie the other " + "members' to it.", + ) + if rbd._crews_couple() and long_run.route != r.REFUSED: + conditioned = dataclasses.replace( + long_run, + reason=long_run.reason + + " With the crews, Birnbaum's measure, the improvement " + "potential, RAW and RRW hold each node working and failed " + "in the chain, solved without it; the criticality and " + "Fussell-Vesely measures are probabilities over its " + "states.", + ) + give( + ( + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + ), + conditioned, + ) + give( + ("joint_importance",), + ( + conditioned + if conditioned.route == r.REFUSED + else dataclasses.replace( + conditioned, + reason=( + "The system's availability given each of the pair up " + "and down, a common-cause group's members conditioned " + "on their states through the groups' chains. " + if rbd.ccf_groups + else "Each component's Birnbaum importance with the " + "other held up, less with it held down. " + ) + + conditioned.reason, + ) + ), + ) + give( + ("differential_importance",), + ( + conditioned + if conditioned.route == r.REFUSED + else dataclasses.replace( + conditioned, + reason="Birnbaum's measure (or, for a proportional " + "change, the criticality importance), shared out; " + "over='parameters', parameter_sensitivity's " + "derivatives. " + conditioned.reason, + ) + ), + ) + give( + ("fussell_vesely",), + from_long_run( + r.EXACT, + "The exact probability that a minimal cut set containing " + "each node is down, over the system's unavailability, from " + "the exact long-run values (method='rare_event' sums the cut " + "sets' probabilities instead).", + ), + ) + give( + ("parameter_sensitivity",), + from_long_run( + r.NUMERICAL, + "Central differences of the long-run availability (or cost " + "rate) in each lever, the diagram rebuilt with the lever " + "moved; at times or over a window, of the values over time.", + ), + ) + # The cost analyses refuse a diagram with nothing priced (not even + # an acquisition cost) first; with an acquisition cost alone there + # is no running cost, exactly. + unpriced = r.refusal(partial(_requirements._require_costs, rbd)) + acquisition_only = r.AnalysisRoute( + r.EXACT, + "No running cost is priced: the cost is the acquisition cost " + "alone, exactly.", + ) + if unpriced: + give(("expected_cost_rate", "total_cost"), r.refused(unpriced)) + elif not rbd.has_costs: + give(("expected_cost_rate", "total_cost"), acquisition_only) + else: + setups = r.refusal( + partial(_requirements._require_separate_setups, rbd) + ) + give( + ("expected_cost_rate", "total_cost"), + ( + r.refused(setups) + if setups + else from_long_run( + r.EXACT, + "The long-run cost rate, from the exact long-run " + "values (total_cost multiplies it by the time).", + ) + ), + ) + capacity = ( + None + if long_run.route == r.REFUSED + else _repairable_capacity._capacity_refusal(rbd) + ) + out["capacity_distribution"] = ( + r.refused(*capacity) + if capacity + else from_long_run( + r.EXACT, + "The exact long-run distribution of the system's capacity, " + + ( + "averaged over the states of the repair crews' Markov " + "chain." + if rbd._crews_couple() + else "from the components' long-run availabilities." + ), + ) + ) + no_capacity = r.refusal(rbd._require_numbered_capacities) or ( + r.refusal(rbd._require_capacity) + ) + out["system_capacity"] = ( + r.refused(no_capacity) + if no_capacity + else r.AnalysisRoute( + r.EXACT, + "The exact distribution of the system's capacity from the " + "node probabilities given.", + ) + ) + give( + ( + "point_availability", + "mission_availability", + "point_unavailability", + "mission_unavailability", + ), + _curves._over_time(rbd), + ) + give( + ("availability_rate",), + ( + _curves._crew_over_time(rbd, "rate") + if rbd._crews_couple() + else _curves._over_time( + rbd, + "Each component's renewal equation solved on a grid (to " + "about 1e-7) and differentiated on it, times its Birnbaum " + "importance at the components' availabilities; the jumps " + "at scheduled events split along the path between the " + "values either side." + + ( + " A common-cause group's part is its chain's moves, " + "split by cause and by member repair, at the " + "system's availability with the group in each of " + "its states (exact); the jumps at a hidden group's " + "tests are split by the Shapley value of what jumps." + if rbd.ccf_groups + else "" + ), + ) + ), + ) + over_time_capacity = _repairable_capacity._capacity_refusal(rbd) + give( + ("point_capacity", "mission_capacity"), + ( + r.refused(*over_time_capacity) + if over_time_capacity + else _curves._over_time( + rbd, + "Each component's renewal equation solved on a grid (to " + "about 1e-7), a degrading component's stages through its " + "renewals too, and the system's capacity distribution " + "exactly at its components' at each time.", + kind="capacity", + ) + ), + ) + window = _curves._over_time( + rbd, + "Each component's expected events from its renewal equation, " + "solved on a grid (to about 1e-7), and the system's failures by " + "the time-dependent Birnbaum/Vesely formula.", + kind="window", + ) + give(("expected_failures", "expected_events"), window) + out["expected_cost"] = ( + r.refused(unpriced) + if unpriced + else (window if rbd.has_costs else acquisition_only) + ) + # A group's members' MTTF and MTTR are their group's: the + # availability allocations hold them, and a member's copies join its + # group. + held_members = ( + " A common-cause group's members keep their availability: " + "their MTTF and MTTR are their group's." + if rbd.ccf_groups + else "" + ) + out["allocate_redundancy"] = ( + r.refused(allocation) + if allocation + else from_long_run( + r.EXACT, + "A search over the exact long-run values of the candidates." + + ( + " A common-cause group member's copies join its group, " + "whose chain takes them in: a BetaFactor group's (an MGL " + "or AlphaFactor group's member refuses)." + if rbd.ccf_groups + else "" + ), + ) + ) + out["availability_allocation"] = ( + r.refused(allocation) + if allocation + else from_long_run( + r.EXACT, + "A search over the exact long-run values of the candidates." + + held_members, + ) + ) + out["mttf_mttr_allocation"] = ( + r.refused(allocation) + if allocation + else from_long_run( + r.NUMERICAL, + "A solver for the components' MTTF and MTTR targets, over the " + "exact long-run values." + held_members, + ) + ) + if out["availability_allocation"].route != r.REFUSED: + # Which components are allocated one follows from the diagram + # (and fixed, which only holds more). + none = r.refusal( + partial(_repairable_allocation._allocated_levers, rbd, set()) + ) + if none: + give( + ("availability_allocation", "mttf_mttr_allocation"), + r.refused(none), + ) + # The optimisers, called for every node they can choose for. + targets = r.refusal( + partial( + _intervals._interval_targets, + rbd, + None, + None, + ) + ) + interval_crews = r.refusal( + partial(_intervals._require_interval_crews, rbd) + ) + refusal = ( + r.refusal(partial(_intervals._maintained, rbd, None)) + or targets + or interval_crews + ) + out["optimal_replacement_intervals"] = ( + r.refused(refusal) + if refusal + else from_long_run( + r.NUMERICAL, + "An optimiser over the age-replacement intervals, with the " + "exact long-run values at each.", + ) + ) + refusal = ( + r.refusal(partial(_intervals._inspected, rbd, None)) + or targets + or interval_crews + or next( + ( + message + for message in ( + r.refusal( + partial( + _requirements._require_tested_exact, + rbd, + node, + ) + ) + for node in rbd._inspection + ) + if message + ), + None, + ) + or r.refusal(partial(_requirements._require_one_inspected, rbd)) + ) + out["optimal_inspection_intervals"] = ( + r.refused(refusal) + if refusal + else from_long_run( + r.NUMERICAL, + "An optimiser over the inspection interval, with the exact " + "long-run values at each.", + ) + ) + # Spares: each component's replacements, counted as a renewal + # process, in the order the methods check them. + for name, long_run_count in ( + ("spares_demand", False), + ("spares_stock", True), + ): + refusal = r.refusal( + partial(_spares._spares_nodes, rbd, None) + ) or r.refusal( + partial( + _crews._require_unlimited_crews, + rbd, + *( + _spares._STOCK_CREWS + if long_run_count + else _spares._SPARES_CREWS + ), + ) + ) + which: Tuple[Hashable, ...] = () + for node in [] if refusal else _spares._spares_nodes(rbd, None): + if refusal: + break + refusal = r.refusal( + partial(_spares._replacements, rbd, node, long_run_count) + ) + which = (node,) + out[name] = ( + r.refused(refusal, which) + if refusal + else r.AnalysisRoute( + r.NUMERICAL, + "Each component's replacements, a renewal process, " + "counted on a grid (to about 1e-6): under block " + + ( + "replacement, averaged over where the lead time " + "falls in the block interval (with repairs or " + "block replacements that take time, from a typical " + "replacement, a failure at each phase of the " + "interval or a block replacement, in the long run)" + if long_run_count + else "replacement, from one block interval to the " "next" + ) + + "; with hidden failures, on its tests: exactly, tested " + "and repaired in no time by tests that find every " + "failure, and otherwise from the cycle between the tests " + "that find failures, followed test by test (#159).", + ) + ) + plan, streamed = _event_loop._stream_plan(rbd, 1.0, 0, False) + paired = ( + "" + if streamed + else " Antithetic pairs are refused: some components' draws " + "do not come from a stream." + ) + groups = r.refusal(partial(_ccf_groups._require_groups_simulated, rbd)) + given = r.refusal(partial(_requirements._require_capacities_given, rbd)) + engine, why = _runs._engine_choice(rbd, capacity=rbd._has_capacity()) + causes = ( + " Each common-cause group's shared causes strike as Poisson " + "processes, each failing the members it names that are up." + if _ccf_groups._has_ccf(rbd) + else "" + ) + out["availability"] = ( + r.refused(groups) + if groups + else ( + r.refused(given, tuple(rbd._capacity_models())) + if given + else r.AnalysisRoute( + r.SIMULATED, + "A discrete-event simulation of the components' failures, " + "repairs and maintenance." + causes + paired, + engine=engine, + engine_reason=why, + ) + ) + ) + out["simulate_chunk"] = out["availability"] + unsaved = r.refusal(partial(_runs._shard_system, rbd)) + out["shards"] = ( + out["availability"] + if out["availability"].route == r.REFUSED + else ( + r.refused(unsaved) + if unsaved + else r.AnalysisRoute( + r.SIMULATED, + "The run's simulations as shards, to simulate anywhere " + "with run_shard (see shards); availability_from_chunks " + "merges their partials into its result.", + ) + ) + ) + out["availability_from_chunks"] = ( + out["availability"] + if given or groups + else r.AnalysisRoute( + r.SIMULATED, + "Simulated chunks of a run (see simulate_chunk), merged into " + "its result.", + ) + ) + engine, why = _timeline_runs.engine_choice(rbd, plan) + out["simulate_timelines"] = ( + r.refused(groups) + if groups + else r.AnalysisRoute( + r.SIMULATED, + "The simulations availability runs, their histories kept " + "as timelines: recorded by the event loop as it runs." + paired, + engine=engine, + engine_reason=why, + ) + ) + engine, why = _runs._engine_choice(rbd, capacity=False) + if unpriced: + out["cost"] = r.refused(unpriced) + elif not rbd.has_costs: + out["cost"] = acquisition_only + elif groups: + out["cost"] = r.refused(groups) + else: + out["cost"] = r.AnalysisRoute( + r.SIMULATED, + "A discrete-event simulation of the components' failures, " + "repairs and maintenance, and what they cost." + paired, + engine=engine, + engine_reason=why, + ) + twin = _runs._twin_report(rbd) + # The expected values over the window, where the methods over time + # work them out, need no simulation, and a run takes them (#187). + exact_means = "" + if all( + out[name].route in (r.EXACT, r.NUMERICAL) + for name in ("mission_availability", "expected_events") + + (("expected_cost",) if rbd.has_costs else ()) + ): + exact_means = ( + " Its expected values over the window need none: " + "mission_availability, expected_events and expected_cost " + "work them out, and by default a run's means are theirs, " + "with no error, so a run to a tolerance stops at " + "once. Simulate for their spread: each simulation's values, " + "percentiles, the chance of no failure." + ) + # Or a run's means are taken given its modules' histories (#189). + if not exact_means: + modules = _runs._conditional_applies(rbd) + if modules: + names = ", ".join(map(repr, modules)) + exact_means = ( + " By default a run's means take each " + "simulation's expected values given the histories of " + f"{names}, the rest exact given their states, which " + "vary less than its own; with conditional=True only " + f"{names} are simulated, for those means at a fraction " + "of the work, and no spread." + ) + for name in ("availability", "cost"): + if out[name].route == r.SIMULATED: + out[name] = dataclasses.replace( + out[name], reason=out[name].reason + exact_means, twin=twin + ) + # The difference of exact expected values needs no simulation + # (#236): where _exact_means works them out. + needed = [out["mission_availability"]] + ( + [out["expected_cost"]] if rbd.has_costs else [] + ) + # A diagram too meshed to work out has no exact expected values, + # so compare simulates it. + if rbd._too_meshed() is None and all( + route.route in (r.EXACT, r.NUMERICAL) for route in needed + ): + out["compare"] = r.AnalysisRoute( + ( + r.NUMERICAL + if any(route.route == r.NUMERICAL for route in needed) + else r.EXACT + ), + "The difference of the two systems' expected values over " + "the window (mission_availability, and expected_cost their " + "costs), where the other's are worked out too (#236); " + "otherwise, or with control_variate=False, the two systems " + "simulated with common random numbers.", + ) + elif groups: + out["compare"] = r.refused(groups) + elif streamed: + out["compare"] = r.AnalysisRoute( + r.SIMULATED, + "The two systems simulated with common random numbers.", + engine=engine, + engine_reason=why, + ) + else: + out["compare"] = r.refused(_UNSTREAMED) + give( + ("initialize_event_queue", "next_event"), + ( + r.refused(groups) + if groups + else r.AnalysisRoute( + r.SIMULATED, + "One simulation, stepped through event by event, drawing " + "from numpy's global RNG.", + ) + ), + ) + out["structural_importance"] = r.AnalysisRoute( + r.EXACT, "From the structure alone." + ) + out["system_probability"] = r.AnalysisRoute( + r.EXACT, + "The structure function over the node probabilities given.", + ) + out["path_set_probabilities"] = r.AnalysisRoute( + r.EXACT, "From the node probabilities given." + ) + allocation_by_probability = r.AnalysisRoute( + r.NUMERICAL, + "A solver over the exact system probability, from the node " + "probabilities given (not the component models).", + ) + give( + ( + "improvement_allocation", + "equal_allocation", + "simple_allocation", + "cost_based_allocation", + ), + allocation_by_probability, + ) + series = r.refusal(rbd._require_series) + out["minimum_effort_allocation"] = ( + r.refused(series) if series else allocation_by_probability + ) + # Each component's own values, and the long-run costs but the + # system's downtime, need no structure; nor the expected cost when + # no running cost is priced. + free = {"node_availability", "spares_demand", "spares_stock"} + if not rbd.downtime_cost_rate: + free |= {"expected_cost_rate", "total_cost"} + if not rbd.has_costs: + free.add("expected_cost") + meshed = rbd._too_meshed() + if meshed is not None and rbd._has_capacity(): + # The simulations that follow the system's capacity do so + # through its reduced diagram, which a structure too meshed has + # not. + for name in ( + "availability", + "simulate_chunk", + "availability_from_chunks", + "shards", + ): + out[name] = r.refused(meshed) + out = rbd._meshed_routes(out, free) + # Parameter uncertainty (#200): each draw's diagram rebuilt and its + # value worked out as the diagram's own, so refused where that is + # (a structure too meshed included). + # By default every fitted model is drawn: without one, the uncertain + # nodes must be given (a nested RBD's fits are not drawn here). + unfitted = r.refusal(partial(_repairable_uncertainty.require_fitted, rbd)) + for name, method in ( + ("mean_availability_uncertainty", "mean_availability"), + ("point_availability_uncertainty", "point_availability"), + ("mission_availability_uncertainty", "mission_availability"), + ("expected_cost_rate_uncertainty", "expected_cost_rate"), + ): + own = out[method] + out[name] = ( + own + if own.route == r.REFUSED + else r.AnalysisRoute( + r.SIMULATED, + "The components' models drawn from their uncertainty, " + f"and each draw's diagram's {method} worked out as its " + f"own ({own.route})." + + ( + f" Given no uncertainty, it refuses: {unfitted}" + if unfitted + else "" + ), + ) + ) + sensitivity = out["parameter_sensitivity"] + out["uncertainty_importance"] = ( + sensitivity + if sensitivity.route == r.REFUSED + else r.AnalysisRoute( + r.NUMERICAL, + "The delta method: parameter_sensitivity's derivatives in " + "each uncertain parameter, with their covariance " + "(method='sobol' estimates Sobol indices from draws " + "instead).", + ) + ) + return r.AnalysisRoutes(sorted(out.items())) diff --git a/repyability/rbd/_repairable_uncertainty.py b/repyability/rbd/_repairable_uncertainty.py index 1d0ad8bb..39ac4f16 100644 --- a/repyability/rbd/_repairable_uncertainty.py +++ b/repyability/rbd/_repairable_uncertainty.py @@ -30,22 +30,36 @@ import difflib import warnings from collections.abc import Mapping -from typing import Any, Dict, Hashable, List, NamedTuple, Optional, Tuple +from typing import ( + Any, + Dict, + Hashable, + List, + NamedTuple, + Optional, + Tuple, +) import numpy as np -from repyability.utils.checks import seed as check_seed +from repyability.rbd.results import ( + UncertaintyImportance, + UncertaintyResult, +) +from repyability.rbd.routes import Refused +from repyability.utils.checks import ( + one_of, +) from repyability.utils.wrappers import outside_level from .uncertainty import ( FIT, - Counter, - SobolPoints, - check_sampling, + check_draws, draw_ccf_models, draw_models, + draw_with, is_fit, - sobol_table, + split_group_levers, varied_ccf_parameters, varied_parameters, ) @@ -190,7 +204,7 @@ def fitted(rbd) -> Dict[Hashable, Dict[str, str]]: """The uncertainty drawn when none is given: ``"fit"`` for every one of the components' models that is a surpyval fit with a parameter covariance, drawn once for all the nodes holding that fitted object in - that role (#214), and for one common-cause group's members together: + that role, and for one common-cause group's members together: ``{node or tuple of nodes: {role: "fit"}}``, by first node and role, a node under as many keys as it has populations.""" order = {node: i for i, node in enumerate(rbd.components)} @@ -232,9 +246,48 @@ def root(node): } +def nested_fits(rbd) -> list: + """The nested RBDs (by node) holding a fitted model the default + uncertainty would draw if they were this diagram's own components: + they are not drawn here (a nested diagram's own uncertainty methods + take them).""" + from .repairable_rbd import RepairableRBD + + return [ + node + for node, component in rbd.components.items() + if isinstance(component, RepairableRBD) + and (fitted(component) or nested_fits(component)) + ] + + +def require_fitted(rbd) -> None: + """Raise (``Refused``) unless some component's model is a surpyval + fit with a parameter covariance, which the uncertainty is drawn from + by default (``fitted``): the message names the nested RBDs whose + fits are not drawn here, if any.""" + if fitted(rbd): + return + inside = nested_fits(rbd) + nested = ( + "" + if not inside + else ( + f" (the fits inside nested RBD {inside[0]!r} are not drawn " + "here: its own uncertainty methods take them)" + ) + ) + raise Refused( + "No component's model is a surpyval fit with a parameter " + f"covariance, which the uncertainty is drawn from by default{nested}: " + "give the uncertain nodes, uncertainty={node: 'fit'} or {node: " + "{'repairability': {parameter: distribution}}}, for example." + ) + + def half_named(label: str, others: list, what: str, together) -> None: """Warn that an input is drawn without ``others``, which hold the same - fitted ``what`` and keep it as it is in every draw (#214).""" + fitted ``what`` and keep it as it is in every draw.""" names = ", ".join(repr(n) for n in others) warnings.warn( f"{label} is drawn without {names}, which hold the same {what} " @@ -259,15 +312,8 @@ def sources( given = uncertainty is not None if uncertainty is None: + require_fitted(rbd) uncertainty = fitted(rbd) - if not uncertainty: - raise ValueError( - "No component's model is a surpyval fit with a parameter " - "covariance, which the uncertainty is drawn from by " - "default: give the uncertain nodes, uncertainty={node: " - "'fit'} or {node: {'repairability': {parameter: " - "distribution}}}, for example." - ) if not uncertainty: raise ValueError( "Give the uncertain nodes: uncertainty={node: 'fit'}, for " @@ -282,8 +328,27 @@ def sources( (i for i, g in enumerate(rbd.ccf_groups) if key is g), None ) if index is not None: + if index in groups: + raise ValueError( + f"Common-cause group {list(key.members)!r} is given an " + "uncertainty twice: under the CCFGroup and under its " + "members." + ) groups[index] = (key, spec) continue + # A group's own parameters under its members' tuple, named as its + # levers are (ccf_beta, ...), go to its model. + index, own, spec = split_group_levers(key, spec, rbd.ccf_groups) + if index is not None and own: + if index in groups: + raise ValueError( + f"Common-cause group {list(key)!r} is given an " + "uncertainty twice: under the CCFGroup and as " + f"{list(own)!r} under its members." + ) + groups[index] = (rbd.ccf_groups[index], own) + if not spec: + continue members = ( key if isinstance(key, tuple) and key not in rbd.components @@ -297,8 +362,9 @@ def sources( if _spec(rbd, node) is None: raise ValueError( f"Node {node!r} has no models of its own to draw (a " - "nested RBD or a junction): give its own components' " - "uncertainty to the nested RBD." + "nested RBD or a junction): the models inside a nested " + "RBD are not drawn here. Give its own uncertainty " + "methods its components' uncertainty." ) label = _label(members) models = _models(rbd, members[0]) @@ -380,12 +446,7 @@ def draws( after the other. With ``sampling="sobol"``, from a scrambled Sobol sequence's points, a pair's sets from dimensions of their own (as ``NonRepairableRBD._uncertain_draws``).""" - if isinstance(n, bool) or not isinstance(n, (int, np.integer)): - raise ValueError(f"n_draws must be an integer, got {n!r}.") - if n < 1: - raise ValueError(f"n_draws must be at least 1, got {n}.") - check_sampling(sampling) - rng = np.random.default_rng(check_seed(seed)) + check_draws(n) def draw(count: int, source) -> Tuple[List[Dict[str, list]], list]: drawn = [] @@ -416,23 +477,19 @@ def draw(count: int, source) -> Tuple[List[Dict[str, list]], list]: ] return drawn, drawn_groups - if sampling == "random": - return draw(2 * n if paired else n, rng) - counter = Counter() - draw(1, counter) - width = counter.dimensions - table = sobol_table(n, 2 * width if paired else width, rng) - first = draw(n, SobolPoints(table[:, :width])) - if not paired: - return first - second = draw(n, SobolPoints(table[:, width:])) - return ( - [ - {role: a[role] + b[role] for role in a} - for a, b in zip(first[0], second[0]) - ], - [None if a is None else a + b for a, b in zip(first[1], second[1])], - ) + def join(first, second): + return ( + [ + {role: a[role] + b[role] for role in a} + for a, b in zip(first[0], second[0]) + ], + [ + None if a is None else a + b + for a, b in zip(first[1], second[1]) + ], + ) + + return draw_with(draw, n, seed, sampling, paired, join) def build(rbd, inputs, drawn, drawn_groups, i: int): @@ -585,7 +642,9 @@ def gradient(keys, names) -> np.ndarray: spec, f"The {role} of {label[0].lower()}{label[1:]}", ) - names = parametric_spec(models[role])[2] + spec_of = parametric_spec(models[role]) + assert spec_of is not None # a varied model is parametric + names = spec_of.names g = gradient(keys, [f"{role}.{names[p]}" for p in positions]) part = part + np.einsum("it,ij,jt->t", g, covariance, g) parts.append(part) @@ -648,3 +707,171 @@ def quantity(models, models_groups) -> np.ndarray: ) ) return first, total, variance + + +def _uncertainty( + rbd, of: str, x, uncertainty, n_draws, seed, sampling, state +) -> UncertaintyResult: + """The quantity ``of`` over draws of the uncertain models (see + ``_repairable_uncertainty``), as an ``UncertaintyResult``.""" + from repyability.rbd import _repairable_uncertainty as drawn + + drawn.check(rbd, of, x, state) + # The diagram's own value first: what it refuses, every draw would. + nominal = drawn.value(rbd, of, x, state) + inputs, groups = drawn.sources(rbd, uncertainty) + models, group_models = drawn.draws( + rbd, inputs, groups, n_draws, seed, sampling + ) + samples = drawn.samples( + rbd, of, x, state, inputs, models, group_models, n_draws + ) + if x is None or np.ndim(x) == 0: + return UncertaintyResult( + samples=samples[:, 0], + nominal=float(nominal[0]), + n_draws=n_draws, + ) + return UncertaintyResult(samples=samples, nominal=nominal, n_draws=n_draws) + + +def mean_availability_uncertainty( + rbd, + uncertainty: Optional[Dict[Hashable, Any]], + *, + n_draws: int, + seed, + sampling: str, +) -> UncertaintyResult: + """See ``RepairableRBD.mean_availability_uncertainty``.""" + return _uncertainty( + rbd, + "mean_availability", + None, + uncertainty, + n_draws, + seed, + sampling, + None, + ) + + +def point_availability_uncertainty( + rbd, + x, + uncertainty: Optional[Dict[Hashable, Any]], + *, + n_draws: int, + seed, + sampling: str, + state, +) -> UncertaintyResult: + """See ``RepairableRBD.point_availability_uncertainty``.""" + return _uncertainty( + rbd, + "point_availability", + x, + uncertainty, + n_draws, + seed, + sampling, + state, + ) + + +def mission_availability_uncertainty( + rbd, + t, + uncertainty: Optional[Dict[Hashable, Any]], + *, + n_draws: int, + seed, + sampling: str, + state, +) -> UncertaintyResult: + """See ``RepairableRBD.mission_availability_uncertainty``.""" + return _uncertainty( + rbd, + "mission_availability", + t, + uncertainty, + n_draws, + seed, + sampling, + state, + ) + + +def expected_cost_rate_uncertainty( + rbd, + uncertainty: Optional[Dict[Hashable, Any]], + *, + n_draws: int, + seed, + sampling: str, +) -> UncertaintyResult: + """See ``RepairableRBD.expected_cost_rate_uncertainty``.""" + return _uncertainty( + rbd, + "expected_cost_rate", + None, + uncertainty, + n_draws, + seed, + sampling, + None, + ) + + +def uncertainty_importance( + rbd, + x, + uncertainty: Optional[Dict[Hashable, Any]], + *, + of: str, + method: str, + n_draws: int, + seed, + sampling: str, + rel_step: Optional[float], + state, +) -> UncertaintyImportance: + """See ``RepairableRBD.uncertainty_importance``.""" + from repyability.rbd import _repairable_uncertainty as drawn + + # parameter_sensitivity's name for the cost rate is taken too (#232). + of = drawn.QUANTITY_NAMES.get(of, of) if isinstance(of, str) else of + drawn.check(rbd, of, x, state) + one_of("method", method, ("delta", "sobol")) + inputs, groups = drawn.sources(rbd, uncertainty) + keys = [item.key for item in inputs] + [ + groups[i][0] for i in sorted(groups) + ] + if method == "delta": + parts = drawn.delta(rbd, of, x, state, inputs, groups, rel_step) + variance = np.sum(parts, axis=0) + with np.errstate(divide="ignore", invalid="ignore"): + shares = [ + np.where(variance > 0.0, part / variance, np.nan) + for part in parts + ] + first = total = dict(zip(keys, shares)) + else: + firsts, totals, variance = drawn.sobol( + rbd, of, x, state, inputs, groups, n_draws, seed, sampling + ) + first, total = dict(zip(keys, firsts)), dict(zip(keys, totals)) + scalar = x is None or np.ndim(x) == 0 + + def shaped(values): + values = np.asarray(values, dtype=float).reshape(-1) + if scalar: + return float(values[0]) + return values.reshape(np.shape(x)) + + return UncertaintyImportance( + method=method, + variance=shaped(variance), + first_order={key: shaped(v) for key, v in first.items()}, + total={key: shaped(v) for key, v in total.items()}, + ) diff --git a/repyability/rbd/_requirements.py b/repyability/rbd/_requirements.py new file mode 100644 index 00000000..0406fba1 --- /dev/null +++ b/repyability/rbd/_requirements.py @@ -0,0 +1,578 @@ +"""The checks a ``RepairableRBD``'s analyses refuse through +(``_require_*``), which ``analysis_routes`` calls too so that its +reasons are the methods' own messages, and the tested units' terms +(``_tested_*``: the life between tests, the rate and phase of the +tests) that the checks and the exact methods share. +""" + +import math +from typing import ( + List, + Optional, + Tuple, +) + +import numpy as np + +from repyability.non_repairable import NonRepairable +from repyability.rbd import ( + _long_run, +) +from repyability.rbd._block_replacement import ( + _check_life, + _Duration, +) +from repyability.rbd._common import ( + _common_period, + _constant_rate, + _sf_values, +) +from repyability.rbd._hidden_life import ( + TestedLife, +) +from repyability.rbd._hidden_tests import TestedUnit +from repyability.rbd._hidden_tests import check as check_tested +from repyability.rbd._model_utils import ( + failure_time_scale, + is_fixed_probability, + model_mean, +) +from repyability.rbd.load_sharing_node import LoadSharingModel +from repyability.rbd.rbd import _close_name +from repyability.rbd.routes import Refused +from repyability.rbd.standby_node import StandbyModel + + +def _mean_lives(rbd) -> List[float]: + """The components' mean lives (their failure-time scales), against + which an endless horizon's discount rate is judged (#231).""" + return [ + failure_time_scale(component.reliability) + for component in rbd.components.values() + if isinstance(component, NonRepairable) + ] + + +def _require_component(rbd, node, given: str) -> None: + """Raise if ``node``, given in ``given``, is not a component (see + ``_not_a_component``).""" + if node not in rbd.components: + raise _not_a_component(rbd, node, given) + + +def _not_a_component(rbd, node, given: str) -> ValueError: + """The error for ``node``, given in ``given``, that is not a + component: a junction, which never fails, or an unknown name, with + the closest component's name if one is close (#232), and those + that are (#222).""" + if node in rbd._junction_nodes: + return ValueError( + f"Node {node!r} given in {given} is a junction " + "(PerfectReliability): it never fails, so it is no component " + "to fail, repair, maintain or stock spares for." + ) + close = _close_name(node, rbd.components) + hint = "" if close is None else f" Did you mean {close!r}?" + return ValueError( + f"Unknown node {node!r} given in {given}; it is not a component " + f"of the RBD.{hint} Its components are: {list(rbd.components)}." + ) + + +def _require_costs(rbd) -> None: + """Raise (``Refused``) if nothing is priced, not even an acquisition + cost (see ``has_costs``): the cost analyses have no cost model to + work on. (With an ``acquisition_cost`` alone they have the one-off + purchase, and no running cost, exactly.)""" + if not (rbd.has_costs or rbd.acquisition_cost): + raise Refused( + "Nothing is priced: give the components a repair_cost, " + "replace_cost, downtime_cost or acquisition_cost, their " + "maintenance or tests a cost, a maintenance group a " + "setup_cost, or the system a downtime_cost_rate." + ) + + +def _require_one_inspected(rbd) -> None: + """Raise if more than one component has hidden failures, when no + intervals are given to choose from (``optimal_inspection_intervals`` + with ``allowed=None``).""" + if len(rbd._inspection) > 1: + raise Refused( + "More than one component has hidden failures: give the " + "intervals to choose from in allowed (tests are made on " + "a calendar, and the long-run values depend on how the " + "schedules line up)." + ) + + +def _require_capacities_given(rbd) -> None: + """Raise if a node takes its capacity from its model (a + ``DegradingNode``'s stages, or a nested RBD's capacities), which + the availability simulation does not follow.""" + own = rbd._capacity_models() + if own: + raise NotImplementedError( + f"Node(s) {sorted(own, key=str)} take their capacity from " + "their models (a DegradingNode's stages, or a nested RBD's " + "capacities), which the simulation does not follow. Give " + "them a capacity, or use capacity_distribution() for the " + "long run." + ) + + +def _require_reliabilities(rbd, node) -> None: + """Raise, as the model itself does, if a component's life or repair + model has no exact or numerical reliability: a ``StandbyModel`` or + ``LoadSharingModel`` that only simulations take (#149).""" + component = rbd.components[node] + for model in ( + getattr(component, "reliability", None), + getattr(component, "time_to_replace", None), + ): + if ( + isinstance(model, (StandbyModel, LoadSharingModel)) + and model.is_simulated + ): + raise model._no_reliability() + + +def _require_time_models(rbd, node) -> None: + """Raise if a component's life or repair model is a probability, + not a distribution of times: its long-run and time-dependent values + then have no exact value (its mean is no mean time). (First, if + one has no reliability: see ``_require_reliabilities``.)""" + _require_reliabilities(rbd, node) + _require_times(rbd, node) + + +def _require_times(rbd, node) -> None: + """Raise if a component's life or repair model is a probability, + not a distribution of times: it has no mean time, nor a value over + time (see ``_require_time_models``).""" + component = rbd.components[node] + for what, model in [ + ("reliability", getattr(component, "reliability", None)), + ("repairability", getattr(component, "time_to_replace", None)), + ]: + if is_fixed_probability(model): + raise NotImplementedError( + f"Component {node!r}: its {what} model is a probability, " + "not a distribution of times, so it has no mean time or " + "availability over time: a unit fails at once with that " + "probability, or never. Estimate the system by " + "simulation, with availability()." + ) + + +def _require_unscheduled_stages(rbd, node) -> None: + """Raise if a degrading component is maintained or inspected on a + schedule: its long-run time in each stage has no exact value.""" + if node in rbd._preventive or node in rbd._inspection: + raise NotImplementedError( + f"Component {node!r} degrades through stages and is " + "maintained or inspected on a schedule: its long-run time in " + "each stage has no exact value here. Give it a capacity " + "instead." + ) + + +def _imperfect_phrase(rbd, node) -> str: + """How a component is repaired imperfectly, in words.""" + imperfect = rbd._imperfect[node] + kind = "I" if imperfect.kijima == "kijima1" else "II" + phrase = f"Kijima {kind}, q = {imperfect.q:g}" + if imperfect.replace_after is not None: + phrase += ( + f", replaced at failure {imperfect.replace_after} since it " + "was renewed" + ) + return phrase + + +def _repairs_take_time(component) -> bool: + """Whether a component's repairs take time (some of them, at least): + its repair time's survival function is above 0 at 0.""" + repair = component.time_to_replace + return float(np.ravel(_sf_values(repair.sf, np.zeros(1)))[0]) > 0.0 + + +def _require_perfect_repair(rbd, node) -> None: + """Raise if a component is repaired imperfectly (a spec's + ``"repair"``): a repair does not renew it, so its long-run values + are known only by simulation (its values over time, only by + simulation too, but for minimal repair: see + ``_require_minimal_repair``).""" + if node in rbd._imperfect: + if _minimal_repair_blocker(rbd, node) is not None: + over_time = "nor its values over time" + elif _repairs_take_time(rbd.components[node]): + over_time = ( + "its values over a window from new are numerical, its " + "operating age followed on a grid" + ) + else: + over_time = ( + "its values over a window from new are exact, as it fails " + "as often as its life's cumulative hazard" + ) + raise NotImplementedError( + f"Component {node!r} is repaired imperfectly " + f"({_imperfect_phrase(rbd, node)}), so a repair does not " + f"renew it: its long-run values have no exact value here " + f"({over_time}). Estimate them by simulation, with " + "availability() or cost()." + ) + + +def _minimal_repair_blocker(rbd, node) -> Optional[str]: + """Why an imperfectly repaired component's values over time have no + exact or numerical value, or None if they have: when it is minimally + repaired (Kijima, ``q = 1``), with no ``replace_after``, preventive + maintenance or tests, and a life that works at 0. Its failures are + then a non-homogeneous Poisson process in its operating time whose + intensity is its life's hazard at its age, the time it has been up, + which its repairs leave as it is: repaired in no time it is up + throughout and fails ``H(t)`` times by ``t`` on average, ``H`` the + life's cumulative hazard, and with repairs that take time its + operating age is followed on a grid (see ``MinimalRepairCurve``).""" + imperfect = rbd._imperfect[node] + component = rbd.components[node] + if imperfect.q < 1.0: + return "its repairs take away some of its age (q below 1)" + if imperfect.replace_after is not None: + return ( + f"it is replaced at its failure {imperfect.replace_after} " + "since it was renewed" + ) + if node in rbd._preventive: + return "it has preventive maintenance" + if node in rbd._inspection: + return "its failures are found only by its tests" + life = component.reliability + if float(np.ravel(_sf_values(life.sf, np.zeros(1)))[0]) < 1.0: + return "its life may end at 0 (dead on arrival)" + return None + + +def _require_minimal_repair(rbd, node) -> None: + """Raise unless an imperfectly repaired component's values over time + are exact or numerical: minimal repair (see + ``_minimal_repair_blocker``).""" + blocker = _minimal_repair_blocker(rbd, node) + if blocker is not None: + raise NotImplementedError( + f"Component {node!r} is repaired imperfectly " + f"({_imperfect_phrase(rbd, node)}), and {blocker}: its " + "availability over time has no exact value here (it has " + "for minimal repair, q = 1, with no replace_after, " + "maintenance or tests). Estimate it by simulation, with " + "availability() or cost()." + ) + + +def _require_no_opportunities(rbd, node) -> None: + """Raise if a component can be renewed early at the stops of its + maintenance group (an ``"opportunity"`` below its interval): when + depends on the other members, so its long-run values and its + availability over time are known only by simulation. So too a + component on a ``"calendar"`` policy (#319).""" + schedule = rbd._preventive.get(node) + if schedule is not None and schedule.policy == "calendar": + raise NotImplementedError( + f"Component {node!r} is replaced on a calendar policy, at " + "times that depend on when each unit was put in, so its " + "long-run values and its availability over time have no " + "exact value here. Estimate them by simulation, with " + "availability() or cost()." + ) + if node in rbd._early_members: + raise NotImplementedError( + f"Component {node!r} is renewed early at the stops of its " + f"maintenance group {rbd._member_group[node]!r}, which " + "depend on the other members, so its long-run values and " + "its availability over time have no exact value here. " + "Estimate them by simulation, with availability() or cost()." + ) + + +def _require_separate_setups(rbd) -> None: + """Raise if two members of a maintenance group with a set-up cost + can be replaced at the same instants, again and again: on block + schedules, or never failing before an age replacement in zero time. + Such replacements share one stop, and its set-up, which the exact + cost rate, charging a set-up for each member's failures and + replacements, does not count.""" + for group, spec in rbd._maintenance.items(): + if not spec.setup_cost: + continue + clocked = [node for node in spec.members if _on_a_clock(rbd, node)] + if len(clocked) > 1: + raise NotImplementedError( + f"Components {clocked} of maintenance group {group!r} " + "are replaced on a clock (on block schedules, or never " + "failing before an age replacement in zero time), so " + "their replacements can fall at the same instants and " + "share a set-up, which the exact cost rate does not " + "count: estimate it by simulation, with cost()." + ) + + +def _on_a_clock(rbd, node) -> bool: + """Whether a component's replacements fall on a fixed lattice of + times: under block replacement, or under age replacement in zero + time without a failure before it.""" + schedule = rbd._preventive.get(node) + if schedule is None or not math.isfinite(schedule.interval): + return False + if schedule.policy == "block": + return True + if schedule.policy != "age" or schedule.duration is not None: + return False + survives = rbd.components[node].reliability_function(schedule.interval) + return bool(np.ravel(survives)[0] >= 1.0) + + +def _require_block_models(rbd, node) -> None: + """Raise unless the exact block-replacement values cover the + component's models (the checks ``block_cycle`` makes first).""" + + _refuse_level(rbd, node) + component = rbd.components[node] + _check_life(component.reliability, node) + _Duration(component.time_to_replace, "repair", node) + _Duration(rbd._preventive[node].duration, "replacement", node) + + +def _refuse_level(rbd, node) -> None: + """Raise for a component replaced on condition by its measured + degradation level (#271): its inspections follow the level, which + only the simulation draws.""" + schedule = rbd._preventive.get(node) + if schedule is not None and schedule.level is not None: + raise NotImplementedError( + f"Component {node!r} is replaced on condition by its " + "measured degradation level, which only the simulation " + "follows: estimate it by simulation, with availability() or " + "cost()." + ) + + +def _require_calendars(rbd) -> None: + """Raise if the exact long-run values cannot average over the + components' calendars (block replacement and inspection): a nested + RBD's calendar with another here, intervals with no common period, + or one repeating too often in it (before any computation).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + nested = [ + node + for node, c in rbd.components.items() + if isinstance(c, RepairableRBD) and _long_run._has_calendar(c) + ] + blocks = _long_run._block_nodes(rbd) + if nested and len(nested) + len(rbd._inspection) + len(blocks) > 1: + raise NotImplementedError( + f"Node(s) {sorted(nested, key=str)} are RBDs with hidden " + "failures or block replacement, and other nodes' inspections " + "or block replacements here fall at the same times: " + "estimate the long-run values by simulation, with " + "availability() or cost()." + ) + intervals = {rbd._preventive[node].interval for node in blocks} + intervals |= {rbd._inspection[node].interval for node in rbd._inspection} + if not intervals: + return + period = _common_period(intervals) + if any(round(period / interval) > 100_000 for interval in intervals): + if blocks: + raise NotImplementedError( + f"The block-replacement and inspection intervals " + f"{sorted(intervals)} repeat together only after too many " + "intervals to average over: estimate the long-run values " + "by simulation, with availability() or cost()." + ) + raise NotImplementedError( + f"The inspection intervals {sorted(intervals)} repeat " + "together only after too many inspections to average " + "over: estimate the long-run values by simulation, with " + "availability() or cost()." + ) + + +def _instant_tests(rbd, node) -> bool: + """Whether a component with hidden failures is tested, and + repaired, in no time.""" + return ( + rbd._inspection[node].duration is None + and model_mean(rbd.components[node].time_to_replace) == 0.0 + ) + + +def _tested_kind(rbd, node) -> str: + """How a component with hidden failures' values are worked out: + ``"closed"``, closed forms, for a constant failure rate and tests + and repair in no time; ``"life"``, summed over the test intervals + (``TestedLife``, #144), for any other life with those and tests + that find every failure; and ``"unit"``, its renewal cycle followed + test by test on a grid (``TestedUnit``, #159), for tests or repairs + that take time, or tests that can miss a failure of a life that is + not exponential.""" + instant = _instant_tests(rbd, node) + if instant and _constant_rate(rbd.components[node].reliability): + return "closed" + if instant and not rbd._inspection[node].partial: + return "life" + return "unit" + + +def _require_tested_exact(rbd, node) -> None: + """Raise unless a component with hidden failures has exact or + numerical values (long-run, or over time): with tests and repair in + no time, any life (#144); with tests or repairs that take time, or + tests that can miss a failure, a surpyval parametric life with a + density and repair and test times that end, the tests within the + interval (#159).""" + if _tested_kind(rbd, node) != "unit": + return + component = rbd.components[node] + inspection = rbd._inspection[node] + check_tested( + component.reliability, + component.time_to_replace, + inspection.duration, + inspection.interval, + node, + ) + + +def _inspected_rate(rbd, node) -> Tuple[float, float]: + """The constant failure rate and the inspection interval of a + component with hidden failures, for the closed forms of an + exponential life tested and repaired in no time (see + ``_tested_life`` and ``_tested_unit`` for the others).""" + _require_tested_exact(rbd, node) + if _tested_kind(rbd, node) != "closed": + raise NotImplementedError( + f"Component {node!r} has hidden failures, and a life that is " + "not exponential or tests or repairs that take time, which " + "this does not take: estimate it by simulation, with " + "availability() or cost()." + ) + rate = _constant_rate(rbd.components[node].reliability) + assert rate is not None + return rate, rbd._inspection[node].interval + + +def _tested_life(rbd, node) -> Optional[TestedLife]: + """For a component with hidden failures, a life other than + exponential, and tests and repair in no time that find every + failure, its long run under its tests (see ``_hidden_life``), + worked out once; None for any other (see ``_tested_kind``).""" + _require_tested_exact(rbd, node) + if _tested_kind(rbd, node) != "life": + return None + component = rbd.components[node] + interval = float(rbd._inspection[node].interval) + cache = rbd._cache.kept("tested_lives") + key = (node, id(component.reliability), interval) + if key not in cache: + cache[key] = TestedLife(component.reliability, interval) + return cache[key] + + +def _tested_unit(rbd, node, any_kind: bool = False) -> Optional[TestedUnit]: + """For a component with hidden failures whose tests or repairs + take time, or whose tests can miss a failure of a life that is not + exponential, its values under its tests (see ``_hidden_tests``), + worked out once; None for any other (see ``_tested_kind``), but + with ``any_kind``, for its spares (whose tests the caller has + checked).""" + if not any_kind: + _require_tested_exact(rbd, node) + if _tested_kind(rbd, node) != "unit": + return None + component = rbd.components[node] + inspection = rbd._inspection[node] + cache = rbd._cache.kept("tested_units") + key = ( + node, + id(component.reliability), + id(component.time_to_replace), + id(inspection.duration), + float(inspection.interval), + float(inspection.coverage), + inspection.per_full_test, + ) + if key not in cache: + cache[key] = TestedUnit( + component.reliability, + component.time_to_replace, + inspection.duration, + inspection.interval, + inspection.coverage, + inspection.per_full_test, + node, + _constant_rate(component.reliability), + ) + return cache[key] + + +def _unit_phase(rbd, node, times: np.ndarray) -> np.ndarray: + """The time since a tested component's last full test at each of + ``times`` on its calendar (its full tests at its offset and every + full test's interval from it): where its long run's profile is + read (see ``TestedLongRun``).""" + inspection = rbd._inspection[node] + position = times - inspection.offset if inspection.offset else times + period = inspection.period + return position - period * np.floor(position / period) + + +def _tested_rate(rbd, node) -> float: + """How fast a tested component's long-run availability falls + between tests: its failure rate, or for any other life the rate of + its profile's mean decay (see ``TestedLife.rate``), or its failures + per unit of up time (``TestedLongRun.rate``), for the long-run + grid's spacing.""" + unit = _tested_unit(rbd, node) + if unit is not None: + return unit.long_run.rate + life = _tested_life(rbd, node) + return _inspected_rate(rbd, node)[0] if life is None else life.rate + + +def _tested_scale(rbd, node) -> float: + """The rate that sets the scale of a tested component's interval: + its failure rate, or one over the mean of any other life.""" + _require_tested_exact(rbd, node) + life = rbd.components[node].reliability + rate = _constant_rate(life) + return float(rate) if rate else 1.0 / float(model_mean(life)) + + +def _tested_phase(rbd, node, times: np.ndarray) -> np.ndarray: + """The time since a tested component's last test at each of + ``times`` on its calendar (its tests at its offset and every + interval from it).""" + inspection = rbd._inspection[node] + position = times - inspection.offset if inspection.offset else times + interval = inspection.interval + return position - interval * np.floor(position / interval) + + +def _tested_intensity( + rbd, node, times: np.ndarray, availability +) -> np.ndarray: + """A tested component's long-run rate of failing at each of + ``times`` (``availability`` its availability there): its constant + rate while it is up, or for any other life the rate of its profile + (see ``TestedLife.intensity``).""" + unit = _tested_unit(rbd, node) + if unit is not None: + return unit.long_run.intensity(_unit_phase(rbd, node, times)) + life = _tested_life(rbd, node) + if life is None: + rate, _ = _inspected_rate(rbd, node) + return rate * availability + return life.intensity(_tested_phase(rbd, node, times)) diff --git a/repyability/rbd/_runs.py b/repyability/rbd/_runs.py new file mode 100644 index 00000000..6404135a --- /dev/null +++ b/repyability/rbd/_runs.py @@ -0,0 +1,2372 @@ +"""A ``RepairableRBD``'s simulated runs: ``availability`` and ``cost``, +their result, the engine that runs them, the variance reductions (the +exact twin as a control variate, conditional runs over the dependent +modules), a run in chunks or shards and its merge +(``simulate_chunk``, ``shards``, ``availability_from_chunks``), the +timelines (``simulate_timelines``) and ``compare``. The event loop +itself (``_run``, ``_replicate``) stays with the diagram. The methods of +``RepairableRBD`` of those names call these. +""" + +import dataclasses +import hashlib +import json +from functools import partial +from typing import ( + TYPE_CHECKING, + Any, + Callable, + Collection, + Dict, + Hashable, + List, + Optional, + Tuple, +) + +import numpy as np + +from repyability._version import __version__ +from repyability.rbd import ( + _conditional, +) +from repyability.rbd import _montecarlo as montecarlo +from repyability.rbd import _streams, _timeline_runs +from repyability.rbd._exact import ( + bin_totals, +) +from repyability.rbd._model_utils import ( + SAVE_ERRORS, +) +from repyability.rbd.node_state import NodeState +from repyability.rbd.results import ( + AvailabilityResult, + ConditionalRun, + ConfidenceInterval, + ControlVariate, + CostResult, + Criticalities, + FailureCriticalityIndex, + RestorationCriticalityIndex, + TimelineSimulation, + UpDownImportance, +) +from repyability.rbd.routes import Refused + +if TYPE_CHECKING: + from repyability.rbd.chunks import SimulationChunk + +from repyability.rbd import ( + _ccf_groups, + _crews, + _curves, + _long_run, + _requirements, + _windows, +) +from repyability.rbd._common import ( + _curve_points, + _safe_ratio, +) +from repyability.rbd._tally import ( + _acquisition_only, + _Breakdown, + _CapacityRecorder, + _check_shard_map, + _chunk_means, + _exact_controls, + _Exacts, + _ModuleRun, + _shard_bytes, + _Tally, +) +from repyability.rbd._time_order import ( + _capacity_totals, + _group_totals, + _working_over_time, +) +from repyability.utils.checks import ( + one_of, + simulation_window, +) + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + + +def _state_key(states: dict) -> Optional[str]: + """The components' ``states`` at the start of a run (checked; see + ``RepairableRBD._simulation_states``) as JSON, the same in every + process, for a chunk's settings (see ``simulate_chunk``); None for + every component new.""" + + def plain(states: dict) -> list: + out = [] + for node, start in states.items(): + if isinstance(start, NodeState): + if start.new: + continue + value: Any = dataclasses.asdict(start) + else: + value = plain(start) + if not value: + continue + out.append([repr(node), value]) + return sorted(out, key=lambda item: item[0]) + + entries = plain(states) + return json.dumps(entries, sort_keys=True) if entries else None + + +def _hot_units(spec: dict) -> "RepairableRBD": + """A standby group's twin (see ``RepairableRBD._twin``): its units + operating together, each failing and repaired on its own, ``k`` of them + needed, as a nested RBD whose components are named ``0, 1, ...``, as + the group's units are, so that their streams are the units'.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + standby = spec["standby"] + units, k = int(standby.get("units", 2)), int(standby.get("k", 1)) + keys = ("reliability", "repairability") + RepairableRBD.COST_KEYS + unit = {key: spec[key] for key in keys if key in spec} + return RepairableRBD( + [("s", u) for u in range(units)] + [(u, "t") for u in range(units)], + {u: dict(unit) for u in range(units)}, + k={"t": k} if k > 1 else None, + ) + + +def _states_from_key(rbd, key: Optional[str]) -> dict: + """The components' states ``_state_key`` saved, for ``rbd`` (a copy of + the system the key was made for, as a shard rebuilds it): each node by + its ``repr``, a nested RBD's own states in turn.""" + + def states(rbd, entries: list) -> dict: + nodes = {repr(node): node for node in rbd.components} + out: dict = {} + for name, value in entries: + node = nodes[name] + if isinstance(value, dict): + out[node] = NodeState(**value) + else: + out[node] = states(rbd.components[node], value) + return out + + return {} if key is None else states(rbd, json.loads(key)) + + +def _stopping_rule( + N: int, + tolerance: Optional[float], + confidence: float, + max_N: Optional[int], + antithetic: bool, + target: str, + t_simulation: float, +) -> Optional[Callable[..., int]]: + """``None`` for a fixed number of replications; otherwise a function + of the tally so far giving how many more replications to run (0 once + the confidence interval of the mean availability over the window, or of + the mean cost, is at most ``tolerance`` either side, or ``max_N`` have + run: then with a warning). Given the simulations' ``values`` too (a + controlled run's, see ``_controlled_run``), it judges those; given why + they cannot be judged yet (``unjudged``), it runs on to the limit.""" + montecarlo.check_confidence(confidence) + limit = montecarlo.sample_limit( + N, tolerance, max_N, antithetic, ("mc_samples", "max_samples") + ) + if limit is None: + return None + + def stop( + tally: "_Tally", values=None, unjudged: Optional[str] = None + ) -> int: + if values is not None: + # The controlled values of a run with an exact twin. + values = np.asarray(values, dtype=float) + elif target == "cost": + values = np.asarray(tally.cost_samples, dtype=float) + else: + values = np.asarray(tally.uptimes, dtype=float) / t_simulation + return montecarlo.more_samples( + values, + N, + tolerance, # type: ignore[arg-type] + confidence, + limit, + antithetic, + target, + "max_samples", + unjudged, + ) + + return stop + + +#: Why antithetic pairs and common random numbers (``compare``) are refused +#: when some component's draws do not come from a stream. +_UNSTREAMED = ( + "Antithetic and common random numbers need every component's draws to " + "be replayable (surpyval parametric distributions and the composite " + "models built from them)." +) + + +def failure_criticality_index_per_system_failures(FCI, system_failures): + """Share of all system failures that each node caused. + + A node causes a system failure when its own failure takes the system + from up to down. For each node this is the number of system failures + it caused divided by ``system_failures``; every node gets 0 when there + were no system failures. + + Parameters + ---------- + FCI : dict + Node -> counts, with the number of system failures the node caused + under ``"system_failures"`` (as accumulated by + ``RepairableRBD.availability``). + system_failures : int + The total number of system failures. + + Returns + ------- + dict + Node -> share of the system failures, in ``[0, 1]``. + + Examples + -------- + >>> from repyability.rbd.repairable_rbd import ( + ... failure_criticality_index_per_system_failures as per_system, + ... ) + >>> counts = {"a": {"system_failures": 3}, "b": {"system_failures": 1}} + >>> per_system(counts, 4) + {'a': 0.75, 'b': 0.25} + """ + fci = {} + for node in FCI.keys(): + if system_failures == 0: + # No system failures occurred (e.g. a redundant node was forced + # working, or the system was highly reliable over the simulated + # window), so no node can be credited with causing one. + fci[node] = 0 + else: + fci[node] = FCI[node]["system_failures"] / system_failures + return fci + + +def failure_criticality_index_per_component_failures(FCI): + """Share of each node's own failures that caused a system failure. + + For each node: the number of system failures it caused (its failures + that took the system from up to down) divided by its number of + failures, or 0 if it never failed. + + Parameters + ---------- + FCI : dict + Node -> counts, with the system failures the node caused under + ``"system_failures"`` and its own failures under + ``"component_failures"`` (as accumulated by + ``RepairableRBD.availability``). + + Returns + ------- + dict + Node -> share of the node's failures, in ``[0, 1]``. + + Examples + -------- + >>> from repyability.rbd.repairable_rbd import ( + ... failure_criticality_index_per_component_failures as per_component, + ... ) + >>> per_component({"a": {"system_failures": 3, "component_failures": 4}}) + {'a': 0.75} + """ + fci = {} + for node in FCI.keys(): + try: + fci[node] = ( + FCI[node]["system_failures"] / FCI[node]["component_failures"] + ) + except ZeroDivisionError: + # If there were no component failures then there were no + # system failures caused by that node + fci[node] = 0 + return fci + + +def restoration_criticality_index_by_system(RCI, system_restorations): + """Share of all system restorations that each node caused. + + A node causes a system restoration when its own restoration takes the + system from down to up. For each node this is the number of system + restorations it caused divided by ``system_restorations``; every node + gets 0 when there were no system restorations. + + Parameters + ---------- + RCI : dict + Node -> counts, with the number of system restorations the node + caused under ``"system_restorations"`` (as accumulated by + ``RepairableRBD.availability``). + system_restorations : int + The total number of system restorations. + + Returns + ------- + dict + Node -> share of the system restorations, in ``[0, 1]``. + + Examples + -------- + >>> from repyability.rbd.repairable_rbd import ( + ... restoration_criticality_index_by_system as by_system, + ... ) + >>> by_system({"a": {"system_restorations": 1}}, 2) + {'a': 0.5} + """ + rci = {} + for node in RCI.keys(): + if system_restorations == 0: + # No system restorations occurred, so no node can be credited with + # causing one. + rci[node] = 0 + else: + rci[node] = RCI[node]["system_restorations"] / system_restorations + return rci + + +def restoration_criticality_index_by_component(RCI): + """Share of each node's own restorations that restored the system. + + For each node: the number of system restorations it caused (its + restorations that took the system from down to up) divided by its + number of restorations, or 0 if it was never restored. + + Parameters + ---------- + RCI : dict + Node -> counts, with the system restorations the node caused under + ``"system_restorations"`` and its own restorations under + ``"component_restorations"`` (as accumulated by + ``RepairableRBD.availability``). + + Returns + ------- + dict + Node -> share of the node's restorations, in ``[0, 1]``. + + Examples + -------- + >>> from repyability.rbd.repairable_rbd import ( + ... restoration_criticality_index_by_component as by_component, + ... ) + >>> counts = {"system_restorations": 1, "component_restorations": 4} + >>> by_component({"a": counts}) + {'a': 0.25} + """ + rci = {} + for node in RCI.keys(): + try: + rci[node] = ( + RCI[node]["system_restorations"] + / RCI[node]["component_restorations"] + ) + except ZeroDivisionError: + # If there were no component restorations then there were no + # times the system was restored by restoring this node. + rci[node] = 0 + return rci + + +def _engine_choice(diagram, capacity: bool) -> Tuple[str, str]: + """The engine ``engine="auto"`` runs a long simulation on, and why + (see ``_simulation_engine``).""" + from repyability.rbd import _compiled + + from . import _event_loop + + plan = _event_loop._stream_plan(diagram, 1.0, 0, False)[0] + engine, reason = _compiled.choice( + diagram, plan, object() if capacity else None + ) + if reason is not None: + return "python", f"the compiled engine does not simulate {reason}" + if engine is None: + return ( + "python", + "numba is not installed; pip install 'repyability[fast]' " + "for the compiled engine", + ) + return ( + engine, + "compiled for a run long enough to repay loading it; a short " + "one runs in Python", + ) + + +def availability( + diagram, + t_simulation: float, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + mc_samples: Optional[int], + verbose: bool, + seed: Optional[int], + *, + tolerance: Optional[float], + confidence: float, + max_samples: Optional[int], + antithetic: bool, + n_jobs: Optional[int], + demand: Optional[float], + engine: str, + state, + curve_points: Optional[int], + shard_map: Optional[Callable], + shard_size: Optional[int], + control_variate: Optional[bool], + conditional: Optional[bool], +) -> AvailabilityResult: + """See ``RepairableRBD.availability``.""" + N = 10_000 if mc_samples is None else mc_samples + + return _simulated( + diagram, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + verbose, + seed, + tolerance=tolerance, + confidence=confidence, + max_N=max_samples, + antithetic=antithetic, + n_jobs=n_jobs, + target="availability", + demand=demand, + engine=engine, + state=state, + curve_points=_curve_points(curve_points), + shard_map=shard_map, + shard_size=shard_size, + control_variate=control_variate, + conditional=conditional, + ) + + +def simulate_timelines( + diagram, + t_simulation: float, + mc_samples: Optional[int], + seed: Optional[int], + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + antithetic: bool, + engine: str, + n_jobs: Optional[int], + start: int, + state, +) -> "TimelineSimulation": + """See ``RepairableRBD.simulate_timelines``.""" + return _timeline_runs.simulate( + diagram, + t_simulation, + 1000 if mc_samples is None else mc_samples, + seed, + working_nodes, + broken_nodes, + antithetic, + engine, + n_jobs, + start, + state, + ) + + +def simulate_chunk( + diagram, + t_simulation: float, + start: int, + stop: int, + *, + seed: int, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + antithetic: bool, + demand: Optional[float], + engine: str, + n_jobs: Optional[int], + verbose: bool, + state, + curve_points: Optional[int], + control_variate: Optional[bool], + conditional: Optional[bool], +) -> "SimulationChunk": + """See ``RepairableRBD.simulate_chunk``.""" + if seed is None: + raise ValueError( + "A chunk needs the run's seed: the chunks of a run share it." + ) + for name, value in (("start", start), ("stop", stop)): + if isinstance(value, bool) or not isinstance(value, (int, np.integer)): + raise ValueError(f"{name} must be a whole number, got {value!r}.") + if not 0 <= start < stop: + raise ValueError( + f"The chunk must be simulations start to stop - 1, with 0 <= " + f"start < stop; got start={start!r}, stop={stop!r}." + ) + if antithetic and (start % 2 or stop % 2): + raise ValueError( + "With antithetic pairs, start and stop must be even, so that " + "the chunk holds whole pairs." + ) + means = _chunk_means(control_variate, conditional) + working, broken = diagram._held_nodes(working_nodes, broken_nodes) + states = _curves._simulation_states(diagram, state, working | broken) + chunk = _chunk( + diagram, + t_simulation, + int(start), + int(stop), + _streams.entropy_of(seed), + working, + broken, + method, + antithetic, + demand, + engine, + None if n_jobs is None else montecarlo.jobs(n_jobs), + verbose, + states, + _curve_points(curve_points), + ) + # Kept only where they differ from availability's defaults, so that + # a chunk is saved as it was before them. + chunk.settings.update( + {name: value for name, value in means.items() if value is False} + ) + return chunk + + +def _chunk( + diagram, + t_simulation: float, + start: int, + stop: int, + entropy: int, + working: set, + broken: set, + method: str, + antithetic: bool, + demand: Optional[float], + engine: str, + jobs: Optional[int], + verbose: bool, + states: dict, + curve_points: Optional[int], +) -> "SimulationChunk": + """Simulations ``start`` to ``stop - 1`` of the run of ``entropy`` + (see ``simulate_chunk``, whose arguments, checked, these are), as + a chunk with the run's settings.""" + from repyability.rbd.chunks import SimulationChunk + + from . import _event_loop + + capacity = _chunk_capacity(diagram, broken, method, demand) + tally = _event_loop._run( + diagram, + t_simulation, + working, + broken, + method, + stop - start, + verbose, + None, + antithetic, + capacity=capacity, + jobs=jobs, + engine=engine, + entropy=entropy, + first=start, + states=states, + curve_points=curve_points, + ) + settings = { + "t_simulation": float(t_simulation), + "entropy": entropy, + "method": method, + "working_nodes": sorted(working, key=repr), + "broken_nodes": sorted(broken, key=repr), + "antithetic": bool(antithetic), + "demand": None if demand is None else float(demand), + "fingerprint": _fingerprint(diagram), + "state": _state_key(states), + "curve_points": tally.curve_points, + } + return SimulationChunk([(start, stop)], settings, tally) + + +def _chunk_capacity( + diagram, broken: set, method: str, demand: Optional[float] +) -> Optional[_CapacityRecorder]: + """Check a chunk's (or shard's) ``method`` and ``demand``, and give + the recorder of the capacities it follows, if the system has + them.""" + diagram.is_system_working( + {c: c not in broken for c in diagram.components}, method + ) + if diagram._has_capacity(): + return _CapacityRecorder(diagram, demand) + if demand is not None: + raise ValueError( + "A demand is measured against capacities, and no node has " + "one: give them with capacity={node: capacity}." + ) + return None + + +def shards( + diagram, + t_simulation: float, + mc_samples: Optional[int], + *, + seed: Optional[int], + size: Optional[int], + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + antithetic: bool, + demand: Optional[float], + engine: str, + state, + curve_points: Optional[int], +) -> List[bytes]: + """See ``RepairableRBD.shards``.""" + from . import _event_loop + + N = 10_000 if mc_samples is None else mc_samples + montecarlo.check_count(N, antithetic, "mc_samples") + working, broken = diagram._held_nodes(working_nodes, broken_nodes) + states = _curves._simulation_states(diagram, state, working | broken) + capacity = _chunk_capacity(diagram, broken, method, demand) + _ccf_groups._require_free_members(diagram, working, broken) + _ccf_groups._require_groups_simulated(diagram, states) + t_simulation = simulation_window(t_simulation) + entropy = _streams.entropy_of(seed) + template = _shard_template( + diagram, + t_simulation, + entropy, + working, + broken, + method, + antithetic, + demand, + engine, + states, + _curve_points(curve_points), + ) + # What a run checks before it simulates (see _run). + plan, complete = _event_loop._stream_plan( + diagram, t_simulation, entropy, antithetic, states + ) + if antithetic and not complete: + raise NotImplementedError(_UNSTREAMED) + _simulation_engine( + diagram, engine, plan, capacity, N, here=False, states=states + ) + step = _shard_size(plan, size) + return [ + _shard_bytes(template, first, min(first + step, N)) + for first in range(0, N, step) + ] + + +def _shard_template( + diagram, + t_simulation: float, + entropy: int, + working: set, + broken: set, + method: str, + antithetic: bool, + demand: Optional[float], + engine: str, + states: dict, + curve_points: Optional[int], +) -> dict: + """What every shard of a run holds (see ``shards``): all but its + range.""" + from repyability.rbd.shards import KIND + + return { + "kind": KIND, + "repyability_version": __version__, + "system": _shard_system(diagram), + "t_simulation": float(t_simulation), + "entropy": int(entropy), + "method": method, + "working_nodes": sorted(working, key=repr), + "broken_nodes": sorted(broken, key=repr), + "antithetic": bool(antithetic), + "demand": None if demand is None else float(demand), + "engine": engine, + "state": _state_key(states), + "curve_points": curve_points, + } + + +def _shard_system(diagram) -> dict: + """This system as a shard carries it: ``to_dict``'s JSON data, with + every model of a class that loads back as itself (see + ``serialisation.exactly``), so that a worker simulates this + system.""" + from repyability.rbd.repairable_rbd import RepairableRBD + from repyability.rbd.serialisation import exactly + + try: + with exactly(): + system = diagram.to_dict() + json.dumps(system) + if type(diagram) is not RepairableRBD: + raise NotImplementedError( + f"it is a {type(diagram).__name__}, which loads as a " + "RepairableRBD" + ) + except SAVE_ERRORS as error: + raise NotImplementedError( + "A shard carries the system as JSON, and this one cannot be " + f"saved: {str(error).rstrip('.')}. Run it with " + "availability(n_jobs=...) instead." + ) from None + return system + + +def _shard_size(plan: _streams.Plan, size: Optional[int]) -> int: + """Simulations to a shard of the run of ``plan``: ``size`` (by + default 1024), rounded up to a whole number of its widest block of + draws, so that no two shards draw the same block.""" + wanted = 1024 if size is None else size + if ( + isinstance(wanted, bool) + or not isinstance(wanted, (int, np.integer)) + or wanted < 1 + ): + raise ValueError( + "The size of a shard must be a whole number of simulations, " + f"at least 1; got {size!r}." + ) + widest = max( + [plan.columns(spec) for spec in plan.specs.values()] + + [2 if plan.antithetic else 1] + ) + return int(-(-int(wanted) // widest) * widest) + + +def availability_from_chunks( + diagram, + chunks, + mc_samples: Optional[int], + *, + allow_gaps: bool, + control_variate: Optional[bool], + conditional: Optional[bool], +) -> AvailabilityResult: + """See ``RepairableRBD.availability_from_chunks``.""" + from repyability.rbd.chunks import SimulationChunk + + if isinstance(chunks, (SimulationChunk, dict, bytes, bytearray)): + chunks = [chunks] + chunk = SimulationChunk.merge(chunks) + if mc_samples is not None and chunk.ranges != [(0, int(mc_samples))]: + raise ValueError( + f"The chunks hold simulations {chunk.ranges} (as " + "(start, stop) ranges), not all of simulations 0 to " + f"{int(mc_samples) - 1}: some are missing." + ) + if not allow_gaps and chunk.ranges != [(0, chunk.n_simulations)]: + held = ", ".join(f"{a} to {b - 1}" for a, b in chunk.ranges) + raise ValueError( + f"The chunks hold simulations {held}: those between or " + "before them are missing (a chunk that never came back?). " + "Give them all, or pass allow_gaps=True for the result of " + "the simulations held." + ) + settings = chunk.settings + means = _chunk_means(control_variate, conditional) + for name in ("control_variate", "conditional"): + if settings.get(name) is False: + means[name] = False + fingerprint = _fingerprint(diagram) + if ( + settings["fingerprint"] is not None + and fingerprint is not None + and settings["fingerprint"] != fingerprint + ): + raise ValueError( + "The chunks were simulated with another system (or with " + "this one saved by another RePyability version)." + ) + working = set(settings["working_nodes"]) + broken = set(settings["broken_nodes"]) + capacity = None + if diagram._has_capacity(): + capacity = _CapacityRecorder(diagram, settings["demand"]) + initial_up = bool( + diagram.is_system_working( + {c: c not in broken for c in diagram.components}, + settings["method"], + ) + ) + # The means the run's result takes by default (#187, #189). + controls, record, breakdown = _default_means( + diagram, + chunk._tally, + float(settings["t_simulation"]), + working, + broken, + settings["method"], + int(settings["entropy"]), + bool(settings["antithetic"]), + _states_from_key(diagram, settings.get("state")), + chunk.ranges, + demand=None if capacity is None else capacity.demand, + **means, + ) + return _availability_result( + diagram, + chunk._tally, + settings["t_simulation"], + initial_up, + settings["antithetic"], + capacity, + controls, + conditional=record, + breakdown=breakdown, + ) + + +def _default_means( + diagram, + tally: "_Tally", + t_simulation: float, + working: set, + broken: set, + method: str, + entropy: int, + antithetic: bool, + state, + ranges: List[Tuple[int, int]], + control_variate: Optional[bool] = None, + conditional: Optional[bool] = None, + demand: Optional[float] = None, +) -> Tuple[ + Tuple[Optional[ControlVariate], ...], + Optional[ConditionalRun], + Optional[_Breakdown], +]: + """The means the result of a plain run takes by default, for its + simulations ``ranges`` in ``tally`` (chunks merged, see + ``availability_from_chunks``), as ``availability`` takes them: the + exact methods', where they work them out (#187, see + ``_exact_controls``), or given the histories of its modules, + simulated again from the run's ``entropy`` (#189, see + ``_conditioned_run``), with the expected cost's split found so; + else none. The fraction of the ``demand`` delivered, if one was + measured, is taken the same way. ``control_variate=False`` takes + neither, and ``conditional=False`` not the second (#236), as + ``availability`` does.""" + T = t_simulation + if control_variate is False: + return (None, None), None, None + exact_means = _exact_means( + diagram, T, working, broken, method, state, demand + ) + if exact_means is not None: + return ( + _exact_controls(tally, T, exact_means, antithetic), + None, + exact_means.breakdown, + ) + if conditional is False: + return (None, None), None, None + modules = _conditional_applies(diagram, working, broken, state) + if not modules: + return (None, None), None, None + run = _ModuleRun( + diagram, + T, + working, + broken, + method, + entropy, + antithetic=antithetic, + n_jobs=None, + engine="auto", + curve_points=1, + state=state, + demand=demand, + capacities=demand is not None, + modules=modules, + ) + try: + run.prepare() + except NotImplementedError: + return (None, None), None, None + for start, stop in ranges: + run.simulate(start, stop - start) + record = run.means() + return (None, None), record, run.split(record) + + +def _fingerprint(diagram) -> Optional[str]: + """A hash of this system saved as JSON (with the RePyability + version), which chunks of its runs carry; None if it cannot be + saved.""" + try: + text = json.dumps(diagram.to_dict(), sort_keys=True) + except SAVE_ERRORS: + return None + return hashlib.sha256(text.encode()).hexdigest() + + +def compare( + diagram, + other: "RepairableRBD", + t_simulation: float, + mc_samples: Optional[int], + seed: Optional[int], + *, + quantity: str, + confidence: float, + n_jobs: Optional[int], + engine: str, + state, + control_variate: Optional[bool], +) -> ConfidenceInterval: + """See ``RepairableRBD.compare``.""" + from . import _event_loop + + N = 10_000 if mc_samples is None else mc_samples + t_simulation = simulation_window(t_simulation) + + one_of("quantity", quantity, ("availability", "cost")) + montecarlo.check_confidence(confidence) + montecarlo.check_count(N, False, "mc_samples") + if control_variate not in (None, False): + raise ValueError( + "compare takes control_variate=None (the exact difference " + "where the exact methods work out both systems' expected " + "values) or False (the simulated difference), got " + f"{control_variate!r}: its common random numbers are its " + "own variance reduction." + ) + if quantity == "cost": + for rbd in (diagram, other): + _requirements._require_costs(rbd) + if control_variate is None: + exact = _exact_difference( + diagram, other, t_simulation, quantity, state + ) + if exact is not None: + return ConfidenceInterval( + estimate=exact, + lower=exact, + upper=exact, + confidence=confidence, + standard_error=0.0, + n_samples=0, + method="exact", + ) + jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) + states = [ + _curves._simulation_states(rbd, state) for rbd in (diagram, other) + ] + entropy = _streams.entropy_of(seed) + values = [] + for rbd, start in zip((diagram, other), states): + tally = _event_loop._run( + rbd, + t_simulation, + set(), + set(), + "p", + N, + False, + None, + jobs=jobs, + engine=engine, + entropy=entropy, + common=True, + states=start, + ) + if quantity == "cost": + running = ( + np.asarray(tally.cost_samples, dtype=float) + if rbd.has_costs + else np.zeros(N) + ) + values.append(running + rbd.acquisition_cost) + else: + values.append(np.asarray(tally.uptimes) / t_simulation) + differences = values[0] - values[1] + estimate = float(np.mean(differences)) + standard_error = montecarlo.standard_error(differences, False) + z = montecarlo.z_value(confidence) + return ConfidenceInterval( + estimate=estimate, + lower=estimate - z * standard_error, + upper=estimate + z * standard_error, + confidence=confidence, + standard_error=standard_error, + n_samples=N, + method="simulated", + ) + + +def _exact_difference( + diagram, + other: "RepairableRBD", + t_simulation: float, + quantity: str, + state, +) -> Optional[float]: + """``compare``'s difference worked out exactly (#236): this + system's expected value of ``quantity`` over the window less + ``other``'s, where the exact methods work out both (see + ``_exact_means``); else None.""" + values = [] + for rbd in (diagram, other): + exacts = _exact_means(rbd, t_simulation, set(), set(), "p", state) + if exacts is None: + return None + if quantity == "availability": + values.append(exacts.availability) + continue + if exacts.cost is not None: + running = exacts.cost + elif rbd.has_costs: + return None + else: + running = 0.0 + values.append(running + rbd.acquisition_cost) + return float(values[0] - values[1]) + + +def _twin(diagram) -> Tuple["RepairableRBD", List[str]]: + """This system's exact twin (#154), and what it changes, in words. + + The twin has this system's diagram and components, with their + models, failing and repaired independently: without the limit on + repair crews or the maintenance groups, which tie components + together, and, component by component, without what the exact + methods over time do not take (see ``_twin_simpler``). Its expected + values over a window are then exact (``mission_availability``, + ``expected_cost``), and each of its streams is named as this + system's is (a standby group's units as the twin's nested units), + so that, simulated with common random numbers, it follows this + system closely: see ``availability``'s ``control_variate``. + + Raises + ------ + NotImplementedError + If no such twin has exact values over time: a component's model + is a probability, or a life is simulated. + """ + from repyability.rbd import routes as r + from repyability.rbd.repairable_rbd import RepairableRBD + + args = diagram._init_args + changes = [] + if _crews._crews_limited(diagram): + changes.append("the limit on repair crews") + if diagram._maintenance: + changes.append("the maintenance groups") + specs: Dict[Any, Any] = {} + for node, value in args["components"].items(): + if isinstance(value, RepairableRBD): + nested, inner = _twin(value) + # Its costs are not the twin's to count, as the system's + # construction has said. + nested._costs_reported = True + specs[node] = nested + changes.extend(f"{what} in {node!r}" for what in inner) + elif isinstance(value, dict): + spec = { + key: item + for key, item in value.items() + if key not in ("priority", "group") + } + preventive = value.get("preventive") + if preventive and "opportunity" in preventive: + # Opportunities come from a group's stops. + spec["preventive"] = { + key: item + for key, item in preventive.items() + if key != "opportunity" + } + specs[node] = spec + else: + specs[node] = value + twin = _twin_from(diagram, specs) + simpler = {} + for node, spec in specs.items(): + if isinstance(spec, dict): + simpler[node], change = _twin_simpler(twin, node, spec) + if change: + changes.append(change) + if any(simpler[node] is not specs[node] for node in simpler): + twin = _twin_from(diagram, {**specs, **simpler}) + if twin.ccf_groups and r.refusal( + partial(_ccf_groups._require_groups_over_time, twin, {}) + ): + # Its groups' chains cannot follow them over time (#158): its + # members fail independently. + twin = _twin_from(diagram, {**specs, **simpler}, groups=False) + changes.append("the common-cause groups") + route = _curves._over_time(twin) + if route.route == r.REFUSED: + raise NotImplementedError( + "This system has no exact twin to control its simulation " + f"by: {route.reason}" + ) + if route.route == r.SIMULATED: + raise NotImplementedError( + "This system has no exact twin to control its simulation " + "by: its components' values over time are simulated " + f"({route.reason})" + ) + return twin, changes + + +def _twin_report(diagram) -> str: + """What ``analysis_routes`` says of the exact twin a run with + ``control_variate`` is controlled by (see ``AnalysisRoute.twin``): + what it leaves out of this system, or why there is none.""" + from . import _event_loop + + try: + _, changes = _twin(diagram) + except NotImplementedError as error: + return f"none. {error}" + _, complete = _event_loop._stream_specs(diagram, 1.0) + if not complete: + return f"none. {_UNSTREAMED}" + if not changes: + return ( + "the system itself, whose values over the window are exact " + "(mission_availability, expected_cost)." + ) + return f"the system without {', '.join(changes)}." + + +def _twin_from( + diagram, components: dict, groups: bool = True +) -> "RepairableRBD": + """A system of this one's diagram with ``components`` (see + ``_twin``), and its common-cause groups unless not ``groups``.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + args = diagram._init_args + return RepairableRBD( + args["edges"], + components, + k=args["k"], + input_node=args["input_node"], + output_node=args["output_node"], + on_infeasible_rbd=args["on_infeasible_rbd"], + downtime_cost_rate=args["downtime_cost_rate"], + ccf_groups=args.get("ccf_groups") if groups else None, + ) + + +def _twin_simpler(diagram, node, spec: dict) -> Tuple[Any, str]: + """A component's spec in an exact twin (see ``_twin``), this system + being the twin so far, and what it changes (or ""): a standby group + whose own chain cannot follow it over time becomes its units + operating together; imperfect repair becomes perfect (but for + minimal repair, whose values over time are exact or numerical); + replacement on condition, inspections and block replacement that + the exact methods do not take go.""" + from repyability.rbd import routes as r + + if node in diagram._standby: + if r.refusal(partial(_long_run._standby_rates, diagram, node)): + return ( + _hot_units(spec), + f"the switching of standby group {node!r} (its units " + "operate together)", + ) + return spec, "" + drop: Dict[str, str] = {} + if node in diagram._imperfect and r.refusal( + partial( + _requirements._require_minimal_repair, + diagram, + node, + ) + ): + drop["repair"] = drop["replace_after"] = "imperfect repair" + if node in diagram._inspection and r.refusal( + partial( + _requirements._require_tested_exact, + diagram, + node, + ) + ): + drop["inspection"] = "inspections" + schedule = diagram._preventive.get(node) + if ( + schedule is not None + and schedule.policy in ("block", "condition") + and r.refusal( + partial( + _requirements._require_block_models, + diagram, + node, + ) + ) + ): + drop["preventive"] = ( + "block replacement" + if schedule.policy == "block" + else "replacement on condition" + ) + if not drop: + return spec, "" + what = list(dict.fromkeys(drop[key] for key in drop if key in spec)) + note = " (its failures are revealed)" if "inspection" in drop else "" + return ( + {key: value for key, value in spec.items() if key not in drop}, + f"the {' and '.join(what)} of {node!r}{note}", + ) + + +def _simulated( + diagram, + t_simulation: float, + working_nodes, + broken_nodes, + method: str, + N: int, + verbose: bool, + seed, + *, + tolerance: Optional[float], + confidence: float, + max_N: Optional[int], + antithetic: bool, + n_jobs: Optional[int], + target: str, + demand: Optional[float] = None, + engine: str = "auto", + state=None, + curve_points: Optional[int] = None, + shard_map: Optional[Callable] = None, + shard_size: Optional[int] = None, + control_variate: Optional[bool] = None, + conditional: Optional[bool] = None, +) -> AvailabilityResult: + """``availability`` (and ``cost``): validate, run the replications + (serially, in parallel, or as shards through ``shard_map``, until + converged if asked; with ``control_variate``, alongside the system's + exact twin; with ``conditional``, of the dependent modules alone, + see ``_conditional_run``) and build the result, its curve on a grid + of ``curve_points`` steps if given. By default its expected values + are exact where the system is its own exact twin (#187), and taken + given the modules' histories where a conditional run applies + (#189, see ``_conditioned_run``).""" + from . import _event_loop + + t_simulation = simulation_window(t_simulation) + working_nodes, broken_nodes = diagram._held_nodes( + working_nodes, broken_nodes + ) + states = _curves._simulation_states( + diagram, state, working_nodes | broken_nodes + ) + # The initial system state with every component up but those forced + # broken, which can make the system start down; a simulation that + # starts down from the components' states records the change at 0 + # (see _replicate). + initial_status = {c: c not in broken_nodes for c in diagram.components} + initial_up = bool(diagram.is_system_working(initial_status, method)) + montecarlo.check_count(N, antithetic, "mc_samples") + stop = _stopping_rule( + N, tolerance, confidence, max_N, antithetic, target, t_simulation + ) + _check_shard_map(shard_map, shard_size, n_jobs) + if control_variate is not None and not isinstance( + control_variate, (bool, np.bool_) + ): + raise ValueError( + f"control_variate must be True or False, got " + f"{control_variate!r}." + ) + if conditional is not None and not isinstance( + conditional, (bool, np.bool_) + ): + raise ValueError( + f"conditional must be True, False or None, got " + f"{conditional!r}." + ) + if conditional: + if demand is not None and not diagram._has_capacity(): + raise ValueError( + "A demand is measured against capacities, and no node " + "has one: give them with capacity={node: capacity}." + ) + return _conditional_run( + diagram, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + seed, + stop=stop, + antithetic=antithetic, + n_jobs=n_jobs, + engine=engine, + curve_points=curve_points if target == "availability" else 1, + target=target, + state=state, + shard_map=shard_map, + shard_size=shard_size, + demand=demand, + capacities=target == "availability" and diagram._has_capacity(), + control_variate=bool(control_variate), + ) + capacity = None + # Shards follow the capacities whatever the target, as chunks do. + if ( + target == "availability" or shard_map is not None + ) and diagram._has_capacity(): + capacity = _CapacityRecorder(diagram, demand) + elif demand is not None: + raise ValueError( + "A demand is measured against capacities, and no node has " + "one: give them with capacity={node: capacity}." + ) + measured = None if capacity is None else capacity.demand + # By default the expected values are the exact methods', where they + # work them out (#187): a run to a tolerance stops at once. + auto = control_variate is None + exact_means = ( + _exact_means( + diagram, + t_simulation, + working_nodes, + broken_nodes, + method, + state, + measured, + ) + if auto + else None + ) + twin = None + changes: List[str] = [] + exacts: Optional[_Exacts] = None + if control_variate: + twin, changes = _twin(diagram) + if shard_map is not None and changes: + raise ValueError( + "A run with control_variate simulates the system's twin " + "alongside it, here: leave out shard_map." + ) + exacts = _twin_exact( + diagram, + twin, + t_simulation, + working_nodes, + broken_nodes, + method, + state, + measured if not changes else None, + ) + # Otherwise, by default, its means are taken given the histories of + # its modules, where a conditional run applies (#189). + modules = ( + _conditional_applies(diagram, working_nodes, broken_nodes, state) + if auto and exact_means is None and conditional is None + else [] + ) + entropy = sharded = None + if shard_map is not None: + _ccf_groups._require_groups_simulated(diagram) + entropy = _streams.entropy_of(seed) + template = _shard_template( + diagram, + t_simulation, + entropy, + working_nodes, + broken_nodes, + method, + antithetic, + demand, + engine, + states, + curve_points, + ) + plan, _ = _event_loop._stream_plan( + diagram, t_simulation, entropy, antithetic, states + ) + step = _shard_size(plan, shard_size) + sharded = (shard_map, template, step) + jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) + controls: Tuple[Optional[ControlVariate], ...] = (None, None, None) + # The expected cost's split, where the means are exact (#223). + breakdown: Optional[_Breakdown] = None + if modules: + done = _conditioned_run( + diagram, + modules, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + verbose, + seed, + antithetic, + stop, + capacity=capacity, + jobs=jobs, + n_jobs=n_jobs, + engine=engine, + state=state, + states=states, + curve_points=curve_points, + target=target, + sharded=sharded, + shard_map=shard_map, + shard_size=shard_size, + ) + if done is not None: + tally, record, breakdown = done + return _availability_result( + diagram, + tally, + t_simulation, + initial_up, + antithetic, + capacity, + conditional=record, + breakdown=breakdown, + ) + if twin is not None: + assert exacts is not None + tally, controls = _controlled_run( + diagram, + twin, + exacts, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + verbose, + seed, + antithetic, + stop, + capacity=capacity, + jobs=jobs, + engine=engine, + state=state, + states=states, + curve_points=curve_points, + target=target, + itself=not changes, + sharded=sharded, + ) + if not changes: + # Its means are the system's exact ones (#223). + breakdown = exacts.breakdown + else: + tally = _event_loop._run( + diagram, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + verbose, + seed, + antithetic, + None if exact_means is not None else stop, + capacity=capacity, + jobs=jobs, + engine=engine, + entropy=entropy, + states=states, + curve_points=curve_points, + sharded=sharded, + ) + if exact_means is not None: + controls = _exact_controls( + tally, t_simulation, exact_means, antithetic + ) + breakdown = exact_means.breakdown + return _availability_result( + diagram, + tally, + t_simulation, + initial_up, + antithetic, + capacity, + controls, + breakdown=breakdown, + ) + + +def _exact_means( + diagram, + t_simulation: float, + working: set, + broken: set, + method: str, + state, + demand: Optional[float] = None, +) -> Optional[_Exacts]: + """The system's own expected values over the window, where the + exact methods work them out (#187): its mean availability, if it is + priced its expected cost, and against a ``demand`` the fraction of + it delivered (see ``_twin_exact``); else None.""" + try: + return _twin_exact( + diagram, + diagram, + t_simulation, + working, + broken, + method, + state, + demand, + ) + except NotImplementedError: + return None + + +def _conditional_applies( + diagram, working: Collection = (), broken: Collection = (), state=None +) -> list: + """The modules a conditional run would simulate (see + ``_conditional_modules``) with the nodes ``working`` and ``broken`` + held and from the components' ``state``, or none if it would + refuse: the others must be taken exactly given them, which a + structure too meshed for its decision diagram, or common-cause + groups the exact methods over time refuse, prevent.""" + from repyability.rbd import routes as r + + if diagram._too_meshed() is not None: + return [] + if diagram.ccf_groups and r.refusal( + partial( + _ccf_groups._require_groups_over_time, + diagram, + state or {}, + ) + ): + return [] + try: + return diagram._conditional_modules(set(working), set(broken)) + except NotImplementedError: + return [] + + +def _modules_rbd(diagram, modules: list) -> "RepairableRBD": + """The ``modules`` of a conditional run (see + ``_conditional_modules``) as a diagram of their own, each between + the input and output nodes with its own spec: their streams are + named as in this system, so its simulations are this system's + modules'. It has this system's repair crews (whose jobs are the + modules' alone: crews that tie the components together make every + node they serve a module), the maintenance groups of its members, + and no system downtime cost.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + args = diagram._init_args + options = args["maintenance_groups"] or {} + groups = { + name: spec + for name, spec in options.items() + if set(diagram._maintenance[name].members) <= set(modules) + } + return RepairableRBD( + [(diagram.input_node, node) for node in modules] + + [(node, diagram.output_node) for node in modules], + {node: args["components"][node] for node in modules}, + input_node=diagram.input_node, + output_node=diagram.output_node, + repair_crews=args["repair_crews"], + maintenance_groups=groups or None, + ) + + +def _capacity_given( + diagram, + modules: list, + state: int, + working: set, + broken: set, + x: np.ndarray, + others, + demand: Optional[float], +) -> "_conditional.CapacityGiven": + """The system's capacity given the modules in one joint ``state`` + (see ``_conditional_given``), on the grid ``x``: from its exact + distribution over time with the modules held so (a module up at its + levels by their probabilities, as the simulation weighs them), and + the other nodes from their states ``others``, against ``demand`` + (see ``_conditional.capacity_given``).""" + up = {node for j, node in enumerate(modules) if state >> j & 1} + down = set(modules) - up + distribution = diagram.point_capacity( + x, working | up, broken | down, state=others + ) + return _conditional.capacity_given( + x, distribution.levels, distribution.probabilities, demand + ) + + +def _up_at_start(diagram, node, start) -> bool: + """Whether ``node`` is up just before 0 from its checked ``start`` + (see ``_states``; None, new): a component unless it starts down, a + nested RBD as its system, from its components'.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if start is None: + return True + component = diagram.components[node] + if isinstance(component, RepairableRBD): + status = { + inner: _up_at_start(component, inner, start.get(inner)) + for inner in component.components + } + return bool(component.is_system_working(status, "p")) + return bool(start.alive) + + +def _crew_free(diagram) -> "RepairableRBD": + """This system without its limit on repair crews: where every node + the crews serve is held (a conditional run's modules), they tie + nothing together, and the rest is worked out as if they were + unlimited (a nested RBD keeps its own).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if not _crews._crews_limited(diagram): + return diagram + return RepairableRBD(**{**diagram._init_args, "repair_crews": None}) + + +def _conditional_given( + diagram, + modules: list, + state: int, + working: set, + broken: set, + method: str, + x: np.ndarray, + others=None, +) -> "_conditional.Given": + """The system given the modules in one joint ``state`` (bit ``j`` + set while module ``j`` is up), on the grid ``x``: with the modules + held so, and the other nodes from their states ``others`` (new by + default), its expected up time, failures and planned outages before + each time, and its point availability (see + ``_conditional.Given``).""" + up = {node for j, node in enumerate(modules) if state >> j & 1} + down = set(modules) - up + _, _, _, counts, _, _ = _windows._window( + diagram, x, working | up, broken | down, method, state=others + ) + available = diagram.point_availability( + x, working | up, broken | down, method, state=others + ) + # Just before 0, every other node is up, but one started down. + started = _curves._states(diagram, others, working | broken | up | down) + status = { + node: node not in broken | down + and _up_at_start(diagram, node, started.get(node)) + for node in diagram.components + } + return _conditional.Given( + x, + np.asarray(counts["uptime"], dtype=float), + np.asarray(counts["failures"], dtype=float), + np.asarray(counts["planned"], dtype=float), + np.asarray(available, dtype=float), + float(bool(diagram.is_system_working(status, method))), + ) + + +def _conditional_run( + diagram, + t_simulation: float, + working: set, + broken: set, + method: str, + N: int, + seed, + *, + stop: Optional[Callable[..., int]], + antithetic: bool, + n_jobs: Optional[int], + engine: str, + curve_points: Optional[int], + target: str, + state=None, + shard_map: Optional[Callable] = None, + shard_size: Optional[int] = None, + demand: Optional[float] = None, + capacities: bool = False, + control_variate: bool = False, +) -> AvailabilityResult: + """``availability`` (and ``cost``) with ``conditional=True`` (#189, + see ``_conditional`` and ``_ModuleRun``): simulate the modules + alone, in rounds while ``stop`` asks for more, and take every other + node exactly given their joint states. From the components' + ``state``, the modules start from theirs and the others from + theirs. With ``shard_map``, the modules' simulations run as shards + (see ``shards``) wherever it sends them. With ``capacities``, the + system's capacity given the modules' states too, against + ``demand``. With ``control_variate``, the exact twin's stand-ins for + the modules (see ``_twin``) are simulated alongside them, with + common random numbers, and each simulation's expected values given + theirs, whose mean is the twin's exact one, control the run's (see + ``ControlVariate``).""" + run = _ModuleRun( + diagram, + t_simulation, + working, + broken, + method, + _streams.entropy_of(seed), + antithetic=antithetic, + n_jobs=n_jobs, + engine=engine, + curve_points=curve_points, + state=state, + shard_map=shard_map, + shard_size=shard_size, + demand=demand, + capacities=capacities, + control_variate=control_variate, + ) + run.prepare() + first, count = 0, N + while True: + run.simulate(first, count) + first += count + if stop is None: + break + # A run to a tolerance judges the mean asked for (see + # _stopping_rule), controlled if it is: not while the modules + # have not changed state (#215). + count = stop(None, run.judged(target), run.unjudged()) + if not count: + break + return run.result() + + +def _conditioned_run( + diagram, + modules: list, + t_simulation: float, + working: set, + broken: set, + method: str, + N: int, + verbose: bool, + seed, + antithetic: bool, + stop: Optional[Callable[..., int]], + *, + capacity: Optional[_CapacityRecorder], + jobs: Optional[int], + n_jobs: Optional[int], + engine: str, + state, + states: dict, + curve_points: Optional[int], + target: str, + sharded: Optional[tuple], + shard_map: Optional[Callable], + shard_size: Optional[int], +) -> Optional[Tuple["_Tally", ConditionalRun, Optional[_Breakdown]]]: + """A plain run whose means are taken given its ``modules``' + histories (#189), by default where a conditional run applies: the + simulations, in rounds while ``stop`` asks for more (judged by + those means), and of the same simulations each one's expected up + time and cost given the histories of its modules, worked out as a + conditional run works them out (see ``_ModuleRun``): the modules' + simulations are the plain run's, drawn from the same streams. The + simulations' totals, a record holding those values, and the + expected cost's split given the modules (None unpriced, or while + they have not changed state); or None, before simulating anything, + if the rest cannot be worked out exactly given the modules.""" + from . import _event_loop + + entropy = _streams.entropy_of(seed) + demand = None if capacity is None else capacity.demand + run = _ModuleRun( + diagram, + t_simulation, + working, + broken, + method, + entropy, + antithetic=antithetic, + n_jobs=n_jobs, + # The modules' histories are the event loop's, on every engine. + engine=engine if engine in ("python", "numba") else "auto", + curve_points=1, + state=state, + shard_map=shard_map, + shard_size=shard_size, + demand=demand, + capacities=demand is not None, + modules=modules, + ) + try: + run.prepare() + except NotImplementedError: + return None + tally: Optional[_Tally] = None + first, count = 0, N + while True: + part = _event_loop._run( + diagram, + t_simulation, + working, + broken, + method, + count, + verbose, + None, + antithetic, + capacity=capacity, + jobs=jobs, + engine=engine, + entropy=entropy, + first=first, + states=states, + curve_points=curve_points, + sharded=sharded, + ) + if tally is None: + tally = part + else: + tally.merge(part) + run.simulate(first, count) + first += count + if stop is None: + break + # Its means given the modules, or the simulations' own while + # the modules have not changed state (#215). + more = stop(tally, None if run.unjudged() else run.judged(target)) + if not more: + break + count = more + assert tally is not None + record = run.means() + return tally, record, run.split(record) + + +def _twin_exact( + diagram, + twin: "RepairableRBD", + t_simulation: float, + working: set, + broken: set, + method: str, + state, + demand: Optional[float] = None, +) -> _Exacts: + """The exact ``twin``'s (see ``_twin``) mean availability over the + window, if both systems are priced its expected cost, by category + and by component (else None), and against a ``demand`` the fraction + of it delivered (``mission_capacity``; None where that refuses), + building each component's curve once for all (#185).""" + priced = diagram.has_costs and twin.has_costs + delivered = None + # The cost's curves, which count events, serve the availability too. + with twin._sharing_curves(): + expected = ( + twin.expected_cost( + t_simulation, working, broken, method, state=state + ) + if priced + else None + ) + exact = float( + np.ravel( + twin.mission_availability( + t_simulation, working, broken, method, state=state + ) + )[0] + ) + if demand is not None: + try: + delivered = float( + np.ravel( + twin.mission_capacity( + t_simulation, working, broken, state=state + ).delivered_fraction(demand) + )[0] + ) + except (NotImplementedError, Refused): + delivered = None + if expected is None: + return _Exacts(exact, delivered=delivered) + + def first(value) -> float: + return float(np.ravel(value)[0]) + + return _Exacts( + exact, + first(expected.mean), + ( + {key: first(v) for key, v in expected.by_category.items()}, + {node: first(v) for node, v in expected.by_component.items()}, + ), + delivered, + ) + + +def _controlled_run( + diagram, + twin: "RepairableRBD", + exacts: _Exacts, + t_simulation: float, + working: set, + broken: set, + method: str, + N: int, + verbose: bool, + seed, + antithetic: bool, + stop: Optional[Callable[..., int]], + *, + capacity: Optional[_CapacityRecorder], + jobs: Optional[int], + engine: str, + state, + states: dict, + curve_points: Optional[int], + target: str, + itself: bool = False, + sharded: Optional[tuple] = None, +) -> Tuple["_Tally", Tuple[Optional[ControlVariate], ...]]: + """Run this system and its exact ``twin`` (see ``_twin``) with + common random numbers, as ``compare`` does, in rounds while + ``stop`` asks for more, judged by the controlled values; return + this system's totals and the controls of its fractions up and of + its costs (None without costs) by the twin's (see + ``ControlVariate``), ``itself`` if the twin is this system: by its + exact values ``exacts``, its mean availability and its expected + cost (None unpriced; see ``_twin_exact``). A twin that is this + system is not simulated again, so its run can be ``sharded`` (see + ``_run``).""" + from . import _event_loop + + assert sharded is None or itself + twin_states = _curves._simulation_states(twin, state, working | broken) + exact, exact_cost = exacts.availability, exacts.cost + entropy = _streams.entropy_of(seed) + tally: Optional[_Tally] = None + twin_tally: Optional[_Tally] = None + first, count = 0, N + while True: + part = _event_loop._run( + diagram, + t_simulation, + working, + broken, + method, + count, + verbose, + None, + antithetic, + capacity=capacity, + jobs=jobs, + engine=engine, + entropy=entropy, + common=True, + first=first, + states=states, + curve_points=curve_points, + sharded=sharded, + ) + # A twin that is this system, drawn from the same streams, runs + # as this system does, to the last bit: its run is this one. + twin_part = ( + None + if itself + else _event_loop._run( + twin, + t_simulation, + working, + broken, + method, + count, + False, + None, + antithetic, + jobs=jobs, + engine=engine, + entropy=entropy, + common=True, + first=first, + states=twin_states, + curve_points=1, + ) + ) + if tally is None: + tally, twin_tally = part, twin_part + else: + tally.merge(part) + if twin_tally is not None and twin_part is not None: + twin_tally.merge(twin_part) + twins = tally if twin_tally is None else twin_tally + fractions = np.asarray(tally.uptimes, dtype=float) / t_simulation + control = ControlVariate.of( + fractions, + np.asarray(twins.uptimes, dtype=float) / t_simulation, + exact, + antithetic, + itself=itself, + ) + cost_control = None + if exact_cost is not None: + cost_control = ControlVariate.of( + tally.cost_samples, + twins.cost_samples, + exact_cost, + antithetic, + itself=itself, + ) + if stop is None: + break + if target == "cost": + values = ( + tally.cost_samples + if cost_control is None + else cost_control.controlled(tally.cost_samples) + ) + else: + values = control.controlled(fractions) + more = stop(tally, values) + if not more: + break + first, count = first + count, more + delivered_control = None + if itself and exacts.delivered is not None and tally.delivered: + delivered = np.asarray(tally.delivered, dtype=float) + delivered_control = ControlVariate.of( + delivered, delivered, exacts.delivered, antithetic, itself=True + ) + return tally, (control, cost_control, delivered_control) + + +def _simulation_engine( + diagram, + engine: str, + plan: _streams.Plan, + capacity: Optional[_CapacityRecorder], + N: int, + here: bool = True, + states: Optional[dict] = None, +) -> str: + """The engine that runs a simulation: ``"python"``, ``"numba"`` or + one another package adds (see ``availability``'s ``engine``), for a + run from the components' ``states``. Not ``here`` (the simulations + run as shards elsewhere), only whether it can simulate the system + is checked, and ``engine`` is kept, for each shard's worker to + choose by.""" + from repyability.rbd import _compiled, engines + + added = engines.registered() + if engine not in ("auto", "python", "numba") and engine not in added: + names = ", ".join( + repr(name) for name in ["auto", "python", "numba", *added] + ) + raise ValueError(f"engine must be one of {names}, got {engine!r}.") + if engine == "python": + return engine + if engine != "auto": + # What the engine does not simulate is refused first: installing + # it would not help. + reason = _compiled.unsupported( + diagram, plan, capacity, numba=engine == "numba", states=states + ) + if reason is not None: + raise NotImplementedError( + f"The compiled engine does not simulate {reason}: use " + "engine='python', or 'auto', which chooses the engine " + "that can." + ) + if here and engine == "numba": + _compiled.require() + elif here and not added[engine].available(): + raise ImportError( + f"The {engine!r} simulation engine cannot run here." + ) + return _compiled.ready(engine, auto=False) if here else engine + if not here: + return engine + name, _ = _compiled.choice(diagram, plan, capacity, states) + if name is not None and _compiled.worthwhile(plan, N, name): + return _compiled.ready(name, auto=True) + return "python" + + +def _availability_result( + diagram, + tally: "_Tally", + t_simulation: float, + initial_up: bool, + antithetic: bool, + capacity: Optional[_CapacityRecorder] = None, + controls: Tuple[Optional[ControlVariate], ...] = (None, None, None), + conditional: Optional[ConditionalRun] = None, + breakdown: Optional[_Breakdown] = None, +) -> AvailabilityResult: + """The ``AvailabilityResult`` of the replications in ``tally``: its + exact totals rounded, once; with ``controls``, the controls of its + fractions up, its costs and its delivered fractions by an exact twin + (see ``_controlled_run``); with ``conditional``, the record of its means + taken given its modules' histories (see ``_conditioned_run``); with + ``breakdown``, the split of the expected cost its mean is found + with, exact or given the modules (#223), in place of the + simulations' own.""" + N = tally.n + nodes = tally.nodes + tally._fold() + + def rounded(totals) -> list: + return [float(total) for total in totals] + + system_uptime = float(tally.system_uptime) + system_downtime = float(tally.system_downtime) + both_up = rounded(tally.intersection_uptime) + both_down = rounded(tally.intersection_downtime) + # Collect Importance/Criticality measures from the simulation + # reference: https://www.weibull.com/pubs/2004rm_05B_02.pdf + # Operational Criticality Index + oci_down = { + k: _safe_ratio(v, system_downtime) for k, v in zip(nodes, both_down) + } + oci_up = {k: _safe_ratio(v, system_uptime) for k, v in zip(nodes, both_up)} + # Intersection Over Union Importance + iou_up = { + k: _safe_ratio(both, either) + for k, both, either in zip(nodes, both_up, rounded(tally.union_uptime)) + } + iou_down = { + k: _safe_ratio(both, either) + for k, both, either in zip( + nodes, both_down, rounded(tally.union_downtime) + ) + } + FCI, RCI = tally.criticality_counts() + # Failure Criticality Index Importance + fci_sys = failure_criticality_index_per_system_failures( + FCI, tally.system_failures + ) + fci_comp = failure_criticality_index_per_component_failures(FCI) + # Restoration Criticality Index Importance + rci_sys = restoration_criticality_index_by_system( + RCI, tally.system_restorations + ) + rci_comp = restoration_criticality_index_by_component(RCI) + criticalities = Criticalities( + operational_criticality_index=UpDownImportance( + up=oci_up, down=oci_down + ), + iou=UpDownImportance(up=iou_up, down=iou_down), + failure_criticality_index=FailureCriticalityIndex( + per_system_failure=fci_sys, per_component_failure=fci_comp + ), + restoration_criticality_index=RestorationCriticalityIndex( + by_system=rci_sys, by_component=rci_comp + ), + ) + + # The availability from t=0..t_simulation: how many of the N + # simulated systems work after each time at which one changed state + # (and at 0 and t_simulation whether or not any did), over N. + if tally.binned is not None: + # On the grid (#153): how many work at each of its times. + assert tally.edges is not None + time = tally.edges.copy() + working = (N if initial_up else 0) + np.cumsum(tally.binned) + else: + changed_at, deltas = tally.state_changes() + time, working = _working_over_time( + changed_at, + deltas, + t_simulation, + N if initial_up else 0, + ) + system_availability = working / N + + cost_result = _acquisition_only(diagram, N, t_simulation, antithetic) + if diagram.has_costs: + # Per-component means cover each node's repair, replace, + # preventive and own downtime cost. (System downtime is a + # system-level quantity and is not attributed to components.) + by_category, by_component = ( + breakdown + if breakdown is not None + else ( + {k: float(v) / N for k, v in tally.cost_by_category.items()}, + {k: float(v) / N for k, v in tally.cost_by_component.items()}, + ) + ) + cost_result = CostResult( + samples=np.asarray(tally.cost_samples, dtype=float), + t_simulation=t_simulation, + n_simulations=N, + acquisition_cost=diagram.acquisition_cost, + by_category=by_category, + by_component=by_component, + antithetic=antithetic, + control_variate=controls[1], + conditional=conditional, + ) + + capacity_fields: dict = {} + if capacity is not None: + # The mean capacity from t=0..t_simulation: the simulated + # systems' total expected capacity after each time at which one + # changed (unlimited while one can carry an unlimited amount). + if tally.capacity_binned is not None: + # On the grid (#190): the total after each of its times. + assert tally.edges is not None + assert tally.unlimited_binned is not None + times = tally.edges.copy() + totals = np.cumsum(bin_totals(tally.capacity_binned, len(times))) + unlimited = np.cumsum(tally.unlimited_binned) > 0 + else: + at, steps, starts, free_at, counts = tally.capacity_changes() + change = _group_totals(steps, starts) + del steps, starts # (their memory serves what follows) + times, totals, unlimited = _capacity_totals( + at, change, free_at, counts, t_simulation + ) + del at, change + # The mean over the systems, never below 0, and unlimited while + # one can carry an unlimited amount: np.where(unlimited, inf, + # np.maximum(totals / N, 0.0)), worked out in place. + curve = np.divide(totals, N, out=totals) + np.maximum(curve, 0.0, out=curve) + curve[unlimited] = np.inf + capacity_fields = dict( + capacity_timeline=times, + capacity=curve, + capacity_time={ + level: float(tally.capacity_time[level]) + for level in sorted(tally.capacity_time) + }, + demand=capacity.demand, + delivered=( + None + if capacity.demand is None + else np.asarray(tally.delivered, dtype=float) + ), + delivered_control=controls[2] if len(controls) > 2 else None, + ) + + return AvailabilityResult( + timeline=time, + availability=system_availability, + system_uptime=system_uptime, + time_simulated_to=t_simulation, + criticalities=criticalities, + node_uptime=dict(zip(nodes, rounded(tally.node_uptime))), + node_downtime=dict(zip(nodes, rounded(tally.node_downtime))), + system_downtime=system_downtime, + system_failures=tally.system_failures, + system_restorations=tally.system_restorations, + n_simulations=N, + cost=cost_result, + system_planned_outages=tally.system_planned_outages, + uptimes=np.asarray(tally.uptimes, dtype=float), + antithetic=antithetic, + opportunistic_renewals=( + dict(zip(nodes, tally.opportunistic)) + if diagram._maintenance + else None + ), + control_variate=controls[0], + conditional=conditional, + **capacity_fields, + ) + + +def cost( + diagram, + t_simulation: float, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + mc_samples: Optional[int], + verbose: bool, + seed: Optional[int], + *, + tolerance: Optional[float], + confidence: float, + max_samples: Optional[int], + antithetic: bool, + n_jobs: Optional[int], + engine: str, + state, + shard_map: Optional[Callable], + shard_size: Optional[int], + control_variate: Optional[bool], + conditional: Optional[bool], +) -> CostResult: + """See ``RepairableRBD.cost``.""" + _requirements._require_costs(diagram) + N = 10_000 if mc_samples is None else mc_samples + t_simulation = simulation_window(t_simulation) + if not diagram.has_costs: + # An acquisition cost alone: no running cost, exactly. + montecarlo.check_count(N, antithetic, "mc_samples") + alone = _acquisition_only(diagram, N, t_simulation, antithetic) + assert alone is not None + return alone + result = _simulated( + diagram, + t_simulation, + working_nodes, + broken_nodes, + method, + N, + verbose, + seed, + tolerance=tolerance, + confidence=confidence, + max_N=max_samples, + antithetic=antithetic, + n_jobs=n_jobs, + target="cost", + engine=engine, + state=state, + # The cost result has no curve: count the changes, not keep them. + curve_points=1, + shard_map=shard_map, + shard_size=shard_size, + control_variate=control_variate, + conditional=conditional, + ).cost + assert result is not None # something is priced + return result diff --git a/repyability/rbd/_sampling.py b/repyability/rbd/_sampling.py index b401e3f1..45d40649 100644 --- a/repyability/rbd/_sampling.py +++ b/repyability/rbd/_sampling.py @@ -22,6 +22,7 @@ :func:`inverse_sampler` replays that too. """ +import math import weakref from dataclasses import dataclass from typing import Callable, Optional @@ -29,7 +30,13 @@ import numpy as np from surpyval import Parametric -from ._model_utils import is_mixture, lfp_p +from ._degradation import is_degradation +from ._model_utils import ( + MODEL_ERRORS, + is_fixed_probability, + is_mixture, + lfp_p, +) from .helper_classes import PerfectReliability, PerfectUnreliability Sampler = Callable[[np.ndarray], np.ndarray] @@ -78,7 +85,6 @@ def lifetime_sampler(model) -> Optional[RowSampler]: :func:`row_sampler`), or None if they cannot be drawn in a block. A fixed probability, which surpyval draws as an event indicator, is a unit that fails at the start (a lifetime of 0) or never (``inf``).""" - from repyability.rbd._model_utils import is_fixed_probability if is_fixed_probability(model): failure = float(np.ravel(model.ff(1.0))[0]) @@ -120,7 +126,8 @@ def inverse_sampler(model) -> Optional[Sampler]: def _components(model) -> Callable[[str, np.ndarray], np.ndarray]: """``values(name, x)``: a mixture's components' function ``name`` - (``"ff"``, ``"sf"``, ``"df"`` or ``"qf"``) at the 1-d ``x``, one column + (``"ff"``, ``"sf"``, ``"df"``, ``"qf"``, ``"Hf"`` or ``"hf"``) at the + 1-d ``x``, one column a component. In one call over every component where the distribution's functions broadcast over their parameters (checked once, against a call a component), else one call each.""" @@ -148,17 +155,21 @@ def together(name: str, x: np.ndarray) -> np.ndarray: ("ff", points), ("sf", points), ("df", points), + ("Hf", points), + ("hf", points), ) ) - except Exception: + except MODEL_ERRORS: ok = False return together if ok else one_by_one def mixture_quantile(model) -> Sampler: - """A surpyval ``MixtureModel``'s quantile function, which it does not - have (surpyval #651): for each ``u``, the ``x`` with ``F(x) = u``, to - the last bit or two. The mixture's quantile lies between its + """A surpyval ``MixtureModel``'s quantile function, which its own + (from surpyval 0.24) is not yet fit to replace: that loses the upper + tail, inverting ``F`` there, and takes some 200 times as long (surpyval + #821). For each ``u``, the ``x`` with ``F(x) = u``, to the last bit or + two. The mixture's quantile lies between its components' (``F`` is their weighted sum), which bracket it; Newton's steps, with the density, go from the bracket's middle (geometric while its lower end is above 0), and a step that would leave the bracket @@ -228,22 +239,99 @@ def quantile(u): class MixtureLife: - """A surpyval ``MixtureModel`` with the quantile function it lacks - (:func:`mixture_quantile`), for surpyval's ``conditional_gaps`` to - draw a life given an age with: its cumulative hazard and quantile. - (Without the mixture's ``p``, its EM responsibilities, which that - function would take for a limited failure population's, surpyval - #626.) :meth:`of` keeps one a mixture.""" + """A surpyval ``MixtureModel`` with :func:`mixture_quantile` for its + quantile function (surpyval #821), for surpyval's ``conditional_gaps`` + to draw a life given an age with: its cumulative hazard and quantile. + :meth:`of` keeps one a mixture.""" _made: "weakref.WeakKeyDictionary" = weakref.WeakKeyDictionary() def __init__(self, model): self._model = model self.qf = mixture_quantile(model) + self._values = _components(model) + self._log_w = np.log(np.ravel(np.asarray(model.w, dtype=float))) + self._w = np.exp(self._log_w) + self._start: Optional[tuple] = None def Hf(self, x): return self._model.Hf(x) + def _cumulative(self, x: np.ndarray) -> tuple: + """The mixture's cumulative hazard and hazard at the 1-d ``x``, + from its components' (``H = -log sum w_i exp(-H_i)``, and ``h`` + the components' hazards weighed by their shares of the survivors, + ``w_i S_i``): from the share that has failed while it is under a + half, where the cumulative hazard is small, else in logs, so that + both tails keep their precision.""" + with np.errstate(all="ignore"): + each = self._values("Hf", x) + rates = self._values("hf", x) + logs = self._log_w - each + top = np.max(logs, axis=1, keepdims=True) + alive = np.exp(logs - top) + total = np.sum(alive, axis=1) + failed = -np.expm1(-each) @ self._w + H = np.where( + failed < 0.5, + -np.log1p(-failed), + -(top[:, 0] + np.log(total)), + ) + h = np.sum(alive * rates, axis=1) / total + return H, h + + def aged(self, age: float, u: float) -> float: + """The life left at virtual age ``age``, from the uniform ``u``: the + ``x`` with ``H(age + x) = H(age) - log(u)``, as surpyval's + ``conditional_gaps`` draws it (#295), to the last bit or two. + Newton's steps on ``H``, with its hazard, from a table of ``H`` + made once a model, inside a bracket that a step leaving it halves + (about 3 evaluations a draw, where ``conditional_gaps`` takes the + quantile's or some 150 of ``H`` deep in the tail).""" + if not u > 0.0: + return math.inf + H_age = float(self._cumulative(np.array([age]))[0][0]) if age else 0.0 + if math.isinf(H_age): + return 0.0 # no survival past its age: it fails at once + target = H_age - math.log(u) + if not math.isfinite(target): + return math.inf + if target <= H_age: + return 0.0 + lo, hi = age, math.inf + x = max(self._first_guess(target), age) + for _ in range(_STEPS): + H, h = (float(v[0]) for v in self._cumulative(np.array([x]))) + if H < target: + lo = x + else: + hi = x + step = (H - target) / h if h > 0.0 else math.nan + if abs(step) <= _SETTLED * x or H == target: + return max(x - step - age, 0.0) + new = x - step + if not lo < new < hi: + # Out of the bracket: its middle, or further out. + new = 0.5 * (lo + hi) if math.isfinite(hi) else 2.0 * x + 1.0 + x = new + return max(x - age, 0.0) + + def _first_guess(self, target: float) -> float: + """Where ``H`` reaches ``target``, read off a table of ``H`` on a + log grid over the components' lives (made once a model).""" + if self._start is None: + with np.errstate(all="ignore"): + scale = float(np.max(self._values("qf", np.array([0.5])))) + grid = scale * np.logspace(-9.0, 9.0, 2048) + H, _ = self._cumulative(grid) + kept = np.isfinite(H) & (H > 0.0) + kept[kept] &= np.diff(np.r_[-np.inf, H[kept]]) > 0.0 + self._start = (np.log(H[kept]), np.log(grid[kept])) + log_H, log_x = self._start + if not log_H.size: + return 0.0 + return math.exp(np.interp(math.log(target), log_H, log_x)) + @classmethod def of(cls, model) -> "MixtureLife": """The one for ``model``, made again if its parameters or weights @@ -273,16 +361,8 @@ def stream_sampler(model) -> Optional[Sampler]: sampler = inverse_sampler(model) if sampler is None and is_mixture(model): return mixture_quantile(model) + if sampler is None and is_degradation(model): + # A degradation process's first-passage time from its starting + # level (#271), by its own quantile function. + return lambda u: np.asarray(model.qf(u), dtype=float) return sampler - - -def draw_rows(samplers: list[Sampler], size: int) -> list[np.ndarray]: - """``size`` rounds of one draw from each sampler, in order, as a list of - ``size``-long arrays (one per sampler). - - The uniforms are taken row by row -- round 0's draws first -- which is - the order a loop over rounds making one ``random(1)`` call per model - would consume them in. - """ - u = np.random.random_sample((size, len(samplers))) - return [column(u, j, sampler) for j, sampler in enumerate(samplers)] diff --git a/repyability/rbd/_sensitivity.py b/repyability/rbd/_sensitivity.py index ac6dd9b8..30d7dca8 100644 --- a/repyability/rbd/_sensitivity.py +++ b/repyability/rbd/_sensitivity.py @@ -38,6 +38,7 @@ from typing import ( Any, Callable, + Collection, Dict, Hashable, List, @@ -48,10 +49,11 @@ import numpy as np +from repyability.rbd import _curves, _long_run from repyability.utils.checks import is_number, is_whole, number_or_nan from . import _importance_time -from ._model_utils import parametric_spec +from ._model_utils import CovariateSpec, lever_spec #: What the sensitivities can be of. QUANTITIES = ("availability", "cost_rate") @@ -70,16 +72,15 @@ #: The range of a probability, a fraction or a share. _UNIT = (0.0, 1.0) -#: The range of a positive number, such as an interval. -_POSITIVE = (0.0, math.inf) class _Lever(NamedTuple): """One lever: whose (a node, a common-cause group's members, or None for the system), its name, its value, whether it is discrete (one more), the constructor's arguments with it at a value (``build``), - the node whose curve alone it moves (None if it moves more), and the - range of its values (see ``Lever``).""" + the node whose curve alone it moves (None if it moves more), the + range of its values (see ``Lever``), and what a proportional change + moves it by, per unit of the change (see ``_proportional_scale``).""" key: Hashable name: str @@ -88,15 +89,13 @@ class _Lever(NamedTuple): build: Callable[[float], dict] node: Optional[Hashable] bounds: Tuple[float, float] + scale: Optional[float] -def _model_bounds(cls, n: int) -> List[Tuple[float, float]]: - """The range of each of a surpyval model's ``n`` parameters, as its +def _model_bounds(given) -> List[Tuple[float, float]]: + """The range of each of a surpyval model's parameters, as its distribution bounds them (``from_params`` refuses a value outside), ``-inf`` and ``inf`` where it does not.""" - given = getattr(cls, "bounds", None) - if given is None or len(given) != n: - return [(-math.inf, math.inf)] * n return [ ( -math.inf if low is None else float(low), @@ -107,39 +106,64 @@ def _model_bounds(cls, n: int) -> List[Tuple[float, float]]: def _model_levers(prefix: str, model, rebuild) -> List[tuple]: - """``(name, value, set, bounds)`` for each parameter of a parametric - ``model``, named ``prefix.`` (or the parameter's own name, - without a prefix), ``set(v)`` giving ``rebuild`` of the model with it - at ``v``.""" - spec = parametric_spec(model) + """``(name, value, set, bounds, scale)`` for each parameter of a + parametric ``model``, named ``prefix.`` (or the parameter's + own name, without a prefix), ``set(v)`` giving ``rebuild`` of the model + with it at ``v``: a parametric model's parameters, or a regression + node's covariates (#272); ``scale`` as ``_proportional_scale``.""" + spec = lever_spec(model) if spec is None: return [] - cls, params, names, extras = spec - ranges = _model_bounds(cls, len(params)) + ranges = _model_bounds(spec.bounds) out = [] - for j, name in enumerate(names): + for j, name in enumerate(spec.names): def moved(v, j=j): - trial = list(params) + trial = list(spec.params) trial[j] = v - return rebuild(cls.from_params(trial, **extras)) + return rebuild(spec.build(trial)) named = f"{prefix}.{name}" if prefix else name - out.append((named, float(params[j]), moved, ranges[j])) + scale = _proportional_scale(spec, name, spec.params[j]) + out.append((named, spec.params[j], moved, ranges[j], scale)) return out +#: Parameters on a log scale, by distribution: the log of a time (a +#: LogNormal's ``mu``, Galton its other name) or of a rate (a Cox-Lewis +#: process's ``alpha``). Changing the unit of time adds to them, so they +#: have no zero to be a fraction of. +_LOG_SCALE = {"LogNormal": {"mu"}, "Galton": {"mu"}, "Cox-Lewis": {"alpha"}} + + +def _proportional_scale(spec, name: str, value: float) -> Optional[float]: + """What a proportional change of a model's parameter moves it by, per + unit of the change, so that it does not depend on the units of time + or of a covariate: its value (a scale, a rate, a shape, a time); 1 for + a parameter on a log scale (``_LOG_SCALE``), whose change moves the + time it is the log of by that fraction; None for a regression node's + covariate, whose zero is its unit's (0 degrees C is not 0 K), so that a + proportional change of it means nothing.""" + + if isinstance(spec, CovariateSpec): + return None + if name in _LOG_SCALE.get(getattr(spec.cls, "name", ""), ()): + return 1.0 + return float(value) + + def _option_bounds(key: str, option: str, options: dict) -> tuple: """The range of a maintenance option (as the constructor checks it): an interval past its offset or opportunity, an offset or opportunity - within the interval, a coverage or threshold a probability.""" + within the interval (any age under a calendar policy, which has + none), a coverage or threshold a probability.""" if option == "interval": floor = number_or_nan( options.get("offset" if key == "inspection" else "opportunity") ) return (0.0 if math.isnan(floor) else floor, math.inf) if option in ("offset", "opportunity"): - return (0.0, float(options["interval"])) + return (0.0, float(options.get("interval", math.inf))) return _UNIT @@ -151,10 +175,10 @@ def _spec_levers(spec: dict) -> List[tuple]: model = spec.get(key) if model is None or isinstance(model, str): continue - for name, value, moved, bounds in _model_levers( + for name, value, moved, bounds, scale in _model_levers( key, model, lambda m, key=key: {**spec, key: m} ): - out.append((name, value, moved, False, bounds)) + out.append((name, value, moved, False, bounds, scale)) for key, numeric in ( ("preventive", _PREVENTIVE), ("inspection", _INSPECTION), @@ -184,11 +208,12 @@ def moved(v, key=key, option=option, options=options): moved, False, _option_bounds(key, option, options), + float(value), ) ) duration = options.get("duration") if duration is not None and not isinstance(duration, str): - for name, value, moved, bounds in _model_levers( + for name, value, moved, bounds, scale in _model_levers( f"{key}.duration", duration, lambda m, key=key, options=options: { @@ -196,7 +221,7 @@ def moved(v, key=key, option=option, options=options): key: {**options, "duration": m}, }, ): - out.append((name, value, moved, False, bounds)) + out.append((name, value, moved, False, bounds, scale)) standby = spec.get("standby") if isinstance(standby, dict): for option in ("dormancy_factor", "switching_probability"): @@ -212,6 +237,7 @@ def moved(v, option=option): moved, False, _UNIT, + float(standby[option]), ) ) @@ -227,6 +253,7 @@ def more(v): more, True, spare, + float(standby.get("units", 2)), ) ) repair = spec.get("repair") @@ -235,7 +262,8 @@ def more(v): def moved_q(v): return {**spec, "repair": {**repair, "q": v}} - out.append(("repair.q", float(repair["q"]), moved_q, False, _UNIT)) + q = float(repair["q"]) + out.append(("repair.q", q, moved_q, False, _UNIT, q)) return out @@ -270,31 +298,92 @@ def _levers(rbd) -> List[_Lever]: spec = _as_spec(component) if spec is None: continue - for name, value, moved, discrete, bounds in _spec_levers(spec): + for name, value, moved, discrete, bounds, scale in _spec_levers(spec): def build(v, node=node, moved=moved): return {**args, "components": {**components, node: moved(v)}} out.append( - _Lever(node, name, value, discrete, build, node, bounds) + _Lever(node, name, value, discrete, build, node, bounds, scale) ) for g, group in enumerate(groups): names = tuple(group.members) - spec = _as_spec(components[names[0]]) - if spec is not None: - for name, value, moved, discrete, bounds in _spec_levers(spec): + # Each member's own levers, by name: a lever the members hold at + # one value is the group's, moved in each member's own spec (so + # that what else differs between them, an offset, is kept); a + # member's test offset, its own place on the calendar, and a + # lever whose values differ between them, are each member's own, + # listed after the group's (the sensitivities report by key). + own: Dict[Hashable, Dict[str, tuple]] = {} + members_own: List[_Lever] = [] + for member in names: + spec = _as_spec(components[member]) + own[member] = ( + {} + if spec is None + else {lever[0]: lever for lever in _spec_levers(spec)} + ) + order = dict.fromkeys(name for m in names for name in own[m]) + for name in order: + each = {m: own[m][name] for m in names if name in own[m]} + values = {lever[1] for lever in each.values()} + if ( + name == "inspection.offset" + or len(each) < len(names) + or len(values) > 1 + ): + for member, ( + _, + value, + moved, + discrete, + bounds, + scale, + ) in each.items(): + + def build_member(v, member=member, moved=moved): + return { + **args, + "components": {**components, member: moved(v)}, + } + + members_own.append( + _Lever( + member, + name, + value, + discrete, + build_member, + None, + bounds, + scale, + ) + ) + continue - def build_group(v, moved=moved, names=names): - changed = dict(components) - for member in names: - changed[member] = moved(v) - return {**args, "components": changed} + def build_group(v, each=each): + changed = dict(components) + for member, (_, _, moved, _, _, _) in each.items(): + changed[member] = moved(v) + return {**args, "components": changed} - out.append( - _Lever( - names, name, value, discrete, build_group, None, bounds - ) + _, value, _, discrete, _, scale = each[names[0]] + bounds = ( + max(lever[4][0] for lever in each.values()), + min(lever[4][1] for lever in each.values()), + ) + out.append( + _Lever( + names, + name, + value, + discrete, + build_group, + None, + bounds, + scale, ) + ) for letter, value in ccf_parameters(group.model).items(): def build_ccf(v, g=g, letter=letter, group=group): @@ -312,8 +401,10 @@ def build_ccf(v, g=g, letter=letter, group=group): build_ccf, None, _UNIT, + value, ) ) + out += members_own crews = args.get("repair_crews") if crews is not None: @@ -329,6 +420,7 @@ def more_crews(v): more_crews, None, (1.0, math.inf), + float(crews), ) ) return out @@ -340,22 +432,53 @@ def _nonrepairable_levers(rbd) -> List[_Lever]: by their own names, and a common-cause group's (its members' one model's, then its own, ``ccf_beta``, ...) under the tuple of its members, at its first member's place, in the order of the nodes.""" + return _model_levers_of( + rbd._init_args, "reliabilities", rbd.reliabilities, rbd.ccf_groups + ) + + +def _fault_tree_levers(tree) -> List[_Lever]: + """Every lever of a fault ``tree``, as its ``parameter_sensitivity`` + reports them (as ``_nonrepairable_levers`` lists a diagram's): each + event's model's parameters, an event given as a probability having + the one, ``p``, and a common-cause group's under the tuple of its + members; a moved model goes back into the tree as the event was given + (``_as_event``: a probability stays one).""" + return _model_levers_of( + tree._init_args, + "events", + {event: tree._event_model(event) for event in tree.events}, + tree.ccf_groups, + tree._as_event, + ) + + +def _model_levers_of( + args: dict, field: str, models: Mapping, groups, entry=lambda key, m: m +) -> List[_Lever]: + """The levers of a structure built from ``args``, its constructor's + arguments, whose ``args[field]`` holds its nodes' ``models`` (the + diagram's ``reliabilities``, or a tree's events as models): each + node's model's parameters, by their own names, and a common-cause + group's (its members' one model's, then its own, ``ccf_beta``, ...) + under the tuple of its members, at its first member's place, in the + order of the nodes. ``entry(key, model)`` is a moved model as the + constructor takes it.""" from .ccf import CCFGroup from .ccf import parameters as ccf_parameters from .ccf import with_parameters as with_ccf_parameters - args = rbd._init_args - models = dict(args["reliabilities"]) - groups = list(rbd.ccf_groups) + given = dict(args[field]) + groups = list(groups) group_of = {m: g for g, group in enumerate(groups) for m in group.members} out: List[_Lever] = [] - for node, model in rbd.reliabilities.items(): + for node, model in models.items(): g = group_of.get(node) if g is not None and node != groups[g].members[0]: continue members = (node,) if g is None else tuple(groups[g].members) key = node if g is None else members - for name, value, moved, bounds in _model_levers( + for name, value, moved, bounds, scale in _model_levers( "", model, lambda m: m ): @@ -363,9 +486,9 @@ def build(v, moved=moved, members=members): changed = moved(v) return { **args, - "reliabilities": { - **models, - **{member: changed for member in members}, + field: { + **given, + **{m: entry(m, changed) for m in members}, }, } @@ -378,6 +501,7 @@ def build(v, moved=moved, members=members): build, node if g is None else None, bounds, + scale, ) ) if g is None: @@ -400,15 +524,20 @@ def build_ccf(v, g=g, letter=letter, group=group): build_ccf, None, _UNIT, + float(value), ) ) return out def _levers_of(rbd) -> List[_Lever]: - """``rbd``'s levers, repairable or not.""" + """``rbd``'s levers, repairable or not, or a fault tree's.""" + from repyability.fault_tree import FaultTree + from .non_repairable_rbd import NonRepairableRBD + if isinstance(rbd, FaultTree): + return _fault_tree_levers(rbd) if isinstance(rbd, NonRepairableRBD): return _nonrepairable_levers(rbd) return _levers(rbd) @@ -493,13 +622,22 @@ def _difference(at: Callable, lever: _Lever, rel_step: float, base): return np.full_like(base, np.nan) +#: What a diagram keeps that a diagram rebuilt with a lever moved reuses +#: for the components it shares (see ``_rebuilt``): the cycles and tested +#: units keyed by their models' identities and their own schedules. +_REUSED = ("age_cycles", "block_cycles", "tested_lives", "tested_units") + + def _rebuilt(rbd, lever: _Lever, value: float): """The diagram with the lever at ``value``, or None if that is not a - valid value.""" + valid value. It reuses what ``rbd`` has worked out for the + components the lever leaves as they are (``_REUSED``).""" try: - return type(rbd)(**lever.build(value)) + changed = type(rbd)(**lever.build(value)) except (ValueError, TypeError): return None + changed.__dict__["_kept"] = rbd._cache.copied(*_REUSED) + return changed def _system_at(rbd, lever: _Lever, quantities, asked: _Asked): @@ -535,11 +673,11 @@ def _calendar_lever(rbd, lever: _Lever) -> bool: block replacement's or a test's interval) that others share: the long-run values jump as it leaves their common calendar, and are averaged over a common period that it would make too long to follow - (see ``repairable_rbd._common_period``).""" + (see ``_common._common_period``).""" if lever.name not in ("preventive.interval", "inspection.interval"): return False members = set(lever.key) if isinstance(lever.key, tuple) else {lever.key} - calendar = set(rbd._block_nodes()) | set(rbd._inspection) + calendar = set(_long_run._block_nodes(rbd)) | set(rbd._inspection) return bool(members & calendar) and bool(calendar - members) @@ -573,7 +711,8 @@ def one(v): ) part = importance * float(alone.mean_unavailability()) if quantity == "cost_rate": - part = float(alone.expected_cost_rate()) + rate * part + own = alone.expected_cost_rate() if alone.has_costs else 0.0 + part = float(own) + rate * part except (ValueError, TypeError, NotImplementedError): return None return np.atleast_1d(part) @@ -608,12 +747,15 @@ def __init__(self, rbd, asked: _Asked): ) def _window_points(self): - from .repairable_rbd import _MISSION_POINTS + from ._curves import _MISSION_POINTS rbd, asked = self.rbd, self.asked + assert asked.window is not None forced = asked.working | asked.broken - states = rbd._states(asked.state, forced) - curves = rbd._availability_curves(asked.window, forced, state=states) + states = _curves._states(rbd, asked.state, forced) + curves = _curves._availability_curves( + rbd, asked.window, forced, state=states + ) def integrands(x): p, q = _importance_time._independent( @@ -657,6 +799,7 @@ def one(v): moved = -importance * curve if self.weights is None: return moved.reshape(self.shape) + assert self.asked.window is not None return np.atleast_1d( (self.weights @ moved) / float(self.asked.window) ) @@ -677,7 +820,7 @@ def sensitivity( unit_costs: Optional[dict], ) -> dict: """``RepairableRBD.parameter_sensitivity``: see there.""" - from .repairable_rbd import _check_times + from repyability.utils.checks import nonnegative_times quantities = (of,) if isinstance(of, str) else tuple(of) if not quantities or any(q not in QUANTITIES for q in quantities): @@ -717,7 +860,7 @@ def sensitivity( f"rel_step must be a number in (0, 0.5), got {rel_step!r}." ) if x is not None: - _check_times(x) + nonnegative_times(x) if window is not None: if ( isinstance(window, bool) @@ -728,9 +871,7 @@ def sensitivity( f"window must be a positive, finite length, got {window!r}." ) window = float(window) - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) - rbd._validate_node_overrides(working, broken) + working, broken = rbd._held_nodes(working_nodes, broken_nodes) held = working | broken asked = _Asked(x, window, state, working, broken) found = _levers(rbd) @@ -764,6 +905,11 @@ def sensitivity( pinned = lever.key in held or ( isinstance(lever.key, tuple) and bool(held & set(lever.key)) ) + if timed and lever.name == "repair.q" and lever.value >= 1.0: + # Minimal repair (q = 1) is the one imperfect repair the + # values over time take: the derivative in q does not exist + # there, so the lever is left out of the report. + continue system = None for quantity in quantities: if pinned: @@ -828,11 +974,14 @@ def _named(lever) -> Tuple[Hashable, str]: ) -def _no_lever(key, name: str, found: dict) -> ValueError: - """The error for a lever the diagram does not have, with the closest - name if one is close (#232): of the levers of ``key``, or else of the - keys.""" +def _no_lever(rbd, key, name: str, found: dict) -> ValueError: + """The error for a lever ``rbd`` does not have, with the closest name + if one is close (#232): of the levers of ``key``, or else of the keys, + which are named by the kind of diagram.""" + from repyability.fault_tree import FaultTree + from .rbd import _close_name + from .repairable_rbd import RepairableRBD names = [n for k, n in found if k == key] if names: @@ -844,10 +993,17 @@ def _no_lever(key, name: str, found: dict) -> ValueError: keys = list(dict.fromkeys(k for k, _ in found)) close = _close_name(key, keys) hint = "" if close is None else f" Did you mean {close!r}?" + if isinstance(rbd, FaultTree): + holders = "event with a model or common-cause group" + elif isinstance(rbd, RepairableRBD): + holders = ( + "node with a model, common-cause group or the repair crews (None)" + ) + else: + holders = "node with a model or common-cause group" return ValueError( - f"{key!r} has no levers: it is no node with a model, common-cause " - f"group or the repair crews (None).{hint} The keys with levers are: " - f"{keys}." + f"{key!r} has no levers: it is no {holders}.{hint} The keys with " + f"levers are: {keys}." ) @@ -867,7 +1023,7 @@ def with_levers(rbd, values) -> Any: (lever.key, lever.name): lever for lever in _levers_of(changed) } if (key, name) not in found: - raise _no_lever(key, name, found) + raise _no_lever(rbd, key, name, found) lever = found[(key, name)] if not is_number(value): raise TypeError( @@ -893,8 +1049,27 @@ def with_levers(rbd, values) -> Any: return changed -#: The names callers used before the levers were public (#244), kept as -#: they were for 0.13; from 0.14 they may change or go. Use ``levers()``, -#: ``Lever`` and ``with_levers``. -Lever = _Lever -levers = _levers +def parameter_sensitivity( + rbd, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + *, + x, + window, + state, + rel_step: Optional[float], + of, + unit_costs: Optional[dict], +) -> dict: + """See ``RepairableRBD.parameter_sensitivity``.""" + return sensitivity( + rbd, + x=x, + window=window, + state=state, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + rel_step=rel_step, + of=of, + unit_costs=unit_costs, + ) diff --git a/repyability/rbd/_spares.py b/repyability/rbd/_spares.py index f2beb8e8..cb8013f0 100644 --- a/repyability/rbd/_spares.py +++ b/repyability/rbd/_spares.py @@ -68,7 +68,21 @@ end, the block replacement there and ``C`` past it; after a block replacement, ``C``; over ``M(T) + 1`` replacements an interval. The life's sums are kept as the bands of grid points that hold their mass, - as a block interval may hold many lives. + as a block interval may hold many lives. A unit dead on arrival (its + life ending at 0 with probability ``q``) put in at a replacement fails + at its instant, and its replacement is put in there too (#322): each + replacement of the units that last, a failure or a block replacement, + brings ``G`` more at its instant, ``P(G = g) = q^g (1 - q)``, + independent of everything else, so the count above is that of the + lasting lives (the life given that it lasts) compounded + (``_compounded``): ``s`` or more if, of the units put in one after + another, the ``s``-th has fewer than that count lasting before it (a + binomial). A typical replacement is one of its instant's, with ``d`` + of them before it as often as ``q^d (1 - q)``, so before a replacement + it is one more lasting replacement compounded, less one. (With repairs + or block replacements that take time, a unit dead on arrival after one + that took time fails on its own, and its replacement goes on as the + recursion below has it: on the grid, where it follows the atoms.) With repairs or block replacements that take time (#160), a unit down at a block time is not replaced there, and an interval need not start @@ -121,14 +135,55 @@ ``l`` tests back from a place ``q`` with probability ``pi_q' P(C_q' = l) / pi_q``, ``q'`` the place ``l`` tests before and ``pi`` the long-run shares of the places. + +``spares_demand`` and ``spares_stock`` here are the bodies of +``RepairableRBD``'s methods of those names, which call them. """ +import dataclasses import functools import math -from typing import Any, Callable, List, NamedTuple, Optional, Tuple +from collections.abc import Mapping +from typing import ( + Any, + Callable, + Collection, + Dict, + Hashable, + Iterable, + List, + NamedTuple, + Optional, + Tuple, +) import numpy as np +from repyability.rbd import _crews +from repyability.rbd import _montecarlo as montecarlo +from repyability.rbd import _requirements +from repyability.rbd._common import ( + _cdf, + _fleet, + _fractions, + _horizon, + _safe_mean, + _sf_values, + _summed, + _whole, +) +from repyability.rbd._hidden_life import ( + TestedLife, +) +from repyability.rbd._hidden_tests import check as check_tested +from repyability.rbd.results import ( + SparesDemand, + SparesStock, +) +from repyability.rbd.routes import Refused +from repyability.utils.checks import ( + one_of, +) from repyability.utils.vectors import dot #: How closely the counts' probabilities are computed. @@ -191,33 +246,64 @@ class Tested(NamedTuple): position: int = 0 -class _Dist(NamedTuple): +class _Dist: """A distribution on the grid, rounded one way: its continuous part as - masses on the grid points, and its atoms (on grid points), exactly.""" + masses on the grid points, and its atoms (on grid points), exactly. + Its Fourier transforms, for the convolutions of ``_sum``, are kept + once taken (``spectrum``): one convolved with many (a cycle with + each count's sum so far) is transformed once. Its masses are not + changed once a transform is taken.""" + + __slots__ = ("rest", "atoms", "_spectra") - rest: np.ndarray - atoms: np.ndarray + def __init__(self, rest: np.ndarray, atoms: np.ndarray): + self.rest = rest + self.atoms = atoms + self._spectra: Dict[Tuple[bool, int], np.ndarray] = {} - def __add__(self, other): # type: ignore[override] + def __add__(self, other: "_Dist") -> "_Dist": return _Dist(self.rest + other.rest, self.atoms + other.atoms) def scaled(self, factor: float) -> "_Dist": return _Dist(factor * self.rest, factor * self.atoms) - -def _fft(a: np.ndarray, b: np.ndarray) -> np.ndarray: - from scipy.signal import fftconvolve - - if not a.any() or not b.any(): - return np.zeros(len(a)) - return np.maximum(fftconvolve(a, b)[: len(a)], 0.0) + def spectrum(self, atoms: bool, size: int) -> np.ndarray: + """The real FFT of length ``size`` of its atoms, or of all its + masses, taken once.""" + from scipy.fft import rfft + + found = self._spectra.get((atoms, size)) + if found is None: + values = self.atoms if atoms else self.rest + self.atoms + found = self._spectra[(atoms, size)] = rfft(values, size) + return found + + +def _fft(a: _Dist, b: _Dist, atoms: bool) -> np.ndarray: + """The convolution of the masses of ``a`` and ``b`` (their atoms, or + all their masses), on the grid: by FFT, as ``scipy.signal.fftconvolve`` + does it, with each distribution's transform taken once.""" + from scipy.fft import irfft, next_fast_len + + n = len(a.rest) + if atoms: + empty = not (a.atoms.any() and b.atoms.any()) + else: + empty = not ( + (a.rest.any() or a.atoms.any()) and (b.rest.any() or b.atoms.any()) + ) + if empty: + return np.zeros(n) + size = next_fast_len(2 * n - 1, True) + product = a.spectrum(atoms, size) * b.spectrum(atoms, size) + return np.maximum(irfft(product, size)[:n], 0.0) def _sum(a: _Dist, b: _Dist) -> _Dist: """The distribution of the sum of two independent ones: its atoms are sums of atoms, the rest continuous.""" - total = _fft(a.rest + a.atoms, b.rest + b.atoms) - atoms = _fft(a.atoms, b.atoms) + total = _fft(a, b, False) + atoms = _fft(a, b, True) return _Dist(np.maximum(total - atoms, 0.0), atoms) @@ -456,7 +542,10 @@ def _block_step( (an atom there). Rounded up, a failure on the block time itself came before it; rounded down, after it. ``before`` holds units put into service on a block time but just before it (rounded up onto it), which - are replaced there unless dead on arrival.""" + are replaced there unless dead on arrival, a failure just before it + too (rounded up onto it). The failures that are atoms are those at + the instant of a start that is one: of units dead on arrival put in + at a block time, after its replacement (#322).""" size = len(start.rest) failed = _Dist(np.zeros(size), np.zeros(size)) blocks = np.zeros(size) @@ -468,15 +557,14 @@ def _block_step( for lo in range(0, size, period): hi = lo + period if hi < size and before[hi]: - failed.atoms[hi] += before[hi] * dead + failed.rest[hi] += before[hi] * dead blocks[hi] += before[hi] * (1.0 - dead) rest, atoms = start.rest[lo:hi], start.atoms[lo:hi] if not (rest.any() or atoms.any()): continue - part = _Dist( - np.pad(rest, (0, width - len(rest))), - np.pad(atoms, (0, width - len(atoms))), - ) + part = _Dist(np.zeros(width), np.zeros(width)) + part.rest[: len(rest)] = rest + part.atoms[: len(atoms)] = atoms out = _sum(part, cut) keep = width if up else width - 1 stop = min(lo + keep, size) @@ -501,8 +589,10 @@ def _block_starts( """The next units' starts: after each failure's repair and each block replacement's. Rounded up, a start on a block time is before it (its true time is at most the grid's), unless it is the unit put in there - by that block's replacement in no time (or, split, in under a step): - those are kept apart.""" + by that block's replacement in no time (or, split, in under a step), + or by the repair in no time (or in under a step) of a failure at the + block time itself, the atoms (a unit dead on arrival after that + block's replacement, #322): those are kept apart, as atoms.""" start = _sum(failed, repair) + _sum(blocks, maintenance) size = len(start.rest) before = np.zeros(size) @@ -510,12 +600,12 @@ def _block_starts( return start, before on = np.arange(period, size, period) total = start.rest[on] + start.atoms[on] - renewed = blocks.atoms[on] * float( - maintenance.atoms[0] + maintenance.rest[0] - ) - before[on] = np.maximum(total - renewed, 0.0) + after = failed.atoms[on] * float( + repair.atoms[0] + repair.rest[0] + ) + blocks.atoms[on] * float(maintenance.atoms[0] + maintenance.rest[0]) + before[on] = np.maximum(total - after, 0.0) start.rest[on] = 0.0 - start.atoms[on] = np.minimum(renewed, total) + start.atoms[on] = np.minimum(after, total) return start, before @@ -964,7 +1054,7 @@ def _assembled( def carries_over(model: Block) -> bool: """Whether a block-replaced unit's repairs or block replacements take - time, so that an interval need not start with a new unit (#160).""" + time, so that an interval need not start with a new unit.""" zero = np.zeros(1) if float(np.ravel(model.repair(zero))[0]) < 1.0: return True @@ -973,6 +1063,55 @@ def carries_over(model: Block) -> bool: ) +def _alive(model: Block) -> Tuple[Block, float]: + """``model`` with its life given that it lasts, and the share of its + units dead on arrival (#322): with repairs and block replacements in + no time, a replacement is followed at its instant by the failure of + a unit dead on arrival, whose replacement is too, and so on; the + replacements of the units that last are the lasting lives' alone.""" + dead = float(np.ravel(model.life(np.zeros(1)))[0]) + if dead <= 0.0: + return model, 0.0 + life = model.life + + def alive(x: np.ndarray) -> np.ndarray: + return np.maximum(life(x) - dead, 0.0) / (1.0 - dead) + + return model._replace(life=alive), dead + + +def _compounded(tails: np.ndarray, dead: float, kind: str) -> np.ndarray: + """``P(N >= s)``, ``s = 1, 2, ...``, for the replacements ``N`` when + each of the lasting units' replacements, counted by ``tails`` (their + ``P(B >= s)``), brings ``G`` more at its instant, ``P(G = g) = dead^g + (1 - dead)``: the units dead on arrival put in one after another + there. ``N`` is ``s`` or more if the ``s``-th of the units put in, in + ``s - 1`` Bernoulli trials with the chance ``1 - dead`` of lasting, + has fewer than ``B`` lasting before it (#322). Before a replacement + (``"arrival"``), the ones before it at its instant count too: as + often as it has ``d`` of its instant's before it, ``P(d) = dead^d (1 - + dead)``, so ``N + 1`` is ``B + 1`` replacements compounded.""" + if dead <= 0.0: + return tails + from scipy.stats import binom + + chances = -np.diff(np.concatenate(([1.0], tails, [0.0]))) + counts = np.arange(len(chances)) + more = 1 if kind == "arrival" else 0 + out: List[float] = [] + for s in range(1, MAX_COUNT + 1): + tail = float( + chances @ binom.cdf(counts - 1 + more, s - 1 + more, 1.0 - dead) + ) + out.append(min(max(tail, 0.0), 1.0)) + if tail < _TAIL: + return np.array(out) + raise NotImplementedError( + f"More than {MAX_COUNT} replacements are likely in the time: count " + "them by simulation instead." + ) + + def _rows_fft(a: np.ndarray, b: np.ndarray, size: int = 0) -> np.ndarray: """Each row of ``a`` convolved with ``b`` and cut to ``size`` (the rows' length by default), clipped at 0 (as ``_fft``), on every core. @@ -1029,7 +1168,7 @@ def _rows_step( for lo in range(0, size, period): hi = lo + period if hi < size: - failed_atoms[:, hi] += before[:, hi] * dead + failed_rest[:, hi] += before[:, hi] * dead blocks[:, hi] += before[:, hi] * (1.0 - dead) part_rest, part_atoms = rest[:, lo:hi], atoms[:, lo:hi] if not (part_rest.any() or part_atoms.any()): @@ -1086,10 +1225,12 @@ def _rows_starts( return start_rest, start_atoms, before on = np.arange(period, size, period) total = start_rest[:, on] + start_atoms[:, on] - renewed = blocks[:, on] * float(maintenance.atoms[0] + maintenance.rest[0]) - before[:, on] = np.maximum(total - renewed, 0.0) + after = failed_atoms[:, on] * float( + repair.atoms[0] + repair.rest[0] + ) + blocks[:, on] * float(maintenance.atoms[0] + maintenance.rest[0]) + before[:, on] = np.maximum(total - after, 0.0) start_rest[:, on] = 0.0 - start_atoms[:, on] = np.minimum(renewed, total) + start_atoms[:, on] = np.minimum(after, total) return start_rest, start_atoms, before @@ -1636,7 +1777,10 @@ def tails_on(steps: int) -> np.ndarray: if isinstance(model, Block): if kind == "new": return _block_tails(model, end, steps) - return _block_lead_tails(model, end, kind, steps) + alive, dead = _alive(model) + return _compounded( + _block_lead_tails(alive, end, kind, steps), dead, kind + ) return _count_tails(model, end, kind, steps) steps = FIRST_STEPS @@ -1656,3 +1800,464 @@ def tails_on(steps: int) -> np.ndarray: break tails = np.minimum.accumulate(np.clip(previous, 0.0, 1.0)) return -np.diff(np.concatenate(([1.0], tails, [0.0]))) + + +#: What the spares counts say when a component can wait for a repair crew. +_SPARES_CREWS = ( + "the spares counts assume", + "Count them by simulation: spares_demand(method='simulate').", +) +#: What the stock levels say then: they have no simulation of their own to +#: point to (#232). +_STOCK_CREWS = ( + "the stock levels assume", + "spares_stock has no simulation to fall back on; the diagram without " + "the limit (repair_crews=None) gives the stock when a crew is always " + "free.", +) + + +def _spares_nodes(rbd, nodes, given: str = "nodes") -> list: + """The components whose spares are counted: ``nodes`` (given in + ``given``) checked, or every component; refused when none is given + and one is a nested RBD, whose components' spares are not counted + here (its own ``spares_demand`` counts them).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + if nodes is None: + nested = [ + node + for node, component in rbd.components.items() + if isinstance(component, RepairableRBD) + ] + if nested: + raise Refused( + f"Node {nested[0]!r} is a nested RBD, whose components' " + "spares are not counted here (its own spares_demand and " + "spares_stock count them): give the components counted, " + f"nodes={[n for n in rbd.components if n not in nested]!r}." + ) + return list(rbd.components) + chosen = list(dict.fromkeys(nodes)) + for node in chosen: + _requirements._require_component(rbd, node, given) + if isinstance(rbd.components[node], RepairableRBD): + raise ValueError( + f"Node {node!r} is a nested RBD: its spares are its " + "components', which its own spares_demand counts." + ) + return chosen + + +def _spares_parts(rbd, parts) -> Dict[Hashable, Tuple[Hashable, ...]]: + """The parts whose spares are pooled (#183), ``{part: members}``, + checked: each a collection of components (no nested RBD), a node + in one part at most, a part's name no component's, and no two + members in one common-cause group, whose shared causes replace + them together.""" + if parts is None: + return {} + if not isinstance(parts, Mapping) or not parts: + raise ValueError( + "parts must be a dict of part name: the components that use " + f"it, e.g. {{'seal': ['seal1', 'seal2']}}; got {parts!r}." + ) + out: Dict[Hashable, Tuple[Hashable, ...]] = {} + owner: Dict[Hashable, Hashable] = {} + for part, members in parts.items(): + if part in rbd.components: + raise ValueError( + f"Part {part!r} has the name of a component: name the " + "part apart, as both can be counted." + ) + if isinstance(members, (str, bytes)) or not isinstance( + members, Iterable + ): + raise ValueError( + f"Part {part!r}: give the components that use it as a " + f"list, got {members!r}." + ) + listed = list(members) + twice = sorted( + {repr(n) for n in listed if listed.count(n) > 1}, key=str + ) + if twice: + raise ValueError( + f"Part {part!r} lists {', '.join(twice)} more than once: " + "name each component that uses it once." + ) + chosen = _spares_nodes(rbd, listed, f"part {part!r}") + if not chosen: + raise ValueError(f"Part {part!r} has no components.") + for node in chosen: + if node in owner: + raise ValueError( + f"Node {node!r} is in parts {owner[node]!r} and " + f"{part!r}: its spares come from one shelf." + ) + owner[node] = part + for group in rbd.ccf_groups: + shared = [m for m in group.members if m in chosen] + if len(shared) > 1: + raise NotImplementedError( + f"Part {part!r}: {shared} are a common-cause group, " + "whose shared causes replace them together, so " + "their pooled demand is not the sum of independent " + "ones: count them apart." + ) + out[part] = tuple(chosen) + return out + + +def _spares_counted(rbd, nodes, parts) -> Tuple[list, dict, list]: + """The components counted on their own (``nodes``, or every one + but a nested RBD; none when only ``parts`` are asked for), the + parts (see ``_spares_parts``), and every component either counts, + in order.""" + pools = _spares_parts(rbd, parts) + chosen = [] if (nodes is None and pools) else _spares_nodes(rbd, nodes) + if nodes is not None: + for part, members in pools.items(): + both = [node for node in chosen if node in members] + if both: + raise ValueError( + f"Node {both[0]!r} is in nodes and in part {part!r}: " + "its spares come from one shelf, the part's or its " + "own." + ) + counted = list( + dict.fromkeys( + [*chosen, *(m for members in pools.values() for m in members)] + ) + ) + return chosen, pools, counted + + +def _replacements(rbd, node, long_run: bool = False): + """What a component's replacements follow, for counting them + exactly (see ``_spares``): a renewal process, a block schedule's + (``Block``) or a test lattice's (``Tested``); raise + if they are not counted exactly. ``long_run``: for the counts in a + lead time, which need a demand that goes on.""" + simulate = ( + # The stock in a lead time has no simulation of its own to + # point to (#232). + "spares_stock has no simulation to count them by; " + "spares_demand(method='simulate') counts the spares used over a " + "window from new." + if long_run + else "count its spares by simulation: " + "spares_demand(method='simulate')." + ) + _requirements._require_reliabilities(rbd, node) + if node in rbd._standby: + raise NotImplementedError( + f"Component {node!r} is a standby group, whose units' " + f"failures depend on each other: {simulate}" + ) + if node in rbd._inspection: + return _tested_replacements(rbd, node, simulate) + schedule = rbd._preventive.get(node) + if schedule is not None and schedule.policy == "block": + component = rbd.components[node] + duration = schedule.duration + return Block( + _cdf(component.reliability), + _cdf(component.time_to_replace), + None if duration is None else _cdf(duration), + float(schedule.interval), + ) + if schedule is not None and schedule.policy == "condition": + raise NotImplementedError( + f"Component {node!r} is replaced on condition, at " + f"inspections on a calendar that do not renew it: {simulate}" + ) + if schedule is not None and schedule.policy == "calendar": + raise NotImplementedError( + f"Component {node!r} is replaced on a calendar policy, at " + f"times that depend on when each unit was put in: {simulate}" + ) + if node in rbd._early_members: + raise NotImplementedError( + f"Component {node!r} is renewed early at the stops of its " + f"maintenance group, which depend on the other members: " + f"{simulate}" + ) + if node in rbd._imperfect: + raise NotImplementedError( + f"Component {node!r} is repaired imperfectly " + f"({_requirements._imperfect_phrase(rbd, node)}), so its " + "replacements are " + f"not a renewal process of lives as new: {simulate}" + ) + component = rbd.components[node] + life, repair = component.reliability, component.time_to_replace + age = math.inf if schedule is None else float(schedule.interval) + maintenance = None if schedule is None else schedule.duration + if math.isinf(age): + up = _safe_mean(life) + fails = 1.0 + else: + up = float(component.avg_replacement_time(age)) + fails = float(np.ravel(_cdf(life)(np.array([age])))[0]) + down = fails * _safe_mean(repair) + (1.0 - fails) * ( + 0.0 if maintenance is None else _safe_mean(maintenance) + ) + mean_cycle = up + down + if long_run and not (0.0 < mean_cycle < math.inf): + raise NotImplementedError( + f"The life of component {node!r} may never end, or has no " + "mean, " + "so its demand has no long run: it has no stock level." + ) + return Replacements( + _cdf(life), + _cdf(repair), + None if maintenance is None else _cdf(maintenance), + age, + mean_cycle, + ) + + +def _tested_replacements(rbd, node, simulate: str) -> "Tested": + """A component with hidden failures: its replacements fall on its + tests (see ``Tested``): exactly, tested and repaired in no + time by tests that find every failure, or from its cycle on a grid + when its tests or repairs take time or its tests can miss a failure + (#159); raise if they cannot be counted so.""" + inspection = rbd._inspection[node] + component = rbd.components[node] + if inspection.partial or _requirements._tested_kind(rbd, node) == "unit": + # The replacements still fall on the tests that find failures, + # whatever the tests and repairs take; with tests that can miss + # a failure, a cycle depends on the place in the full tests' + # period of the test that starts it (#159). + instead = simulate[0].upper() + simulate[1:] + check_tested( + component.reliability, + component.time_to_replace, + inspection.duration, + inspection.interval, + node, + instead, + ) + unit = _requirements._tested_unit(rbd, node, any_kind=True) + assert unit is not None + first = float(inspection.offset or inspection.interval) + # The full tests are at the offset and every full test's + # interval from it: the first test is one, or the first after + # 0, an interval on, is the one after it. + position = 0 if inspection.offset else 1 % unit.per + found_at, cycle = unit.renewals(first, position, instead) + return Tested( + first, float(inspection.interval), found_at, cycle, position + ) + life = _requirements._tested_life(rbd, node) or TestedLife( + component.reliability, inspection.interval + ) + cycle = np.where( + life.down[:-1] <= 0.5, + np.diff(life.down), + life.up[:-1] - life.up[1:], + ) + model = component.reliability + first = float(inspection.offset or inspection.interval) + interval = float(inspection.interval) + + def found(tests: int) -> np.ndarray: + # The unit new at 0 fails before the first test, or between two. + times = first + interval * np.arange(tests) + failed = np.clip(_cdf(model)(times), 0.0, 1.0) + survive = np.clip(_sf_values(model.sf, times), 0.0, 1.0) + before = np.concatenate([[0.0], failed[:-1]]) + alive = np.concatenate([[1.0], survive[:-1]]) + return np.maximum( + np.where(before <= 0.5, failed - before, alive - survive), + 0.0, + ) + + return Tested(first, interval, found, np.maximum(cycle, 0.0)) + + +def spares_demand( + rbd, + horizon: float, + *, + nodes: Optional[Collection[Hashable]], + fleet: int, + method: str, + mc_samples: Optional[int], + seed, + parts: Optional[Mapping[Hashable, Collection[Hashable]]], +) -> Dict[Hashable, SparesDemand]: + """See ``RepairableRBD.spares_demand``.""" + end = _horizon(horizon) + fleet = _whole("fleet", fleet) + one_of("method", method, ("exact", "simulate")) + chosen, pools, counted = _spares_counted(rbd, nodes, parts) + if method == "simulate": + runs = _simulated_replacements( + rbd, + end, + counted, + 10_000 if mc_samples is None else mc_samples, + seed, + ) + counts = {node: _fractions(runs[node]) for node in chosen} + pooled = { + part: _fractions(np.sum([runs[m] for m in members], axis=0)) + for part, members in pools.items() + } + else: + _crews._require_unlimited_crews(rbd, *_SPARES_CREWS) + models = {node: _replacements(rbd, node) for node in counted} + each = { + node: count(model, end, "new") for node, model in models.items() + } + counts = {node: each[node] for node in chosen} + pooled = { + part: _summed(each[m] for m in members) + for part, members in pools.items() + } + out = { + node: SparesDemand(_fleet(number, fleet), end, fleet, method) + for node, number in counts.items() + } + for part, number in pooled.items(): + out[part] = SparesDemand( + _fleet(number, fleet), end, fleet, method, pools[part] + ) + return out + + +def _simulated_replacements( + rbd, horizon: float, nodes: list, mc_samples: int, seed +) -> Dict[Any, np.ndarray]: + """How many times each of ``nodes`` was replaced in each of + ``mc_samples`` simulations of ``[0, horizon)``.""" + from . import _event_loop + + montecarlo.check_count(mc_samples, False, "mc_samples") + if horizon == 0.0: + # Nothing is replaced in no time (as the exact count says). + return {node: np.zeros(mc_samples, dtype=np.int64) for node in nodes} + tally = _event_loop._run( + rbd, + horizon, + set(), + set(), + "p", + mc_samples, + False, + seed, + replacements=True, + ) + counts = np.array(tally.replacements, dtype=np.int64) + column = {node: c for c, node in enumerate(rbd.components)} + return {node: counts[:, column[node]] for node in nodes} + + +def spares_stock( + rbd, + lead_time: float, + *, + fill_rate: Optional[float], + stockout_probability: Optional[float], + nodes: Optional[Collection[Hashable]], + fleet: int, + parts: Optional[Mapping[Hashable, Collection[Hashable]]], +) -> Dict[Hashable, SparesStock]: + """See ``RepairableRBD.spares_stock``.""" + tau = _horizon(lead_time) + fleet = _whole("fleet", fleet) + if fill_rate is None and stockout_probability is None: + raise ValueError("Give a fill_rate, a stockout_probability, or both.") + for name, target in ( + ("fill_rate", fill_rate), + ("stockout_probability", stockout_probability), + ): + if target is not None and not 0.0 < target < 1.0: + raise ValueError(f"{name} must be in (0, 1), got {target!r}.") + chosen, pools, counted = _spares_counted(rbd, nodes, parts) + _crews._require_unlimited_crews(rbd, *_STOCK_CREWS) + models = {node: _replacements(rbd, node, True) for node in counted} + for part, members in pools.items(): + blocks = [m for m in members if isinstance(models[m], Block)] + if len(blocks) > 1: + raise NotImplementedError( + f"Part {part!r}: {blocks} are replaced on block " + "schedules, which keep step with each other, so their " + "demands in a lead time are not independent: count " + "them apart." + ) + random = { + node: count(model, tau, "random") for node, model in models.items() + } + arrival = { + node: count(model, tau, "arrival") for node, model in models.items() + } + + def stocked(members: Tuple[Hashable, ...]) -> SparesStock: + if len(members) == 1: + node = members[0] + on_order = _fleet(random[node], fleet) + at_demand = ( + arrival[node] + if fleet == 1 + else _fleet( + np.convolve( + arrival[node], _fleet(random[node], fleet - 1) + ), + 1, + ) + ) + else: + # A demand finds its own component's replacements as they + # come (``arrival``), and the others' as at a random time, + # each component's share of the demands its share of the + # long-run rates. + one = _summed(random[m] for m in members) + others = _fleet(one, fleet - 1) + rates = np.array([rate(models[m]) for m in members]) + on_order = _fleet(one, fleet) + at_demand = np.zeros(1) + for i, member in enumerate(members): + seen = _summed( + [arrival[member], others] + + [random[m] for j, m in enumerate(members) if j != i] + ) + size = max(len(at_demand), len(seen)) + at_demand = np.pad(at_demand, (0, size - len(at_demand))) + at_demand += ( + rates[i] + / rates.sum() + * np.pad(seen, (0, size - len(seen))) + ) + at_demand = at_demand / at_demand.sum() + result = SparesStock(0, 0.0, 1.0, tau, fleet, on_order, at_demand) + stock = 0 + while not ( + ( + fill_rate is None + or result.fill_rate_for(stock) >= fill_rate - 1e-12 + ) + and ( + stockout_probability is None + or result.stockout_probability_for(stock) + <= stockout_probability + 1e-12 + ) + ): + stock += 1 + return dataclasses.replace( + result, + stock=stock, + fill_rate=result.fill_rate_for(stock), + stockout_probability=result.stockout_probability_for(stock), + ) + + out: Dict[Hashable, SparesStock] = { + node: stocked((node,)) for node in chosen + } + for part, members in pools.items(): + out[part] = dataclasses.replace(stocked(members), members=members) + return out diff --git a/repyability/rbd/_spec.py b/repyability/rbd/_spec.py new file mode 100644 index 00000000..0d913a83 --- /dev/null +++ b/repyability/rbd/_spec.py @@ -0,0 +1,853 @@ +"""A ``RepairableRBD``'s inputs, checked: the keys a component's spec may +carry and the rules each takes (preventive maintenance, inspection, +standby groups, imperfect repair, costs), the maintenance groups and the +common-cause groups. The constructor calls these; the keys are also the +class's attributes (``RepairableRBD.PREVENTIVE_KEYS``, ...). +""" + +import math +import warnings +from typing import ( + TYPE_CHECKING, + Any, + Dict, + Hashable, + Optional, + Tuple, +) + +import numpy as np + +from repyability.rbd import _requirements +from repyability.rbd._degradation import ( + is_degradation, +) +from repyability.rbd._model_utils import ( + is_mixture, +) + +if TYPE_CHECKING: + pass +from repyability.rbd._events import ( + _Imperfect, + _Inspection, + _MaintenanceGroup, + _Preventive, + _Standby, +) +from repyability.utils.checks import ( + is_number, + number_or_nan, + unfitted_distribution, +) +from repyability.utils.wrappers import outside_level + +#: Optional per-component cost fields accepted in a component spec dict. +COST_KEYS = ("repair_cost", "replace_cost", "downtime_cost") + +#: The costs charged per failure, which may also be given as a +#: distribution of the cost (drawn afresh at each failure). +PER_FAILURE_COST_KEYS = ("repair_cost", "replace_cost") + +#: Every key a component spec dict may carry. +COMPONENT_SPEC_KEYS = ( + ("reliability", "repairability") + + COST_KEYS + + ( + "preventive", + "inspection", + "acquisition_cost", + "priority", + "standby", + "group", + "repair", + "replace_after", + "duty", + ) +) + +#: The models of imperfect repair a spec's ``"repair"`` can name. +REPAIR_MODELS = ("kijima1", "kijima2") + +#: The keys of a component's ``"preventive"`` spec. +PREVENTIVE_KEYS = ( + "interval", + "policy", + "duration", + "cost", + "threshold", + "inspection_cost", + "opportunity", + "level", + "calendar", +) + +#: The policies a ``"preventive"`` spec can name: replacement at an +#: age, on a calendar (block), on the condition an inspection finds, or +#: on a ``CalendarPolicy``'s threshold at each time (#319). +PREVENTIVE_POLICIES = ("age", "block", "condition", "calendar") + +#: The keys of a maintenance group's options. +GROUP_KEYS = ("setup_cost", "system_down") + +#: The keys of a component's ``"inspection"`` spec. +INSPECTION_KEYS = ( + "interval", + "duration", + "cost", + "offset", + "coverage", + "full_test", +) + +#: The keys of a component's ``"standby"`` spec. +STANDBY_KEYS = ("units", "k", "dormancy_factor", "switching_probability") + +#: The costs charged per action (per failure, per preventive +#: replacement or per inspection), which may be distributions. +_PER_ACTION_COST_KEYS = PER_FAILURE_COST_KEYS + ( + "preventive_cost", + "inspection_cost", +) + + +#: The share of its interval below which a test offset is taken as 0: the +#: first test at the interval, not one at the start of a new unit (#237). +_TINY_OFFSET = 1e-9 + + +def _mean_cost(cost) -> float: + """A declared cost's expected value: the number itself, or the mean of a + cost distribution.""" + if isinstance(cost, float): + return cost + return float(np.ravel(cost.mean())[0]) + + +def _warn_nested_costs(components: dict) -> None: + """Warn of a nested ``RepairableRBD`` that carries costs, which this + diagram does not count (its cost analyses price its own components + alone), naming the node: once per nested object, so that a diagram + rebuilt with the same one (a lever moved, a draw) says nothing new.""" + from repyability.rbd.repairable_rbd import RepairableRBD + + for name, component in components.items(): + if not isinstance(component, RepairableRBD): + continue + if component._costs_reported or not ( + component.has_costs or component.acquisition_cost + ): + continue + component._costs_reported = True + warnings.warn( + f"Node {name!r} is a nested RBD that carries costs, which this " + "diagram does not count: its cost analyses (expected_cost_rate, " + "expected_cost, cost, total_cost) price its own components " + "alone. Count the nested diagram's costs with its own methods.", + UserWarning, + stacklevel=outside_level(), + ) + + +def _refuse_shared_nested(components: dict) -> None: + """Raise if one ``RepairableRBD`` object is given for two nodes: a + nested diagram carries its simulation state (its event queue), so + it is one node's. Give each node its own (``copy.deepcopy`` one + for nodes alike).""" + from repyability.rbd.repairable_rbd import RepairableRBD + + first: dict = {} + for name, component in components.items(): + if not isinstance(component, RepairableRBD): + continue + other = first.setdefault(id(component), name) + if other != name: + raise ValueError( + f"Nodes {other!r} and {name!r} are given one RepairableRBD " + "object: a nested diagram carries its simulation state, " + "so it is one node's. Give each node its own " + "(copy.deepcopy it for nodes alike)." + ) + + +def _validate_ccf_groups(rbd, ccf_groups) -> list: + """The common-cause groups, checked: each a ``CCFGroup`` of + components of this RBD (not a nested RBD or a standby group), each + component in one group at most, the members of a group identical + (the same life and repair models, compared through their saved + form).""" + if not ccf_groups: + return [] + from repyability.rbd.ccf import checked_groups + from repyability.rbd.repairable_rbd import RepairableRBD + from repyability.rbd.serialisation import serialise_model + + def check_member(member, group): + if member not in rbd.components: + raise _requirements._not_a_component( + rbd, member, f"common-cause group {list(group.members)}" + ) + if ( + isinstance(rbd.components[member], RepairableRBD) + or member in rbd._standby + ): + raise ValueError( + f"CCF group member {member!r} is a nested RBD or a " + "standby group: a group's members are single " + "components." + ) + + def saved(member): + component = rbd.components[member] + return ( + serialise_model(component.reliability), + serialise_model(component.time_to_replace), + ) + + return checked_groups( + ccf_groups, + check_member, + saved, + lambda group: ( + f"The members of a CCF group, {list(group.members)}, " + "must be identical components, with the same life and " + "repair models." + ), + ) + + +def _validate_imperfect(rbd, node, spec: dict) -> Optional[_Imperfect]: + """A component spec's imperfect repair (``"repair"`` and + ``"replace_after"``), validated; None if it is renewed by every + repair, as by default (a restoration factor of 0).""" + repair, limit = spec.get("repair"), spec.get("replace_after") + if repair is None and limit is None: + return None + q, model = 0.0, "" + if repair is not None: + if ( + not isinstance(repair, dict) + or set(repair) != {"model", "q"} + or repair["model"] not in REPAIR_MODELS + ): + raise ValueError( + f"Component {node!r}: repair must be a dict of a " + f"'model' ({' or '.join(map(repr, REPAIR_MODELS))}) " + f"and its restoration factor 'q', got {repair!r}." + ) + q = repair["q"] + if ( + isinstance(q, bool) + or not isinstance(q, (int, float, np.number)) + or not 0.0 <= float(q) <= 1.0 + ): + raise ValueError( + f"Component {node!r}: the restoration factor q must be " + "a number in [0, 1] (0 renews the unit, 1 is minimal " + f"repair), got {q!r}." + ) + q, model = float(q), repair["model"] + if limit is not None: + if ( + isinstance(limit, bool) + or not isinstance(limit, (int, np.integer)) + or limit < 1 + ): + raise ValueError( + f"Component {node!r}: replace_after must be a whole " + f"number of failures of at least 1, got {limit!r}." + ) + if q == 0.0: + raise ValueError( + f"Component {node!r}: replace_after needs imperfect " + "repair (a 'repair' with q above 0): renewed by every " + "repair, the unit is as new after each failure anyway." + ) + limit = int(limit) + if q == 0.0: + return None + if spec.get("standby") is not None: + raise ValueError( + f"Component {node!r} is a standby group, whose units are " + "renewed by their repairs, so it cannot be repaired " + "imperfectly." + ) + preventive = spec.get("preventive") + if isinstance(preventive, dict) and preventive.get("policy") == ( + "condition" + ): + raise ValueError( + f"Component {node!r} is replaced on condition, judged by its " + "age as new, so it cannot be repaired imperfectly." + ) + life = spec["reliability"] + if not (is_mixture(life) or (hasattr(life, "Hf") and hasattr(life, "qf"))): + raise ValueError( + f"Component {node!r}: an imperfectly repaired unit's lives " + "are drawn given its virtual age, which needs a lifetime " + "model with a cumulative hazard (Hf) and quantiles (qf), " + "such as a surpyval distribution." + ) + return _Imperfect(model, q, limit) + + +def _validate_groups( + rbd, options: Optional[dict] +) -> Dict[Hashable, _MaintenanceGroup]: + """The maintenance groups (#108), from the components' ``"group"`` + and the options given for each, checked; and the members that can + be renewed early (``_early_members``).""" + members: Dict[Hashable, list] = {} + for node, group in rbd._member_group.items(): + try: + hash(group) + except TypeError: + raise ValueError( + f"Component {node!r}: a group is named by a hashable " + f"value, got {group!r}." + ) from None + for spec_key, what in ( + ("_inspection", "hidden failures"), + ("_standby", "a standby group"), + ): + if node in getattr(rbd, spec_key): + raise ValueError( + f"Component {node!r} has {what}, so it cannot be in " + "a maintenance group." + ) + members.setdefault(group, []).append(node) + rbd._early_members = { + node + for node, schedule in rbd._preventive.items() + if schedule.opportunity < schedule.interval + } + for node in rbd._early_members: + if node not in rbd._member_group: + raise ValueError( + f"Component {node!r} has an opportunity, but no " + "maintenance group to give it one: add a 'group'." + ) + options = dict(options or {}) + unknown = set(options) - set(members) + if unknown: + raise ValueError( + f"maintenance_groups names group(s) " + f"{sorted(map(str, unknown))} that no component is in." + ) + out: Dict[Hashable, _MaintenanceGroup] = {} + for group, nodes in members.items(): + given = options.get(group) or {} + if not isinstance(given, dict) or set(given) - set(GROUP_KEYS): + raise ValueError( + f"Maintenance group {group!r}: its options must be a " + f"dict of {', '.join(GROUP_KEYS)}, got {given!r}." + ) + setup = given.get("setup_cost") + setup = ( + 0.0 + if setup is None + else _validate_cost(group, "setup_cost", setup) + ) + system_down = given.get("system_down", False) + if not isinstance(system_down, bool): + raise ValueError( + f"Maintenance group {group!r}: system_down must be True " + f"or False, got {system_down!r}." + ) + out[group] = _MaintenanceGroup(tuple(nodes), setup, system_down) + return out + + +def _unknown_component(node, component) -> str: + """Why ``component`` cannot be node ``node``: it is none of the + kinds a node can be.""" + from repyability.repairable import Repairable + + kinds = ( + "Give a spec dict with 'reliability' and 'repairability', a " + "NonRepairable, or a nested RepairableRBD (or, for a junction " + "that never fails, such as a k-out-of-n vote point, " + "PerfectReliability)." + ) + if isinstance(component, Repairable): + return ( + f"Component {node!r} is a Repairable, which models the " + "minimal or imperfect repair of a single unit: it cannot be " + "a node, whose repairs renew it as new. " + kinds + ) + spec = ( + "{'reliability': , 'repairability': }" + ) + distribution = unfitted_distribution(component) + if distribution is not None: + return ( + f"Component {node!r} is surpyval's {distribution} " + f"distribution itself: fit it to data " + f"(surv.{distribution}.fit(times)) or give its parameters " + f"(surv.{distribution}.from_params([...])), and give it " + f"with its repairs, as {spec}." + ) + if callable(getattr(component, "sf", None)): + return ( + f"Component {node!r} is a model of a life alone (a " + f"{type(component).__name__}); a repairable component needs " + f"its repairs too: give {spec}." + ) + return f"Component {node!r} is a {type(component).__name__}. " + kinds + + +def _validate_component_spec(node, spec: dict) -> None: + """Reject unknown keys in a component spec. + + A mistyped cost key (``repair_costs``) would otherwise be silently + ignored and priced at zero, which is a quiet way to get the money + wrong; surface it at construction instead. + """ + unknown = set(spec) - set(COMPONENT_SPEC_KEYS) + if unknown: + raise ValueError( + f"Component {node!r} has unknown key(s) " + f"{sorted(map(str, unknown))}. A component spec takes " + f"{', '.join(COMPONENT_SPEC_KEYS)}." + ) + for key, what in ( + ("reliability", "its lives (time to failure)"), + ("repairability", "its repair times"), + ): + if key not in spec: + raise ValueError( + f"Component {node!r} needs a {key!r}: the " + f"distribution of {what}." + ) + value = spec[key] + if isinstance(value, (int, float)) and not isinstance(value, bool): + mean = "MTTF" if key == "reliability" else "MTTR" + instant = ( + ", or 'instant' for a repair in zero time" + if key == "repairability" + else "" + ) + raise TypeError( + f"Component {node!r}: {key} is the number {value!r}, " + f"not a distribution of {what}. For an {mean} of " + f"{value:g}, give e.g. " + f"surpyval.Exponential.from_params([1 / {value:g}]); " + "for a fixed time, " + f"surpyval.ExactEventTime.from_params([{value:g}])" + f"{instant}." + ) + + +def _saved_name(saved: dict) -> str: + """A model's saved form in a few words, for a message.""" + if saved.get("kind") == "surpyval": + model = saved["model"] + return f"{model.get('distribution')} {model.get('params')}" + return str(saved.get("kind")) + + +def _validate_calendar(node, spec: dict, policy: str, life): + """A ``"calendar"`` policy's ``CalendarPolicy`` (#319), checked: given + under that policy and no other, with no interval (its thresholds say + when) and no cost (its calendar's prices are charged), and worked out + for the component's own ``life`` (the policy carries the saved form + of the life it was worked out for; one that does not save passes).""" + from repyability.maintenance import CalendarPolicy + from repyability.rbd.serialisation import serialise_model + + given = spec.get("calendar") + if policy != "calendar": + if given is not None: + raise ValueError( + f"Component {node!r}: a calendar applies only to the " + "'calendar' policy." + ) + return None + if not isinstance(given, CalendarPolicy): + raise ValueError( + f"Component {node!r}: the 'calendar' policy needs a calendar, " + "the CalendarPolicy of NonRepairable.optimal_calendar_policy; " + f"got {given!r}." + ) + for key, why in ( + ("interval", "its policy's thresholds say when to replace"), + ("cost", "its calendar's price at each opportunity is charged"), + ): + if spec.get(key) is not None: + raise ValueError( + f"Component {node!r}: a 'calendar' policy takes no {key}: " + f"{why}." + ) + if given.life is not None: + try: + saved = serialise_model(life) + except (NotImplementedError, TypeError, ValueError): + saved = None + if saved is not None and saved != given.life: + raise ValueError( + f"Component {node!r}: its calendar policy was worked out " + f"for another life ({_saved_name(given.life)}), not the " + f"life of the component ({_saved_name(saved)}): work the " + "policy out for it, " + "NonRepairable(life, ...).optimal_calendar_policy(calendar)." + ) + return given + + +def _validate_preventive( + node, spec, life=None +) -> Tuple[_Preventive, Any, Any]: + """A component's ``"preventive"`` spec, validated: its schedule, + and the cost of each replacement and of each inspection (None if + it prices nothing); ``life`` is the component's life model, which a + ``"level"`` needs to be a degradation process (#271).""" + if not isinstance(spec, dict): + raise ValueError( + f"Component {node!r}: preventive must be a dict with " + f"{', '.join(PREVENTIVE_KEYS)}, got {spec!r}." + ) + unknown = set(spec) - set(PREVENTIVE_KEYS) + if unknown: + raise ValueError( + f"Component {node!r} has unknown preventive key(s) " + f"{sorted(map(str, unknown))}. A preventive spec takes " + f"{', '.join(PREVENTIVE_KEYS)}." + ) + policy = spec.get("policy", "age") + if policy not in PREVENTIVE_POLICIES: + raise ValueError( + f"Component {node!r}: the preventive policy must be 'age', " + f"'block', 'condition' or 'calendar', got {policy!r}." + ) + calendar = _validate_calendar(node, spec, policy, life) + if calendar is not None: + interval = math.inf + else: + try: + interval = number_or_nan(spec["interval"]) + except KeyError: + raise ValueError( + f"Component {node!r}: a preventive spec needs an interval." + ) from None + if not interval > 0.0: + raise ValueError( + f"Component {node!r}: the preventive interval must be a " + f"positive number (inf for none), got {spec['interval']!r}." + ) + threshold = 0.0 + level = _validate_level(node, spec, policy, life) + if policy == "condition" and level is None: + if spec.get("threshold") is None: + raise ValueError( + f"Component {node!r}: the 'condition' policy needs a " + "threshold: the probability of failing before the next " + "inspection above which the unit is replaced (or, for " + "a degradation process, a level)." + ) + threshold = spec["threshold"] + if ( + isinstance(threshold, bool) + or not isinstance(threshold, (int, float, np.number)) + or not 0.0 <= float(threshold) <= 1.0 + ): + raise ValueError( + f"Component {node!r}: the threshold must be a " + f"probability, in [0, 1], got {threshold!r}." + ) + threshold = float(threshold) + elif policy != "condition": + for key in ("threshold", "inspection_cost"): + if spec.get(key) is not None: + raise ValueError( + f"Component {node!r}: {key} applies only to the " + "'condition' policy, which inspects the unit." + ) + opportunity = math.inf + if spec.get("opportunity") is not None: + if policy not in ("age", "calendar"): + raise ValueError( + f"Component {node!r}: an opportunity applies only to " + "the 'age' and 'calendar' policies (a unit renewed early " + "at a stop of its maintenance group)." + ) + opportunity = spec["opportunity"] + # Under a calendar policy (no single interval, #349): any age. + within = ( + "at least 0" + if policy == "calendar" + else f"from 0 to the interval ({interval:g})" + ) + if ( + isinstance(opportunity, bool) + or not isinstance(opportunity, (int, float, np.number)) + or not 0.0 <= float(opportunity) <= interval + ): + raise ValueError( + f"Component {node!r}: the opportunity must be an age " + f"{within}, got {opportunity!r}." + ) + opportunity = float(opportunity) + duration = spec.get("duration", "instant") + if isinstance(duration, str) and duration == "instant": + duration = None + elif not hasattr(duration, "random"): + raise ValueError( + f"Component {node!r}: the preventive duration must be a " + "time-to-maintain model (such as a fitted surpyval " + f"distribution) or 'instant', got {duration!r}." + ) + costs = [] + for key, name in ( + ("cost", "preventive_cost"), + ("inspection_cost", "inspection_cost"), + ): + cost = spec.get(key) + if cost is not None: + cost = _validate_component_cost(node, name, cost) + if isinstance(cost, float) and cost == 0.0: + cost = None + costs.append(cost) + schedule = _Preventive( + interval, policy, duration, threshold, opportunity, level, calendar + ) + return schedule, costs[0], costs[1] + + +def _validate_level(node, spec: dict, policy: str, life) -> Optional[float]: + """A ``"preventive"`` spec's ``"level"`` (#271), checked: a + number below the failure threshold of the degradation process the + component's life is, under the ``"condition"`` policy and in + place of its ``"threshold"``; None if not given.""" + level = spec.get("level") + if level is None: + return None + if policy != "condition": + raise ValueError( + f"Component {node!r}: level applies only to the " + "'condition' policy, whose inspections measure it." + ) + if not is_degradation(life): + raise ValueError( + f"Component {node!r}: a level is the measured degradation " + "level at which an inspection replaces the unit, so its " + "life must be a surpyval Wiener or gamma degradation " + "process; replace on its age with a threshold instead." + ) + if spec.get("threshold") is not None: + raise ValueError( + f"Component {node!r}: give the condition policy a level " + "(of the measured degradation) or a threshold (a " + "probability of failing by the next inspection), not both." + ) + level = number_or_nan(level) + if not level < float(life.threshold): + raise ValueError( + f"Component {node!r}: the level must be a number below the " + "degradation process's failure threshold, " + f"{float(life.threshold):g}, got {spec['level']!r}." + ) + return level + + +def _validate_standby(node, spec: dict) -> _Standby: + """A component spec's ``"standby"``, validated: the group of + units it makes the component (see ``_StandbyGroup``).""" + standby = spec["standby"] + if not isinstance(standby, dict): + raise ValueError( + f"Component {node!r}: standby must be a dict of " + f"{', '.join(STANDBY_KEYS)}, got {standby!r}." + ) + unknown = set(standby) - set(STANDBY_KEYS) + if unknown: + raise ValueError( + f"Component {node!r}: unknown standby key(s) " + f"{sorted(map(str, unknown))}; it takes " + f"{', '.join(STANDBY_KEYS)}." + ) + for key in ("preventive", "inspection"): + if spec.get(key) is not None: + raise ValueError( + f"Component {node!r} is a standby group, which takes no " + f"{key!r} schedule." + ) + if isinstance(spec["repairability"], str): + raise ValueError( + f"Component {node!r} is a standby group, whose units need a " + "repair time: repaired instantly, a spare would never be " + "needed." + ) + counts = {"units": standby.get("units", 2), "k": standby.get("k", 1)} + for key, value in counts.items(): + if ( + isinstance(value, bool) + or not isinstance(value, (int, float, np.integer, np.floating)) + or int(value) != value + or value < 1 + ): + raise ValueError( + f"Component {node!r}: standby {key} must be a whole " + f"number, 1 or more; got {value!r}." + ) + units, k = int(counts["units"]), int(counts["k"]) + if k >= units: + raise ValueError( + f"Component {node!r}: a standby group needs a spare, so k " + f"must be less than units; got k={k} and units={units}." + ) + fractions = { + "dormancy_factor": standby.get("dormancy_factor", 0.0), + "switching_probability": standby.get("switching_probability", 1.0), + } + for key, value in fractions.items(): + if ( + isinstance(value, bool) + or not isinstance(value, (int, float, np.integer, np.floating)) + or not 0.0 <= value <= 1.0 + ): + raise ValueError( + f"Component {node!r}: standby {key} must be a number in " + f"[0, 1]; got {value!r}." + ) + return _Standby( + units, + k, + float(fractions["dormancy_factor"]), + float(fractions["switching_probability"]), + ) + + +def _validate_inspection(node, spec) -> Tuple[_Inspection, Any]: + """A component's ``"inspection"`` spec, validated: its schedule, and + its cost (None if it prices nothing).""" + if not isinstance(spec, dict): + raise ValueError( + f"Component {node!r}: inspection must be a dict with " + f"{', '.join(INSPECTION_KEYS)}, got {spec!r}." + ) + unknown = set(spec) - set(INSPECTION_KEYS) + if unknown: + raise ValueError( + f"Component {node!r} has unknown inspection key(s) " + f"{sorted(map(str, unknown))}. An inspection spec takes " + f"{', '.join(INSPECTION_KEYS)}." + ) + try: + interval = number_or_nan(spec["interval"]) + except KeyError: + raise ValueError( + f"Component {node!r}: an inspection spec needs an interval." + ) from None + if not (interval > 0.0 and np.isfinite(interval)): + raise ValueError( + f"Component {node!r}: the inspection interval must be a " + f"positive, finite number, got {spec['interval']!r}. (A " + "hidden failure is found only by an inspection.)" + ) + duration = spec.get("duration", "instant") + if isinstance(duration, str) and duration == "instant": + duration = None + elif not hasattr(duration, "random"): + raise ValueError( + f"Component {node!r}: the inspection duration must be a " + "time-to-test model (such as a fitted surpyval distribution) " + f"or 'instant', got {duration!r}." + ) + cost = spec.get("cost") + if cost is not None: + cost = _validate_component_cost(node, "inspection_cost", cost) + if isinstance(cost, float) and cost == 0.0: + cost = None + offset = number_or_nan(spec.get("offset", 0.0)) + if not 0.0 <= offset < interval: + raise ValueError( + f"Component {node!r}: the inspection offset, the time of its " + "first test, must be at least 0 and less than its interval, " + f"{interval:g}, got {spec['offset']!r}." + ) + if offset <= _TINY_OFFSET * interval: + # An offset of 0 puts the first test at the interval; one this + # close to 0 (a share of the interval worked out as nearly 0, + # say) is taken as 0, not as a test at the start (#237). + offset = 0.0 + coverage = number_or_nan(spec.get("coverage", 1.0)) + if not 0.0 <= coverage <= 1.0: + raise ValueError( + f"Component {node!r}: the inspection coverage, the chance " + "that a test finds a failure, must be from 0 to 1, got " + f"{spec['coverage']!r}." + ) + full_test = spec.get("full_test") + if full_test is not None: + given = full_test + full_test = number_or_nan(full_test) + count = full_test / interval + if not ( + np.isfinite(count) + and round(count) >= 1 + and abs(count - round(count)) <= 1e-9 * count + ): + raise ValueError( + f"Component {node!r}: the full_test interval must be a " + f"whole multiple of its inspection interval, {interval:g} " + f"(every so many tests is a full one), got {given!r}." + ) + full_test = round(count) * interval + elif coverage < 1.0: + raise ValueError( + f"Component {node!r}: with a coverage below 1, the failures " + "its tests miss are found only by a full test: give its " + "full_test interval, a whole multiple of its inspection " + "interval (the mission time, say, after which it is " + "renewed)." + ) + schedule = _Inspection(interval, duration, offset, coverage, full_test) + return schedule, cost + + +def _validate_cost(node, key: str, value) -> float: + """Coerce a cost to a finite, non-negative float.""" + if hasattr(value, "qf"): + raise ValueError( + f"{node!r}: {key} must be a number, not a distribution. Only " + "the costs charged per action (repair_cost, replace_cost and " + "the preventive and inspection costs) may be distributions." + ) + if not is_number(value): + # Text that reads as a number, and a bool, are refused (#233). + raise ValueError(f"{node!r}: {key} must be a number, got {value!r}.") + cost = float(value) + if not np.isfinite(cost) or cost < 0.0: + raise ValueError( + f"{node!r}: {key} must be finite and non-negative, got " + f"{value!r}." + ) + return cost + + +def _validate_component_cost(node, key: str, value): + """Validate a per-component cost: a number (coerced to float) or, for + the per-action costs, a distribution of the cost -- anything with + ``qf`` and ``mean``, such as a fitted surpyval model. + + A distribution must have a finite mean and put no more than 1e-12 + probability on a negative cost. + """ + if key not in _PER_ACTION_COST_KEYS or not hasattr(value, "qf"): + return _validate_cost(node, key, value) + mean = _mean_cost(value) + if not np.isfinite(mean): + raise ValueError( + f"{node!r}: the {key} distribution must have a finite mean, " + f"got {mean:g}." + ) + lowest = float(np.ravel(value.qf(np.array([1e-12])))[0]) + if not lowest >= 0.0: + raise ValueError( + f"{node!r}: the {key} distribution puts appreciable " + f"probability on a negative cost (its 1e-12 quantile is " + f"{lowest:g}). Use one on [0, inf), such as a LogNormal, " + "Gamma or Weibull." + ) + return value diff --git a/repyability/rbd/_streams.py b/repyability/rbd/_streams.py index a03e70db..b449fe95 100644 --- a/repyability/rbd/_streams.py +++ b/repyability/rbd/_streams.py @@ -52,6 +52,10 @@ #: uniforms that decide whether the tests that can miss its failures find #: them (one per strike: they are found, or missed, alike). CAUSE, CAUSE_TEST = 11, 12 +#: The uniforms a unit replaced on condition by its measured degradation +#: level (#271) draws its level at each inspection from, and its failure +#: from that level: two an inspection. +LEVEL = 13 #: The kind of each cost that can be a distribution, by its cost key. COST_KINDS = { "repair_cost": 3, diff --git a/repyability/rbd/_tally.py b/repyability/rbd/_tally.py new file mode 100644 index 00000000..05d2cfad --- /dev/null +++ b/repyability/rbd/_tally.py @@ -0,0 +1,1629 @@ +"""A ``RepairableRBD`` run's totals: what its simulations add up to. + +``_Tally`` sums the simulations' results (``_Replication``) into a run's +availability, cost, counts and timelines, exactly however the run is split +into chunks or shards; ``_ModuleRun`` does the same for a conditional run +(#189), whose simulations are its modules' alone; the ``_Capacity*`` +classes record the capacity a run delivers. +""" + +import json +import math +import warnings +from collections import Counter +from contextlib import contextmanager +from typing import ( + TYPE_CHECKING, + Any, + Callable, + Dict, + Hashable, + List, + NamedTuple, + Optional, + Tuple, +) + +import numpy as np + +from repyability.rbd import ( + _conditional, + _curves, + _requirements, + _timeline_runs, + _windows, +) +from repyability.rbd._exact import ( + ExactSum, + add_columns, + binned_parts, + compact_parts, +) +from repyability.rbd._point_availability import ( + curve_breaks, +) +from repyability.rbd.results import ( + AvailabilityResult, + ConditionalRun, + ControlVariate, + CostResult, +) + +if TYPE_CHECKING: + from repyability.rbd.repairable_rbd import RepairableRBD + +from repyability.rbd._time_order import ( + _add_at, + _by_time, + _exact_total, + _group_partials, + _partials, + _zeroed, +) +from repyability.utils.checks import ( + simulation_window, +) +from repyability.utils.wrappers import outside_level + + +def _check_shard_map(shard_map, shard_size, n_jobs) -> None: + """Check ``availability``'s ``shard_map`` and ``shard_size``.""" + if shard_map is None: + if shard_size is not None: + raise ValueError( + "shard_size is the size of the shards shard_map runs: give " + "shard_map too." + ) + return + if not callable(shard_map): + raise ValueError( + "shard_map must be a map: called as shard_map(run_shard, " + f"shards), like map; got {shard_map!r}." + ) + if n_jobs is not None: + raise ValueError( + "shard_map runs the shards wherever it sends them: give its " + "workers the CPUs, and leave out n_jobs." + ) + + +def _module_shards( + shard_map, template: dict, step: int, first: int, count: int, modules: int +): + """A conditional run's simulations ``first`` to ``first + count - 1`` + of its modules (#189) as shards, cut at the multiples of ``step``, + which ``shard_map(run_shard, shards)`` runs wherever it sends them: + the ``modules`` histories joined in order, each simulation's own cost, + the costs by category, and by module (None for a module not costed).""" + from repyability.rbd.shards import run_shard + + stop = first + count + cuts = [first, *range((first // step + 1) * step, stop, step), stop] + ranges = list(zip(cuts, cuts[1:])) + parts = [ + _conditional.from_partial(saved, modules) + for saved in shard_map( + run_shard, [_shard_bytes(template, a, b) for a, b in ranges] + ) + ] + parts.sort(key=lambda part: part[0]) + if [part[0] for part in parts] != ranges: + raise ValueError( + "shard_map gave back other partials than its shards': it must " + "return run_shard's result for each shard it is given." + ) + categories: Dict[str, float] = {} + for part in parts: + for key, value in part[3].items(): + categories[key] = categories.get(key, 0.0) + value + by_module = np.sum([part[4] for part in parts], axis=0) + costed = np.any([part[5] for part in parts], axis=0) + return ( + _conditional.joined([part[1] for part in parts]), + np.concatenate([part[2] for part in parts]), + categories, + [float(v) if c else None for v, c in zip(by_module, costed)], + ) + + +#: A window's expected cost split by category and by component (see +#: ``CostResult.by_category`` and ``by_component``). +_Breakdown = Tuple[Dict[str, float], Dict[Hashable, float]] + + +class _Exacts(NamedTuple): + """A system's expected values over a window, worked out exactly (see + ``RepairableRBD._twin_exact``): its mean availability, if it is + priced its expected cost, with that cost's split (see + ``ExpectedCost``), and against a demand the fraction of it delivered + (``mission_capacity``), where that is worked out.""" + + availability: float + cost: Optional[float] = None + breakdown: Optional[_Breakdown] = None + delivered: Optional[float] = None + + +class _ModuleRun: + """A system's dependent modules simulated alone, and every other node + taken exactly given their joint states (#189, see ``_conditional``): + the machinery of a conditional run (``RepairableRBD._conditional_run``) + and of a plain run's means taken given its modules + (``RepairableRBD._conditioned_run``). + + The modules (by default ``_conditional_modules``) are simulated as a + diagram of their own (``_modules_rbd``), from their states, whose + streams are named as in the system, so that its simulations are the + system's modules'. ``prepare`` works out the other nodes' own expected + values over the window; ``simulate`` runs a range of simulations, + working out the system given each joint state its modules meet once, + on a grid; and ``result`` and ``means`` give what they make. The + components' curves are built once for the whole run (see + ``_curves``).""" + + def __init__( + self, + rbd: "RepairableRBD", + t_simulation: float, + working: set, + broken: set, + method: str, + entropy: int, + *, + antithetic: bool, + n_jobs: Optional[int], + engine: str, + curve_points: Optional[int], + state=None, + shard_map: Optional[Callable] = None, + shard_size: Optional[int] = None, + demand: Optional[float] = None, + capacities: bool = False, + control_variate: bool = False, + modules: Optional[list] = None, + ) -> None: + from . import _event_loop, _runs + + T = simulation_window(t_simulation) + self.rbd = rbd + self.T = T + self.working, self.broken, self.method = working, broken, method + self.antithetic, self.n_jobs = antithetic, n_jobs + states = _curves._simulation_states(rbd, state, working | broken) + if modules is None: + modules = rbd._conditional_modules(working, broken) + self.modules = modules + self.module_states = { + node: states[node] for node in modules if node in states + } + self.entropy = entropy + self.engine = engine + self.sub = _runs._modules_rbd(rbd, modules) if modules else None + self.recorder = _CapacityRecorder(rbd, demand) if capacities else None + self.capacity_given: Dict[int, _conditional.CapacityGiven] = {} + self.capacity_parts: List[_conditional.CapacityValues] = [] + self.shards = None + if self.sub is not None and shard_map is not None: + plan, _ = _event_loop._stream_plan( + self.sub, T, entropy, antithetic, self.module_states + ) + template = { + **_runs._shard_template( + self.sub, + T, + entropy, + set(), + set(), + "p", + antithetic, + None, + self.engine, + self.module_states, + None, + ), + "histories": True, + } + self.shards = ( + shard_map, + template, + _runs._shard_size(plan, shard_size), + ) + # The exact twin's stand-ins for the modules, with common random + # numbers, and its exact expected values. + self.twin_sub: Optional["RepairableRBD"] = None + self.twin_module_states: dict = {} + self.exacts = _Exacts(0.0) + if control_variate and self.sub is not None: + if self.shards is not None: + raise ValueError( + "A run with control_variate simulates the system's twin " + "alongside it, here: leave out shard_map." + ) + twin, _ = _runs._twin(rbd) + self.exacts = _runs._twin_exact( + rbd, twin, T, working, broken, method, state + ) + self.twin_sub = _runs._modules_rbd(twin, modules) + twin_states = _curves._simulation_states( + twin, state, working | broken + ) + self.twin_module_states = { + node: twin_states[node] + for node in modules + if node in twin_states + } + self.twin_uptimes: List[np.ndarray] = [] + self.twin_costs: List[np.ndarray] = [] + self.curve_x = _conditional.steps(curve_points, T) + self.given: Dict[int, _conditional.Given] = {} + self.parts: List[Tuple[int, _conditional.Values]] = [] + self.module_up = np.zeros(len(modules)) + self.priced = rbd.has_costs + self.own_costs: List[np.ndarray] = [] + self.own_categories = dict.fromkeys(_CATEGORIES, 0.0) + self.own_components: Dict[Hashable, float] = {} + self.fixed = set(modules) + # The other nodes' states, as given (the modules are held). + self.other_states = ( + None + if state is None + else { + node: value + for node, value in state.items() + if node not in self.fixed + } + ) + # The rest given the modules, every node the crews serve among them. + self.exact = _runs._crew_free(rbd) + self._memo: dict = {} + self.n = 0 + + @contextmanager + def _curves(self): + """Within it, the exact part's curves are shared, as within + ``RepairableRBD._sharing_curves``, and kept from one round of + simulations to the next; out of it, the system holds none, so that + a plain run of it between rounds can go to other processes.""" + exact = self.exact + if exact._curve_memo is not None: + yield + return + exact._curve_memo = self._memo + try: + yield + finally: + exact._curve_memo = None + + def prepare(self) -> None: + """The other nodes' own expected down times and costs over the + window, which do not depend on the modules, and the grid the + system given each joint state is worked out on.""" + exact, T = self.exact, self.T + with self._curves(): + _, _, curves, self.totals, _, _ = _windows._window( + exact, + np.array([T]), + self.working | self.fixed, + self.broken, + self.method, + nodes=True, + state=self.other_states, + ) + self.exact_cost = ( + exact.expected_cost( + T, + self.working | self.fixed, + self.broken, + self.method, + state=self.other_states, + ) + if self.priced + else None + ) + self.others = ( + 0.0 + if self.exact_cost is None + else math.fsum( + float(np.ravel(value)[0]) + for key, value in self.exact_cost.by_category.items() + if key != "system_downtime" + ) + ) + self.x = _conditional.grid( + T, [curve_breaks(curve, 0.0, T) for curve in curves.values()] + ) + + def _given(self, paths: "_conditional.Paths") -> None: + """The system given each joint state ``paths`` meet, not worked + out yet.""" + from . import _runs + + new = [ + state for state in paths.states.tolist() if state not in self.given + ] + if not new: + return + with self._curves(): + for state in new: + self.given[state] = _runs._conditional_given( + self.exact, + self.modules, + state, + self.working, + self.broken, + self.method, + self.x, + self.other_states, + ) + if self.recorder is not None: + self.capacity_given[state] = _runs._capacity_given( + self.exact, + self.modules, + state, + self.working, + self.broken, + self.x, + self.other_states, + self.recorder.demand, + ) + + def simulate(self, first: int, count: int) -> None: + """Simulations ``first`` to ``first + count - 1``: the modules' + histories and own costs, and each simulation's expected values + given them (and its twin's, if controlled).""" + T, modules = self.T, self.modules + histories: List[Any] = [] + simulated = np.zeros(count) + if self.shards is not None: + histories, simulated, spent_on, by_module = _module_shards( + *self.shards, first, count, len(modules) + ) + for key, spent in spent_on.items(): + self.own_categories[key] += spent + for node, spent in zip(modules, by_module): + if spent is not None: + self.own_components[node] = ( + self.own_components.get(node, 0.0) + spent + ) + elif self.sub is not None: + # The modules' histories, and their own costs in the same + # simulations. + data, tally = _timeline_runs.with_costs( + self.sub, + T, + count, + None, + self.antithetic, + self.engine, + self.n_jobs, + first, + self.module_states, + entropy=self.entropy, + common=self.twin_sub is not None, + ) + histories = [data[node] for node in modules] + if self.sub.has_costs: + tally._fold() + simulated = np.asarray(tally.cost_samples, dtype=float) + for key, spent in tally.cost_by_category.items(): + self.own_categories[key] += float(spent) + for node, spent in tally.cost_by_component.items(): + self.own_components[node] = self.own_components.get( + node, 0.0 + ) + float(spent) + paths = _conditional.paths(histories, count, T) + self._given(paths) + self.parts.append( + (count, _conditional.values(paths, self.given, self.curve_x)) + ) + if self.recorder is not None: + self.capacity_parts.append( + _conditional.capacity_values( + paths, self.capacity_given, self.curve_x + ) + ) + if self.twin_sub is not None: + # The twin's modules in the same simulations: the other nodes + # are the same, and so is the system given them. + data, tally = _timeline_runs.with_costs( + self.twin_sub, + T, + count, + None, + self.antithetic, + self.engine, + self.n_jobs, + first, + self.twin_module_states, + entropy=self.entropy, + common=self.twin_sub is not None, + ) + twin_paths = _conditional.paths( + [data[node] for node in modules], count, T + ) + self._given(twin_paths) + twin_uptime = _conditional.values( + twin_paths, self.given, self.curve_x + ).uptime + self.twin_uptimes.append(twin_uptime) + spent = np.zeros(count) + if self.twin_sub.has_costs: + tally._fold() + spent = np.asarray(tally.cost_samples, dtype=float) + self.twin_costs.append( + spent + + self.others + + self.rbd.downtime_cost_rate * (T - twin_uptime) + ) + self.module_up += _conditional.module_totals(histories, count, T)[0] + self.own_costs.append(simulated) + self.n = first + count + + def uptimes(self) -> np.ndarray: + """Each simulation's expected up time given its modules.""" + return np.concatenate([values.uptime for _, values in self.parts]) + + def costs(self) -> np.ndarray: + """Each simulation's expected cost given its modules: their own + costs, the other nodes' expected costs, and the system's expected + down time's.""" + return ( + np.concatenate(self.own_costs) + + self.others + + self.rbd.downtime_cost_rate * (self.T - self.uptimes()) + ) + + def breakdown(self) -> _Breakdown: + """The expected cost of a window given the modules' histories (the + mean of ``costs``), by category and by component: the modules' own + costs in the simulations, the other nodes' expected costs, and the + system's expected down time's.""" + n, exact_cost = self.n, self.exact_cost + assert exact_cost is not None + by_category = { + key: self.own_categories[key] / n + + ( + 0.0 + if key == "system_downtime" + else float(np.ravel(exact_cost.by_category.get(key, 0.0))[0]) + ) + for key in _CATEGORIES + } + by_category["system_downtime"] = float( + np.mean(self.rbd.downtime_cost_rate * (self.T - self.uptimes())) + ) + by_component = { + node: float(np.ravel(value)[0]) + for node, value in exact_cost.by_component.items() + if node not in self.fixed + } + for node, amount in self.own_components.items(): + by_component[node] = amount / n + return by_category, by_component + + def controls( + self, + ) -> Tuple[Optional[ControlVariate], Optional[ControlVariate]]: + """The controls of the fractions up and of the costs, if the run is + controlled (the latter if priced).""" + if self.twin_sub is None: + return None, None + control = ControlVariate.of( + self.uptimes() / self.T, + np.concatenate(self.twin_uptimes) / self.T, + self.exacts[0], + self.antithetic, + ) + cost_control = None + if self.priced and self.exacts[1] is not None: + cost_control = ControlVariate.of( + self.costs(), + np.concatenate(self.twin_costs), + self.exacts[1], + self.antithetic, + ) + return control, cost_control + + def judged(self, target: str) -> np.ndarray: + """The values a run to a tolerance judges: of the ``target``'s mean + (see ``_stopping_rule``), controlled if the run is.""" + control, cost_control = self.controls() + if target == "cost": + values = self.costs() + return ( + values + if cost_control is None + else cost_control.controlled(values) + ) + values = self.uptimes() / self.T + return values if control is None else control.controlled(values) + + def unjudged(self) -> Optional[str]: + """Why the spread of the simulations' expected values given the + modules does not show the error (#215): they never changed state, + so their outages were not sampled, and every simulation's values + are the same. None once they have, or with no modules, when the + values are exact.""" + if len(self.given) > 1 or not self.modules: + return None + return ( + f"The modules {list(self.modules)} never changed state in the " + f"{self.n} simulations, so their outages were not sampled: run " + "more simulations to sample them." + ) + + def _warn_unjudged(self, instead: str) -> None: + reason = self.unjudged() + if reason is not None: + warnings.warn( + f"{reason} {instead}", + RuntimeWarning, + stacklevel=outside_level(), + ) + + def split(self, record: ConditionalRun) -> Optional[_Breakdown]: + """The expected cost's split a plain run takes with ``record``, its + means given the modules (see ``means``): ``breakdown``, where the + cost's mean is taken given them (priced, and the modules changed + state, see ``ConditionalRun.informative``); else None, and the + split is the simulations' own.""" + if not self.priced or not record.informative: + return None + return self.breakdown() + + def delivered(self) -> Optional[np.ndarray]: + """Each simulation's expected fraction of the demand delivered + given its modules' histories, in order; None without a demand.""" + if self.recorder is None or self.recorder.demand is None: + return None + return np.concatenate( + [piece.delivered for piece in self.capacity_parts] + ) / (self.recorder.demand * self.T) + + def means(self) -> ConditionalRun: + """The record of a plain run whose means are taken given its + modules (see ``RepairableRBD._conditioned_run``).""" + self._warn_unjudged( + "The means and their intervals are the simulations' own " + "(method='simulated')." + ) + return ConditionalRun( + tuple(self.modules), + len(self.given), + whole=True, + uptimes=self.uptimes(), + costs=self.costs() if self.priced else None, + delivered=self.delivered(), + ) + + def result(self) -> AvailabilityResult: + """The conditional run's result (see + ``RepairableRBD._conditional_run``).""" + self._warn_unjudged( + "The mean intervals have no error to give (nan): a plain run " + "(conditional=False) gives one." + ) + rbd, T, n = self.rbd, self.T, self.n + parts, curve_x = self.parts, self.curve_x + uptimes = self.uptimes() + failures = np.concatenate([v.failures for _, v in parts]) + planned = np.concatenate([v.planned for _, v in parts]) + restorations = np.concatenate([v.restorations for _, v in parts]) + curve = np.zeros(curve_x.size) + square = np.zeros(curve_x.size) + for size, part in parts: + curve += size * part.curve / n + square += size * part.curve_square / n + down = self.totals["downtime"] + node_downtime: Dict[Hashable, float] = {} + for node in rbd.components: + if node in self.working: + node_downtime[node] = 0.0 + elif node in self.broken: + node_downtime[node] = n * T + elif node in self.fixed: + j = self.modules.index(node) + node_downtime[node] = n * T - float(self.module_up[j]) + else: + node_downtime[node] = n * float(np.ravel(down[node])[0]) + node_uptime = { + node: n * T - value for node, value in node_downtime.items() + } + record = ConditionalRun(tuple(self.modules), len(self.given), square) + control, cost_control = self.controls() + cost_result = _acquisition_only(rbd, n, T, self.antithetic) + if self.priced: + by_category, by_component = self.breakdown() + cost_result = CostResult( + samples=self.costs(), + t_simulation=T, + n_simulations=n, + acquisition_cost=rbd.acquisition_cost, + by_category=by_category, + by_component=by_component, + antithetic=self.antithetic, + conditional=record, + control_variate=cost_control, + ) + capacity_fields: dict = {} + if self.recorder is not None: + time_at: Dict[float, float] = {} + for piece in self.capacity_parts: + for level, spent in piece.time_at.items(): + time_at[level] = time_at.get(level, 0.0) + spent + mean = np.sum( + [piece.curve for piece in self.capacity_parts], axis=0 + ) + unlimited = np.sum( + [piece.unlimited for piece in self.capacity_parts], axis=0 + ) + capacity_fields = dict( + capacity_timeline=curve_x.copy(), + capacity=np.where(unlimited > 0, np.inf, mean / n), + capacity_time={ + level: time_at[level] for level in sorted(time_at) + }, + demand=self.recorder.demand, + delivered=self.delivered(), + ) + return AvailabilityResult( + timeline=curve_x, + availability=curve, + system_uptime=float(math.fsum(uptimes)), + time_simulated_to=T, + criticalities=None, + node_uptime=node_uptime, + node_downtime=node_downtime, + system_downtime=float(n * T - math.fsum(uptimes)), + system_failures=float(math.fsum(failures)), + system_restorations=float(math.fsum(restorations)), + n_simulations=n, + cost=cost_result, + system_planned_outages=float(math.fsum(planned)), + uptimes=uptimes, + antithetic=self.antithetic, + conditional=record, + control_variate=control, + **capacity_fields, + ) + + +def _exact_controls( + tally: "_Tally", + t_simulation: float, + exacts: _Exacts, + antithetic: bool, +) -> Tuple[ControlVariate, Optional[ControlVariate], Optional[ControlVariate]]: + """The controls of a run's fractions up, costs and delivered + fractions by the system itself, whose expected values the exact + methods work out (#187): ``exacts``, its mean availability over the + window, its expected cost (None unpriced) and the fraction of the + demand delivered (None without one, or where ``mission_capacity`` + refuses). Its intervals are then those values, with no error (see + ``ControlVariate.itself``).""" + exact, exact_cost = exacts.availability, exacts.cost + fractions = np.asarray(tally.uptimes, dtype=float) / t_simulation + control = ControlVariate.of( + fractions, fractions, exact, antithetic, itself=True + ) + cost_control = None + if exact_cost is not None: + costs = np.asarray(tally.cost_samples, dtype=float) + cost_control = ControlVariate.of( + costs, costs, exact_cost, antithetic, itself=True + ) + delivered_control = None + if exacts.delivered is not None and tally.delivered: + delivered = np.asarray(tally.delivered, dtype=float) + delivered_control = ControlVariate.of( + delivered, delivered, exacts.delivered, antithetic, itself=True + ) + return control, cost_control, delivered_control + + +def _chunk_means(control_variate, conditional) -> Dict[str, Optional[bool]]: + """A chunk's (or a merge's) ``control_variate`` and ``conditional``, + checked (#236): None, the means ``availability`` takes by default, or + False, the simulations' own; a chunk holds the system's simulations + alone, not a twin's or its modules' alone, which True would take.""" + out: Dict[str, Optional[bool]] = {} + for name, value, alone in ( + ("control_variate", control_variate, "its exact twin's"), + ("conditional", conditional, "its modules' alone"), + ): + if isinstance(value, (bool, np.bool_)) and not value: + value = False + elif value is not None: + raise ValueError( + f"{name} must be None or False for chunks, got {value!r}: a " + f"chunk holds the system's simulations, not {alone}, which " + f"{name}=True simulates. Run the whole run with " + "availability() for that." + ) + out[name] = value + return out + + +def _acquisition_only( + rbd: "RepairableRBD", N: int, t_simulation: float, antithetic: bool +) -> Optional[CostResult]: + """The cost result of ``N`` simulations of ``rbd`` when nothing but its + ``acquisition_cost`` is priced (#234): no running cost, exactly (its + mean interval ``"exact"``), with the acquisition beside it; None when + nothing at all is.""" + if not rbd.acquisition_cost: + return None + zeros = np.zeros(N) + return CostResult( + samples=zeros, + t_simulation=t_simulation, + n_simulations=N, + acquisition_cost=rbd.acquisition_cost, + by_category={category: 0.0 for category in _CATEGORIES}, + by_component={}, + antithetic=antithetic, + control_variate=ControlVariate( + twin=zeros, + exact=0.0, + coefficient=0.0, + correlation=0.0, + itself=True, + ), + ) + + +def _shard_bytes(template: dict, start: int, stop: int) -> bytes: + """A shard (see ``RepairableRBD.shards``): the run's ``template`` with + the range of its simulations, as JSON.""" + return json.dumps({**template, "start": start, "stop": stop}).encode() + + +#: The cost categories, in the order ``CostResult.by_category`` lists them. +_CATEGORIES = ( + "repair", + "replace", + "preventive", + "inspection", + "component_downtime", + "system_downtime", + "setup", +) + + +# The category of a repair's own cost (all an imperfect repair is charged). +_REPAIR = _CATEGORIES.index("repair") + + +class _Replication: + """One simulation's results, which a :class:`_Tally` adds to its totals + in the order of the simulations. + + ``node_up``, ``both_up`` and ``both_down`` hold each component's up + time, and the time it and the system were both up and both down; + ``counts`` each component's failures, the system failures they caused, + its restorations and the system restorations they caused; and + ``changes`` the times the system changed state, with ``deltas`` +1 for + a restoration and -1 for a failure or planned outage. ``cost`` is the + simulation's total cost (None when nothing is priced), ``by_category`` + and ``by_node`` its breakdown, and the ``capacity`` fields what the + capacity trace recorded (see ``_CapacityTrace``). ``replacements`` + counts each component's replacements (its failures, found ones for a + component with hidden failures, its preventive replacements, and a + standby group's units' failures): the spares it used. + """ + + __slots__ = ( + "uptime", + "node_up", + "both_up", + "both_down", + "counts", + "failures", + "restorations", + "planned", + "changes", + "deltas", + "cost", + "by_category", + "by_node", + "capacity_changes", + "capacity_time", + "delivered", + "replacements", + "opportunistic", + "history", + ) + uptime: float + node_up: List[float] + both_up: List[float] + both_down: List[float] + counts: Tuple[List[int], List[int], List[int], List[int]] + failures: int + restorations: int + planned: int + changes: List[float] + deltas: List[int] + cost: Optional[float] + by_category: List[float] + by_node: Dict[Any, float] + capacity_changes: Optional[list] + capacity_time: Optional[list] + delivered: Optional[float] + replacements: List[int] + opportunistic: List[int] + history: Optional[tuple] + + +class _Tally: + """The running totals of an availability simulation. + + Every total that is a sum of floats is kept exactly, as an ``ExactSum`` + (see ``_exact``), and rounded once when the result is built (#151): so + the totals are the same to the last bit however the simulations ran, + one after another in this process, in blocks in other processes, + compiled, or in chunks merged later. Per-node totals are lists in the + order of ``nodes``; the per-simulation values (``uptimes``, + ``cost_samples``, ``delivered``) are kept in the simulations' order. + """ + + #: Simulations whose per-node values wait to be folded into the + #: per-node totals (see ``_fold``). + _ROWS = 256 + + def __init__( + self, + nodes: list, + costs: dict, + t_simulation: float, + curve_points: Optional[int] = None, + ): + n = len(nodes) + self.nodes = nodes + self.t_simulation = t_simulation + self.n = 0 + # The times the simulated systems changed state, and +1 or -1; and + # the same, as arrays, from the compiled engine. + self.changes: list = [] + self.deltas: list = [] + self.change_arrays: list = [] + # With curve_points (#153), the changes are counted on a grid of + # times instead of kept: the net change in how many systems work + # in each bin up to each grid time (from just after the one + # before), the grid's first time 0. + self.curve_points = curve_points + self.edges: Optional[np.ndarray] = None + self.binned: Optional[np.ndarray] = None + if curve_points is not None: + steps = np.arange(curve_points + 1) + self.edges = t_simulation * steps / curve_points + self.binned = np.zeros(curve_points + 1, dtype=np.int64) + # Per node: its failures, the system failures they caused, its + # restorations and the system restorations they caused. + self.counts = [[0] * n for _ in range(4)] + self.system_restorations = 0 + self.system_failures = 0 + self.system_planned_outages = 0 + self.system_uptime = ExactSum() + self.system_downtime = ExactSum() + self.node_uptime = [ExactSum() for _ in range(n)] + self.node_downtime = [ExactSum() for _ in range(n)] + self.intersection_uptime = [ExactSum() for _ in range(n)] + self.intersection_downtime = [ExactSum() for _ in range(n)] + self.union_uptime = [ExactSum() for _ in range(n)] + self.union_downtime = [ExactSum() for _ in range(n)] + # Each simulation's up time, its nodes' up times and times up and + # down with the system, and its costs, until they are folded into + # the totals (see _fold). + self._rows: list = [] + self._cost_rows: list = [] + # One system uptime (and, when priced, one total cost) per + # replication, in order. + self.uptimes: List[float] = [] + self.cost_samples: List[float] = [] + self.cost_by_category = {key: ExactSum() for key in _CATEGORIES} + self.cost_by_component = {node: ExactSum() for node in costs} + # With capacities (see _CapacityRecorder): the times a simulated + # system's expected capacity changed and by how much, and the times + # whether it can carry an unlimited amount did (+1 or -1), as lists + # from the Python engine and arrays from the compiled one (see + # capacity_records); the time spent at each capacity (exactly, see + # _add_at); and each replication's delivered fraction of the + # demand, in order. + self.capacity_times: list = [] + self.capacity_steps: list = [] + self.unlimited_times: list = [] + self.unlimited_steps: list = [] + self.capacity_arrays: list = [] + # With curve_points (#190), the changes of the expected capacity are + # counted on the grid instead, as the changes of state are: in each + # bin, exactly (arrays whose exact sum, bin by bin, is the bin's + # total change), and the net change in how many can carry an + # unlimited amount. + self.capacity_binned: Optional[list] = None + self.unlimited_binned: Optional[np.ndarray] = None + if self.edges is not None: + self.capacity_binned = [] + self.unlimited_binned = np.zeros(len(self.edges), dtype=np.int64) + self.capacity_time: dict = {} + self.delivered: List[float] = [] + # Each replication's replacements of each node, when asked for + # (see RepairableRBD.spares_demand). + self.replacements: Optional[List[List[int]]] = None + # Per node: its early renewals at stops of its maintenance group. + self.opportunistic = [0] * n + # The simulations' histories, when asked for (see + # RepairableRBD.simulate_timelines): kept, in order, in place of + # every total. + self.histories: Optional[_timeline_runs.Records] = None + + def add(self, rec: _Replication) -> None: + """Add the next simulation's results.""" + if self.histories is not None: + assert rec.history is not None # recorded (see _replicate) + self.histories.add(rec.history) + self.n += 1 + # Each simulation's cost is kept beside its histories (for a + # conditional run's modules, see _conditional_run). + if rec.cost is not None: + self.cost_samples.append(rec.cost) + self._cost_rows.append( + [*rec.by_category, *rec.by_node.values()] + ) + return + self.n += 1 + self.uptimes.append(rec.uptime) + self.system_failures += rec.failures + self.system_restorations += rec.restorations + self.system_planned_outages += rec.planned + self.changes.extend(rec.changes) + self.deltas.extend(rec.deltas) + self._rows.append( + [rec.uptime, *rec.node_up, *rec.both_up, *rec.both_down] + ) + for totals, counts in zip(self.counts, rec.counts): + for c, count in enumerate(counts): + if count: + totals[c] += count + if rec.cost is not None: + self.cost_samples.append(rec.cost) + self._cost_rows.append([*rec.by_category, *rec.by_node.values()]) + if len(self._rows) >= self._ROWS: + self._fold() + if rec.capacity_changes is not None: + for time, mean, limitless in rec.capacity_changes: + if mean: + self.capacity_times.append(time) + self.capacity_steps.append(mean) + if limitless: + self.unlimited_times.append(time) + self.unlimited_steps.append(limitless) + spent = self.capacity_time + for level, time in rec.capacity_time or (): + _add_at(spent, level, time) + if rec.delivered is not None: + self.delivered.append(rec.delivered) + if self.replacements is not None: + self.replacements.append(rec.replacements) + for c, count in enumerate(rec.opportunistic): + if count: + self.opportunistic[c] += count + + def add_changes(self, times: np.ndarray, deltas: np.ndarray) -> None: + """Add simulations' changes of state (times, and +1 or -1), as an + engine gives them: kept, or counted on the grid.""" + self.change_arrays.append((times, deltas)) + if self.binned is not None: + self._fold_changes() + + def add_capacity(self, arrays: tuple) -> None: + """Add simulations' changes of capacity, as an engine gives them + (see ``capacity_records``): kept, or counted on the grid.""" + self.capacity_arrays.append(arrays) + if self.capacity_binned is not None: + self._fold_capacity() + + def _grid_bins(self, times: np.ndarray) -> np.ndarray: + """The grid's bin of each of ``times``: that of the first grid time + at or after it (``-0.0`` taken as ``0.0``).""" + assert self.edges is not None + return np.minimum( + np.searchsorted(self.edges, times + 0.0, side="left"), + len(self.edges) - 1, + ) + + def _fold_capacity(self) -> None: + """Count the changes of capacity kept so far in the grid's bins, + exactly, and drop them.""" + assert self.capacity_binned is not None + assert self.unlimited_binned is not None + size = len(self.unlimited_binned) + for times, steps, free_times, free_steps in self.capacity_arrays: + self.capacity_binned.extend( + binned_parts(self._grid_bins(times), steps, size) + ) + if len(free_times): + self.unlimited_binned += np.bincount( + self._grid_bins(free_times), + weights=free_steps, + minlength=size, + ).astype(np.int64) + self.capacity_arrays = [] + if len(self.capacity_binned) > 64: + self.capacity_binned = compact_parts(self.capacity_binned, size) + + def _fold_changes(self) -> None: + """Count the changes kept so far in the grid's bins (each time in + the bin of the first grid time at or after it), and drop them.""" + assert self.binned is not None and self.edges is not None + if not (self.changes or self.change_arrays): + return + times, deltas = self.state_changes() + self.changes, self.deltas, self.change_arrays = [], [], [] + bins = np.minimum( + np.searchsorted(self.edges, times, side="left"), + len(self.edges) - 1, + ) + counted = np.bincount(bins, weights=deltas, minlength=len(self.edges)) + self.binned += counted.astype(np.int64) + + def _fold(self) -> None: + """Fold the values of the simulations added since the last fold + into the totals, all at once (see ``fold_columns``), and the + changes into the grid's bins, with ``curve_points``.""" + if self.binned is not None: + self._fold_changes() + if self._rows: + table = np.asarray(self._rows, dtype=float) + self._rows = [] + n = len(self.nodes) + self.fold_columns( + table[:, 0], + table[:, 1 : 1 + n], # noqa: E203 + table[:, 1 + n : 1 + 2 * n], # noqa: E203 + table[:, 1 + 2 * n :], # noqa: E203 + ) + if self._cost_rows: + self.fold_costs(np.asarray(self._cost_rows, dtype=float)) + self._cost_rows = [] + if self.capacity_times or self.unlimited_times: + self.capacity_arrays.append( + ( + np.asarray(self.capacity_times, dtype=float), + np.asarray(self.capacity_steps, dtype=float), + np.asarray(self.unlimited_times, dtype=float), + np.asarray(self.unlimited_steps, dtype=np.int64), + ) + ) + self.capacity_times, self.capacity_steps = [], [] + self.unlimited_times, self.unlimited_steps = [], [] + if self.capacity_binned is not None and self.capacity_arrays: + self._fold_capacity() + + def compact(self) -> "_Tally": + """This tally with its simulations' values folded into the totals + and its changes of state (and of capacity) in arrays: small to send + from a worker process to the parent (see ``_simulate_block``).""" + if self.histories is not None: + self.histories.compact() + if self._cost_rows: + self.fold_costs(np.asarray(self._cost_rows, dtype=float)) + self._cost_rows = [] + return self + self._fold() + if self.changes: + times, deltas = self.state_changes() + self.changes, self.deltas = [], [] + self.change_arrays = [(times, deltas)] + return self + + def fold_columns(self, uptime, up, both_up, both_down) -> None: + """Add simulations' values to the totals, exactly (one row per + simulation): the system's up time (and so its down time), each + node's up time (and down time), and its times up and down with the + system (and so either up or either down).""" + t_end = self.t_simulation + uptime = np.asarray(uptime, dtype=float).reshape(-1, 1) + add_columns( + [ + self.system_uptime, + self.system_downtime, + *self.node_uptime, + *self.node_downtime, + *self.intersection_uptime, + *self.intersection_downtime, + *self.union_uptime, + *self.union_downtime, + ], + np.hstack( + [ + uptime, + t_end - uptime, + up, + t_end - up, + both_up, + both_down, + t_end - both_down, + t_end - both_up, + ] + ), + ) + + def fold_costs(self, table) -> None: + """Add simulations' costs to the totals, exactly: one row per + simulation, its costs by category and then by component (in the + order of ``cost_by_component``).""" + add_columns( + [ + *self.cost_by_category.values(), + *self.cost_by_component.values(), + ], + table, + ) + + def state_changes(self) -> Tuple[np.ndarray, np.ndarray]: + """Every time a simulated system changed state (``-0.0`` made + ``0.0``), and +1 for a restoration or -1 for a failure or planned + outage: new arrays, which the caller may overwrite.""" + times = [np.asarray(self.changes, dtype=float)] + deltas = [np.asarray(self.deltas, dtype=np.int64)] + for chunk_times, chunk_deltas in self.change_arrays: + times.append(chunk_times) + deltas.append(chunk_deltas) + return _zeroed(np.concatenate(times)), np.concatenate(deltas) + + def capacity_records(self) -> Tuple[np.ndarray, ...]: + """Every change of a simulated system's expected capacity, as + times (``-0.0`` made ``0.0``) and changes, whose exact sum at a + time is the total change then; and every change of whether it can + carry an unlimited amount, as times and changes in how many systems + can (see ``capacity_changes``): new arrays, which the caller may + overwrite.""" + times = [np.asarray(self.capacity_times, dtype=float)] + steps = [np.asarray(self.capacity_steps, dtype=float)] + free_times = [np.asarray(self.unlimited_times, dtype=float)] + free_steps = [np.asarray(self.unlimited_steps, dtype=np.int64)] + for arrays in self.capacity_arrays: + for kept, array in zip( + (times, steps, free_times, free_steps), arrays + ): + kept.append(array) + return ( + _zeroed(np.concatenate(times)), + np.concatenate(steps), + _zeroed(np.concatenate(free_times)), + np.concatenate(free_steps), + ) + + def capacity_changes(self) -> Tuple[np.ndarray, ...]: + """The times a simulated system's expected capacity changed, in + order, the changes then, in the order of their times, and where + each time's changes start (their exact sum is the total change + then, see ``_group_totals``); and the times whether it can carry an + unlimited amount changed, in order, and the net change then in how + many can.""" + times, steps, free_times, free_steps = self.capacity_records() + at, steps, starts = _by_time(times, steps) + # Whole numbers, each time's added exactly (as floats, below 2**53). + free_at, free_steps, firsts = _by_time(free_times, free_steps) + counts = ( + np.add.reduceat(free_steps, firsts) if firsts.size else free_steps + ) + return at, steps, starts, free_at, counts.astype(np.int64) + + # The totals that are lists of floats, one per node, and the counts. + _NODE_SUMS = ( + "node_uptime", + "node_downtime", + "intersection_uptime", + "intersection_downtime", + "union_uptime", + "union_downtime", + ) + _COUNTS = ( + "system_restorations", + "system_failures", + "system_planned_outages", + ) + + def merge(self, other: "_Tally") -> None: + """Add ``other``'s simulations, which come after this tally's (see + ``SimulationChunk``). The per-simulation values follow on in order, + and the totals, kept exactly, are those of one run of them all, to + the last bit.""" + if self.histories is not None: + assert other.histories is not None # blocks of one run + self.histories.merge(other.histories) + self.n += other.n + self.cost_samples.extend(other.cost_samples) + for rows in (self, other): + if rows._cost_rows: + rows.fold_costs(np.asarray(rows._cost_rows, dtype=float)) + rows._cost_rows = [] + for key, amount in other.cost_by_category.items(): + self.cost_by_category[key].add(amount) + for node, amount in other.cost_by_component.items(): + self.cost_by_component[node].add(amount) + return + self._fold() + other._fold() + self.n += other.n + self.system_uptime.add(other.system_uptime) + self.system_downtime.add(other.system_downtime) + for name in self._COUNTS: + setattr(self, name, getattr(self, name) + getattr(other, name)) + for name in self._NODE_SUMS: + for mine, theirs in zip(getattr(self, name), getattr(other, name)): + mine.add(theirs) + for totals, counts in zip(self.counts, other.counts): + for c, count in enumerate(counts): + totals[c] += count + for c, count in enumerate(other.opportunistic): + self.opportunistic[c] += count + if self.binned is not None: + assert other.binned is not None # chunks of one run + self.binned += other.binned + else: + times, deltas = other.state_changes() + self.change_arrays.append((times, deltas)) + self.uptimes.extend(other.uptimes) + self.cost_samples.extend(other.cost_samples) + for key, amount in other.cost_by_category.items(): + self.cost_by_category[key].add(amount) + for node, amount in other.cost_by_component.items(): + self.cost_by_component[node].add(amount) + if self.capacity_binned is not None: + # Chunks of one run, on one grid. + assert other.capacity_binned is not None + assert self.unlimited_binned is not None + self.capacity_binned.extend(other.capacity_binned) + self.unlimited_binned += other.unlimited_binned + else: + self.capacity_arrays.append(other.capacity_records()) + for level, time in other.capacity_time.items(): + _add_at(self.capacity_time, level, time) + self.delivered.extend(other.delivered) + if self.replacements is not None: + assert other.replacements is not None # blocks of one run + self.replacements.extend(other.replacements) + + def to_dict(self) -> dict: + """The totals, as JSON data (see ``SimulationChunk.to_dict``): each + exact total as the floats whose exact sum it is.""" + self._fold() + times, deltas = self.state_changes() + at, steps, starts, free_at, counts = self.capacity_changes() + return { + "nodes": list(self.nodes), + "t_simulation": self.t_simulation, + "n": self.n, + "changes": times.tolist(), + "deltas": deltas.tolist(), + "curve_points": self.curve_points, + "binned": None if self.binned is None else self.binned.tolist(), + "counts": [list(counts) for counts in self.counts], + **{name: getattr(self, name) for name in self._COUNTS}, + "system_uptime": self.system_uptime.partials, + "system_downtime": self.system_downtime.partials, + **{ + name: [total.partials for total in getattr(self, name)] + for name in self._NODE_SUMS + }, + "uptimes": list(self.uptimes), + "cost_samples": list(self.cost_samples), + "cost_by_category": { + key: amount.partials + for key, amount in self.cost_by_category.items() + }, + "cost_by_component": [ + [node, amount.partials] + for node, amount in self.cost_by_component.items() + ], + "capacity_changes": list( + zip(at.tolist(), _group_partials(steps, starts)) + ), + "unlimited_changes": list(zip(free_at.tolist(), counts.tolist())), + "capacity_binned": ( + None + if self.capacity_binned is None + or self.unlimited_binned is None + else [ + part.tolist() + for part in compact_parts( + self.capacity_binned, len(self.unlimited_binned) + ) + ] + ), + "unlimited_binned": ( + None + if self.unlimited_binned is None + else self.unlimited_binned.tolist() + ), + "capacity_time": sorted( + (level, _partials(time)) + for level, time in self.capacity_time.items() + ), + "delivered": list(self.delivered), + "opportunistic": list(self.opportunistic), + } + + @classmethod + def from_dict(cls, data: dict) -> "_Tally": + """The totals ``to_dict`` gave (node names as JSON gives them + back).""" + from repyability.rbd.serialisation import _node_name + + nodes = [_node_name(node) for node in data["nodes"]] + components = [ + _node_name(node) for node, _ in data["cost_by_component"] + ] + tally = cls( + nodes, + dict.fromkeys(components), + data["t_simulation"], + data.get("curve_points"), + ) + tally.n = int(data["n"]) + tally.changes = [float(t) for t in data["changes"]] + tally.deltas = [int(d) for d in data["deltas"]] + if data.get("binned") is not None: + tally.binned = np.array(data["binned"], dtype=np.int64) + tally.counts = [[int(c) for c in counts] for counts in data["counts"]] + for name in cls._COUNTS: + setattr(tally, name, int(data[name])) + tally.system_uptime = _exact_total(data["system_uptime"]) + tally.system_downtime = _exact_total(data["system_downtime"]) + for name in cls._NODE_SUMS: + setattr(tally, name, [_exact_total(v) for v in data[name]]) + tally.uptimes = [float(v) for v in data["uptimes"]] + tally.cost_samples = [float(v) for v in data["cost_samples"]] + tally.cost_by_category.update( + { + key: _exact_total(v) + for key, v in data["cost_by_category"].items() + } + ) + tally.cost_by_component = { + _node_name(node): _exact_total(amount) + for node, amount in data["cost_by_component"] + } + for level, value in data["capacity_time"]: + tally.capacity_time[float(level)] = _exact_total(value) + for time, value in data["capacity_changes"]: + # A total change of 0 still marks its time (as one 0.0). + for part in _exact_total(value).partials or [0.0]: + tally.capacity_times.append(float(time)) + tally.capacity_steps.append(part) + for time, change in data["unlimited_changes"]: + tally.unlimited_times.append(float(time)) + tally.unlimited_steps.append(int(change)) + if data.get("capacity_binned") is not None: + tally.capacity_binned = [ + np.array(part, dtype=float) for part in data["capacity_binned"] + ] + if data.get("unlimited_binned") is not None: + tally.unlimited_binned = np.array( + data["unlimited_binned"], dtype=np.int64 + ) + tally.delivered = [float(v) for v in data["delivered"]] + tally.opportunistic = [int(c) for c in data["opportunistic"]] + return tally + + def criticality_counts(self) -> Tuple[dict, dict]: + """The failure and restoration counts behind the criticality + indices: node -> Counter, for each node that failed (FCI) or was + restored (RCI), in the order of the nodes.""" + failed, caused_down, restored, caused_up = self.counts + FCI = { + node: Counter( + component_failures=failed[c], system_failures=caused_down[c] + ) + for c, node in enumerate(self.nodes) + if failed[c] + } + RCI = { + node: Counter( + component_restorations=restored[c], + system_restorations=caused_up[c], + ) + for c, node in enumerate(self.nodes) + if restored[c] + } + return FCI, RCI + + +class _CapacityState(NamedTuple): + """The system's capacity in one state of the simulation (see + ``_CapacityRecorder``).""" + + levels: tuple + probabilities: tuple + # The expected capacity when it is finite (0 otherwise), and whether it + # can be unlimited. + finite_mean: float + unlimited: int + # The expected fraction of the demand met, if there is a demand. + delivered: float + + +class _CapacityRecorder: + """The system's capacity in each state of an availability simulation. + + Given which components are down, the capacity has an exact + distribution: one level, unless some nodes work at several (a node's + capacity levels, given as a dict), which the simulation, drawing only + whether each component is up, then weighs by their probabilities. It is + worked out once per state and kept.""" + + def __init__(self, rbd: "RepairableRBD", demand: Optional[float]): + _requirements._require_capacities_given(rbd) + self._rbd = rbd + self._nodes = list(rbd.nodes) + self._states: Dict[frozenset, _CapacityState] = {} + # Whether each node works at one level: a state's capacity is then + # one level for sure (see evaluate). + self._single = all( + not isinstance(levels, dict) or len(levels) == 1 + for levels in rbd.capacity.values() + ) + if demand is None: + # Everything up at its highest level: the design capacity. + top = float(self._distribution(frozenset())[0][-1]) + self.demand: Optional[float] = top if np.isfinite(top) else None + else: + demand = float(demand) + if not (np.isfinite(demand) and demand > 0.0): + raise ValueError( + f"demand must be a positive, finite number, got " + f"{demand!r}." + ) + self.demand = demand + + def _distribution(self, down: frozenset) -> Tuple[np.ndarray, np.ndarray]: + arrays = { + node: np.zeros(1) if node in down else np.ones(1) + for node in self._nodes + } + levels, rows = self._rbd._capacity_arrays(arrays, 1) + return levels, rows[:, 0] + + def __call__(self, down: frozenset) -> _CapacityState: + state = self._states.get(down) + if state is None: + state = self._states[down] = self._state(*self._distribution(down)) + return state + + def _state( + self, levels: np.ndarray, probabilities: np.ndarray + ) -> _CapacityState: + """The state whose capacity has ``levels`` with ``probabilities`` + (those that can be reached).""" + finite = np.isfinite(levels) + unlimited = int(np.any(probabilities[~finite] > 0.0)) + delivered = 0.0 + if self.demand is not None: + delivered = ( + float(np.minimum(levels, self.demand) @ probabilities) + / self.demand + ) + return _CapacityState( + tuple(levels.tolist()), + tuple(probabilities.tolist()), + ( + 0.0 + if unlimited + else float(levels[finite] @ probabilities[finite]) + ), + unlimited, + delivered, + ) + + def evaluate(self, components: list, up: np.ndarray) -> list: + """The states with ``components`` up as each row of ``up`` (1 or 0 + for each) says, the others up. When each node works at one level, + they are worked out together, one column of probabilities each: + each probability of each is then a product of 0s and 1s, or a sum + with one 1 at most, so exact, and the same as in the state's own + distribution (``__call__``), whatever else is worked out with it. A + state that does not come out as one level for sure is worked out + on its own all the same.""" + states: list = [None] * len(up) + if self._single and len(up) > 1: + ones = np.ones(len(up)) + given = dict(zip(components, np.asarray(up, dtype=float).T)) + levels, rows = self._rbd._capacity_arrays( + {node: given.get(node, ones) for node in self._nodes}, + len(up), + ) + top = rows.argmax(axis=0) + sure = (np.count_nonzero(rows, axis=0) == 1) & ( + rows[top, np.arange(len(up))] == 1.0 + ) + # A state of one level for sure is its level's. + alone: dict = {} + for i in np.flatnonzero(sure).tolist(): + level = int(top[i]) + state = alone.get(level) + if state is None: + state = alone[level] = self._state( + levels[level : level + 1], np.ones(1) # noqa: E203 + ) + states[i] = state + for i, state in enumerate(states): + if state is None: + states[i] = self( + frozenset( + c for c, works in zip(components, up[i]) if not works + ) + ) + return states + + def states(self, downs: list) -> list: + """The state of each of ``downs`` (the components down), those not + met yet worked out together (see ``evaluate``).""" + known = self._states + new = [down for down in dict.fromkeys(downs) if down not in known] + if len(new) > 1 and self._single: + up = [[node not in down for node in self._nodes] for down in new] + for down, state in zip( + new, self.evaluate(self._nodes, np.array(up)) + ): + known[down] = state + return [self(down) for down in downs] + + def trace(self, status: dict) -> "_CapacityTrace": + """The capacity of one simulation, which starts with ``status``.""" + return _CapacityTrace(self, status) + + +class _CapacityTrace: + """One simulation's capacity over time (for the tally to add up in + order): each change of the expected capacity, and the time spent at + each level and the fraction of the demand delivered. The components + down after each change are recorded as it goes, and the states they + are in worked out at the end, those met first together (see + ``_CapacityRecorder.states``).""" + + def __init__(self, recorder: _CapacityRecorder, status): + self.recorder = recorder + self.down = {node for node, up in status.items() if not up} + # The time of each change and the components down after it, from + # the start. + self.records: list = [(0.0, frozenset(self.down))] + + def change(self, time: float, node, up: bool) -> None: + """Component ``node`` went up (or down) at ``time``.""" + if up: + self.down.discard(node) + else: + self.down.add(node) + self.records.append((time, frozenset(self.down))) + + def finish(self, t_simulation: float, rec: _Replication) -> None: + """The trace, closed at the end of the window, into ``rec``: (time, + change of the expected capacity, change of whether it can be + unlimited), from the start's, and (level, time spent at it), in + order.""" + records = self.records + [(t_simulation, None)] + states = self.recorder.states([down for _, down in self.records]) + state = states[0] + since = delivered = 0.0 + changes = [(0.0, state.finite_mean, state.unlimited)] + spent = [] + for (time, _), new in zip(records[1:], states[1:] + [None]): + span = time - since + if span > 0.0: + for level, p in zip(state.levels, state.probabilities): + spent.append((level, p * span)) + delivered += state.delivered * span + since = time + if new is not None: + mean = new.finite_mean - state.finite_mean + unlimited = new.unlimited - state.unlimited + if mean or unlimited: + changes.append((time, mean, unlimited)) + state = new + rec.capacity_changes = changes + rec.capacity_time = spent + rec.delivered = ( + None if self.recorder.demand is None else delivered / t_simulation + ) diff --git a/repyability/rbd/_time_order.py b/repyability/rbd/_time_order.py new file mode 100644 index 00000000..007254b5 --- /dev/null +++ b/repyability/rbd/_time_order.py @@ -0,0 +1,208 @@ +"""A run's changes put in time order, by numpy alone (#201, #208). + +``_net_by_time`` nets the +1 and -1 changes of state with one sort of +integers, each time's bits above the change's sign (a time is never +negative); ``_by_time`` sorts the capacity's (time, change) pairs, and the +changes in how many systems can carry an unlimited amount, as complex +numbers; ``_capacity_totals`` merges the two the first way. What is made +of a time's changes is exact (whole counts, exact sums), so their order +within a time does not matter. ``test_time_order.py`` checks each against +its definition written out plainly. +""" + +from typing import ( + TYPE_CHECKING, + Tuple, +) + +import numpy as np + +from repyability.rbd._exact import ( + ExactSum, +) + +if TYPE_CHECKING: + pass + + +def _net_by_time( + times: np.ndarray, steps: np.ndarray +) -> Tuple[np.ndarray, np.ndarray]: + """Changes of +1 or -1 (``steps``) at ``times`` (none negative), netted + by time: the distinct times, in order, and the net change at each. + + A time is never negative, so its sign bit is free: each change is one + integer, its time's bits above whether it is +1, and one sort of the + integers puts the changes in time order (``-0.0`` sorts as ``0.0``).""" + keys = times.view(np.uint64) << np.uint64(1) + keys |= steps > 0 + keys.sort() + if not keys.size: + return np.zeros(0), np.zeros(0, dtype=np.int64) + at = keys >> np.uint64(1) + new = np.empty(keys.size, dtype=bool) + new[0] = True + np.not_equal(at[1:], at[:-1], out=new[1:]) + firsts = np.flatnonzero(new) + ups = np.add.reduceat((keys & np.uint64(1)).astype(np.int64), firsts) + sizes = np.diff(np.append(firsts, keys.size)) + return at[firsts].view(np.float64), 2 * ups - sizes + + +def _working_over_time( + changed_at: np.ndarray, + deltas: np.ndarray, + t_end: float, + start: int, +) -> Tuple[np.ndarray, np.ndarray]: + """How many simulated systems work after each time at which one changed + state (and at 0 and ``t_end`` whether or not any did): the times, in + order, and the counts, ``start`` working at 0. Each change is +1 or -1 + (``deltas``); the counts are whole numbers added exactly.""" + times, net = _net_by_time(changed_at, deltas) + time, weights = _with_ends(times, net.astype(float), t_end) + weights[0] += start + return time, weights.cumsum() + + +def _with_ends( + times: np.ndarray, values: np.ndarray, t_end: float +) -> Tuple[np.ndarray, np.ndarray]: + """``times`` (in order, each once) with 0 and ``t_end`` among them, and + ``values`` at their times (0 at an end that had none).""" + for end in (0.0, t_end): + i = int(np.searchsorted(times, end)) + if i == times.size or times[i] != end: + times = np.insert(times, i, end) + values = np.insert(values, i, 0) + return times, values + + +def _add_at(totals: dict, key, value) -> None: + """Add ``value`` (a float or an ``ExactSum``) to ``totals[key]``, + exactly: a float while only one value has come, an ``ExactSum`` once + another does.""" + held = totals.get(key) + if held is None: + totals[key] = ExactSum(value) if isinstance(value, ExactSum) else value + elif isinstance(held, ExactSum): + held.add(value) + else: + totals[key] = ExactSum((held,)).add(value) + + +def _partials(value) -> list: + """The floats whose exact sum is ``value`` (see ``_add_at``).""" + return value.partials if isinstance(value, ExactSum) else [value] + + +def _by_time( + times: np.ndarray, values: np.ndarray +) -> Tuple[np.ndarray, np.ndarray, np.ndarray]: + """``values`` grouped by their ``times``: the distinct times, in order; + the values, in the order of their times; and where each time's values + start. A time's values are in no particular order: what is made of + them (``_group_totals``, ``_group_partials``) is exact. + + Each (time, value) pair is a complex number, which numpy sorts by its + real part, then its imaginary one: one sort of the pairs.""" + pairs = np.empty(times.size, dtype=complex) + pairs.real = times + pairs.imag = values + pairs.sort() + times, values = pairs.real, pairs.imag + new = np.ones(times.size, dtype=bool) + new[1:] = times[1:] != times[:-1] + starts = np.flatnonzero(new) + return times[starts], values, starts + + +def _zeroed(times: np.ndarray) -> np.ndarray: + """``times``, a new array, with ``-0.0`` made ``0.0`` (``+ 0.0``, in + place).""" + return np.add(times, 0.0, out=times) + + +def _capacity_totals( + at: np.ndarray, + change: np.ndarray, + free_at: np.ndarray, + counts: np.ndarray, + t_end: float, +) -> Tuple[np.ndarray, np.ndarray, np.ndarray]: + """The times at which a simulated system's expected capacity (by + ``change``, at the times ``at``) or whether it can carry an unlimited + amount (by the net ``counts`` of systems that can, at ``free_at``) + changed, and 0 and ``t_end``: in order; the systems' total expected + capacity after each (the running total of the changes, as + ``np.cumsum`` adds them); and whether any can carry an unlimited amount + then. ``at`` and ``free_at`` are in order, each time once (as + ``_by_time`` gives them). + + The two are merged as ``_net_by_time`` merges its changes, each time's + bits above whether it is ``free_at``'s: the merged times of each keep + their order, so their changes follow them in turn.""" + keys = np.concatenate( + ( + at.view(np.uint64) << np.uint64(1), + (free_at.view(np.uint64) << np.uint64(1)) | np.uint64(1), + ) + ) + keys.sort() + bits = keys >> np.uint64(1) + new = np.ones(keys.size, dtype=bool) + new[1:] = bits[1:] != bits[:-1] + group = np.cumsum(new) - 1 + free = (keys & np.uint64(1)).astype(bool) + times = bits[new].view(np.float64) + step = np.zeros(times.size) + step[group[~free]] = change + freed = np.zeros(times.size, dtype=np.int64) + freed[group[free]] = counts + times, step = _with_ends(times, step, t_end) + _, freed = _with_ends(bits[new].view(np.float64), freed, t_end) + return times, np.cumsum(step), np.cumsum(freed) > 0 + + +def _group_stops(values: np.ndarray, starts: np.ndarray) -> np.ndarray: + """Where each group of ``values`` that starts at one of ``starts`` + stops.""" + stops = np.empty_like(starts) + stops[:-1] = starts[1:] + stops[-1:] = values.size + return stops + + +def _group_totals(values: np.ndarray, starts: np.ndarray) -> np.ndarray: + """The exact sum of each group of ``values`` (from each of ``starts`` + to the next), rounded once: a group of one value, the value.""" + totals = values[starts] + several = np.flatnonzero(_group_stops(values, starts) - starts > 1) + for i in several.tolist(): + stop = starts[i + 1] if i + 1 < starts.size else values.size + totals[i] = float(ExactSum(values[starts[i] : stop])) # noqa: E203 + return totals + + +def _group_partials(values: np.ndarray, starts: np.ndarray) -> list: + """For each group of ``values`` (from each of ``starts`` to the next), + the floats whose exact sum is the group's (``ExactSum.partials``: none + for a sum of 0).""" + stops = _group_stops(values, starts) + out = [] + for a, b, first in zip( + starts.tolist(), stops.tolist(), values[starts].tolist() + ): + if b - a == 1: + out.append([first] if first else []) + else: + out.append(ExactSum(values[a:b]).partials) + return out + + +def _exact_total(data) -> ExactSum: + """An exact total from ``_Tally.to_dict``'s data: the floats whose + exact sum it is (or a number).""" + if isinstance(data, (int, float)): + return ExactSum(float(data)) + return ExactSum([float(v) for v in data]) diff --git a/repyability/rbd/_timeline_runs.py b/repyability/rbd/_timeline_runs.py index b62e198a..467c9602 100644 --- a/repyability/rbd/_timeline_runs.py +++ b/repyability/rbd/_timeline_runs.py @@ -17,6 +17,7 @@ import numpy as np +from repyability.rbd import _ccf_groups, _curves from repyability.rbd import _montecarlo as montecarlo from repyability.rbd import _streams from repyability.timelines import Timelines, _Data @@ -197,10 +198,12 @@ def simulate( ): """``simulate_timelines``: validated, run, and returned as a ``TimelineSimulation``.""" - from repyability.rbd.repairable_rbd import _UNSTREAMED + from repyability.rbd._runs import _UNSTREAMED from repyability.rbd.results import TimelineSimulation - rbd._require_groups_simulated() + from . import _event_loop + + _ccf_groups._require_groups_simulated(rbd) if ( isinstance(t_simulation, bool) or not isinstance(t_simulation, (int, float, np.integer, np.floating)) @@ -236,24 +239,23 @@ def simulate( f"record histories), got {engine!r}." ) jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) - working = set() if working_nodes is None else set(working_nodes) - broken = set() if broken_nodes is None else set(broken_nodes) - rbd._validate_node_overrides(working, broken) + working, broken = rbd._held_nodes(working_nodes, broken_nodes) # The components' states at 0, as availability takes them (#163). - states = rbd._simulation_states(state, working | broken) + states = _curves._simulation_states(rbd, state, working | broken) rng = np.random.get_state() after = rng try: entropy = _streams.entropy_of(seed) if seed is None: after = np.random.get_state() - plan, complete = rbd._stream_plan( - t_simulation, entropy, antithetic, states=states + plan, complete = _event_loop._stream_plan( + rbd, t_simulation, entropy, antithetic, states=states ) if antithetic and not complete: raise NotImplementedError(_UNSTREAMED) chosen = _engine(rbd, plan, engine, N, states) - tally = rbd._run( + tally = _event_loop._run( + rbd, t_simulation, working, broken, @@ -269,6 +271,7 @@ def simulate( states=states, histories=True, ) + assert tally.histories is not None parts, recorded = tally.histories.data(t_simulation) data = dict(zip(rbd.components, parts)) system = _named(rbd, recorded) @@ -309,7 +312,9 @@ def with_costs( see ``RepairableRBD._conditional_run``). With ``common``, every draw must come from a stream, for common random numbers with another system's run.""" - from repyability.rbd.repairable_rbd import _UNSTREAMED + from repyability.rbd._runs import _UNSTREAMED + + from . import _event_loop if engine not in ("auto", "python", "numba"): raise ValueError( @@ -320,12 +325,13 @@ def with_costs( if entropy is None: entropy = _streams.entropy_of(seed) states = states or {} - plan, complete = rbd._stream_plan( - t_simulation, entropy, antithetic, states + plan, complete = _event_loop._stream_plan( + rbd, t_simulation, entropy, antithetic, states ) if antithetic and not complete: raise NotImplementedError(_UNSTREAMED) - tally = rbd._run( + tally = _event_loop._run( + rbd, t_simulation, set(), set(), @@ -342,6 +348,7 @@ def with_costs( states=states, histories=True, ) + assert tally.histories is not None parts, _ = tally.histories.data(t_simulation) return dict(zip(rbd.components, parts)), tally @@ -353,10 +360,14 @@ def _engine(rbd, plan: _streams.Plan, engine: str, N: int, states=None) -> str: histories).""" from repyability.rbd import _compiled + from . import _runs + if engine == "python": return engine if engine == "numba": - return rbd._simulation_engine("numba", plan, None, N, states=states) + return _runs._simulation_engine( + rbd, "numba", plan, None, N, states=states + ) if ( _compiled.available() and _compiled.unsupported(rbd, plan, None, numba=True, states=states) diff --git a/repyability/rbd/_windows.py b/repyability/rbd/_windows.py new file mode 100644 index 00000000..3a111efe --- /dev/null +++ b/repyability/rbd/_windows.py @@ -0,0 +1,1087 @@ +"""A ``RepairableRBD``'s values over time from its components' curves: the +availability and unavailability at points (``point_availability``, +``point_unavailability``) and over missions (``mission_availability``, +``mission_unavailability``), and the expected failures, events and cost +in a window (``expected_failures``, ``expected_events``, +``expected_cost``), integrated on coarse pieces (``_quadrature``). The +methods of ``RepairableRBD`` of those names call these. +""" + +import math +from functools import partial +from typing import ( + TYPE_CHECKING, + Any, + Collection, + Dict, + Hashable, + List, + NamedTuple, + Optional, +) + +import numpy as np + +from repyability.rbd import ( + _ccf_groups, + _curves, + _quadrature, + _rates, + _requirements, +) +from repyability.rbd._common import ( + _DISCOUNT_ROUNDS, + _DISCOUNT_TOLERANCE, + _discount_pieces, + _discount_rate, + _horizons, + _matched, + _present_horizon, + _shaped, +) +from repyability.rbd._point_availability import ( + Atoms, +) +from repyability.rbd._spec import _mean_cost +from repyability.rbd._tally import ( + _CATEGORIES, +) +from repyability.rbd.results import ( + ExpectedCost, + ExpectedEvents, +) +from repyability.utils.checks import ( + nonnegative_times, + structure_method, +) + +from ._mean_lifetime import _G7_W, _GK_W, _GK_X + +if TYPE_CHECKING: + pass + + +class _Jumps(NamedTuple): + """What the system does at the times its nodes fail or are taken down + at exact times (see ``RepairableRBD._atom_groups``): the times, the + probability at each that the system fails there, and that a planned + outage takes it down there, and how much its point availability there + falls with them (``drop``); if asked for, each node's part in the + failures (``caused``).""" + + times: np.ndarray + failures: np.ndarray + planned: np.ndarray + drop: np.ndarray + caused: Optional[dict] = None + + +def point_availability( + rbd, + x, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +): + """See ``RepairableRBD.point_availability``.""" + return _point(rbd, x, working_nodes, broken_nodes, method, state) + + +def point_unavailability( + rbd, + x, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +): + """See ``RepairableRBD.point_unavailability``.""" + return _point( + rbd, x, working_nodes, broken_nodes, method, state, down=True + ) + + +def _point(rbd, x, working_nodes, broken_nodes, method, state, down=False): + """``point_availability``, or with ``down``, ``point_unavailability`` + (#237).""" + times = nonnegative_times(x) + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + horizon = float(times.max()) if times.size else 0.0 + states = _curves._states(rbd, state, working_nodes | broken_nodes) + system_at, _, _ = _system_at( + rbd, horizon, working_nodes, broken_nodes, method, states, down + ) + values = system_at(times.ravel()) + if np.ndim(x) == 0: + return float(values[0]) + return values.reshape(times.shape) + + +def _system_at( + rbd, horizon, working_nodes, broken_nodes, method, states, down=False +): + """The system's point availability (with ``down``, unavailability) + over ``[0, horizon]``, as a function of the times; the curves whose + bends its integral's pieces follow; and the crews' curve, where a + component can wait for a crew (else None).""" + if rbd._crews_couple(): + crew = _curves._crew_curve( + rbd, horizon, working_nodes, broken_nodes, method, states + ) + if not down: + return crew.at, [crew], crew + + def crew_down(x): + return np.clip(1.0 - crew.at(x), 0.0, 1.0) + + return crew_down, [crew], crew + if rbd.ccf_groups: + grouped = _ccf_groups._groups_curve( + rbd, horizon, working_nodes, broken_nodes, method, states + ) + return (grouped.down_at if down else grouped.at), [grouped], None + forced = working_nodes | broken_nodes + curves = _curves._availability_curves(rbd, horizon, forced, state=states) + + def system_at(x): + if down: + return _curves._curves_down_at( + rbd, curves, x, working_nodes, broken_nodes, states + ) + return _curves._curves_at( + rbd, curves, x, working_nodes, broken_nodes, method + ) + + return system_at, list(curves.values()), None + + +def mission_availability( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +): + """See ``RepairableRBD.mission_availability``.""" + return _mission(rbd, t, working_nodes, broken_nodes, method, state) + + +def mission_unavailability( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +): + """See ``RepairableRBD.mission_unavailability``.""" + return _mission( + rbd, t, working_nodes, broken_nodes, method, state, down=True + ) + + +def _mission(rbd, t, working_nodes, broken_nodes, method, state, down=False): + """``mission_availability``, or with ``down``, + ``mission_unavailability`` (#237).""" + windows = nonnegative_times(t) + working_nodes, broken_nodes = rbd._held_nodes(working_nodes, broken_nodes) + ends = windows.ravel() + horizon = float(ends.max()) if ends.size else 0.0 + states = _curves._states(rbd, state, working_nodes | broken_nodes) + system_at, pieces, crew = _system_at( + rbd, horizon, working_nodes, broken_nodes, method, states, down + ) + if crew is not None and not crew.nested: + # The crews' chain integrates it exactly. + totals = crew.integral(ends) + if down: + totals = np.maximum(ends - totals, 0.0) + else: + scale = None + if down: + # To the unavailability's own size, and the time scale of + # the fastest closed form in it (see _closed_form_down, and + # the common-cause groups' chains' tests and repairs of an + # exponential length, #220); a unit repaired at once is never + # down, and sets none. + rates = list(_curves._closed_form_rates(rbd, states).values()) + rates += _ccf_groups._timing_rates(rbd) + scale = ( + min( + [horizon or 1.0] + + [1.0 / rate for rate in rates if 0.0 < rate < math.inf] + ) + / 4.0 + ) + totals = _integrated(rbd, system_at, pieces, ends, horizon, scale) + averages = np.empty(len(ends)) + positive = ends > 0.0 + averages[positive] = totals[positive] / ends[positive] + if not positive.all(): + averages[~positive] = system_at(np.zeros(1))[0] + if np.ndim(t) == 0: + return float(averages[0]) + return averages.reshape(windows.shape) + + +def _integrated( + rbd, + system_at, + curves: list, + ends: np.ndarray, + horizon: float, + relative: Optional[float] = None, +) -> np.ndarray: + """The integral from 0 to each of ``ends`` of the system's point + availability, ``system_at``, from its nodes' ``curves`` (over the + pieces of ``_quadrature``, and settling into a constant or a period + it is extended over: see ``mission_availability``). With + ``relative``, a time scale, to a tolerance relative to the integral + itself, pieces being halved down to that scale where the grids are + coarser: for an unavailability, small, and changing in closed form + faster than the grids (#237).""" + from repyability.rbd._curves import _MISSION_POINTS, _settling + + # After ``settle`` the system's availability is constant, or repeats + # with ``period``: it is integrated up to ``reach``, and extended. + settle, period = _settling(curves) + reach = min(horizon, settle if period is None else settle + period) + beyond = ends > reach + fixed = [ends[~beyond]] + if period is not None and beyond.any(): + cycles = np.floor((ends[beyond] - settle) / period) + rest = ends[beyond] - settle - cycles * period + rest = np.clip(rest, 0.0, period) + fixed += [np.array([settle]), settle + rest] + + def estimate(a, b): + x, half = _quadrature.points(a, b) + return {"up": _quadrature.summed(system_at(x), half)} + + try: + edges, finest = _quadrature.pieces( + curves, np.concatenate(fixed), reach, _MISSION_POINTS + ) + keys: tuple = () + if relative is not None: + keys, finest = ("up",), np.minimum(finest, relative) + edges, integrals = _quadrature.refined( + estimate, edges, finest, _MISSION_POINTS, keys + ) + except _quadrature.TooMany as error: + raise NotImplementedError( + f"Integrating the availability over [0, {reach}] takes " + f"{error.count} pieces (the components' curves bend that " + "often), more than the limit: estimate it by simulation, " + "with availability()." + ) from None + running = _quadrature.running(integrals.get("up"), len(edges) - 1) + + def uptime(x): + return running[np.searchsorted(edges, x)] + + totals = np.empty(len(ends)) + totals[~beyond] = uptime(ends[~beyond]) + if beyond.any(): + if period is None: + level = system_at(np.array([horizon]))[0] + totals[beyond] = uptime(reach) + (ends[beyond] - reach) * level + else: + base = uptime(settle) + totals[beyond] = ( + base + + cycles * (uptime(reach) - base) + + (uptime(settle + rest) - base) + ) + return totals + + +def _window( + rbd, + t, + working_nodes, + broken_nodes, + method: str, + nodes: bool = False, + setups: bool = False, + state=None, + causes: bool = False, +): + """The windows ``t`` checked, flat, and the nodes' curves and the + system's expected events over them (see ``_window_counts``): the + forced nodes have no curves. With ``setups``, the stops of each + maintenance group with a set-up cost are counted too; with + ``causes``, the system failures each node (or common cause) + causes.""" + method = structure_method(method) + windows = nonnegative_times(t) + working, broken = rbd._held_nodes(working_nodes, broken_nodes) + forced = working | broken + ends = windows.ravel() + horizon = float(ends.max()) if ends.size else 0.0 + states = _curves._states(rbd, state, forced) + groups = None + if setups: + groups = { + name: [node for node in spec.members if node not in forced] + for name, spec in rbd._maintenance.items() + if spec.setup_cost + } + if rbd._crews_couple(): + curves, counts = _curves._crew_window( + rbd, ends, working, broken, method, states, nodes, groups, causes + ) + return windows, ends, curves, counts, working, broken + if rbd.ccf_groups: + curves, counts = _curves._groups_window( + rbd, ends, working, broken, method, states, nodes, groups, causes + ) + return windows, ends, curves, counts, working, broken + curves = _curves._availability_curves( + rbd, horizon, forced, counts=True, state=states + ) + counts = _window_counts( + rbd, + curves, + ends, + working, + broken, + method, + nodes=nodes, + groups=groups, + causes=causes, + ) + return windows, ends, curves, counts, working, broken + + +def expected_failures( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +): + """See ``RepairableRBD.expected_failures``.""" + windows, _, _, counts, _, _ = _window( + rbd, t, working_nodes, broken_nodes, method, state=state + ) + return _shaped(counts["failures"], t, windows) + + +def expected_events( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, +) -> ExpectedEvents: + """See ``RepairableRBD.expected_events``.""" + windows, ends, curves, counts, working, broken = _window( + rbd, t, working_nodes, broken_nodes, method, nodes=True, state=state + ) + node_events = {node: curve.events(ends) for node, curve in curves.items()} + zero = np.zeros(len(ends)) + + def shaped(values) -> Any: + return _shaped(values, t, windows) + + def per_node(key: str) -> dict: + return { + node: shaped(node_events[node][key] if node in curves else zero) + for node in rbd.components + } + + downtime = { + node: shaped( + counts["downtime"][node] + if node in curves + else (ends if node in broken else zero) + ) + for node in rbd.components + } + return ExpectedEvents( + window=shaped(ends), + system_failures=shaped(counts["failures"]), + system_planned_outages=shaped(counts["planned"]), + system_downtime=shaped(np.maximum(ends - counts["uptime"], 0.0)), + node_failures=per_node("failures"), + node_corrective=per_node("corrective"), + node_preventive=per_node("preventive"), + node_inspections=per_node("inspections"), + node_downtime=downtime, + ) + + +def expected_cost( + rbd, + t, + working_nodes: Optional[Collection[Hashable]], + broken_nodes: Optional[Collection[Hashable]], + method: str, + state, + *, + discount_rate: float, +) -> ExpectedCost: + """See ``RepairableRBD.expected_cost``.""" + _requirements._require_costs(rbd) + rate = _discount_rate(discount_rate) + if rate > 0.0 and rbd.has_costs: + _present_horizon( + _horizons(t, 0.0), rate, partial(_requirements._mean_lives, rbd) + ) # warns of a rate that discounts the costs away + return _discounted_cost( + rbd, t, rate, working_nodes, broken_nodes, method, state + ) + if not rbd.has_costs: + # An acquisition cost alone: no running cost, exactly. + windows = nonnegative_times(t) + zero = _shaped(np.zeros(windows.size), t, windows) + return ExpectedCost( + window=_shaped(windows.ravel(), t, windows), + mean=zero, + by_category={category: zero for category in _CATEGORIES}, + by_component={}, + acquisition_cost=rbd.acquisition_cost, + ) + windows, ends, curves, counts, working, broken = _window( + rbd, + t, + working_nodes, + broken_nodes, + method, + nodes=True, + setups=True, + state=state, + ) + node_events = {node: curve.events(ends) for node, curve in curves.items()} + categories = {category: np.zeros(len(ends)) for category in _CATEGORIES} + by_component: Dict[Hashable, Any] = {} + for node, node_costs in rbd.costs.items(): + own = np.zeros(len(ends)) + charges = [] + events = node_events.get(node) + if events is not None: + for key, category, kind in ( + ("repair_cost", "repair", "corrective"), + ("replace_cost", "replace", "corrective"), + ("preventive_cost", "preventive", "preventive"), + ("inspection_cost", "inspection", "inspections"), + ): + if category == "replace" and node in rbd._imperfect: + # Minimally repaired (the only imperfect repair + # with exact values), it is never replaced: its + # repairs are charged only as repairs, as the + # simulation charges them. + continue + if key in node_costs: + charges.append( + ( + category, + _mean_cost(node_costs[key]) * events[kind], + ) + ) + rate = node_costs.get("downtime_cost", 0.0) + if rate: + down = ( + counts["downtime"][node] + if node in curves + else (ends if node in broken else np.zeros(len(ends))) + ) + charges.append(("component_downtime", rate * down)) + for category, charge in charges: + categories[category] = categories[category] + charge + own = own + charge + by_component[node] = _shaped(own, t, windows) + if rbd.downtime_cost_rate: + categories["system_downtime"] = rbd.downtime_cost_rate * ( + np.maximum(ends - counts["uptime"], 0.0) + ) + for name, spec in rbd._maintenance.items(): + if not spec.setup_cost: + continue + # Each failure or replacement of a member is a stop, but those + # at one instant are one. + stops = -counts["overlaps"].get(name, np.zeros(len(ends))) + for node in spec.members: + if node in node_events: + stops = stops + ( + node_events[node]["failures"] + + node_events[node]["preventive"] + ) + categories["setup"] = categories["setup"] + spec.setup_cost * stops + mean = np.zeros(len(ends)) + for values in categories.values(): + mean = mean + values + return ExpectedCost( + window=_shaped(ends, t, windows), + mean=_shaped(mean, t, windows), + by_category={ + category: _shaped(values, t, windows) + for category, values in categories.items() + }, + by_component=by_component, + acquisition_cost=rbd.acquisition_cost, + ) + + +def _discounted_cost( + rbd, t, rate: float, working_nodes, broken_nodes, method: str, state +) -> ExpectedCost: + """``expected_cost`` at ``t`` discounted at ``rate`` (#231): each + window's ``exp(-r T) C(T) + r * integral_0^T exp(-r s) C(s) ds``, + ``C`` the expected cost from new, the integral by the Gauss-Kronrod + (7, 15) rule on pieces from 0, with the windows among their ends. + The pieces whose two rules differ most are halved until the + differences add up to ``_DISCOUNT_TOLERANCE`` of the integral: + ``C`` jumps at scheduled events (tests, block replacements), where + a piece's error only halves with it.""" + windows = nonnegative_times(t) + ends = windows.ravel() + horizon = float(ends.max()) if ends.size else 0.0 + keys: list = [] + + def values(times: np.ndarray) -> np.ndarray: + """``C`` at ``times``: its mean, each category's and each + component's, as rows. The longest window is worked out too, so + that every call's curves reach it, and are shared.""" + got = rbd.expected_cost( + np.append(times, horizon), + working_nodes, + broken_nodes, + method, + state, + ) + if not keys: + keys.extend(got.by_component) + rows = [got.mean] + rows += [got.by_category[category] for category in _CATEGORIES] + rows += [got.by_component[node] for node in keys] + return np.vstack([np.ravel(row)[:-1] for row in rows]) + + def rules(a: np.ndarray, b: np.ndarray): + """The pieces' integrals of ``exp(-r s) C(s)`` by the Kronrod + rule, as rows, and the mean's error, against the Gauss rule.""" + half = 0.5 * (b - a) + x = (0.5 * (a + b))[:, None] + half[:, None] * _GK_X + g = np.exp(-rate * x) * values(x.ravel()).reshape(-1, *x.shape) + kronrod = (g @ _GK_W) * half + return kronrod, np.abs(kronrod[0] - (g[0] @ _G7_W) * half) + + with rbd._sharing_curves(): + at_ends = values(ends) + a, b = _discount_pieces(ends, horizon) + pieces, error = rules(a, b) + for _ in range(_DISCOUNT_ROUNDS): + budget = _DISCOUNT_TOLERANCE * abs(float(pieces[0].sum())) + if error.sum() <= budget: + break + # Halve the pieces over their share of the error allowed. + split = error > budget / error.size + middle = 0.5 * (a + b)[split] + halves, errors = rules( + np.concatenate([a[split], middle]), + np.concatenate([middle, b[split]]), + ) + a = np.concatenate([a[~split], a[split], middle]) + b = np.concatenate([b[~split], middle, b[split]]) + pieces = np.concatenate([pieces[:, ~split], halves], axis=1) + error = np.concatenate([error[~split], errors]) + order = np.argsort(b, kind="stable") + running = np.concatenate( + [np.zeros((len(at_ends), 1)), np.cumsum(pieces[:, order], axis=1)], + axis=1, + ) + # Every window is a piece's end: the pieces up to it are within it. + upto = np.searchsorted(b[order], ends, side="right") + present = np.exp(-rate * ends) * at_ends + rate * running[:, upto] + rows = iter(present) + mean = next(rows) + categories = {category: next(rows) for category in _CATEGORIES} + components = {node: next(rows) for node in keys} + return ExpectedCost( + window=_shaped(ends, t, windows), + mean=_shaped(mean, t, windows), + by_category={ + category: _shaped(v, t, windows) + for category, v in categories.items() + }, + by_component={ + node: _shaped(v, t, windows) for node, v in components.items() + }, + acquisition_cost=rbd.acquisition_cost, + discount_rate=rate, + ) + + +def _filled( + rbd, probabilities: dict, size: int, working_nodes, broken_nodes +) -> dict: + """The nodes' ``probabilities`` (arrays of ``size``), with the input + and output nodes, and the forced nodes held at 1 or 0, added.""" + out = dict(probabilities) + for node in list(rbd.in_or_out) + list(working_nodes | broken_nodes): + out[node] = np.ones(size) + return rbd._probabilities_with_overrides(out, working_nodes, broken_nodes) + + +def _atom_groups( + rbd, + curves: dict, + atoms: dict, + stop: float, + working_nodes, + broken_nodes, + crew=None, + causes: bool = False, +) -> _Jumps: + """The system at the times before ``stop`` at which its nodes fail + or are taken down at exact times (their ``atoms``): a unit dead on + arrival at 0, an exact lifetime, a replacement at its age or at a + block time. + + Nodes due at one time go down together, so the system fails there + if it is up just before and down just after: since they only go + down, that is the fall in its availability, worked out as the rise + in its unavailability (a sum of products, which keeps a small one's + precision). A node's value just before is its point availability + there with its own events' fall (``drop``) added back; just after, + less the probabilities of its failures, then of its planned outages + (the failures are taken to come first). The others are at their + point availability there: tests and restorations due then are + done. One node alone fails the system with the probability of its + failure times its Birnbaum importance there. With limited repair + crews (``crew``, see ``_chain_transient.CrewSystem``), the nodes are + the nested RBDs, and the system is worked out over the crews' chain + at those times; with common-cause groups (a + ``_ccf_chain.GroupsSystem``), over the groups' chains, whose own + jumps (``jump_times``, the tests that take a working member off + line, #220) are planned outages too: the chains are taken just + before those times for the values before, and at them for the + values after. With ``causes``, the failures are split among the + nodes failing together as ``_rates.split_jumps`` splits the fall + in availability from just before to just after their failures.""" + down = [ + a.times[(a.failure > 0.0) | (a.planned > 0.0)] for a in atoms.values() + ] + if crew is not None: + down.append(crew.jump_times(stop)) + times = np.unique(np.concatenate(down)) if down else np.empty(0) + if not times.size: + empty = np.empty(0) + return _Jumps( + empty, + empty, + empty, + empty, + {node: empty for node in curves} if causes else None, + ) + at, before, failed, planned = {}, {}, {}, {} + for node, curve in curves.items(): + value = np.asarray(curve.at(times.copy()), dtype=float) + a = atoms[node] + drop = _matched(times, a.times, a.drop) + failing = _matched(times, a.times, a.failure) + stopping = _matched(times, a.times, a.planned) + at[node] = value + before[node] = np.clip(value + drop, 0.0, 1.0) + failed[node] = np.clip(before[node] - failing, 0.0, 1.0) + planned[node] = np.clip(failed[node] - stopping, 0.0, 1.0) + + # The chains just before the times, for the values before (and the + # failures, which they take no part in), and at them for the rest. + just = np.nextafter(times, -np.inf) + + def unreliability(values: dict, when: np.ndarray) -> np.ndarray: + if crew is not None: + return crew.probabilities(values, when)[1] + return rbd._system_unreliability( + _filled(rbd, values, len(times), working_nodes, broken_nodes) + ) + + now, first = unreliability(at, times), unreliability(before, just) + after, last = unreliability(failed, just), unreliability(planned, times) + caused = None + if causes: + + def importances(values: dict) -> dict: + size = len(next(iter(values.values()))) + if crew is not None: + # Each jump's time, for each point of its path. + points = size // len(times) + return crew.importance(values, np.repeat(just, points)) + return rbd._importances( + _filled(rbd, values, size, working_nodes, broken_nodes) + )[0] + + falls = _rates.split_jumps(importances, before, failed) + caused = {node: -fall for node, fall in falls.items()} + return _Jumps( + times, + np.maximum(after - first, 0.0), + np.maximum(last - after, 0.0), + np.maximum(now - first, 0.0), + caused, + ) + + +def _system_atoms(rbd, curves: dict, stop: float, crew=None) -> Atoms: + """This RBD's own failures and planned outages at exact times before + ``stop``, from its nodes' ``curves`` (and its crews' chain, + ``crew``), as a node of another (see ``Atoms``).""" + atoms = {node: curve.atoms(stop) for node, curve in curves.items()} + jumps = _atom_groups(rbd, curves, atoms, stop, set(), set(), crew) + return Atoms( + jumps.times, + jumps.failures, + jumps.planned, + np.zeros(len(jumps.times)), + jumps.drop, + ) + + +def _before_period_end(reach: float, curves: dict, crew=None) -> float: + """``reach``, the end of the period the counts are extended by, moved + down to an event at an exact time worked out a rounding short of it + (a test's time as a curve adds it up, against the settling time plus + the period): the event is the next period's first, and would + otherwise be counted in this one and again in the extension.""" + near = [ + times[np.isclose(times, reach, rtol=1e-12, atol=0.0)] + for times in [c.atoms(reach).times for c in curves.values()] + + ([crew.jump_times(reach)] if crew is not None else []) + ] + found = np.concatenate(near) if near else np.empty(0) + return float(found.min()) if found.size else reach + + +def _window_counts( + rbd, + curves: dict, + ends, + working_nodes, + broken_nodes, + method: str, + nodes: bool = False, + groups: Optional[dict] = None, + crew=None, + causes: bool = False, +) -> dict: + """The system's expected up time, failures and planned outages in + ``[0, end)`` for each of ``ends``, from its nodes' ``curves`` (which + count their events); with ``nodes``, each node's expected down time + too; with ``causes``, the system failures each node causes (its + terms of the failures' sum, below, under ``"caused"``; with the + crews' chain or common-cause groups, each of their components' or + causes' too, from their rates, #199); and for each of ``groups`` (a + name and members), the + expected number of its members' failures and replacements that fall + at one instant with another's (each stop of a maintenance group is + one). + With limited repair crews (``crew``, a + ``_chain_transient.CrewSystem``, #162), the ``curves`` are the nested + RBDs', independent of the crews' chain: the system is worked out + over the chain at each time, and the chain's own components' + failures, at their rate in each state, add to its failures. + + The up time integrates the system's point availability as + ``mission_availability`` does. A node's failure takes the system + down if the node is critical then, which, the nodes being + independent, it is at ``t`` with probability ``I_B(t)``, its + Birnbaum importance at the nodes' availabilities then: the system's + failures are ``integral of sum_i I_B^i(t) dM_i(t)``, ``M_i(t)`` the + node's expected failures by ``t``, and its planned outages likewise. + It is summed over the pieces of ``_quadrature``, on each with the + importance taken as the cubic through its values at the points the + up time is integrated at (see ``_quadrature.stieltjes``); the events + at exact times are kept apart and taken together (see + ``_atom_groups``), and start pieces. The ends do not: a total at + an end within a piece adds the integral from the piece's start, + by the same quadrature, so that counting at many times (a nested + RBD's, for the RBD it is in) costs no more pieces. Past the time the + nodes have all settled, every count is extended exactly: at its + long-run rate, or a period at a time. + """ + from repyability.rbd._curves import _MISSION_POINTS, _settling + + ends = np.asarray(ends, dtype=float).ravel() + horizon = float(ends.max()) if ends.size else 0.0 + followed = list(curves.values()) + ([crew.curve] if crew else []) + settle, period = _settling(followed) + reach = min(horizon, settle if period is None else settle + period) + if period is not None: + reach = _before_period_end(reach, curves, crew) + beyond = ends > reach + atoms = {node: curve.atoms(reach) for node, curve in curves.items()} + jumps = _atom_groups( + rbd, curves, atoms, reach, working_nodes, broken_nodes, crew, causes + ) + # The events at exact times start pieces; the ends need not (see + # ``totals_at``), so that a nested RBD counted at many times costs + # no more pieces. + fixed = [jumps.times] + [own.times for own in atoms.values()] + cycles = rest = np.empty(0) + if period is not None and beyond.any(): + cycles = np.floor((ends[beyond] - settle) / period) + rest = np.clip(ends[beyond] - settle - cycles * period, 0.0, period) + fixed += [np.array([settle]), settle + rest] + + # Each node's events at exact times, kept apart (taken together, + # below): their running totals, by time. + exact = {} + for node, own in atoms.items(): + order = np.argsort(own.times, kind="stable") + exact[node] = ( + own.times[order], + _quadrature.running(own.failure[order], len(order)), + _quadrature.running(own.planned[order], len(order)), + ) + + def spread(node, x: np.ndarray): + """The node's expected failures and planned outages before + each time ``x``, less those at exact times.""" + events = curves[node].events(x) + times, failures, planned = exact[node] + before = np.searchsorted(times, x, side="left") + return ( + events["failures"] - failures[before], + events["planned"] - planned[before], + ) + + def estimate(a: np.ndarray, b: np.ndarray) -> dict: + x, half = _quadrature.points(a, b) + at_points = {node: curve.at(x) for node, curve in curves.items()} + # The system's availability and every node's importance at each + # point, in one pass. + own: dict = {} + if crew is not None: + importance, up, _, rate, own = crew.evaluate(at_points, x) + else: + importance, works, fails, _, _ = rbd._importances( + _filled(rbd, at_points, len(x), working_nodes, broken_nodes) + ) + up = works if structure_method(method) == "p" else 1.0 - fails + out: Dict[Hashable, np.ndarray] = { + "uptime": _quadrature.summed(up, half) + } + if nodes: + for node, values in [*at_points.items(), *own.items()]: + out[("downtime", node)] = _quadrature.summed( + 1.0 - values, half + ) + n = len(a) + failures, planned = np.zeros(n), np.zeros(n) + if crew is not None: + # The chain's components' failures, at their rate in each + # state, where they take the system down. + failures += _quadrature.summed(rate, half) + if causes: + for key, value in crew.caused(at_points, x).items(): + out[("caused", key)] = _quadrature.summed(value, half) + for node in curves: + counted = spread(node, np.concatenate([a, b, x])) + for total, values, key in zip( + (failures, planned), counted, ("caused", None) + ): + term = _quadrature.stieltjes( + importance[node], + values[:n], + values[n : 2 * n], # noqa: E203 + values[2 * n :], # noqa: E203 + ) + total += term + if causes and key is not None: + out[(key, node)] = term + out["failures"], out["planned"] = failures, planned + return out + + try: + edges, finest = _quadrature.pieces( + followed, np.concatenate(fixed), reach, _MISSION_POINTS + ) + edges, integrals = _quadrature.refined( + estimate, + edges, + finest, + _MISSION_POINTS, + relative={"failures", "planned"}, + ) + except _quadrature.TooMany as error: + raise NotImplementedError( + f"Counting the expected events over [0, {reach}] takes " + f"{error.count} pieces (the components' curves bend that " + "often), more than the limit: estimate them by simulation, " + "with availability() or cost()." + ) from None + pieces = len(edges) - 1 + # The events at exact times, at the start of the piece they fall + # in (each is at an edge). + starting: Dict[Any, np.ndarray] = { + "failures": np.zeros(pieces), + "planned": np.zeros(pieces), + } + if causes: + for node in curves: + starting[("caused", node)] = np.zeros(pieces) + if jumps.times.size: + piece = np.searchsorted(edges, jumps.times, side="right") - 1 + np.add.at(starting["failures"], piece, jumps.failures) + np.add.at(starting["planned"], piece, jumps.planned) + if causes: + assert jumps.caused is not None + for node, values in jumps.caused.items(): + np.add.at(starting[("caused", node)], piece, values) + for name, members in (groups or {}).items(): + starting[("overlaps", name)] = _overlaps(atoms, members, edges) + keys: List[Any] = ["uptime", "failures", "planned"] + if causes: + keys += [("caused", node) for node in curves] + if crew is not None: + keys += [("caused", key) for key in crew.cause_keys] + if nodes: + keys += [("downtime", node) for node in curves] + if crew is not None: + keys += [("downtime", node) for node in crew.served] + keys += [("overlaps", name) for name in groups or {}] + series: Dict[Any, np.ndarray] = { + key: _quadrature.running( + integrals.get(key, np.zeros(pieces)) + + starting.get(key, np.zeros(pieces)), + pieces, + ) + for key in keys + } + + def totals_at(x: np.ndarray) -> Dict[Any, np.ndarray]: + """Each series before each time ``x`` (within ``[0, reach]``): + at an edge, its running total; within a piece, the running + total at its start, the events there, and its integral from + there, by the quadrature of the piece's start to ``x``.""" + at = np.minimum(np.searchsorted(edges, x), len(edges) - 1) + on_edge = edges[at] == x + out = {key: series[key][at] for key in keys} + if pieces and not on_edge.all(): + inside = ~on_edge + k = np.searchsorted(edges, x[inside], side="right") - 1 + k = np.clip(k, 0, pieces - 1) + partial = _quadrature.in_blocks(estimate, edges[k], x[inside]) + for key in keys: + out[key][inside] = ( + series[key][k] + + starting.get(key, np.zeros(pieces))[k] + + partial.get(key, 0.0) + ) + return out + + # Past ``reach``: constant rates, or a period at a time. + rates: dict = {} + if period is None and beyond.any(): + rates = _settled_rates( + rbd, + curves, + reach, + horizon, + working_nodes, + broken_nodes, + method, + crew, + ) + out: dict = {"downtime": {}, "overlaps": {}, "caused": {}} + inside = totals_at(ends[~beyond]) + for key, values in series.items(): + totals = np.empty(len(ends)) + totals[~beyond] = inside[key] + if beyond.any(): + if period is None: + rate = rates.get(key, 0.0) + totals[beyond] = values[-1] + (ends[beyond] - reach) * rate + else: + base = values[np.searchsorted(edges, settle)] + partial = values[np.searchsorted(edges, settle + rest)] + totals[beyond] = ( + base + cycles * (values[-1] - base) + (partial - base) + ) + if isinstance(key, tuple): + out[key[0]][key[1]] = totals + else: + out[key] = totals + return out + + +def _settled_rates( + rbd, + curves: dict, + reach, + horizon, + working_nodes, + broken_nodes, + method, + crew=None, +) -> dict: + """What ``_window_counts`` counts, per unit time, once every node's + curve is constant: the system's availability, its failures and + planned outages (each node's rates, from its counts' slope then, + times its Birnbaum importance), and each node's unavailability; + with the crews' chain (``crew``), its own components' failures + too.""" + x = np.array([horizon]) + values = {node: curve.at(x) for node, curve in curves.items()} + failures = planned = 0.0 + if crew is not None: + importance, up, _, rate, own = crew.evaluate(values, x) + rates: dict = {"uptime": float(up[0])} + failures += float(rate[0]) + for node, value in own.items(): + rates[("downtime", node)] = 1.0 - float(value[0]) + for key, value in crew.caused(values, x).items(): + rates[("caused", key)] = float(np.ravel(value)[0]) + else: + filled = _filled(rbd, values, 1, working_nodes, broken_nodes) + rates = { + "uptime": float( + np.ravel(rbd.system_probability(filled, method=method))[0] + ) + } + importance = rbd._importances(filled)[0] + span = np.array([reach, reach + max(reach, 1.0)]) + for node, curve in curves.items(): + events = curve.events(span) + width = span[1] - span[0] + weight = float(importance[node][0]) + caused = weight * np.diff(events["failures"])[0] / width + failures += caused + planned += weight * np.diff(events["planned"])[0] / width + rates[("downtime", node)] = 1.0 - float(values[node][0]) + rates[("caused", node)] = caused + rates["failures"], rates["planned"] = failures, planned + return rates + + +def _overlaps(atoms: dict, members, edges: np.ndarray) -> np.ndarray: + """In each piece between ``edges``, the expected number of the + ``members``' failures and replacements at exact times that fall at + one instant with another member's: at each such instant the + members' expected number, less the probability of at least one + (they are independent). A maintenance group charges one set-up per + instant.""" + pieces = len(edges) - 1 + own = [atoms[m] for m in members if m in atoms] + acting = [a.times[(a.failure + a.preventive) > 0.0] for a in own] + if len(own) < 2 or not any(t.size for t in acting): + return np.zeros(pieces) + times = np.unique(np.concatenate(acting)) + expected = np.zeros(len(times)) + none = np.ones(len(times)) + for a in own: + chance = np.clip( + _matched(times, a.times, a.failure + a.preventive), 0.0, 1.0 + ) + expected += chance + none *= 1.0 - chance + overlap = np.maximum(expected - (1.0 - none), 0.0) + piece = np.searchsorted(edges, times, side="right") - 1 + keep = (piece >= 0) & (piece < pieces) + return np.bincount(piece[keep], overlap[keep], pieces) diff --git a/repyability/rbd/ccf.py b/repyability/rbd/ccf.py index 63b6c058..06ffd1ee 100644 --- a/repyability/rbd/ccf.py +++ b/repyability/rbd/ccf.py @@ -6,8 +6,8 @@ exact RBD engine assumes independence, and therefore over-estimates redundant systems; a CCF model injects the shared-cause coupling. -Two models are provided, both consumed by ``NonRepairableRBD``'s ``ccf_groups`` -argument via a [`CCFGroup`][repyability.CCFGroup]: +Three models are provided, all consumed by ``NonRepairableRBD``'s +``ccf_groups`` argument via a [`CCFGroup`][repyability.CCFGroup]: * [`BetaFactor`][repyability.BetaFactor] — a fraction ``beta`` of a component's failures come from a cause shared across the *whole* group (all @@ -17,8 +17,13 @@ additionally captures *partial* common causes (a cause failing some but not all of the group) via a cascade of conditional probabilities ``beta, gamma, delta, ...``. Beta-factor is the two-unit special case. +* [`AlphaFactor`][repyability.AlphaFactor] — the alpha-factor model: the + same multiplicities as MGL's, parameterised by ``alpha_1, ..., alpha_m``, + the shares of a group's failure events that fail exactly ``1, ..., m`` + members, which event counts estimate directly (NUREG/CR-5485), under a + staggered or a non-staggered testing scheme. -Both expose ``decompose``, which splits a group's failure probability into +All expose ``decompose``, which splits a group's failure probability into an independent part plus a set of mutually-exclusive *shock* outcomes (each a subset of members failing together); ``NonRepairableRBD`` evaluates the exact system reliability by conditioning on those outcomes. See issue #44. @@ -34,17 +39,18 @@ members' own, ``H(t) = -log R(t)``. Each member then keeps its own life distribution, whatever it is, and the model holds over the whole life; it agrees with the probability split to first order in ``Q``. - -Alpha-factor (a data-estimable reparameterisation of the same multiplicities) -is a planned extension. """ from itertools import combinations +from itertools import combinations as subsets_of +from itertools import product from math import comb from typing import Any, Collection, Dict, Hashable, List, Optional, Tuple import numpy as np +from repyability.utils.checks import number_or_nan, one_of + # A group's failure decomposition: the per-component independent failure # probability, and a list of (members-failing-together, probability) shocks. Decomposition = Tuple[np.ndarray, List[Tuple[frozenset, np.ndarray]]] @@ -67,6 +73,21 @@ def _check_basis(basis: str) -> str: return basis +def _check_shocks(basis: str, shocks: Optional[str]) -> str: + """How a model's shared causes combine, from its ``shocks`` option: + None for the ``basis``'s default (independent by rate, exclusive + splitting the probability); by rate, only independent.""" + if shocks is None: + return "independent" if basis == "rate" else "exclusive" + one_of("shocks", shocks, ("exclusive", "independent")) + if basis == "rate" and shocks != "independent": + raise ValueError( + "By rate, every cause strikes independently: shocks=" + "'exclusive' splits the probability (basis='probability')." + ) + return shocks + + def _hazard(Q: np.ndarray, R: np.ndarray) -> np.ndarray: """The cumulative hazard ``-log R``, from whichever of the probability of failing ``Q`` and of surviving ``R`` keeps its precision: a small @@ -189,7 +210,6 @@ def _as_independent( keeps its precision. None where some ``A_S`` is negative (a set the shocks never fail, while two causes could; or a large probability of failing): no independent causes do the same.""" - from itertools import combinations as subsets_of struck = {frozenset(subset): p for subset, p in shocks} none = 1.0 - sum(struck.values()) @@ -227,6 +247,14 @@ def _causes(self, members) -> Tuple[float, list]: def _split(self, members, Q, R) -> tuple: raise NotImplementedError + def _by_rate(self) -> "_Model": + """This model splitting the failure rate (``basis="rate"``).""" + raise NotImplementedError + + def required_group_size(self) -> Any: + """The group size this model requires: None for any size.""" + raise NotImplementedError + @property def shocks(self) -> str: """How the shared causes combine: ``"independent"`` events (by @@ -360,9 +388,10 @@ class BetaFactor(_Model): """ def __init__(self, beta: float, basis: str = "probability"): - if not (0.0 <= beta <= 1.0): - raise ValueError(f"beta must be in [0, 1], got {beta!r}.") - self.beta = float(beta) + value = number_or_nan(beta) + if not (0.0 <= value <= 1.0): + raise ValueError(f"beta must be a number in [0, 1], got {beta!r}.") + self.beta = value self.basis = _check_basis(basis) def required_group_size(self) -> Any: @@ -430,6 +459,9 @@ def _causes(self, members) -> Tuple[float, list]: each shared cause, as the members it fails and its fraction.""" return 1.0 - self.beta, [(frozenset(members), self.beta)] + def _by_rate(self) -> "BetaFactor": + return BetaFactor(self.beta, basis="rate") + def __eq__(self, other: object) -> bool: return ( isinstance(other, BetaFactor) @@ -445,7 +477,136 @@ def __repr__(self) -> str: return f"BetaFactor(beta={self.beta}{basis})" -class MGL(_Model): +class _SpecificSets(_Model): + """What the MGL and alpha-factor models share: each *specific* set of + ``k`` of the ``m`` members has a cause of its own, which fails it with + the basic-event probability ``Q_k``, a fraction of each member's ``Q`` + given by the model's parameters (``_fractions``); the ``Q_k`` of the + sets holding a member sum to its ``Q``. Splitting the probability, the + shocks are exclusive or independent (``shocks``); by rate, each cause + is a shock with that fraction of the members' hazard.""" + + def __init__(self, basis: str, shocks: Optional[str]): + self.basis = _check_basis(basis) + self._shocks = _check_shocks(self.basis, shocks) + + @property + def shocks(self) -> str: + """How the shared causes combine: ``"exclusive"`` (one at most, + the probability split's default) or ``"independent"`` (as + independent basic events; by rate, always).""" + return self._shocks + + @property + def group_size(self) -> int: + """The number of members this model describes.""" + raise NotImplementedError + + def required_group_size(self) -> int: + """The group size this model requires, ``group_size``. + + [`CCFGroup`][repyability.CCFGroup] rejects a group whose number of + members differs. + + Returns + ------- + int + ``group_size``: an ``MGL`` model's ``len(letters) + 1``, an + ``AlphaFactor`` model's ``len(alphas)``. + """ + return self.group_size + + def _fractions(self, m: int) -> List[float]: + """``Q_k / Q`` for ``k = 1, ..., m``: the fraction of a member's + probability of failing (or of its hazard, by rate) that fails each + specific set of ``k`` members, this one among them.""" + raise NotImplementedError + + def decompose(self, members: Collection[Hashable], Q) -> Decomposition: + """Split the failure probability into independent and shared parts. + + With ``m = len(members)``, each member fails alone with ``Q_1`` + and each specific set of ``k >= 2`` members fails together with + the basic-event probability ``Q_k`` given in the class + description. The RBD calls this at each evaluation time with + ``Q``, the probability that the group's first member has failed + by then. + + By rate, or with ``shocks="independent"``, the shocks are the sets + of members the causes that have struck fail between them: several + causes may strike, and a member fails if any of its causes has, so + these are the distinct unions, mutually exclusive as the RBD needs, + each with its probability. + + Parameters + ---------- + members : collection of hashable + The group's member node names: ``group_size`` of them, as + [`CCFGroup`][repyability.CCFGroup] ensures. + Q : float or array_like + Each member's total failure probability (the same for every + member of a symmetric group), at one or more times. + + Returns + ------- + q_independent : numpy.ndarray + ``Q_1``, each member's independent failure probability, at + least 1-d (by rate, ``1 - (1 - Q) ** (Q_1 / Q)``). + shocks : list of (frozenset, numpy.ndarray) + ``(subset, Q_k)`` for every subset of ``k = 2, ..., m`` members, + smaller subsets first; by rate, ``(subset, probability)`` for + every set the struck causes can fail together. + """ + Q = np.atleast_1d(np.asarray(Q, dtype=float)) + q_independent, _, shocks = self._split(members, Q, 1.0 - Q) + return q_independent, shocks + + def _split(self, members, Q, R): + """Each member's probability of failing on its own and of not + doing so, and the shocks (see ``decompose``), at the members' + probabilities of failing ``Q`` and of surviving ``R`` (both 1-d + arrays: by rate, whichever keeps its precision is used).""" + members = tuple(members) + if self.basis == "rate": + independent, causes = self._causes(members) + return _rate_outcomes(members, independent, causes, _hazard(Q, R)) + if self.shocks == "independent": + independent, causes = self._causes(members) + return _event_outcomes(members, independent, causes, Q) + m = len(members) + fractions = self._fractions(m) + q_independent = fractions[0] * Q + shocks: List[Tuple[frozenset, np.ndarray]] = [] + for k in range(2, m + 1): + q_k = fractions[k - 1] * Q + for subset in combinations(members, k): + shocks.append((frozenset(subset), q_k)) + return q_independent, 1.0 - q_independent, shocks + + def _causes(self, members) -> Tuple[float, list]: + """By rate: each member's own causes' fraction of its hazard, and + each shared cause, as the members it fails and its fraction.""" + members = tuple(members) + m = len(members) + fractions = self._fractions(m) + causes = [ + (frozenset(subset), fractions[k - 1]) + for k in range(2, m + 1) + for subset in combinations(members, k) + ] + return fractions[0], causes + + def _options(self) -> List[str]: + """The ``basis`` and ``shocks`` options as ``repr`` shows them: + only those that differ from the defaults.""" + if self.basis == "rate": + return ["basis='rate'"] + if self.shocks == "independent": + return ["shocks='independent'"] + return [] + + +class MGL(_SpecificSets): """The Multiple Greek Letter common-cause model. The parameters are the conditional probabilities of a common-cause failure @@ -465,7 +626,8 @@ class MGL(_Model): total failure probability ``Q``; its own failure and the shared causes being separate events, it fails with probability ``Q`` to first order (``Q - beta (1 - beta) Q**2`` for one shared cause), as in PRA's basic - events. + events. The [`AlphaFactor`][repyability.AlphaFactor] model gives the + same ``Q_k`` from the parameters event counts estimate. That is the default ``basis``, ``"probability"``: the PRA basic-event model, for a small ``Q`` (see [`BetaFactor`][repyability.BetaFactor]). @@ -485,7 +647,7 @@ class MGL(_Model): that several may strike: ``shocks="independent"``. The two differ at second order in ``Q`` (a 2-out-of-3 group of ``MGL(0.2, 0.3)`` at ``Q = 0.0311`` fails with probability 0.010219 one way and 0.010192 - the other), so match the tool you check against (#180). A + the other), so match the tool you check against. A ``BetaFactor``, or an ``MGL`` model with one shared cause, has one shock, and both agree. By rate, the causes strike independently. @@ -542,154 +704,240 @@ def __init__( ): if len(letters) < 1: raise ValueError("MGL needs at least one parameter (beta).") - for value in letters: - if not (0.0 <= value <= 1.0): + for given in letters: + if not (0.0 <= number_or_nan(given) <= 1.0): raise ValueError( - f"MGL parameters must be in [0, 1], got {value!r}." + "MGL parameters must be numbers in [0, 1], got " + f"{given!r}." ) - self.letters = tuple(float(v) for v in letters) - self.basis = _check_basis(basis) - if shocks is None: - shocks = "independent" if self.basis == "rate" else "exclusive" - if shocks not in ("exclusive", "independent"): - raise ValueError( - "shocks must be 'exclusive' or 'independent', got " - f"{shocks!r}." - ) - if self.basis == "rate" and shocks != "independent": - raise ValueError( - "By rate, every cause strikes independently: shocks=" - "'exclusive' splits the probability (basis='probability')." - ) - self._shocks = shocks - - @property - def shocks(self) -> str: - """How the shared causes combine: ``"exclusive"`` (one at most, - the probability split's default) or ``"independent"`` (as - independent basic events; by rate, always).""" - return self._shocks + self.letters = tuple(number_or_nan(v) for v in letters) + super().__init__(basis, shocks) @property def group_size(self) -> int: """The number of members this model describes, ``len(letters) + 1``.""" return len(self.letters) + 1 - def required_group_size(self) -> int: - """The group size this model requires, ``group_size``. + def _fractions(self, m: int) -> List[float]: + # rho[0..m] is rho_1, ..., rho_{m+1}: 1, the letters, then 0. + rho = [1.0, *self.letters, 0.0] + out, product_so_far = [], 1.0 + for k in range(1, m + 1): + product_so_far *= rho[k - 1] + out.append(product_so_far * (1.0 - rho[k]) / comb(m - 1, k - 1)) + return out - [`CCFGroup`][repyability.CCFGroup] rejects a group whose number of - members differs. + def _by_rate(self) -> "MGL": + return MGL(*self.letters, basis="rate") - Returns - ------- - int - ``len(letters) + 1``. - """ - return self.group_size + def __eq__(self, other: object) -> bool: + return ( + isinstance(other, MGL) + and other.letters == self.letters + and other.basis == self.basis + and other.shocks == self.shocks + ) - def _specific_set_prob(self, m: int, k: int, Q: np.ndarray) -> np.ndarray: - # rho[0..m-1] represents rho_1..rho_m (rho_1 = 1, then the letters). - rho = [1.0] + list(self.letters) - prod = 1.0 - for i in range(k): - prod *= rho[i] - rho_next = rho[k] if k < m else 0.0 - return (prod * (1.0 - rho_next) / comb(m - 1, k - 1)) * Q + def __hash__(self) -> int: + return hash( + (type(self).__name__, self.letters, self.basis, self.shocks) + ) - def decompose(self, members: Collection[Hashable], Q) -> Decomposition: - """Split the failure probability into independent and shared parts. + def __repr__(self) -> str: + parts = [repr(letter) for letter in self.letters] + self._options() + return f"MGL({', '.join(parts)})" - With ``m = len(members)``, each member fails alone with - ``Q_1 = (1 - beta) * Q``, and each specific set of ``k >= 2`` - members fails together with the basic-event probability ``Q_k`` - given in the class description. The RBD calls this at each - evaluation time with ``Q``, the probability that the group's first - member has failed by then. - By rate, or with ``shocks="independent"``, the shocks are the sets - of members the causes that have struck fail between them: several - causes may strike, and a member fails if any of its causes has, so - these are the distinct unions, mutually exclusive as the RBD needs, - each with its probability. +#: The testing schemes an alpha-factor model's parameters were estimated +#: under: ``"staggered"``, the members tested one after another, or +#: ``"non-staggered"``, all at once. +TESTING = ("staggered", "non-staggered") - Parameters - ---------- - members : collection of hashable - The group's member node names: ``group_size`` of them, as - [`CCFGroup`][repyability.CCFGroup] ensures. - Q : float or array_like - Each member's total failure probability (the same for every - member of a symmetric group), at one or more times. +#: How far the alpha factors given may sum from 1 (published tables are +#: rounded); they are scaled to sum to 1 exactly. +ALPHA_TOLERANCE = 0.01 - Returns - ------- - q_independent : numpy.ndarray - ``Q_1``, each member's independent failure probability, at - least 1-d (by rate, ``1 - (1 - Q) ** (1 - beta)``). - shocks : list of (frozenset, numpy.ndarray) - ``(subset, Q_k)`` for every subset of ``k = 2, ..., m`` members, - smaller subsets first; by rate, ``(subset, probability)`` for - every set the struck causes can fail together. - """ - Q = np.atleast_1d(np.asarray(Q, dtype=float)) - q_independent, _, shocks = self._split(members, Q, 1.0 - Q) - return q_independent, shocks - def _split(self, members, Q, R): - """Each member's probability of failing on its own and of not - doing so, and the shocks (see ``decompose``), at the members' - probabilities of failing ``Q`` and of surviving ``R`` (both 1-d - arrays: by rate, whichever keeps its precision is used).""" - members = tuple(members) - if self.basis == "rate": - independent, causes = self._causes(members) - return _rate_outcomes(members, independent, causes, _hazard(Q, R)) - if self.shocks == "independent": - independent, causes = self._causes(members) - return _event_outcomes(members, independent, causes, Q) - m = len(members) - q_independent = self._specific_set_prob(m, 1, Q) - shocks: List[Tuple[frozenset, np.ndarray]] = [] - for k in range(2, m + 1): - q_k = self._specific_set_prob(m, k, Q) - for subset in combinations(members, k): - shocks.append((frozenset(subset), q_k)) - return q_independent, 1.0 - q_independent, shocks +class AlphaFactor(_SpecificSets): + """The alpha-factor common-cause model. - def _causes(self, members) -> Tuple[float, list]: - """By rate: each member's own causes' fraction of its hazard, and - each shared cause, as the members it fails and its fraction.""" - members = tuple(members) - m = len(members) - one = np.ones(1) - causes = [ - (frozenset(subset), float(self._specific_set_prob(m, k, one)[0])) - for k in range(2, m + 1) - for subset in combinations(members, k) + The parameters are the shares of a group's failure events by how many + members each failed: ``alpha_k`` is the fraction of the events that + failed exactly ``k`` of the ``m`` members, so ``alpha_1, ..., alpha_m`` + sum to 1. Each is estimated directly from event counts, which is why + common-cause data sources (NUREG/CR-5485, the NRC's CCF parameter + estimates) publish this model's parameters. The multiplicities are the + [`MGL`][repyability.MGL] model's: the probability that a common cause + fails a *specific* set of ``k`` of the ``m`` members is + + ``Q_k = [1 / C(m-1, k-1)] * alpha_k * Q`` + + for a group whose members are tested one after another, in a + staggered scheme (``testing="staggered"``), and + + ``Q_k = [k / C(m-1, k-1)] * (alpha_k / alpha_t) * Q`` + + with ``alpha_t = sum of k * alpha_k``, for one tested all at once, + in a non-staggered scheme (``testing="non-staggered"``): the schemes + count a multiple failure differently (NUREG/CR-5485), so use the one + the estimates were published for. + Either way the ``Q_k`` of the sets holding a component sum to its total + failure probability ``Q``, and everything else is as for ``MGL`` (the + ``basis``, the ``shocks``, and what the diagrams do with the group): + each ``AlphaFactor`` model is an ``MGL`` model with the letters + ``rho_k = sum over j >= k of C(m-1, j-1) Q_j / sum over j >= k-1``. + + The levers of the model (its ``parameters``, for ``levers()``, + ``parameter_sensitivity`` and the parameter uncertainty) are + ``alpha_2, ..., alpha_m``, named ``ccf_alpha_2``, ...; ``alpha_1`` is + the remainder, so a change that would take their sum past 1 is + refused. + + Use it through a [`CCFGroup`][repyability.CCFGroup]; models with equal + alpha factors, ``testing``, ``basis`` and ``shocks`` compare equal. + + Parameters + ---------- + alphas : sequence of float + ``alpha_1, ..., alpha_m``, each in ``[0, 1]``, summing to 1 (a + rounded table's, summing to within 1% of 1, are scaled to 1); at + least two. A group of ``m`` members needs ``m`` alpha factors. + testing : {"staggered", "non-staggered"}, optional + The testing scheme the alpha factors were estimated under, by + default ``"staggered"`` (keyword only). + basis : {"probability", "rate"}, optional + What the alpha factors split, by default ``"probability"`` + (keyword only; see [`MGL`][repyability.MGL]). + shocks : {"exclusive", "independent"}, optional + How the shared causes combine when the alpha factors split the + probability (keyword only): by default ``"exclusive"``, one at + most; ``"independent"``, as independent basic events. By rate + they are always independent. + + Raises + ------ + ValueError + If fewer than two alpha factors are given, any is not a number in + ``[0, 1]``, they do not sum to 1, ``testing`` is neither + ``"staggered"`` nor ``"non-staggered"``, ``basis`` is neither + ``"probability"`` nor ``"rate"``, or ``shocks`` is neither + ``"exclusive"`` nor ``"independent"`` (``"exclusive"`` by rate). + + Examples + -------- + A three-member group in which 95% of the failure events failed one + member, 4% two and 1% all three, each member failing with probability + ``Q = 0.1``, its tests staggered: + + >>> from repyability import AlphaFactor + >>> model = AlphaFactor([0.95, 0.04, 0.01]) + >>> model.group_size + 3 + >>> q_independent, shocks = model.decompose(["a", "b", "c"], 0.1) + >>> round(float(q_independent[0]), 4) # alpha_1 * Q + 0.095 + >>> for members, q in shocks: + ... print(sorted(members), round(float(q[0]), 4)) + ['a', 'b'] 0.002 + ['a', 'c'] 0.002 + ['b', 'c'] 0.002 + ['a', 'b', 'c'] 0.001 + + The same alpha factors estimated under non-staggered testing give + more weight to the multiple failures (``alpha_t = 1.06``): + + >>> model = AlphaFactor([0.95, 0.04, 0.01], testing="non-staggered") + >>> q_independent, shocks = model.decompose(["a", "b", "c"], 0.1) + >>> round(float(q_independent[0]), 4) # 0.95 / 1.06 * Q + 0.0896 + >>> round(float(shocks[0][1][0]), 4) # 2 / 2 * 0.04 / 1.06 * Q + 0.0038 + >>> round(float(shocks[-1][1][0]), 4) # 3 * 0.01 / 1.06 * Q + 0.0028 + """ + + def __init__( + self, + alphas: Collection[float], + *, + testing: str = "staggered", + basis: str = "probability", + shocks: Optional[str] = None, + ): + if isinstance(alphas, (str, bytes)) or not isinstance( + alphas, Collection + ): + raise ValueError( + "alphas must be the alpha factors alpha_1, ..., alpha_m of a " + f"group of m members, in a list; got {alphas!r}." + ) + values = [number_or_nan(alpha) for alpha in alphas] + if len(values) < 2: + raise ValueError( + "AlphaFactor needs an alpha factor for each of a group's " + f"members, at least two (alpha_1, alpha_2); got {len(values)}." + ) + if not all(0.0 <= value <= 1.0 for value in values): + raise ValueError( + "Alpha factors must be numbers in [0, 1], got " + f"{list(alphas)!r}." + ) + total = sum(values) + if abs(total - 1.0) > ALPHA_TOLERANCE: + raise ValueError( + f"Alpha factors must sum to 1, got {list(alphas)!r}, which " + f"sum to {total:.4g}." + ) + self.alphas = tuple(value / total for value in values) + one_of("testing", testing, TESTING) + self.testing = testing + super().__init__(basis, shocks) + + @property + def group_size(self) -> int: + """The number of members this model describes, ``len(alphas)``.""" + return len(self.alphas) + + def _fractions(self, m: int) -> List[float]: + if self.testing == "staggered": + weights = list(self.alphas) + else: + alpha_t = sum(k * alpha for k, alpha in enumerate(self.alphas, 1)) + weights = [ + k * alpha / alpha_t for k, alpha in enumerate(self.alphas, 1) + ] + return [ + weight / comb(m - 1, k - 1) for k, weight in enumerate(weights, 1) ] - return float(self._specific_set_prob(m, 1, one)[0]), causes + + def _by_rate(self) -> "AlphaFactor": + return AlphaFactor(self.alphas, testing=self.testing, basis="rate") def __eq__(self, other: object) -> bool: return ( - isinstance(other, MGL) - and other.letters == self.letters + isinstance(other, AlphaFactor) + and other.alphas == self.alphas + and other.testing == self.testing and other.basis == self.basis and other.shocks == self.shocks ) def __hash__(self) -> int: return hash( - (type(self).__name__, self.letters, self.basis, self.shocks) + ( + type(self).__name__, + self.alphas, + self.testing, + self.basis, + self.shocks, + ) ) def __repr__(self) -> str: - parts = [repr(letter) for letter in self.letters] - if self.basis == "rate": - parts.append("basis='rate'") - elif self.shocks == "independent": - parts.append("shocks='independent'") - return f"MGL({', '.join(parts)})" + parts = [repr(list(self.alphas))] + if self.testing != "staggered": + parts.append(f"testing={self.testing!r}") + return f"AlphaFactor({', '.join(parts + self._options())})" #: The names of an MGL model's letters, in order: ``beta`` first. @@ -723,9 +971,14 @@ def __repr__(self) -> str: def parameters(model) -> Dict[str, float]: """A common-cause model's parameters by name: a ``BetaFactor``'s ``beta``, an ``MGL`` model's letters (``beta``, ``gamma``, ``delta``, - ...).""" + ...), an ``AlphaFactor`` model's ``alpha_2``, ..., ``alpha_m`` + (``alpha_1`` being the remainder).""" if isinstance(model, BetaFactor): return {"beta": model.beta} + if isinstance(model, AlphaFactor): + return { + f"alpha_{k}": value for k, value in enumerate(model.alphas[1:], 2) + } return { LETTERS[j] if j < len(LETTERS) else f"letter{j + 1}": value for j, value in enumerate(model.letters) @@ -735,7 +988,8 @@ def parameters(model) -> Dict[str, float]: def with_parameters(model, values: Dict[str, float]) -> "_Model": """``model`` with the parameters ``values`` names changed (see ``parameters``), its basis kept: a ``ValueError`` for one outside - ``[0, 1]`` or a name it does not have.""" + ``[0, 1]``, a name it does not have, or alpha factors summing past + 1.""" current = parameters(model) unknown = [name for name in values if name not in current] if unknown: @@ -746,6 +1000,20 @@ def with_parameters(model, values: Dict[str, float]) -> "_Model": current.update({name: float(v) for name, v in values.items()}) if isinstance(model, BetaFactor): return BetaFactor(current["beta"], basis=model.basis) + if isinstance(model, AlphaFactor): + rest = list(current.values()) + first = 1.0 - sum(rest) + if first < -1e-12: + raise ValueError( + f"{list(current)} must sum to at most 1 (alpha_1 is the " + f"remainder), got {rest}, which sum to {sum(rest):.4g}." + ) + return AlphaFactor( + [max(first, 0.0), *rest], + testing=model.testing, + basis=model.basis, + shocks=model.shocks, + ) return MGL(*current.values(), basis=model.basis, shocks=model.shocks) @@ -758,11 +1026,7 @@ def validity_warning(group: "CCFGroup", Q: float) -> None: from repyability.utils.wrappers import outside_level model = group.model - rate: _Model = ( - BetaFactor(model.beta, basis="rate") - if isinstance(model, BetaFactor) - else MGL(*model.letters, basis="rate") - ) + rate = model._by_rate() warnings.warn( f"Common-cause group {list(group.members)} ({model!r}): its " f"members' probability of failing reaches Q = {Q:.3g} at the times " @@ -809,17 +1073,20 @@ class CCFGroup: The RBD nodes that share the common cause (at least two, distinct). Standard CCF theory is for *symmetric* groups, so the members must carry identical component models. - model : BetaFactor or MGL + model : BetaFactor, MGL or AlphaFactor The common-cause model coupling the members. An [`MGL`][repyability.MGL] model fixes the group size (``m - 1`` - letters for ``m`` members). + letters for ``m`` members), as an + [`AlphaFactor`][repyability.AlphaFactor] model does (``m`` alpha + factors). Raises ------ ValueError If there are fewer than two members, a member is listed twice, - ``model`` is not a ``BetaFactor`` or ``MGL``, or an ``MGL`` model's - group size differs from the number of members. + ``model`` is not a ``BetaFactor``, ``MGL`` or ``AlphaFactor``, or + an ``MGL`` or ``AlphaFactor`` model's group size differs from the + number of members. Notes ----- @@ -847,6 +1114,13 @@ class CCFGroup: """ def __init__(self, members: Collection[Hashable], model: Any): + if isinstance(members, (str, bytes)) or not isinstance( + members, Collection + ): + raise ValueError( + "CCF group members must be a collection of node names, " + f"not one name: got {members!r}." + ) members = tuple(members) if len(members) < 2: raise ValueError( @@ -856,10 +1130,10 @@ def __init__(self, members: Collection[Hashable], model: Any): raise ValueError( f"CCF group members must be distinct, got {list(members)}." ) - if not isinstance(model, (BetaFactor, MGL)): + if not isinstance(model, _Model): raise ValueError( - "CCFGroup model must be a BetaFactor or MGL; alpha-factor is " - "not supported yet." + "CCFGroup model must be a BetaFactor, MGL or AlphaFactor, " + f"got {model!r}." ) required = model.required_group_size() if required is not None and required != len(members): @@ -876,7 +1150,7 @@ def __repr__(self) -> str: def as_groups(groups) -> list: """A diagram's ``ccf_groups`` as a list: none for None, and one for a - single ``CCFGroup`` (#232); else the groups given, which the diagram + single ``CCFGroup``; else the groups given, which the diagram checks.""" if groups is None: return [] @@ -890,6 +1164,40 @@ def as_groups(groups) -> list: return list(groups) +def checked_groups(groups, check_member, saved, unlike, noun="Node") -> list: + """The common-cause ``groups``, checked as every diagram and tree + checks them: each a ``CCFGroup``, each member passing + ``check_member(member, group)`` (which raises for one that cannot be in + a group) and in one group at + most, and a group's members alike as ``saved`` saves them (the models + assume a symmetric group); a group whose members cannot be saved is + not compared. ``unlike(group)`` is the message for one that is not + symmetric.""" + from ._model_utils import SAVE_ERRORS + + seen: set = set() + for group in groups: + if not isinstance(group, CCFGroup): + raise ValueError( + "ccf_groups must contain CCFGroup instances, got " + f"{type(group).__name__}." + ) + for member in group.members: + check_member(member, group) + if member in seen: + raise ValueError( + f"{noun} {member!r} appears in more than one CCF group." + ) + seen.add(member) + try: + forms = [saved(member) for member in group.members] + except SAVE_ERRORS: + continue + if any(form != forms[0] for form in forms[1:]): + raise ValueError(unlike(group)) + return list(groups) + + def shock_outcomes(groups, base_probabilities, base_failures, check=None): """Every combination of the common-cause ``groups``' mutually exclusive shock outcomes: for each, its probability, the nodes' probabilities of @@ -901,7 +1209,6 @@ def shock_outcomes(groups, base_probabilities, base_failures, check=None): The exact values condition on the groups module by module instead (see ``_ccf_modules``), and must agree with the sum over these, which the capacity distribution and the redundancy allocations still take.""" - from itertools import product # Each group's mutually-exclusive shock outcomes: (weight, {member: # reliability}, {member: unreliability}) for every subset that can diff --git a/repyability/rbd/chunks.py b/repyability/rbd/chunks.py index 327c48a8..f4b380fe 100644 --- a/repyability/rbd/chunks.py +++ b/repyability/rbd/chunks.py @@ -266,7 +266,7 @@ def from_dict(cls, data: dict) -> "SimulationChunk": ValueError If ``data`` is not a saved chunk. """ - from repyability.rbd.repairable_rbd import _Tally + from repyability.rbd._tally import _Tally from repyability.rbd.serialisation import _node_name if not isinstance(data, dict) or data.get("kind") != ( diff --git a/repyability/rbd/degrading_node.py b/repyability/rbd/degrading_node.py index 801956e2..c576b4e4 100644 --- a/repyability/rbd/degrading_node.py +++ b/repyability/rbd/degrading_node.py @@ -215,16 +215,15 @@ def ff(self, x, *args, **kwargs): def mean(self, mc_samples=None, seed=None, *, method=None): """Mean lifetime (MTTF): the sum of the stages' mean times, exactly; infinite if a stage may never end. ``method="simulate"`` estimates - it from ``mc_samples`` draws of ``random`` instead (#233). + it from ``mc_samples`` draws of ``random`` instead. Parameters ---------- mc_samples : int, optional With ``method="simulate"``, the number of draws, 10_000 by - default. The exact mean draws nothing, and ignores it with a - ``FutureWarning`` (0.14 will refuse it). + default. The exact mean draws nothing, and refuses it. seed : int or None, optional - Seed for those draws, ignored as ``mc_samples`` is. + Seed for those draws, refused as ``mc_samples`` is. method : {None, "exact", "simulate"}, optional As for ``StandbyModel.mean``: None and ``"exact"`` give the exact mean. @@ -233,6 +232,11 @@ def mean(self, mc_samples=None, seed=None, *, method=None): ------- float The mean lifetime, or its estimate. + + Raises + ------ + ValueError + If ``mc_samples`` or ``seed`` is given to the exact mean. """ return drawn_mean( self, diff --git a/repyability/rbd/helper_classes.py b/repyability/rbd/helper_classes.py index 1046067c..5f420260 100644 --- a/repyability/rbd/helper_classes.py +++ b/repyability/rbd/helper_classes.py @@ -234,7 +234,7 @@ def random(cls, size): def perfect_class(model): """``model``, but for an instance of ``PerfectReliability`` or ``PerfectUnreliability`` the class itself, which the diagrams - recognise by identity: an instance stands for its class (#232).""" + recognise by identity: an instance stands for its class.""" if isinstance(model, (PerfectReliability, PerfectUnreliability)): return type(model) return model diff --git a/repyability/rbd/load_sharing_node.py b/repyability/rbd/load_sharing_node.py index 89a6074d..aa6d55e5 100644 --- a/repyability/rbd/load_sharing_node.py +++ b/repyability/rbd/load_sharing_node.py @@ -82,6 +82,8 @@ def _identical_exponential_stage_rates(models, load, k, baselines, phi_table): lam = rate elif not np.isclose(rate, lam): return None + if lam is None: + return None # phi must be identical across units at every stage for the closed form. rates = [] for s in range(n, k - 1, -1): @@ -242,7 +244,7 @@ def __init__( # The hypoexponential closed form needs distinct stage rates. If a # load effect collides two of them, surpyval rejects the rates; fall # back to the simulation path rather than fail. - self._sf_model: object = None + self._sf_model: Any = None if rates is not None and _all_distinct(rates): try: self._sf_model = Hypoexponential.from_params(rates) @@ -320,8 +322,9 @@ def _row_sampler(self): """``random(1)`` as a :class:`~._sampling.RowSampler` (one threshold draw per unit, in unit order), so an RBD with this node batches its draws; ``None`` unless every baseline's draws can be replayed.""" - baselines = [inverse_sampler(base) for base in self._baselines] - if any(base is None for base in baselines): + found = [inverse_sampler(base) for base in self._baselines] + baselines = [base for base in found if base is not None] + if len(baselines) < len(found): return None def draw(u): @@ -390,18 +393,17 @@ def mean(self, mc_samples=None, seed=None, *, method=None): integral of the survival function). A group of different units has neither (see ``is_simulated``), and refuses, unless asked for an estimate: the mean of ``mc_samples`` new draws of ``random``. - ``method="simulate"`` estimates it so whatever the group (#233). + ``method="simulate"`` estimates it so whatever the group. Parameters ---------- mc_samples : int, optional The number of draws to estimate the mean from, 10_000 by default: with ``method="simulate"``, or for a group with no - exact mean. An exact mean draws nothing, and ignores it with a - ``FutureWarning`` (0.14 will refuse it). + exact mean. An exact mean draws nothing, and refuses it. seed : int or None, optional Seed for those draws (see ``random``), by default None. - Ignored as ``mc_samples`` is. + Refused as ``mc_samples`` is. method : {None, "exact", "simulate"}, optional As for ``StandbyModel.mean``. @@ -416,9 +418,8 @@ def mean(self, mc_samples=None, seed=None, *, method=None): If the group has no exact mean and neither ``mc_samples`` nor ``seed`` is given (or ``method="exact"``). ValueError - If ``method`` is not one of these. - TypeError - If ``method="exact"`` is given with ``mc_samples`` or ``seed``. + If ``method`` is not one of these, or ``mc_samples`` or ``seed`` + is given to an exact mean. """ def exact() -> float: diff --git a/repyability/rbd/node_state.py b/repyability/rbd/node_state.py index 9c448a53..fe88f816 100644 --- a/repyability/rbd/node_state.py +++ b/repyability/rbd/node_state.py @@ -16,8 +16,18 @@ from dataclasses import dataclass from typing import Optional +from repyability.utils.deprecation import renamed, renamed_attribute -@dataclass(frozen=True) +#: The release that refuses ``phase``, renamed ``since_scheduled`` in 0.14 +#: (#350). +_PHASE_REMOVAL = "0.15" +_PHASE_WHY = ( + "it is the time since the last scheduled replacement or test, and a " + "phased mission has phases of its own" +) + + +@dataclass(frozen=True, init=False) class NodeState: """The current condition of a single RBD node. @@ -32,15 +42,16 @@ class NodeState: is immutable (a frozen dataclass). A [`RepairableRBD`][repyability.RepairableRBD] takes the same states, - with three more fields, to start its analyses from now (``state=`` of + with more fields, to start its analyses from now (``state=`` of ``point_availability``, ``availability`` and the others): a component up is ``age`` since it was last put into service as new; one down has been down for ``down_for``, in a repair, or (``maintenance=True``) in its preventive maintenance; one on a calendar (block replacement, or - tests of hidden failures) is ``phase`` past its last scheduled - replacement or test; and ``stationary=True`` puts it in its long-run - state instead, for a component long in service whose state is not - known. + tests of hidden failures) is ``since_scheduled`` past its last + scheduled replacement or test; one repaired imperfectly has the + ``virtual_age`` it had at its last repair; and ``stationary=True`` + puts it in its long-run state instead, for a component long in + service whose state is not known. Parameters ---------- @@ -48,7 +59,10 @@ class NodeState: The component's current age / accumulated operating time (``>= 0``), by default ``0.0``. Age 0 is the same as no conditioning whenever the component's reliability at time 0 is 1, as for ordinary lifetime - distributions. + distributions. In a ``RepairableRBD`` it is on the diagram's + clock: for a component that operates part of the time (a spec's + ``"duty"``), the calendar time since it was put into service, + its operating age being ``duty * age``. alive : bool, optional Whether the component is currently working, by default ``True``. down_for : float, optional @@ -59,23 +73,39 @@ class NodeState: For a repairable component that is down: whether it is down for its preventive maintenance rather than for a repair, by default ``False``. - phase : float, optional + since_scheduled : float, optional For a repairable component on a calendar: the time since its last block replacement, or its last test (of hidden failures, at which - it was found working; with an ``age`` below the phase, it was put - into service since, and has not been tested), by default None: 0, - as from new. + it was found working; with an ``age`` below it, it was put into + service since, and has not been tested), by default None: 0, as + from new. stationary : bool, optional For a repairable component: start it in its long-run state, by - default ``False``. Its ``phase`` still places it on its calendar; - the other fields must be left at their defaults. + default ``False``. Its ``since_scheduled`` still places it on its + calendar; the other fields must be left at their defaults. + virtual_age : float, optional + For a repairable component repaired imperfectly (a spec's + ``"repair"``, or a fitted ``GeneralizedRenewal`` or Poisson + process): its virtual age at its last repair, by default None (as + new); ``age`` is then its operating time since. One down is at + this virtual age once its repair is over. For a unit of a fitted + surpyval renewal model, ``unit_states()`` gives its + ``virtual_age`` now and ``since_failure``: give ``age= + since_failure`` and ``virtual_age=virtual_age - since_failure``. + The simulations take it. + phase : float, optional + Deprecated: ``since_scheduled``, which it is renamed (#350), as a + phased mission has phases of its own. Refused in 0.15. Raises ------ ValueError - If ``age``, ``down_for`` or ``phase`` is negative or not finite, - ``down_for`` or ``maintenance`` is given for a component that is - alive, or another field is given with ``stationary``. + If ``age``, ``down_for``, ``since_scheduled`` or ``virtual_age`` + is negative or not finite, ``down_for`` or ``maintenance`` is given + for a component that is alive, or another field is given with + ``stationary``. + TypeError + If ``since_scheduled`` is given under both names. Notes ----- @@ -113,27 +143,79 @@ class NodeState: >>> round(rbd.sf_given_state(20, state), 4) 0.7866 - A repairable pump down for 2 hours of a repair, and one 300 hours old: + A repairable pump down for 2 hours of a repair, one 300 hours old, + and one tested weekly, last tested 100 hours ago: >>> NodeState(alive=False, down_for=2.0) NodeState(age=0.0, alive=False, down_for=2.0) >>> NodeState(age=300.0) NodeState(age=300.0, alive=True) + >>> NodeState(age=300.0, since_scheduled=100.0) + NodeState(age=300.0, alive=True, since_scheduled=100.0) """ age: float = 0.0 alive: bool = True down_for: float = 0.0 maintenance: bool = False - phase: Optional[float] = None + since_scheduled: Optional[float] = None stationary: bool = False + virtual_age: Optional[float] = None + + def __init__( + self, + age: float = 0.0, + alive: bool = True, + down_for: float = 0.0, + maintenance: bool = False, + since_scheduled: Optional[float] = None, + stationary: bool = False, + virtual_age: Optional[float] = None, + *, + phase: Optional[float] = None, + ) -> None: + if phase is not None: + if since_scheduled is not None: + raise TypeError( + "NodeState takes since_scheduled alone: phase is its " + "old name." + ) + renamed( + "NodeState", + "phase", + "since_scheduled", + _PHASE_WHY, + removal=_PHASE_REMOVAL, + ) + since_scheduled = phase + object.__setattr__(self, "age", age) + object.__setattr__(self, "alive", alive) + object.__setattr__(self, "down_for", down_for) + object.__setattr__(self, "maintenance", maintenance) + object.__setattr__(self, "since_scheduled", since_scheduled) + object.__setattr__(self, "stationary", stationary) + object.__setattr__(self, "virtual_age", virtual_age) + self.__post_init__() + + @property + def phase(self) -> Optional[float]: + """``since_scheduled``, by its old name: deprecated, and refused in + 0.15 (#350).""" + renamed_attribute( + "NodeState", + "phase", + "since_scheduled", + _PHASE_WHY, + removal=_PHASE_REMOVAL, + ) + return self.since_scheduled def __post_init__(self) -> None: if self.age < 0: raise ValueError( f"NodeState.age must be non-negative, got {self.age!r}." ) - for name in ("age", "down_for", "phase"): + for name in ("age", "down_for", "since_scheduled", "virtual_age"): value = getattr(self, name) if value is not None and not (math.isfinite(value) and value >= 0): raise ValueError( @@ -146,11 +228,16 @@ def __post_init__(self) -> None: "that is down: give alive=False with them." ) if self.stationary and ( - self.age or not self.alive or self.down_for or self.maintenance + self.age + or not self.alive + or self.down_for + or self.maintenance + or self.virtual_age ): raise ValueError( "A stationary NodeState is in its long-run state, which " - "decides its age and whether it is up: give only its phase." + "decides its age and whether it is up: give only its " + "since_scheduled." ) def __repr__(self) -> str: @@ -159,10 +246,12 @@ def __repr__(self) -> str: fields.append(f"down_for={self.down_for!r}") if self.maintenance: fields.append("maintenance=True") - if self.phase is not None: - fields.append(f"phase={self.phase!r}") + if self.since_scheduled is not None: + fields.append(f"since_scheduled={self.since_scheduled!r}") if self.stationary: fields.append("stationary=True") + if self.virtual_age is not None: + fields.append(f"virtual_age={self.virtual_age!r}") return f"NodeState({', '.join(fields)})" @property @@ -173,5 +262,6 @@ def new(self) -> bool: self.alive and not self.age and not self.stationary - and not self.phase + and not self.since_scheduled + and not self.virtual_age ) diff --git a/repyability/rbd/non_repairable_rbd.py b/repyability/rbd/non_repairable_rbd.py index cddf1683..09fecd4a 100644 --- a/repyability/rbd/non_repairable_rbd.py +++ b/repyability/rbd/non_repairable_rbd.py @@ -6,6 +6,7 @@ """ import functools +import inspect import math import pickle import zlib @@ -32,14 +33,14 @@ import numpy as np from numpy.typing import ArrayLike -from scipy.optimize import brentq from repyability.utils.checks import ( no_distribution, + one_of, real_array, + simulation_options, + structure_method, ) -from repyability.utils.checks import seed as check_seed -from repyability.utils.checks import simulation_options, structure_method from repyability.utils.wrappers import conditional_survival, numpy_seed from . import _ccf_modules @@ -47,17 +48,32 @@ from . import _sensitivity from . import capacity as _capacity from . import redundancy_allocation +from ._degradation import check_life, is_degradation from ._mean_lifetime import mean_lifetime, model_kinks, model_knots +from ._measures import ( + barlow_proschan, + cumulative_hazard, + density, + density_by_differences, + difference_steps, + invert_reliability, + life_end, + model_scale, + rate_parts, +) from ._model_utils import ( + MODEL_ERRORS, + always_works, is_fixed_probability, is_mixture, + lever_spec, lfp_p, model_mean, parametric_spec, refuse_nonparametric, ) from ._sampling import RowSampler, column, inverse_sampler, row_sampler -from .ccf import VALIDITY, BetaFactor, CCFGroup, as_groups +from .ccf import VALIDITY, BetaFactor, CCFGroup, as_groups, checked_groups from .ccf import parameters as ccf_parameters from .ccf import shock_outcomes, validity_warning from .ccf import with_parameters as with_ccf_parameters @@ -70,7 +86,13 @@ from .load_sharing_node import LoadSharingModel from .modular import GraphStructure from .node_state import NodeState -from .rbd import RBD, _check_on_infeasible_rbd, leaves_out_junctions +from .rbd import ( + RBD, + Pairs, + _check_on_infeasible_rbd, + _node_collection, + leaves_out_junctions, +) from .redundancy_allocation import ComponentOption, active_unreliability from .repeated_node import RepeatedNode from .repeated_standby_node import RepeatedStandbyNode @@ -84,9 +106,18 @@ UncertaintyImportance, UncertaintyResult, ) -from .routes import AnalysisRoute +from .routes import EXACT, NUMERICAL, AnalysisRoute, Refused, model_route from .standby_node import StandbyModel -from .uncertainty import draw_ccf_models, draw_models +from .uncertainty import ( + check_draws, + draw_ccf_models, + draw_models, + draw_with, + fitted_together, + split_group_levers, + varied_ccf_parameters, + varied_parameters, +) # Event class for simulation #: Lifetimes per block of a parallel draw (see ``NonRepairableRBD.random``): @@ -184,10 +215,17 @@ def check_x(func): ``wrap(self, x=None, *args, **kwargs)``. """ + signature = inspect.signature(func) + @functools.wraps(func) - def wrap(obj, x=None, *args, **kwargs): - x, scalar_in = _times(obj, x) - result = func(obj, x, *args, **kwargs) + def wrap(obj, *args, **kwargs): + # Bound first, so that a misspelt keyword is the TypeError Python + # raises for it, not a complaint that ``x`` is missing. + bound = signature.bind(obj, *args, **kwargs) + bound.apply_defaults() + x, scalar_in = _times(obj, bound.arguments["x"]) + bound.arguments["x"] = x + result = func(*bound.args, **bound.kwargs) if scalar_in: if isinstance(result, dict): return {k: np.asarray(v).item() for k, v in result.items()} @@ -201,42 +239,41 @@ def _times(obj, x) -> tuple: """The time argument of an RBD method as a float array of at least one dimension, and whether it was a scalar. ``x=None`` stands for any time (1.0) in a fixed-probability RBD, and is refused in a time-varying - one.""" + one (``obj._KIND`` names the kind of diagram in the message).""" if x is None: if obj.is_fixed: x = 1.0 else: raise ValueError( - "x is required: this RBD is time-varying (at least one " - "node model's probability depends on time)." + f"x is required: this {obj._KIND} is time-varying (at least " + "one model's probability depends on time)." ) scalar_in = np.ndim(x) == 0 return np.atleast_1d(real_array(x, "x")), scalar_in -def _dsf_dparam(cls, params, j, x_arr, rel_step, extras=None) -> np.ndarray: - """Partial derivative of a distribution's ``sf`` at ``x_arr`` with respect - to its ``j``-th parameter, by finite difference. +def _dsf_dparam(spec, j, x_arr, rel_step) -> np.ndarray: + """Partial derivative of a model's ``sf`` at ``x_arr`` with respect to + its ``j``-th parameter (``spec``, a ``parametric_spec``), by finite + difference. - ``cls.from_params`` rebuilds the distribution with a perturbed parameter - (and the model's offset, limited-failure-population and zero-inflation - ``extras``, kept as they are), so this works for any surpyval parametric + ``spec.build`` rebuilds the model with a perturbed parameter (its + offset kept as it is), so this works for any surpyval parametric distribution without hard-coding per-distribution derivative formulae. A central difference is used where both perturbations are valid; if one perturbation falls outside a parameter's admissible range (e.g. a probability leaving ``[0, 1]``) it falls back to a one-sided difference about the unperturbed value. """ - theta = params[j] + theta = spec.params[j] h = rel_step * abs(theta) if theta != 0.0 else rel_step def perturbed(delta): - trial = list(params) + trial = list(spec.params) trial[j] = theta + delta try: - model = cls.from_params(trial, **(extras or {})) - return np.asarray(model.sf(x_arr), dtype=float) - except Exception: + return np.asarray(spec.build(trial).sf(x_arr), dtype=float) + except MODEL_ERRORS: return None up = perturbed(h) @@ -245,9 +282,7 @@ def perturbed(delta): return (up - down) / (2.0 * h) # A perturbation hit a parameter bound; fall back to a one-sided # difference about the unperturbed value. - base = np.asarray( - cls.from_params(list(params), **(extras or {})).sf(x_arr), dtype=float - ) + base = np.asarray(spec.build(spec.params).sf(x_arr), dtype=float) if up is not None: return (up - base) / h if down is not None: @@ -278,18 +313,52 @@ def _system_difference(at, theta, rel_step, base) -> np.ndarray: return np.where(fails <= works, -d_fails, d_works) -def _never_fails(model) -> bool: - """Whether ``model`` never fails, so that giving it for the input or - output node changes nothing (#217): ``PerfectReliability``, or a fixed - probability of failing of 0 (as a ``node_availability`` of 1 gives).""" - if model is PerfectReliability or isinstance(model, PerfectReliability): - return True - if is_fixed_probability(model): - try: - return float(np.ravel(model.ff(1.0))[0]) == 0.0 - except Exception: - return False - return False +def ccf_group_sensitivity( + system, group, spec, x: np.ndarray, rel_step: float +) -> Dict[str, np.ndarray]: + """The sensitivities of a system's reliability at ``x`` to a + common-cause group's parameters (see + ``NonRepairableRBD.parameter_sensitivity``): its members' model's + (``spec``, their one model's ``lever_spec``; None for none), each + moved for every member at once, and its common-cause model's, named + ``ccf_beta``, ``ccf_gamma``, ..., each by a central difference of the + system's probabilities with the groups (see ``_system_difference``). + ``system(members=None, model=None)`` gives the probabilities that the + system works and that it fails, with the members' probabilities of + working and of failing at ``members``, or the group's model replaced by + ``model``.""" + base = system() + out: Dict[str, np.ndarray] = {} + if spec is not None: + for j, name in enumerate(spec.names): + + def at_param(value, j=j): + trial = list(spec.params) + trial[j] = value + try: + model = spec.build(trial) + works = np.asarray(model.sf(x), dtype=float) + fails = np.asarray(model.ff(x), dtype=float) + except MODEL_ERRORS: + return None + return system(members=(works, fails)) + + out[name] = _system_difference( + at_param, spec.params[j], rel_step, base + ) + for name, value in ccf_parameters(group.model).items(): + + def at_letter(value, name=name): + try: + model = with_ccf_parameters(group.model, {name: value}) + except ValueError: + return None + return system(model=model) + + out[f"ccf_{name}"] = _system_difference( + at_letter, value, rel_step, base + ) + return out def _check_model(node, model) -> None: @@ -298,6 +367,8 @@ def _check_model(node, model) -> None: saying what to give instead; a number most likely means a probability.""" no_distribution(model, f"The model of node {node!r}") + if is_degradation(model): + check_life(model, f"The model of node {node!r}") if callable(getattr(model, "sf", None)): return if isinstance(model, (int, float)) and not isinstance(model, bool): @@ -338,8 +409,9 @@ class NonRepairableRBD(RBD): common-cause groups given in ``ccf_groups``. The system reliability [`sf`][repyability.NonRepairableRBD.sf] is - computed exactly from the node reliabilities, using the minimal path - sets (or cut sets); the other analytic methods build on it. The mean + computed exactly from the node reliabilities, by a decomposition of + the structure into modules (see the guide's concepts); the other + analytic methods build on it. The mean time to failure [`mean`][repyability.NonRepairableRBD.mean] integrates it (``method="simulate"`` estimates it from simulated lifetimes instead), and [`random`][repyability.NonRepairableRBD.random] draws @@ -349,8 +421,10 @@ class NonRepairableRBD(RBD): A node model can be: - a surpyval distribution: parametric (e.g. - ``surpyval.Weibull.from_params([100, 2])`` or a fitted model) or a - fixed per-demand probability (``surpyval.FixedEventProbability``); + ``surpyval.Weibull.from_params([100, 2])`` or a fitted model), a + ``MixtureModel``, a fitted Wiener or gamma degradation process (its + time to the threshold), or a fixed per-demand probability + (``surpyval.FixedEventProbability``); not a non-parametric fit (e.g. a ``surpyval.KaplanMeier`` fit), which is refused: fit a parametric distribution in surpyval; - a composite node: a [`StandbyModel`][repyability.StandbyModel], @@ -442,6 +516,7 @@ class NonRepairableRBD(RBD): has. A repeated node has the capacity of the node it repeats, wherever it is drawn: give that node's. + Attributes ---------- reliabilities : dict @@ -607,10 +682,18 @@ def __init__( # with the names given models, which it checks against the edges. self._aliases = dict(repeated) self._models_given = list(reliabilities) + list(repeated) + # The junctions (see ``_junctions``), known before the base class + # works out the structure: its irrelevant nodes are found with them + # folded in. + self._junctions_given = frozenset( + name + for name, model in reliabilities.items() + if model is PerfectReliability + ) self._perfect_given = frozenset( name for name, model in reliabilities.items() - if _never_fails(model) + if always_works(model) ) super().__init__( edges, @@ -691,12 +774,10 @@ def _repr_details(self) -> List[str]: def _junctions(self) -> frozenset: """The ``PerfectReliability`` nodes: drawing devices, such as a - k-out-of-n vote (see ``RBD._junctions``).""" - reliabilities = getattr(self, "reliabilities", {}) + k-out-of-n vote (see ``RBD._junctions``), among the nodes drawn + (known as the base class works out the structure).""" return frozenset( - node - for node in self.nodes - if reliabilities.get(node) is PerfectReliability + node for node in self.nodes if node in self._junctions_given ) def _validate_node_overrides(self, working_nodes, broken_nodes) -> None: @@ -707,7 +788,7 @@ def _validate_node_overrides(self, working_nodes, broken_nodes) -> None: ("working_nodes", working_nodes), ("broken_nodes", broken_nodes), ): - for node in nodes: + for node in _node_collection(label, nodes): if node in self.repeated: raise ValueError( f"Node {node}, given to {label}, is a repeat of node " @@ -728,12 +809,12 @@ def sf( The probability that the system is still working at ``x``, computed exactly from each node's reliability ``sf(x)`` by a - Shannon decomposition over the minimal path sets (``method="p"``) - or cut sets (``method="c"``). Nodes are independent apart from any + Shannon decomposition of the structure's modules. Nodes are + independent apart from any common-cause groups: with ``ccf_groups`` each group is conditioned on within the smallest module holding its members, or its shared causes written out as events of their own where a module's groups - would multiply their outcomes (#219, see the common-cause guide), + would multiply their outcomes (see the common-cause guide), so the cost grows with the number of groups rather than doubling. ``working_nodes`` and ``broken_nodes`` condition on the state of @@ -754,10 +835,10 @@ def sf( broken_nodes : Collection[Hashable], optional Nodes to treat as failed (reliability 0), by default none. method : str, optional - ``"p"`` or ``"paths"`` (the default) uses the minimal path - sets, which avoids deriving the cut sets; ``"c"`` or ``"cuts"`` - uses the minimal cut sets. Both are exact and give the same - result. + ``"p"`` or ``"paths"`` (the default) works out the probability + that the system works; ``"c"`` or ``"cuts"`` the probability + that it fails, and returns its complement. Both use the same + decomposition, are exact and give the same result. Returns ------- @@ -797,9 +878,9 @@ def sf( 0.9 """ # Normalise the (optional) node overrides into sets for O(1) lookup. - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) + working_nodes, broken_nodes = self._held_nodes( + working_nodes, broken_nodes + ) node_probabilities = self._base_node_probabilities( x, working_nodes, broken_nodes @@ -882,7 +963,7 @@ def _ccf_group_sensitivity( its members' model's, each moved for every member at once (they carry one model), and its common-cause model's, named ``ccf_beta``, ``ccf_gamma``, ..., each by a central difference of the system's - probabilities with the groups (see ``_system_difference``).""" + probabilities with the groups (see ``ccf_group_sensitivity``).""" p, q = self._importance_inputs(x, working_nodes, broken_nodes) groups = list(self.ccf_groups) index = next(i for i, g in enumerate(groups) if g is group) @@ -901,77 +982,13 @@ def system(members=None, model=None): given, failing, working_nodes, broken_nodes, trial ) - base = system() - out: Dict[str, np.ndarray] = {} - spec = parametric_spec(self.reliabilities[group.members[0]]) - if spec is not None: - cls, params, names, extras = spec - for j, name in enumerate(names): - - def at_param(value, j=j): - trial = list(params) - trial[j] = value - try: - model = cls.from_params(trial, **extras) - works = np.asarray(model.sf(x), dtype=float) - fails = np.asarray(model.ff(x), dtype=float) - except Exception: - return None - return system(members=(works, fails)) - - out[name] = _system_difference( - at_param, params[j], rel_step, base - ) - for name, value in ccf_parameters(group.model).items(): - - def at_letter(value, name=name): - try: - model = with_ccf_parameters(group.model, {name: value}) - except ValueError: - return None - return system(model=model) - - out[f"ccf_{name}"] = _system_difference( - at_letter, value, rel_step, base - ) - return out - - def _ccf_importances(self, x, working_nodes, broken_nodes) -> dict: - """With common-cause groups, what the importance measures are worked - out from, at ``x``: for each node, the probabilities that it works - and that it has failed, and that the system works and fails with - it working and with it failed; and the system's own. - - A group member works when neither its own failure nor a shock that - strikes it has happened. Given its group's shock outcome (see - ``_ccf_modules.group_outcomes``) the member is independent of every - other node, so each joint probability is a sum over its group's - outcomes of the outcome's probability times the member's and the - system's given it, the other groups conditioned on within their - modules (#219). A node outside the groups is independent of them, - so its "given" values are the system's with it held working or - failed. A member of a group written out as shock events (see - ``_ccf_modules.Plan``) works while neither its own causes nor its - shocks have fired. Every term is a product of probabilities, the - system's worked out as for ``ff``, so a small one keeps its - precision (see ``_ccf_modules.Evaluation.joints``).""" - working, broken = set(working_nodes or ()), set(broken_nodes or ()) - p, q = self._importance_inputs(x, working, broken) - evaluation = self._ccf_evaluation(p, q, working, broken) - grouped = {m for group in self.ccf_groups for m in group.members} - keys = ("works", "fails", "up_ok", "down_ok", "up_bad", "down_bad") - sums: Dict[str, dict] = {key: {} for key in keys} - R, Q = evaluation.system() - for node in self.nodes: - values, _ = evaluation.joints(node) - for key in keys: - sums[key][node] = values[key] - return { - **sums, - "R": np.atleast_1d(R), - "Q": np.atleast_1d(Q), - "grouped": grouped, - } + return ccf_group_sensitivity( + system, + group, + lever_spec(self.reliabilities[group.members[0]]), + x, + rel_step, + ) def _ccf_measure( self, @@ -983,47 +1000,55 @@ def _ccf_measure( fv_type: str = "c", method: str = "exact", ) -> Dict[Any, Union[float, np.ndarray]]: - """An importance measure with common-cause groups (see - ``_ccf_importances``): ``R(1_i)`` and ``R(0_i)`` are the system's - reliability given that node ``i`` works and given that it has - failed (for a member, conditioned through the shock outcomes), - and the measures follow from them as without groups. Fussell-Vesely - sums each outcome's numerators (see ``_fv_numerators``), the nodes - being independent given it.""" - if kind not in ("failure", "success"): - raise ValueError( - f"kind must be 'failure' or 'success', got {kind!r}." - ) + """An importance measure with common-cause groups, at ``x``: + ``R(1_i)`` and ``R(0_i)`` are the system's reliability given that + node ``i`` works and given that it has failed, and the measures + follow from them as without groups. + + A group member works when neither its own failure nor a shock that + strikes it has happened. Given its group's shock outcome (see + ``_ccf_modules.group_outcomes``) the member is independent of every + other node, so the system given its state is a sum over its + group's outcomes of the outcome's probability times the member's + and the system's given it, over the member's own, the other groups + conditioned on within their modules (#219). A node outside the + groups is independent of them, so its "given" values are the + system's with it held working or failed. A member of a group + written out as shock events (see ``_ccf_modules.Plan``) works + while neither its own causes nor its shocks have fired. Every term + is a product of probabilities, the system's worked out as for + ``ff``, so a small one keeps its precision (see + ``_ccf_modules.Evaluation.given``). Fussell-Vesely sums each + outcome's numerators (see ``_fv_numerators``), the nodes being + independent given it.""" + one_of("kind", kind, ("failure", "success")) if fv_type not in ("c", "p"): raise ValueError( "fv_type must be either 'c' (cut-set) or 'p' (path-set), " f"fv_type={fv_type!r} was given." ) - if method not in ("exact", "rare_event"): - raise ValueError( - f"method must be 'exact' or 'rare_event', got {method!r}." - ) + one_of("method", method, ("exact", "rare_event")) + working, broken = self._held_nodes(working_nodes, broken_nodes) + p, q = self._importance_inputs(x, working, broken) if measure == "fussell_vesely": - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) - p, q = self._importance_inputs(x, working, broken) shares = self._ccf_fv_numerators( p, q, working, broken, fv_type, method ) _, Q = self._ccf_system_pair(p, q, working, broken) with np.errstate(divide="ignore", invalid="ignore"): return {node: shares[node] / Q for node in self.nodes} - c = self._ccf_importances(x, working_nodes, broken_nodes) - R, Q = c["R"], c["Q"] + evaluation = self._ccf_evaluation(p, q, working, broken) + R, Q = (np.atleast_1d(value) for value in evaluation.system()) out: Dict[Any, Union[float, np.ndarray]] = {} with np.errstate(divide="ignore", invalid="ignore"): for node in self.nodes: - works, fails = c["works"][node], c["fails"][node] - held = node not in c["grouped"] - on = 1.0 if held else works - off = 1.0 if held else fails - r1, q1 = c["up_ok"][node] / on, c["down_ok"][node] / on - r0, q0 = c["up_bad"][node] / off, c["down_bad"][node] / off + given = evaluation.given((node,)) + (works, r1, q1), (fails, r0, q0) = ( + given[True,], + given[False,], + ) + if node not in evaluation.group_of: + works, fails = evaluation.p[node], evaluation.q[node] # From whichever end keeps its precision. birnbaum = np.where(Q <= R, q0 - q1, r1 - r0) if measure == "birnbaum": @@ -1041,6 +1066,29 @@ def _ccf_measure( out[node] = np.asarray(value, dtype=float) return out + @check_x + def _ccf_pairs(self, x, working_nodes, broken_nodes) -> "Pairs": + """The joint importance of each pair of nodes with common-cause + groups, at ``x`` (#317): the system's reliability given each of + the pair working and failed (see ``_ccf_measure``, and + ``_ccf_modules.joint_importance``: two members of one group are + conditioned on their joint state), ``R(1_i, 1_j) - R(1_i, 0_j) - + R(0_i, 1_j) + R(0_i, 0_j)``; 0 for a pair with a node held.""" + working, broken = self._held_nodes(working_nodes, broken_nodes) + p, q = self._importance_inputs(x, working, broken) + evaluation = self._ccf_evaluation(p, q, working, broken) + held = working | broken + junctions = self._junctions() + nodes = [node for node in self.nodes if node not in junctions] + return Pairs.of( + nodes, + lambda i, j: ( + np.zeros(evaluation.shape) + if i in held or j in held + else _ccf_modules.joint_importance(evaluation, i, j) + ), + ) + def _validate_ccf_groups(self, ccf_groups) -> list: """Validate common-cause groups against the RBD structure. @@ -1048,54 +1096,33 @@ def _validate_ccf_groups(self, ccf_groups) -> list: output node), each node in at most one group, and each group symmetric (identical component models). Returns the validated list. """ - seen: set = set() - for group in ccf_groups: - if not isinstance(group, CCFGroup): + from repyability.rbd.serialisation import serialise_model + + def check_member(member, group): + if member not in self.reliabilities: raise ValueError( - "ccf_groups must contain CCFGroup instances, got " - f"{type(group).__name__}." + f"CCF group member {member!r} is not a node in the RBD." + ) + if member in (self.input_node, self.output_node): + raise ValueError( + f"CCF group member {member!r} cannot be the input or " + "output node." + ) + if member in self.repeated: + raise ValueError( + f"CCF group member {member!r} cannot be a repeated " + "node." ) - for member in group.members: - if member not in self.reliabilities: - raise ValueError( - f"CCF group member {member!r} is not a node in the " - "RBD." - ) - if member in (self.input_node, self.output_node): - raise ValueError( - f"CCF group member {member!r} cannot be the input or " - "output node." - ) - if member in self.repeated: - raise ValueError( - f"CCF group member {member!r} cannot be a repeated " - "node." - ) - if member in seen: - raise ValueError( - f"Node {member!r} appears in more than one CCF group." - ) - seen.add(member) - self._check_symmetric_group(group) - return list(ccf_groups) - - def _check_symmetric_group(self, group) -> None: - # Standard CCF theory is for symmetric groups, so the members must - # carry identical component models. Compare via the serialised form - # (exact); skip silently if a member is not serialisable. - from repyability.rbd.serialisation import serialise_model - try: - specs = [ - serialise_model(self.reliabilities[m]) for m in group.members - ] - except Exception: - return - if any(spec != specs[0] for spec in specs[1:]): - raise ValueError( + return checked_groups( + ccf_groups, + check_member, + lambda member: serialise_model(self.reliabilities[member]), + lambda group: ( f"CCF group {list(group.members)} is not symmetric: its " "members must carry identical component models." - ) + ), + ) def _capacity_models(self) -> dict: """``{node: model}`` for the nodes with no capacity entry whose model @@ -1182,7 +1209,7 @@ def _ccf_plan(self, groups, structure=None) -> "_ccf_modules.Plan": expand = structure is None structure = self._decomposition() if structure is None else structure key = (id(structure), _ccf_modules.Plan.key(groups), expand) - cache = self.__dict__.setdefault("_ccf_plans", {}) + cache = self._cache.kept("ccf_plans") entry = cache.get(key) if entry is None or entry[0] is not structure: plan = _ccf_modules.Plan( @@ -1373,8 +1400,9 @@ def _allocation_groups(self, nodes, options, ways) -> dict: f"Node {node!r} is in a common-cause group whose model, " f"{group.model!r}, is for a group of " f"{len(group.members)}: its copies would join the group, " - "and an MGL model's letters for a larger group are not " - "known. Give the group a BetaFactor model, whose beta " + "and an MGL model's letters (an AlphaFactor model's " + "alpha factors) for a larger group are not known. Give " + "the group a BetaFactor model, whose beta " "holds at any size, or leave its members out of costs." ) if node in options: @@ -1413,7 +1441,7 @@ def _require_time_varying(self) -> None: """Raise if the system reliability does not vary with time, so the time to a reliability is undefined.""" if self.is_fixed: - raise ValueError( + raise Refused( "System reliability does not vary with time (all nodes are " "fixed-probability); the time to a reliability is undefined." ) @@ -1534,9 +1562,9 @@ def _sf_and_ff( summed over their outcomes as for ``sf`` (``method`` changes nothing here).""" method = structure_method(method) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) + working_nodes, broken_nodes = self._held_nodes( + working_nodes, broken_nodes + ) node_probabilities = self._base_node_probabilities( x, working_nodes, broken_nodes @@ -1636,9 +1664,9 @@ def capacity_distribution( 113.13 """ x, scalar = _times(self, x) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) + working_nodes, broken_nodes = self._held_nodes( + working_nodes, broken_nodes + ) self._require_capacity() own = {} for node, model in self._capacity_models().items(): @@ -1695,11 +1723,12 @@ def sf_uncertainty( in surpyval, provides. A node's uncertainty is one of: - ``"fit"``: its parameters are drawn from the normal approximation - of its maximum-likelihood fit (surpyval's ``hess_inv``), on the - log scale for a parameter that must be positive and the logit - scale for one in (0, 1), so that every draw is valid; an offset, - zero-inflation or limited-failure-population parameter keeps its - fitted value; + of its maximum-likelihood fit (surpyval's ``covariance()``), on + the log scale for a parameter that must be positive and the + logit scale for one in (0, 1), so that every draw is valid: + the distribution's parameters and a limited failure population's + ``lfp_p`` or a zero-inflated fit's ``f0``; an offset, which the + covariance leaves out, keeps its fitted value, with a warning; - ``{parameter name: distribution}``: each named parameter is drawn from its distribution (anything with ``qf`` or ``ppf``, such as a surpyval or scipy.stats distribution), the others kept; @@ -1712,7 +1741,7 @@ def sf_uncertainty( them independently would understate the uncertainty, which is about the one population's parameters; so would drawing one and leaving the others as fitted, which is warned about when they hold - the same model object (#214). + the same model object. With common-cause groups, each draw is worked out as ``sf`` works it out, with the groups. A group's members carry one model, so @@ -1721,10 +1750,12 @@ def sf_uncertainty( model may be uncertain too: give the [`CCFGroup`][repyability.CCFGroup] itself as a key, with ``{parameter name: distribution}`` over its model's parameters - (``"beta"``, and an [`MGL`][repyability.MGL] model's ``"gamma"``, - ``"delta"``, ...), each drawn in ``[0, 1]``, or a list of models - (``BetaFactor`` or ``MGL``, for the group's size and with its - model's ``basis``), drawn with replacement. The common-cause models + (``"beta"``, an [`MGL`][repyability.MGL] model's ``"gamma"``, + ``"delta"``, ..., an [`AlphaFactor`][repyability.AlphaFactor] + model's ``"alpha_2"``, ...), each drawn in ``[0, 1]``, or a list + of models (``BetaFactor``, ``MGL`` or ``AlphaFactor``, for the + group's size and with its model's ``basis``), drawn with + replacement. The common-cause models are drawn after the nodes', so the nodes' draws are the same whether or not they are uncertain. @@ -1751,7 +1782,7 @@ def sf_uncertainty( ``"random"`` (the default) draws with pseudo-random numbers; ``"sobol"`` from the points of a scrambled Sobol sequence, which cover the parameters more evenly and shrink the error of - the summaries for the same number of draws (#200). + the summaries for the same number of draws. Returns ------- @@ -1992,7 +2023,7 @@ def mean_uncertainty( ``"random"`` (the default) draws with pseudo-random numbers; ``"sobol"`` from the points of a scrambled Sobol sequence, which cover the parameters more evenly and shrink the error of - the summaries for the same number of draws (#200). + the summaries for the same number of draws. Returns ------- @@ -2066,7 +2097,7 @@ def time_to_reliability_uncertainty( Parameters ---------- target : float or array_like - The reliability level, in (0, 1), or levels (#226): each a row + The reliability level, in (0, 1), or levels: each a row of draws, from the same parameter draws. uncertainty : dict, optional ``{node or tuple of nodes: uncertainty}`` for the uncertain @@ -2080,7 +2111,7 @@ def time_to_reliability_uncertainty( ``"random"`` (the default) draws with pseudo-random numbers; ``"sobol"`` from the points of a scrambled Sobol sequence, which cover the parameters more evenly and shrink the error of - the summaries for the same number of draws (#200). + the summaries for the same number of draws. upper_bound : float, optional An upper bound for each draw's search, at which its reliability is below ``target``; found by doubling if None. @@ -2165,7 +2196,7 @@ def bx_life_uncertainty( Parameters ---------- x : float or array_like - The percentage failed, in (0, 100), or percentages (#226): each + The percentage failed, in (0, 100), or percentages: each a row of draws, from the same parameter draws. uncertainty : dict, optional ``{node or tuple of nodes: uncertainty}`` for the uncertain @@ -2179,7 +2210,7 @@ def bx_life_uncertainty( ``"random"`` (the default) draws with pseudo-random numbers; ``"sobol"`` from the points of a scrambled Sobol sequence, which cover the parameters more evenly and shrink the error of - the summaries for the same number of draws (#200). + the summaries for the same number of draws. upper_bound : float, optional An upper bound for each draw's search, found by doubling if None. @@ -2245,11 +2276,11 @@ def uncertainty_importance( upper_bound: Optional[float] = None, ) -> UncertaintyImportance: """Which input's parameter uncertainty makes a system quantity - uncertain: each uncertain input's share of the quantity's variance - (#196), the inputs as for ``sf_uncertainty``. + uncertain: each uncertain input's share of the quantity's variance, + the inputs as for ``sf_uncertainty``. In the guide's Greeks it is *vega*: whose uncertainty widens the answer - (see [Sensitivities: the Greeks](guide/greeks.md)). + (see [Sensitivities: the Greeks](../guide/greeks.md)). ``of`` is the quantity: ``"sf"``, the reliability at the time/s ``x`` (as ``sf_uncertainty``); ``"mean"``, the MTTF (as @@ -2262,7 +2293,7 @@ def uncertainty_importance( its gradient in input ``k``'s parameters (central differences of the exact quantity, with the parameters moved by ``rel_step`` of themselves) and ``Sigma_k`` their covariance: a fit's (surpyval's - ``hess_inv``) for ``"fit"``, or the variances of the distributions + ``covariance()``) for ``"fit"``, or the variances of the distributions given. The inputs are independent, so each one's part is its own term and the shares add up to 1. A list of models has no parameters to move: it needs ``method="sobol"``. @@ -2342,17 +2373,13 @@ def uncertainty_importance( >>> {n: round(share, 2) for n, share in parts.first_order.items()} {'pump': 0.86, 'valve': 0.14} """ - from .uncertainty import varied_ccf_parameters, varied_parameters quantities = ("sf", "mean", "bx_life", "time_to_reliability") if of not in quantities: raise ValueError( f"of must be one of {list(quantities)}, got {of!r}." ) - if method not in ("delta", "sobol"): - raise ValueError( - f"method must be 'delta' or 'sobol', got {method!r}." - ) + one_of("method", method, ("delta", "sobol")) value = self._uncertain_quantity(of, x) sources, ccf_specs = self._uncertainty_sources(uncertainty) keys = [key for key, _, _ in sources] + [ @@ -2436,7 +2463,7 @@ def shaped(values): values = np.asarray(values, dtype=float).reshape(-1) if scalar: return float(values[0]) - return values.reshape(np.shape(x)) + return values.reshape(np.shape(np.asarray(x))) return UncertaintyImportance( method=method, @@ -2483,23 +2510,23 @@ def _configured_models(self, sources, ccf_specs, varied, configurations): nominal (``(-1, None)``), or input ``k``'s parameter ``j`` moved to a value (``(k, (j, value))``), as ``_quantity_samples`` takes them.""" - from .ccf import with_parameters as with_ccf_parameters groups = sorted(ccf_specs) drawn: Dict[Hashable, list] = {} for k, (_, members, _) in enumerate(sources): model = self.reliabilities[members[0]] positions = varied[k][0] - params = np.atleast_1d(np.asarray(model.params, dtype=float)) + spec = parametric_spec(model) + assert spec is not None models = [] for which, move in configurations: if which != k: models.append(model) continue j, moved = move - changed = params.copy() + changed = list(spec.params) changed[positions[j]] = moved - models.append(model.with_params(list(changed))) + models.append(spec.build(changed)) for node in members: drawn[node] = models drawn_groups: list = [None] * len(self.ccf_groups) @@ -2605,22 +2632,9 @@ def _uncertain_draws( ``sampling="sobol"`` (#200), the draws are the points of a scrambled Sobol sequence, a pair's two sets from dimensions of their own (see ``uncertainty``).""" - from .uncertainty import ( - Counter, - SobolPoints, - check_sampling, - sobol_table, - ) - if isinstance(n_draws, bool) or not isinstance( - n_draws, (int, np.integer) - ): - raise ValueError(f"n_draws must be an integer, got {n_draws!r}.") - if n_draws < 1: - raise ValueError(f"n_draws must be at least 1, got {n_draws}.") - check_sampling(sampling) + check_draws(n_draws) sources, ccf_specs = self._uncertainty_sources(uncertainty) - rng = np.random.default_rng(check_seed(seed)) def draw(n: int, source) -> Tuple[Dict[Hashable, list], list]: drawn: Dict[Hashable, list] = {} @@ -2647,23 +2661,16 @@ def draw(n: int, source) -> Tuple[Dict[Hashable, list], list]: ] return drawn, groups - if sampling == "random": - return draw(2 * n_draws if paired else n_draws, rng) - counter = Counter() - draw(1, counter) - width = counter.dimensions - table = sobol_table(n_draws, 2 * width if paired else width, rng) - first = draw(n_draws, SobolPoints(table[:, :width])) - if not paired: - return first - second = draw(n_draws, SobolPoints(table[:, width:])) - return ( - {node: first[0][node] + second[0][node] for node in first[0]}, - [ - None if a is None else a + b - for a, b in zip(first[1], second[1]) - ], - ) + def join(first, second): + return ( + {node: first[0][node] + second[0][node] for node in first[0]}, + [ + None if a is None else a + b + for a, b in zip(first[1], second[1]) + ], + ) + + return draw_with(draw, n_draws, seed, sampling, paired, join) def _uncertainty_sources( self, uncertainty @@ -2700,8 +2707,27 @@ def _uncertainty_sources( (i for i, g in enumerate(self.ccf_groups) if key is g), None ) if index is not None: + if index in ccf_specs: + raise ValueError( + f"Common-cause group {list(key.members)!r} is given " + "an uncertainty twice: under the CCFGroup and under " + "its members." + ) ccf_specs[index] = (key, spec) continue + # A group's own parameters under its members' tuple, named as + # its levers are (ccf_beta, ...), go to its model. + index, own, spec = split_group_levers(key, spec, self.ccf_groups) + if index is not None and own: + if index in ccf_specs: + raise ValueError( + f"Common-cause group {list(key)!r} is given an " + "uncertainty twice: under the CCFGroup and as " + f"{list(own)!r} under its members." + ) + ccf_specs[index] = (self.ccf_groups[index], own) + if not spec: + continue if key in components or not isinstance(key, tuple): members = (key,) else: @@ -2777,41 +2803,15 @@ def _fitted_uncertainty(self) -> Dict[Hashable, str]: every component whose model is a surpyval fit with a parameter covariance, the nodes that share one model object, or one common-cause group, given together, so that they share their - draws.""" - from repyability.rbd.uncertainty import is_fit - - fitted = [ - node - for node in self.nodes - if node not in self.repeated and is_fit(self.reliabilities[node]) - ] - # Join the nodes that share a model object or a common-cause group. - parent = {node: node for node in fitted} - - def root(node): - while parent[node] != node: - parent[node] = parent[parent[node]] - node = parent[node] - return node - - def join(a, b): - parent[root(a)] = root(b) - - holder: Dict[int, Hashable] = {} - for node in fitted: - first = holder.setdefault(id(self.reliabilities[node]), node) - join(node, first) - for group in self.ccf_groups: - members = [m for m in group.members if m in parent] - for member in members[1:]: - join(member, members[0]) - together: Dict[Hashable, list] = {} - for node in fitted: - together.setdefault(root(node), []).append(node) - return { - nodes[0] if len(nodes) == 1 else tuple(nodes): "fit" - for nodes in together.values() - } + draws (see ``uncertainty.fitted_together``).""" + return fitted_together( + { + node: self.reliabilities[node] + for node in self.nodes + if node not in self.repeated + }, + (group.members for group in self.ccf_groups), + ) def _drawn_groups(self, drawn_groups: list, i: int) -> Optional[list]: """The common-cause groups of draw ``i`` (see ``_uncertain_draws``), @@ -2958,9 +2958,10 @@ def allocate_redundancy( shared cause fails it too, and each design is scored exactly over the groups' outcomes. A ``BetaFactor`` group's ``beta`` holds at any size, so its members can be copied: active copies of their own - model (with any number of them ``required``). An ``MGL`` model's - letters are for its group's size, so copies of its member are - refused, as are options and cold spares for a member. + model (with any number of them ``required``). An ``MGL`` or + ``AlphaFactor`` model's parameters are for its group's size, so + copies of its member are refused, as are options and cold + spares for a member. ``method="exact"`` returns a proven optimum. When every costed node is in series with the rest of the system (it lies on every path, as @@ -3079,8 +3080,8 @@ def allocate_redundancy( search that is too large. NotImplementedError If a costed node is in a common-cause group whose model is an - ``MGL`` (its letters are for its group's size), or has options - or cold spares (see above). + ``MGL`` or ``AlphaFactor`` (its parameters are for its group's + size), or has options or cold spares (see above). References ---------- @@ -3362,10 +3363,7 @@ def _redundancy_problem( redundancy_front) and set up the problem the searches solve: the costed nodes, their kinds and designs, the limits and the exact system evaluation.""" - if method not in ("exact", "greedy"): - raise ValueError( - f"method must be 'exact' or 'greedy', got {method!r}." - ) + one_of("method", method, ("exact", "greedy")) if not isinstance(mixing, (bool, np.bool_)): raise ValueError(f"mixing must be True or False, got {mixing!r}.") mixing = bool(mixing) @@ -4686,11 +4684,9 @@ def Hf( >>> round(rbd.Hf(50), 6) 0.5 """ - sf, ff = self._sf_and_ff(x, working_nodes, broken_nodes, method) - with np.errstate(divide="ignore"): - return np.where( - ff < 0.5, -np.log1p(-np.minimum(ff, 0.5)), -np.log(sf) - ) + return cumulative_hazard( + *self._sf_and_ff(x, working_nodes, broken_nodes, method) + ) @check_x def df( @@ -4705,9 +4701,10 @@ def df( Computed by a central finite difference of the system reliability from [`sf`][repyability.NonRepairableRBD.sf] (so it honours - common-cause groups), with step ``h = dx * max(|x|, 1)``: relative - to ``x`` for ``|x| >= 1``, absolute below. While the system is more - likely to work than not, the difference is taken of its + common-cause groups), with step ``h = dx * |x|``, relative to + ``x`` (at 0, ``dx`` of the shortest of the nodes' typical failure + times), so that it does not depend on the unit of time. While the + system is more likely to work than not, the difference is taken of its unreliability from [`ff`][repyability.NonRepairableRBD.ff] instead, which keeps its precision where the reliability's change would be lost to rounding. The lower point is clipped at 0, so the @@ -4757,19 +4754,11 @@ def df( 0.006065 """ x = np.atleast_1d(np.asarray(x, dtype=float)) - h = dx * np.maximum(np.abs(x), 1.0) - x_hi = x + h - x_lo = np.maximum(x - h, 0.0) - sf_lo, ff_lo = self._sf_and_ff( - x_lo, working_nodes, broken_nodes, method + return density_by_differences( + lambda t: self._sf_and_ff(t, working_nodes, broken_nodes, method), + x, + difference_steps(x, dx, self._time_scale), ) - sf_hi, ff_hi = self._sf_and_ff( - x_hi, working_nodes, broken_nodes, method - ) - # The change in whichever of the two is the smaller keeps its - # precision; that in the other would be lost to rounding. - change = np.where(ff_hi <= 0.5, ff_hi - ff_lo, sf_lo - sf_hi) - return np.clip(change / (x_hi - x_lo), 0.0, None) @check_x def hf( @@ -4962,7 +4951,6 @@ def is_time_varying(self) -> bool: def _node_is_analytic(self, model) -> bool: """Whether a node's reliability is computed without simulation: exactly or numerically (see ``repyability.rbd.routes``).""" - from repyability.rbd.routes import EXACT, NUMERICAL, model_route return model_route(model)[0] in (EXACT, NUMERICAL) @@ -5172,17 +5160,19 @@ def built(route, reason): ), ), ) - pairs = r.refusal(self._require_no_ccf_pairs) give( ("joint_importance",), - ( - r.refused(pairs) - if pairs - else built( - r.EXACT, - "Each node's exact Birnbaum importance with the other " - "node held working, less with it held failed.", - ) + built( + r.EXACT, + ( + "The exact system reliability given each of the pair " + "working and failed, a common-cause group's members " + "conditioned on their states through the groups' " + "outcomes." + if grouped + else "Each node's exact Birnbaum importance with the " + "other node held working, less with it held failed." + ), ), ) give( @@ -5223,14 +5213,20 @@ def built(route, reason): ), ), ) - give( - ("barlow_proschan_importance",), - built( + densities = r.refusal( + lambda: self._require_varying_lifetimes( + "lifetimes, so no failure density over time to share out" + ) + ) + out["barlow_proschan_importance"] = ( + r.refused(densities) + if densities + else built( r.NUMERICAL, "Each node's part of the system's failure density (see " "reliability_rate), integrated over time by adaptive " "Gauss-Legendre quadrature (to about 1e-9).", - ), + ) ) out["parameter_sensitivity"] = built( r.NUMERICAL, @@ -5610,8 +5606,7 @@ def random( ------ ValueError If ``size`` is not a positive integer (even, with - ``antithetic``) when ``antithetic`` or ``n_jobs`` is given, or - ``n_jobs`` is not a positive integer or -1. + ``antithetic``), or ``n_jobs`` is not a positive integer or -1. NotImplementedError If ``antithetic`` is asked for and a node's draws cannot be replayed from uniforms (a node model sampled its own way). @@ -5631,10 +5626,10 @@ def random( >>> bool((lifetimes == rbd.random(5, seed=1)).all()) True """ + montecarlo.check_count(size, antithetic, "size") if not antithetic and n_jobs is None: with numpy_seed(seed): return self._draw(size) - montecarlo.check_count(size, antithetic, "size") jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) return self._simulate_lifetimes(size, seed, antithetic, jobs, None) @@ -6154,13 +6149,16 @@ def _exact_mean(self) -> float: [model_kinks(m) for m in models], ) - def _require_varying_lifetimes(self) -> None: + def _require_varying_lifetimes( + self, what: str = "lifetimes to average" + ) -> None: """Raise if every node is a fixed probability: the system then fails - at the start or never, and has no lifetimes to average.""" + at the start or never, so it has no lifetimes (no ``what``: none to + average, no failure density over time).""" if self.is_fixed: - raise ValueError( + raise Refused( "System reliability does not vary with time (all nodes are " - "fixed-probability): the system has no lifetimes to average." + f"fixed-probability): the system has no {what}." ) def _require_lifetimes(self) -> None: @@ -6544,8 +6542,9 @@ def mean_time_to_failure_interval( Raises ------ ValueError - If ``confidence`` is not in (0, 1), or another option is invalid - (see ``mean``). + If ``confidence`` is not in (0, 1), ``mc_samples`` is fewer + than 2 (an interval needs two lifetimes; two pairs, with + ``antithetic``), or another option is invalid (see ``mean``). NotImplementedError With ``antithetic``, if a node's draws cannot be replayed from uniforms. @@ -6580,6 +6579,7 @@ def mean_time_to_failure_interval( max_samples, antithetic, n_jobs, + least=2, ) estimate = float(samples.mean()) standard_error = montecarlo.standard_error(samples, antithetic) @@ -6591,6 +6591,7 @@ def mean_time_to_failure_interval( confidence=confidence, standard_error=standard_error, n_samples=len(samples), + method="simulated", ) def _mttf_samples( @@ -6602,9 +6603,12 @@ def _mttf_samples( max_samples, antithetic: bool, n_jobs, + least: int = 1, ) -> np.ndarray: - """The simulated lifetimes an MTTF estimate is the mean of.""" + """The simulated lifetimes an MTTF estimate is the mean of (at + least ``least`` of them).""" self._require_varying_lifetimes() + montecarlo.check_count(mc_samples, antithetic, "mc_samples", least) if ( tolerance is None and max_samples is None @@ -6612,7 +6616,6 @@ def _mttf_samples( and n_jobs is None ): return self.random(mc_samples, seed=seed) - montecarlo.check_count(mc_samples, antithetic, "mc_samples") montecarlo.check_confidence(confidence) limit = montecarlo.sample_limit( mc_samples, @@ -6655,8 +6658,8 @@ def compare( By default, where ``mean`` works out both systems' MTTFs (exactly or numerically), the difference is theirs, with no error - (``method="exact"`` in the result), and nothing is simulated - (#236), as ``mean`` is exact by default. ``method="simulate"`` + (``method="exact"`` in the result), and nothing is simulated, + as ``mean`` is exact by default. ``method="simulate"`` simulates it whatever. Simulated, both systems' lifetimes are drawn ``mc_samples`` times, @@ -6890,43 +6893,11 @@ def _invert_reliability( ``sf_func`` maps a scalar time to a scalar system reliability and is monotonically non-increasing, so the solution is unique. Shared by :meth:`time_to_reliability` and :meth:`remaining_life`; the target is - bracketed automatically (by doubling) unless ``upper_bound`` is given. + bracketed automatically (by doubling) unless ``upper_bound`` is given + (see ``_measures.invert_reliability``). """ - if np.ndim(target) > 0: - targets = np.asarray(target, dtype=float) - return np.array( - [ - self._invert_reliability(sf_func, float(t), upper_bound) - for t in targets.ravel() - ] - ).reshape(targets.shape) - if not 0.0 < target < 1.0: - raise ValueError("target reliability must be in (0, 1).") self._require_time_varying() - - r0 = float(sf_func(0.0)) - if target > r0: - raise ValueError( - f"target reliability {target} exceeds the system reliability " - f"at t=0 ({r0:.6g}); it is never reached." - ) - - def f(t): - return float(sf_func(t)) - target - - hi = upper_bound - if hi is None: - hi = 1.0 - for _ in range(1000): - if f(hi) < 0.0: - break - hi *= 2.0 - else: - raise ValueError( - "Could not bracket the target reliability; pass an " - "explicit upper_bound." - ) - return float(brentq(f, 0.0, hi)) + return invert_reliability(sf_func, target, upper_bound) def bx_life(self, x, **kwargs): """Bx life: the time by which ``x`` percent of systems have failed. @@ -6938,7 +6909,8 @@ def bx_life(self, x, **kwargs): Parameters ---------- x : float or array_like - The percentage failed, in (0, 100), or several. + The percentage failed (10 for the B10 life, not a fraction), in + (0, 100), or several. **kwargs Keyword arguments of ``time_to_reliability`` (``upper_bound``) and ``sf`` (``working_nodes``, ``broken_nodes``, ``method``). @@ -6970,7 +6942,7 @@ def bx_life(self, x, **kwargs): >>> round(rbd.bx_life(10), 2) 32.46 """ - x = np.asarray(x, dtype=float) if np.ndim(x) else x + x = real_array(x, "x") if not np.all((0.0 < x) & (x < 100.0)): raise ValueError("x must be a percentage in (0, 100).") return self.time_to_reliability(1.0 - x / 100.0, **kwargs) @@ -7101,9 +7073,9 @@ def sf_given_state( ``{node: NodeState}``, the current state of some or all of the component nodes. By default empty, which reproduces ``sf(x)``. method : str, optional - ``"p"`` or ``"paths"`` (the default) uses the minimal path sets - and ``"c"`` or ``"cuts"`` the minimal cut sets; both are - exact. + ``"p"`` or ``"paths"`` (the default) works out the probability + that the system works, ``"c"`` or ``"cuts"`` that it fails (and + returns its complement); both are exact. Returns ------- @@ -7246,7 +7218,7 @@ def mean_residual_life( self, state: Optional[Dict[Hashable, NodeState]] = None ) -> float: """The mean remaining life: the expected time from now until the - system fails, given its components' current states (#179). + system fails, given its components' current states. ``remaining_life`` gives the time until the reliability given the state falls to a target, a percentile of the remaining life; this @@ -7510,7 +7482,7 @@ def birnbaum_importance( """Birnbaum importance of each node at time/s ``x``. In the guide's Greeks it is *delta*: how far the system moves with each - node (see [Sensitivities: the Greeks](guide/greeks.md)). + node (see [Sensitivities: the Greeks](../guide/greeks.md)). ``B_i = R_sys(i working) - R_sys(i failed)``: the rate at which the system reliability changes with node ``i``'s reliability, which is @@ -8066,7 +8038,7 @@ def parameter_sensitivity( """Sensitivity of system reliability to each node's parameters. In the guide's Greeks it is the levers' *deltas* (see [Sensitivities: - the Greeks](guide/greeks.md)). + the Greeks](../guide/greeks.md)). For node ``i`` with parameter ``theta``, the sensitivity at time/s ``x`` is @@ -8092,10 +8064,11 @@ def parameter_sensitivity( are included, fixed-probability nodes among them (their parameter is the failure probability); ``levers()`` lists them, with their values and ranges, and ``with_levers`` builds the RBD with them - moved. Composite nodes (a nested RBD, a - standby, load-sharing or repeated node, a regression node) and the - input and output nodes have no parameters to perturb and are - omitted. A node forced via + moved. A regression node's levers are its covariates + (``"covariate."``), at fixed covariates. Composite nodes + (a nested RBD, a standby, load-sharing or repeated node, a + regression node along a schedule) and the input and output nodes + have no parameters to perturb and are omitted. A node forced via ``working_nodes``/``broken_nodes`` is pinned independently of its parameters, so its sensitivities are reported as zero. @@ -8106,7 +8079,9 @@ def parameter_sensitivity( parameter moves for every member at once. So are the parameters of the group's common-cause model, named ``"ccf_beta"`` (and for an [`MGL`][repyability.MGL] model ``"ccf_gamma"``, ``"ccf_delta"``, - ...), by a central difference in each; a parameter at the edge of + ..., and for an [`AlphaFactor`][repyability.AlphaFactor] model + ``"ccf_alpha_2"``, ..., ``alpha_1`` being the remainder), by a + central difference in each; a parameter at the edge of ``[0, 1]`` is differenced on its one side. Each difference is of the system's probability of failing where that is the smaller, so a small one keeps its precision. @@ -8210,29 +8185,28 @@ def _out(value) -> Union[float, np.ndarray]: name: _out(value) for name, value in values.items() } continue - spec = parametric_spec(model) + spec = lever_spec(model) if spec is None: # A composite node: no parameters to perturb. continue - cls, params, names, extras = spec node_out: Dict[str, Union[float, np.ndarray]] = {} if node_name in forced: # Pinned regardless of its parameters -> zero sensitivity. zero = np.zeros_like(x_arr) - for name in names: + for name in spec.names: node_out[name] = _out(zero) sensitivities[node_name] = node_out continue b_i = np.asarray(birnbaum[node_name], dtype=float) - for j, name in enumerate(names): - dsf = _dsf_dparam(cls, params, j, x_arr, rel_step, extras) + for j, name in enumerate(spec.names): + dsf = _dsf_dparam(spec, j, x_arr, rel_step) node_out[name] = _out(b_i * dsf) sensitivities[node_name] = node_out return sensitivities def levers(self) -> List[Lever]: """The parameters that ``parameter_sensitivity`` moves, in the order - it reports them (#244): each node's model's, by surpyval's names, + it reports them: each node's model's, by surpyval's names, and each common-cause group's under the tuple of its members (the parameters of the one model its members carry, then its own, ``"ccf_beta"``, ...). @@ -8242,6 +8216,10 @@ def levers(self) -> List[Lever]: values. Nothing is worked out, so a report can name the parameters and show their values without the sensitivities; [`with_levers`][repyability.NonRepairableRBD.with_levers] sets them. + The levers are the nodes' models' own parameters: those of the + units inside a composite model (a ``StandbyModel``, a + ``LoadSharingModel``, a ``DegradingNode``) or of a nested diagram + are not reached (see the guide's Greeks). Returns ------- @@ -8264,7 +8242,7 @@ def levers(self) -> List[Lever]: return _sensitivity.public_levers(self) def with_levers(self, values) -> "NonRepairableRBD": - """A copy of this RBD with parameters at new values (#244), each + """A copy of this RBD with parameters at new values, each moved as ``parameter_sensitivity`` moves it: a node's model rebuilt with the parameter changed, a common-cause group's members' one model for every member, a group's own (``"ccf_beta"``, ...) in its @@ -8326,10 +8304,10 @@ def differential_importance( ) -> Dict[Any, Union[float, np.ndarray]]: """Each node's (or parameter's) share of the change in the system reliability when they all change together: the differential - importance measure (DIM, Borgonovo & Apostolakis, 2001; #193). + importance measure (DIM, Borgonovo & Apostolakis, 2001). In the guide's Greeks it is *DIM*, the shares of a change (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``DIM_i = dR/dtheta_i dtheta_i / sum_j dR/dtheta_j dtheta_j``, so the shares add up to 1, and a group's share is the sum of its @@ -8429,7 +8407,12 @@ def differential_importance( >>> round(shares["b"], 4) 0.6923 """ - from ._differential import check, flattened, shares + from ._differential import ( + check, + flattened, + proportional_scales, + shares, + ) check(over, change, kind) held = set(working_nodes or ()) | set(broken_nodes or ()) @@ -8455,10 +8438,9 @@ def differential_importance( if change == "uniform": contributions = derivatives else: - scale: Dict[Any, float] = { - (lever.key, lever.name): lever.value - for lever in self.levers() - } + scale = proportional_scales( + _sensitivity._nonrepairable_levers(self), derivatives + ) contributions = { key: np.asarray(d, dtype=float) * scale[key] for key, d in derivatives.items() @@ -8474,10 +8456,10 @@ def joint_importance( ) -> Dict[Tuple[Hashable, Hashable], Union[float, np.ndarray]]: """The joint (second-order) importance of each pair of nodes: whether improving the two together is worth more than improving - each (Hong & Lie, 1993; Armstrong, 1995; #194). + each (Hong & Lie, 1993; Armstrong, 1995). In the guide's Greeks it is *gamma*: complements or substitutes (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``JRI(i, j) = d2R / dR_i dR_j = R(1_i, 1_j) - R(1_i, 0_j) - R(0_i, 1_j) + R(0_i, 0_j)``, with ``R(1_i, 0_j)`` the system @@ -8493,6 +8475,15 @@ def joint_importance( less that with it held failed (twice as many evaluations as there are nodes, not as pairs). + With common-cause groups, the system's reliability given each of + the pair working and failed is conditioned on their states, as + ``birnbaum_importance`` conditions on one: a member's state says + something of its group's shocks, so the system given it is a sum + over the shock outcomes, and two members of one group are + conditioned on their joint state (see [Common-cause + failures](../guide/common-cause.md)). A node outside the groups is + held. ``beta = 0`` gives the values without the group. + Parameters ---------- x : array_like, optional @@ -8514,7 +8505,7 @@ def joint_importance( ValueError As ``birnbaum_importance`` does. NotImplementedError - With common-cause groups, whose members cannot be held. + If a common-cause group's member is held working or broken. Examples -------- @@ -8531,7 +8522,26 @@ def joint_importance( ... ) >>> {pair: round(v, 4) for pair, v in rbd.joint_importance().items()} {('p1', 'p2'): -0.9, ('p1', 'v'): 0.2, ('p2', 'v'): 0.2} + + A shared cause striking both pumps (a tenth of their failures) + makes one's failure a sign of the other's: with ``p1`` failed, + ``p2`` is likelier to have failed too, so the valve is worth less + then, and more of a complement of each pump. The pumps' pair is + as before: given their states, the valve alone remains. + + >>> from repyability import BetaFactor, CCFGroup + >>> grouped = NonRepairableRBD( + ... [("s", "p1"), ("s", "p2"), ("p1", "v"), ("p2", "v"), + ... ("v", "t")], + ... {"p1": F(0.2), "p2": F(0.2), "v": F(0.1)}, + ... ccf_groups=[CCFGroup(["p1", "p2"], BetaFactor(0.1))], + ... ) + >>> joint = grouped.joint_importance() + >>> round(joint[("p1", "p2")], 4), round(joint[("p1", "v")], 4) + (-0.9, 0.2635) """ + if self.ccf_groups: + return Pairs(self._ccf_pairs(x, working_nodes, broken_nodes)) return self._joint_pairs( lambda working, broken: self.birnbaum_importance( x, working, broken @@ -8540,80 +8550,41 @@ def joint_importance( broken_nodes, ) + def _time_scale(self) -> float: + """The diagram's shortest typical failure time over its nodes (see + ``model_scale``), to size a step at time 0 by; NaN if none has + one.""" + scales = [model_scale(m) for m in self.reliabilities.values()] + scales = [x for x in scales if np.isfinite(x) and x > 0.0] + return min(scales) if scales else float("nan") + def _node_density(self, node, x: np.ndarray) -> np.ndarray: """A node's failure density at the times ``x``: its model's own (``df``) where it has one, else by central differences of its probability of failing, as ``df`` takes the system's (one-sided near 0).""" - model = self.reliabilities[node] - try: - with np.errstate(all="ignore"): - return np.asarray(model.df(x), dtype=float).reshape(x.shape) - except (AttributeError, TypeError, ValueError, NotImplementedError): - pass - step = 1e-6 * np.maximum(np.abs(x), 1.0) - hi, lo = x + step, np.maximum(x - step, 0.0) - if hasattr(model, "ff"): - change = np.asarray(model.ff(hi), dtype=float) - np.asarray( - model.ff(lo), dtype=float - ) - else: - change = np.asarray(model.sf(lo), dtype=float) - np.asarray( - model.sf(hi), dtype=float - ) - return np.clip(change.reshape(x.shape) / (hi - lo), 0.0, None) + return density(self.reliabilities[node], x) def _rate_parts( self, x: np.ndarray, working_nodes, broken_nodes ) -> Dict[Any, np.ndarray]: """Each node's part in the system reliability's rate of change at the times ``x`` (see ``reliability_rate``), and each common-cause - group's, under the tuple of its members.""" - importance = self.birnbaum_importance(x, working_nodes, broken_nodes) - held = set(working_nodes or ()) | set(broken_nodes or ()) - members = {m for group in self.ccf_groups for m in group.members} - parts: Dict[Any, np.ndarray] = {} - for node, value in importance.items(): - if node in members: - continue - value = np.asarray(value, dtype=float).reshape(x.shape) - if node in held: - parts[node] = np.zeros(x.shape) - continue - with np.errstate(invalid="ignore"): - part = -value * self._node_density(node, x) - # A node that cannot matter, with an infinite density (a - # Weibull life of shape below 1, at 0), takes no part. - parts[node] = np.where(value == 0.0, 0.0, part) - if self.ccf_groups: - # A group's part: the system's change with its members' lives - # moved on and the rest held, by central differences. - step = 1e-6 * np.maximum(np.abs(x), 1.0) - hi, lo = x + step, np.maximum(x - step, 0.0) - p, q = self._importance_inputs(x, working_nodes, broken_nodes) - ends = [ - self._importance_inputs(t, working_nodes, broken_nodes) - for t in (hi, lo) - ] - for group in self.ccf_groups: - downs = [] - for moved_p, moved_q in ends: - given, failing = dict(p), dict(q) - for member in group.members: - given[member] = moved_p[member] - failing[member] = moved_q[member] - downs.append( - self._ccf_system_pair( - given, - failing, - working_nodes or (), - broken_nodes or (), - )[1].reshape(x.shape) - ) - parts[tuple(group.members)] = -(downs[0] - downs[1]) / ( - hi - lo - ) - return parts + group's, under the tuple of its members (see ``rate_parts``).""" + return rate_parts( + self.birnbaum_importance(x, working_nodes, broken_nodes), + self._node_density, + x, + self.ccf_groups, + inputs=lambda t: self._importance_inputs( + t, working_nodes, broken_nodes + ), + fails=lambda p, q: self._ccf_system_pair( + p, q, working_nodes or (), broken_nodes or () + )[1], + scale=self._time_scale, + held=set(working_nodes or ()) | set(broken_nodes or ()), + ) def reliability_rate( self, @@ -8622,10 +8593,10 @@ def reliability_rate( broken_nodes: Optional[Collection[Hashable]] = None, ) -> RateBreakdown: """How fast the system reliability is falling at each time ``x``, - and which nodes are bringing it down (#195). + and which nodes are bringing it down. In the guide's Greeks it is *theta*: what is moving the system now (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). The nodes failing independently, the system reliability is multilinear in theirs, so @@ -8713,11 +8684,11 @@ def barlow_proschan_importance( broken_nodes: Optional[Collection[Hashable]] = None, ) -> Dict[Any, Union[float, np.ndarray]]: """Each node's Barlow-Proschan importance: the probability that the - system's failure is caused by the node's (Barlow & Proschan, 1975; - #195), given that the system fails by ``x``, or over its whole life. + system's failure is caused by the node's (Barlow & Proschan, + 1975), given that the system fails by ``x``, or over its whole life. In the guide's Greeks it is *theta*, integrated: who caused the - failures (see [Sensitivities: the Greeks](guide/greeks.md)). + failures (see [Sensitivities: the Greeks](../guide/greeks.md)). A node's failure fails the system when the node is critical then, so the probability that the system has failed by ``x``, through @@ -8752,7 +8723,9 @@ def barlow_proschan_importance( Raises ------ ValueError - For a negative time, or as ``birnbaum_importance`` does. + For a negative time, if every node is a fixed probability (the + system then has no lifetimes), or as ``birnbaum_importance`` + does. Examples -------- @@ -8770,81 +8743,22 @@ def barlow_proschan_importance( >>> {node: round(share, 6) for node, share in shares.items()} {'a': 0.25, 'b': 0.75} """ - from . import _rates - - keys = list(self._rate_parts(np.ones(1), working_nodes, broken_nodes)) - - def density(t: np.ndarray) -> np.ndarray: - parts = self._rate_parts(t, working_nodes, broken_nodes) - return -np.column_stack([parts[key] for key in keys]) - - knots = np.concatenate( - [np.empty(0)] - + [model_knots(m) for m in self.reliabilities.values()] + self._require_varying_lifetimes( + "lifetimes, so no failure density over time to share out" + ) + return barlow_proschan( + lambda t: self._rate_parts(t, working_nodes, broken_nodes), + self.reliabilities.values(), + x, + lambda knots: self._life_end(knots, working_nodes, broken_nodes), ) - knots = np.unique(knots[np.isfinite(knots) & (knots > 0.0)]) - if x is None: - ends = np.array( - [self._life_end(knots, working_nodes, broken_nodes)] - ) - else: - ends = np.atleast_1d(np.asarray(x, dtype=float)).ravel() - if np.any(~np.isfinite(ends)) or np.any(ends < 0.0): - raise ValueError( - f"Times must be finite and non-negative, got {x!r}." - ) - order = np.argsort(ends) - totals = np.zeros((len(ends), len(keys))) - running = np.zeros(len(keys)) - start = 0.0 - for k in order: - end = float(ends[k]) - if end > start: - running = running + _rates.integral( - density, _filled_edges(knots, start, end) - ) - start = end - totals[k] = running - whole = totals.sum(axis=1) - with np.errstate(divide="ignore", invalid="ignore"): - shares = np.where( - whole[:, None] > 0.0, totals / whole[:, None], np.nan - ) - if x is None or np.ndim(x) == 0: - return {key: float(shares[0, j]) for j, key in enumerate(keys)} - shape = np.shape(x) - return {key: shares[:, j].reshape(shape) for j, key in enumerate(keys)} def _life_end( self, knots: np.ndarray, working_nodes, broken_nodes ) -> float: """A time by which what is left of the system's failures is negligible: its reliability within ``1e-14`` of what it never - loses.""" - - def sf(t): - return np.asarray( - self._sf_and_ff(np.atleast_1d(t), working_nodes, broken_nodes)[ - 0 - ], - dtype=float, - ).ravel() - - never = float(sf(np.array([1e300]))[0]) - start = float(knots[-1]) if knots.size else 1.0 - tail = start * 2.0 ** np.arange(int(np.log2(1e300 / start)) + 1) - values = sf(tail) - ended = np.flatnonzero(values - never <= 1e-14) - return float(tail[ended[0]] if ended.size else tail[-1]) - - -def _filled_edges(knots: np.ndarray, start: float, end: float) -> np.ndarray: - """The pieces of an integral from ``start`` to ``end``: at the - ``knots`` within, and at least eight to a decade from the first.""" - inside = knots[(knots > start) & (knots < end)] - low = max(start, float(inside[0]) if inside.size else end * 1e-6) - if low <= 0.0: - low = end * 1e-6 - decades = max(np.log10(end / low), 1.0) - filler = np.geomspace(low, end, int(np.ceil(decades * 8)) + 1) - return np.unique(np.concatenate([[start, end], inside, filler])) + loses (see ``_measures.life_end``).""" + return life_end( + lambda t: self._sf_and_ff(t, working_nodes, broken_nodes)[0], knots + ) diff --git a/repyability/rbd/numerical_convolution.py b/repyability/rbd/numerical_convolution.py index 289301f5..2d15be43 100644 --- a/repyability/rbd/numerical_convolution.py +++ b/repyability/rbd/numerical_convolution.py @@ -33,7 +33,7 @@ def _upper_time(model, eps: float = 1e-10) -> float: """ floor = never_fails(model) scale = failure_time_scale(model) - t = max(scale, 1.0) if np.isfinite(scale) else 1.0 + t = scale if np.isfinite(scale) and scale > 0.0 else 1.0 for _ in range(200): if _scalar(model.sf(t)) - floor <= eps: break diff --git a/repyability/rbd/phased_mission.py b/repyability/rbd/phased_mission.py index 5721929d..b930c941 100644 --- a/repyability/rbd/phased_mission.py +++ b/repyability/rbd/phased_mission.py @@ -23,8 +23,19 @@ ``METHOD = "paths"`` decomposes the phases' minimal path sets over the segments instead, as before (``_mission_plan``). The simulation draws each component's life once per mission instead. + +The importance measures (#316) hold a component across every phase: working +throughout (every segment survived) or failed from the start (none), and +replay the plan with its segments so held. The mission's minimal cut sets +are sets of events "component failed by the end of phase j", from the +phases' own cut sets with the usual phased-mission minimisation (a set is +dropped when another is implied by it: fewer components, or later phases); +the exact Fussell-Vesely importance takes the probability that some cut set +naming a component has occurred from a decision diagram of those events +over the segments (``_cut_set_plan``). """ +import math from typing import ( Any, Dict, @@ -39,11 +50,12 @@ import numpy as np from repyability.rbd import _montecarlo as montecarlo -from repyability.rbd import bdd, modular +from repyability.rbd import bdd, modular, routes from repyability.rbd._ordered_bdd import FALSE, TRUE, OrderedBDD from repyability.rbd._sampling import lifetime_sampler from repyability.rbd.results import ConfidenceInterval -from repyability.rbd.shannon import _shannon_value_and_gradient +from repyability.rbd.shannon import _plan_rows, replay, room +from repyability.utils.checks import one_of, simulation_options from repyability.utils.deprecation import refuse_removed_names from repyability.utils.wrappers import numpy_seed @@ -55,7 +67,9 @@ METHOD = "bdd" #: The most nodes of the missions's decision diagram before the exact values -#: refuse, pointing to the simulation. +#: refuse, pointing to the simulation (and of the cut sets' diagram before +#: the exact Fussell-Vesely importance refuses, pointing to the rare-event +#: sum). MAX_NODES = 1_000_000 #: The most distinct sub-problems the decomposition of the path sets @@ -167,12 +181,13 @@ def _phase_diagram( manager: OrderedBDD, rbd, j: int, rank: Dict[Hashable, int], count: int ) -> int: """Phase ``j``'s structure as a decision diagram over the segments: - built from its diagram's modules and core (see ``modular.py``), each - component working when its first ``j + 1`` segments are survived. - Component ``c``'s ``k``-th segment is variable ``rank[c] * count + k``, - so a component's segments are decided together, in phase order.""" + built from its diagram's modules and core (see ``modular.py``, the + junctions folded in as always working), each component working when + its first ``j + 1`` segments are survived. Component ``c``'s ``k``-th + segment is variable ``rank[c] * count + k``, so a component's segments + are decided together, in phase order.""" try: - decomposition = rbd._decomposition() + decomposition = rbd._set_structure() except ValueError: return FALSE # no set of working nodes makes the phase work if decomposition.always_works: @@ -200,6 +215,43 @@ def _phase_diagram( return slots[root] +def _outcomes( + plan: tuple, p: Dict[int, float], q: Dict[int, float] +) -> Tuple[float, float]: + """The probabilities that the mission of ``plan`` succeeds and that it + fails, for the segments' survival probabilities ``p`` and their + complements ``q`` (by element number), each to its own relative + precision: the plan replayed with the terminals of failing and + succeeding at 0 and 1, and again with them swapped.""" + pivots, actives, inactives, names = _plan_rows(plan) + works = [p[name] for name in names] + fails = [q[name] for name in names] + success, failure = ( + replay( + pivots, + actives, + inactives, + plan[1], + works, + fails, + room(len(pivots), fail, work), + ) + for fail, work in ((0.0, 1.0), (1.0, 0.0)) + ) + return float(success), float(failure) + + +def _weaker(one: Dict[Hashable, int], other: Dict[Hashable, int]) -> bool: + """Whether the cut set ``one`` (each component's phase, by number) is + implied by ``other`` and is not it: every event of ``one`` follows from + one of ``other``, the same component failed by that phase or an + earlier one. Then ``other`` is no minimal cut set.""" + return one != other and all( + component in other and other[component] <= phase + for component, phase in one.items() + ) + + @refuse_removed_names class PhasedMission: """A mission through phases, each with its own duration and diagram over @@ -224,7 +276,11 @@ class PhasedMission: ``MAX_NODES`` nodes. ``method="simulate"`` draws each component's life once per mission instead, and ``reliability_interval`` gives the simulated reliability - with a confidence interval, to a tolerance if asked. + with a confidence interval, to a tolerance if asked. The importance + measures (``birnbaum_importance`` and the others) hold a component + across every phase, working throughout or failed from the start, and + ``minimal_cut_sets`` gives the events, a component failed by the end + of a phase, that together fail the mission. Parameters ---------- @@ -314,8 +370,10 @@ def __init__(self, phases: Sequence[Tuple[Hashable, float, Any]]): f"Phase {name!r} has common-cause groups, which a phased " "mission does not model." ) - ends = {rbd.input_node, rbd.output_node} - used = tuple(n for n in rbd._components() if n not in ends) + # A junction (a PerfectReliability node, such as a vote point) + # is no component: the diagram folds it out of its structure. + left_out = {rbd.input_node, rbd.output_node} | rbd._junctions() + used = tuple(n for n in rbd._components() if n not in left_out) for node in used: model = rbd.reliabilities[node] known = self.components.setdefault(node, model) @@ -330,24 +388,37 @@ def __init__(self, phases: Sequence[Tuple[Hashable, float, Any]]): self.duration = end # The decomposition of the first so many phases, once worked out. self._plans: Dict[int, tuple] = {} + # The minimal cut sets, and the plan of their occurrence by + # component (see ``_cut_set_plan``), once worked out. + self._cut_sets: Optional[List[frozenset]] = None + self._cut_plan: Optional[Tuple[tuple, Dict[Hashable, int]]] = None # ------------------------------------------------------------------ # Exact # ------------------------------------------------------------------ + def _at_ends(self) -> Dict[Hashable, Tuple[np.ndarray, np.ndarray]]: + """Each component's probabilities of working and of having failed + at the phases' ends, from its model (each in its own right, so a + small one keeps its precision).""" + ends = np.array([phase.end for phase in self.phases]) + out: Dict[Hashable, Tuple[np.ndarray, np.ndarray]] = {} + for node, model in self.components.items(): + with np.errstate(all="ignore"): + sf = np.clip(np.ravel(model.sf(ends)).astype(float), 0, 1) + ff = np.clip(np.ravel(model.ff(ends)).astype(float), 0, 1) + out[node] = (sf, ff) + return out + def _segments(self) -> Tuple[Dict[int, float], Dict[int, float]]: """Each component's segments' survival probabilities, and their complements, by element number (``component * phases + k``): the ``k``-th survived with ``R(T_k) / R(T_{k-1})``, the first from the mission's start (so a unit dead on arrival fails in it).""" - ends = np.array([phase.end for phase in self.phases]) - count = len(ends) + count = len(self.phases) p: Dict[int, float] = {} q: Dict[int, float] = {} - for index, model in enumerate(self.components.values()): - with np.errstate(all="ignore"): - sf = np.clip(np.ravel(model.sf(ends)).astype(float), 0, 1) - ff = np.clip(np.ravel(model.ff(ends)).astype(float), 0, 1) + for index, (sf, ff) in enumerate(self._at_ends().values()): before_sf, before_ff = 1.0, 0.0 for k in range(count): element = index * count + k @@ -392,7 +463,7 @@ def _order(self) -> List[Hashable]: order: Dict[Hashable, None] = {} for phase in self.phases: try: - decomposition = phase.rbd._decomposition() + decomposition = phase.rbd._set_structure() except ValueError: continue # the phase cannot work: it uses no component terms = decomposition.terms @@ -453,10 +524,17 @@ def pivot(variable: int) -> int: plans.append((steps, slot)) return plans - def _exact(self, phases: int) -> Tuple[float, float]: - """The probability that the mission gets through its first - ``phases`` phases, and that it fails in them, each to its own - relative precision.""" + def _plan(self, phases: int) -> tuple: + """The plan of the probability that the mission gets through its + first ``phases`` phases (see ``_mission_plan``), worked out once. + + Raises + ------ + NotImplementedError + If the diagram needs more than ``MAX_NODES`` nodes (with + ``METHOD = "paths"``, the decomposition more than + ``MAX_STATES`` states). + """ plan = self._plans.get(phases) if plan is None: if METHOD not in ("bdd", "paths"): @@ -471,10 +549,153 @@ def _exact(self, phases: int) -> Tuple[float, float]: for j, made in enumerate(self._diagram_plans()): self._plans[j + 1] = made plan = self._plans[phases] + return plan + + def _exact(self, phases: int) -> Tuple[float, float]: + """The probability that the mission gets through its first + ``phases`` phases, and that it fails in them, each to its own + relative precision.""" + return _outcomes(self._plan(phases), *self._segments()) + + # ------------------------------------------------------------------ + # Importance and cut sets (#316) + # ------------------------------------------------------------------ + + def _held(self) -> Dict[Hashable, Tuple[float, float, float, float]]: + """For each component, the mission's reliability and unreliability + with it working throughout (every segment survived), then with it + failed from the start (none): the plan replayed with its segments + so held.""" + plan = self._plan(len(self.phases)) p, q = self._segments() - success, _ = _shannon_value_and_gradient(plan, p, q, (0.0, 1.0)) - failure, _ = _shannon_value_and_gradient(plan, p, q, (1.0, 0.0)) - return float(success), float(failure) + count = len(self.phases) + out: Dict[Hashable, Tuple[float, float, float, float]] = {} + for index, component in enumerate(self.components): + segments = range(index * count, (index + 1) * count) + working = _outcomes( + plan, + {**p, **dict.fromkeys(segments, 1.0)}, + {**q, **dict.fromkeys(segments, 0.0)}, + ) + failed = _outcomes( + plan, + {**p, **dict.fromkeys(segments, 0.0)}, + {**q, **dict.fromkeys(segments, 1.0)}, + ) + out[component] = (*working, *failed) + return out + + def _measures(self) -> Dict[Hashable, Tuple[float, float, float]]: + """Each component's Birnbaum importance, ``R(works) - R(failed)``, + improvement potential, ``R(works) - R``, and the fall in the + reliability with it failed from the start, ``R - R(failed)``, each + taken from the probabilities of succeeding or of failing, + whichever are the smaller (a difference of values near 1 would + cancel), as a diagram's are (see + ``modular.Decomposition.value_and_gradient``): ``R(works) - + R(failed)`` is ``F(failed) - F(works)``.""" + success, failure = self._exact(len(self.phases)) + out: Dict[Hashable, Tuple[float, float, float]] = {} + for component, ( + r_works, + f_works, + r_failed, + f_failed, + ) in self._held().items(): + if success <= failure: + out[component] = ( + r_works - r_failed, + r_works - success, + success - r_failed, + ) + else: + out[component] = ( + f_failed - f_works, + failure - f_works, + f_failed - failure, + ) + return out + + def _cut_set_plan(self) -> Tuple[tuple, Dict[Hashable, int]]: + """One plan (in ``shannon.py``'s format) of, for each component, + the event that some minimal cut set naming it (in any phase) has + occurred, with the root slot of each component's: a decision + diagram over the segments, an event "failed by the end of phase + ``j``" being the component's first ``j + 1`` segments not all + survived. Worked out once. + + Raises + ------ + NotImplementedError + If the diagram needs more than ``MAX_NODES`` nodes. + """ + if self._cut_plan is None: + count = len(self.phases) + index = {node: i for i, node in enumerate(self.components)} + phase = {ph.name: j for j, ph in enumerate(self.phases)} + manager = OrderedBDD( + MAX_NODES, + "The mission's cut sets are too many to combine exactly " + f"(more than {MAX_NODES:,} nodes of their decision " + "diagram). Sum them instead: method='rare_event'.", + ) + occurred: Dict[Hashable, List[int]] = { + node: [] for node in self.components + } + for cut in self.minimal_cut_sets(): + events = [] + for node, name in cut: + chain = TRUE + for k in range(phase[name], -1, -1): + chain = manager.node( + index[node] * count + k, FALSE, chain + ) + events.append(manager.ite(chain, FALSE, TRUE)) + diagram = manager.conjunction(events) + for node, _ in cut: + occurred[node].append(diagram) + roots = { + node: manager.disjunction(diagrams) + for node, diagrams in occurred.items() + } + steps, slots = manager.plan(list(roots.values()), lambda v: v) + # The plan's own root is no one component's: each's slot is + # read from the replayed values. + self._cut_plan = ((steps, _FAIL), dict(zip(roots, slots))) + return self._cut_plan + + def _fv_numerators(self, method: str) -> Dict[Hashable, float]: + """For each component, the probability that some minimal cut set + naming it has occurred (``method="exact"``), or the sum of the cut + sets' probabilities (``"rare_event"``), each cut set's the product + of its events', ``F_c(T_j)`` for "``c`` failed by the end of phase + ``j``".""" + one_of("method", method, ("exact", "rare_event")) + if method == "exact": + plan, slots = self._cut_set_plan() + pivots, actives, inactives, names = _plan_rows(plan) + p, q = self._segments() + values = room(len(pivots), 0.0, 1.0) + replay( + pivots, + actives, + inactives, + _FAIL, + [p[name] for name in names], + [q[name] for name in names], + values, + ) + return {node: float(values[slot]) for node, slot in slots.items()} + failed = self._at_ends() + phase = {ph.name: j for j, ph in enumerate(self.phases)} + out: Dict[Hashable, float] = dict.fromkeys(self.components, 0.0) + for cut in self.minimal_cut_sets(): + chance = float( + np.prod([failed[node][1][phase[name]] for node, name in cut]) + ) + for node, _ in cut: + out[node] += chance + return out # ------------------------------------------------------------------ # Simulation @@ -522,7 +743,7 @@ def _failure_phases(self, size: int, antithetic: bool) -> np.ndarray: for n in rbd._components() } lifetime = np.asarray( - rbd._decomposition().lifetime(nodes, size), dtype=float + rbd._set_structure().lifetime(nodes, size), dtype=float ) failed[~(lifetime > phase.end)] = j return failed @@ -535,10 +756,12 @@ def _simulated( tolerance=None, confidence: float = 0.95, max_samples=None, + least: int = 1, ) -> np.ndarray: - """The failure phases of ``mc_samples`` simulated missions, then - more while a run to ``tolerance`` needs them.""" - montecarlo.check_count(mc_samples, antithetic, "mc_samples") + """The failure phases of ``mc_samples`` simulated missions (at + least ``least``), then more while a run to ``tolerance`` needs + them.""" + montecarlo.check_count(mc_samples, antithetic, "mc_samples", least) montecarlo.check_confidence(confidence) limit = montecarlo.sample_limit( mc_samples, @@ -601,8 +824,9 @@ def reliability( Raises ------ ValueError - If ``method`` is unknown, or ``mc_samples`` is given with - ``method="exact"`` or is not a positive integer. + If ``method`` is unknown, or ``mc_samples`` or ``seed`` is + given with ``method="exact"``, or ``mc_samples`` is not a + positive integer. NotImplementedError If the mission's decision diagram needs more than ``MAX_NODES`` nodes (with ``METHOD = "paths"``, the @@ -626,7 +850,7 @@ def reliability( >>> round(mission.reliability(), 10) == round(float(pair.sf(50.0)), 10) True """ - return self._outcome(method, mc_samples, seed)[0] + return self._outcome(method, mc_samples, seed, "reliability")[0] def unreliability( self, @@ -653,7 +877,7 @@ def unreliability( float The mission's unreliability. """ - return self._outcome(method, mc_samples, seed)[1] + return self._outcome(method, mc_samples, seed, "unreliability")[1] #: The mission's reliability, by the name the diagrams give it (#179). sf = reliability @@ -697,7 +921,9 @@ def phase_failure_probabilities( >>> {name: round(chance, 5) for name, chance in failing.items()} {'out': 0.09516, 'back': 0.08611} """ - method = self._method(method, mc_samples) + method = self._method( + method, mc_samples, seed, "phase_failure_probabilities" + ) names = [phase.name for phase in self.phases] if method == "simulate": failed = self._simulated( @@ -760,8 +986,9 @@ def reliability_interval( Raises ------ ValueError - If ``mc_samples``, ``confidence``, ``tolerance`` or - ``max_samples`` is invalid. + If ``mc_samples`` is not a whole number of at least 2 (an + interval needs two missions; even, with ``antithetic``), or + ``confidence``, ``tolerance`` or ``max_samples`` is invalid. NotImplementedError If ``antithetic`` is asked for and a component's lives cannot be drawn from uniforms. @@ -778,7 +1005,13 @@ def reliability_interval( True """ failed = self._simulated( - mc_samples, seed, antithetic, tolerance, confidence, max_samples + mc_samples, + seed, + antithetic, + tolerance, + confidence, + max_samples, + least=2, ) succeeded = (failed == len(self.phases)).astype(float) estimate = float(succeeded.mean()) @@ -791,20 +1024,572 @@ def reliability_interval( confidence=confidence, standard_error=error, n_samples=len(succeeded), + method="simulated", + ) + + def birnbaum_importance(self) -> Dict[Hashable, float]: + """Birnbaum importance of each component over the mission. + + ``B_i = R(i works) - R(i failed)``: the mission's reliability with + component ``i`` working throughout (every phase's segment of its + life survived) less its reliability with ``i`` failed from the + start (failed in the first phase, so in every phase), the + probability that ``i`` is critical to the mission. Each is the + mission's decision diagram replayed with the component's segments + so held, and the difference is taken from the probabilities of + succeeding or of failing, whichever are the smaller, so a small + importance in a reliable mission keeps its precision. Exact, at + the mission's end; it assumes independent components. + + Returns + ------- + dict[Hashable, float] + ``{component: importance}`` over the mission's components. + + Raises + ------ + NotImplementedError + If the mission's decision diagram needs more than ``MAX_NODES`` + nodes. + + Examples + -------- + Two engines, both needed to take off and either to cruise: an + engine failed from the start fails the take-off, and one working + throughout carries the mission unless the other fails in the + take-off, so each is critical with the probability that the other + lasts the take-off, ``exp(-0.01 * 0.1)``. + + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> {k: round(v, 6) for k, v in mission.birnbaum_importance().items()} + {'e1': 0.999, 'e2': 0.999} + """ + return { + component: birnbaum + for component, (birnbaum, _, _) in self._measures().items() + } + + def improvement_potential(self) -> Dict[Hashable, float]: + """Improvement potential of each component over the mission. + + ``IP_i = R(i works) - R``: how much the mission's reliability would + rise if component ``i`` worked throughout (every phase's segment + of its life survived). Each is the mission's decision diagram + replayed with the component's segments so held (see + [`birnbaum_importance`][repyability.PhasedMission.birnbaum_importance]), + the difference taken from the probabilities of succeeding or of + failing, whichever are the smaller. Exact, at the mission's end; it + assumes independent components. + + Returns + ------- + dict[Hashable, float] + ``{component: improvement potential}`` over the mission's + components. + + Raises + ------ + NotImplementedError + If the mission's decision diagram needs more than ``MAX_NODES`` + nodes. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> potential = mission.improvement_potential() + >>> {k: round(v, 6) for k, v in potential.items()} + {'e1': 0.003372, 'e2': 0.003372} + """ + return { + component: potential + for component, (_, potential, _) in self._measures().items() + } + + def risk_achievement_worth(self) -> Dict[Hashable, float]: + """Risk achievement worth (RAW) of each component over the mission. + + ``RAW_i = F(i failed) / F``, per Modarres & Kaminskiy: the factor by + which the mission's unreliability would grow if component ``i`` + were failed from the start (failed in the first phase, so in every + phase). It is at least 1. Each unreliability is the mission's + decision diagram replayed with the component's segments so held + (see + [`birnbaum_importance`][repyability.PhasedMission.birnbaum_importance]), + in its own right (not as one less a reliability), so a small one + keeps its precision; where the mission cannot fail the division + gives ``inf`` or ``nan``. Exact, at the mission's end; it assumes + independent components. + + Returns + ------- + dict[Hashable, float] + ``{component: RAW}`` over the mission's components. + + Raises + ------ + NotImplementedError + If the mission's decision diagram needs more than ``MAX_NODES`` + nodes. + + Examples + -------- + Either engine failed from the start fails the take-off, so the + mission: its unreliability becomes 1. + + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> raw = mission.risk_achievement_worth() + >>> {k: round(v, 2) for k, v in raw.items()} + {'e1': 228.74, 'e2': 228.74} + >>> round(1 / mission.unreliability(), 2) + 228.74 + """ + _, failure = self._exact(len(self.phases)) + with np.errstate(divide="ignore", invalid="ignore"): + return { + component: float(np.float64(f_failed) / failure) + for component, (_, _, _, f_failed) in self._held().items() + } + + def risk_reduction_worth(self) -> Dict[Hashable, float]: + """Risk reduction worth (RRW) of each component over the mission. + + ``RRW_i = F / F(i works)``, per Modarres & Kaminskiy: the factor by + which the mission's unreliability would shrink if component ``i`` + worked throughout (every phase's segment of its life survived). It + is at least 1, and ``inf`` where the component working throughout + makes the mission certain. Each unreliability is the mission's + decision diagram replayed with the component's segments so held + (see + [`birnbaum_importance`][repyability.PhasedMission.birnbaum_importance]), + in its own right, so a small one keeps its precision. Exact, at the + mission's end; it assumes independent components. + + Returns + ------- + dict[Hashable, float] + ``{component: RRW}`` over the mission's components. + + Raises + ------ + NotImplementedError + If the mission's decision diagram needs more than ``MAX_NODES`` + nodes. + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> rrw = mission.risk_reduction_worth() + >>> {k: round(v, 4) for k, v in rrw.items()} + {'e1': 4.374, 'e2': 4.374} + """ + _, failure = self._exact(len(self.phases)) + with np.errstate(divide="ignore", invalid="ignore"): + return { + component: float(np.float64(failure) / f_works) + for component, (_, f_works, _, _) in self._held().items() + } + + def criticality_importance( + self, kind: str = "failure" + ) -> Dict[Hashable, float]: + """Criticality importance of each component over the mission. + + With ``R`` and ``F`` the mission's reliability and unreliability, + and ``R(i works)`` and ``R(i failed)`` its reliability with + component ``i`` working throughout and failed from the start (see + [`birnbaum_importance`][repyability.PhasedMission.birnbaum_importance]): + + - ``kind="failure"`` (the default) gives the failure-oriented form + (Rausand & Høyland), ``CI_i = (R(i works) - R) / F``: the + probability, given that the mission has failed, that it would + have succeeded with component ``i`` working throughout; the + share of the mission's failures that ``i`` accounts for. For a + diagram this is ``B_i * (1 - R_i) / (1 - R_sys)``, as + [`criticality_importance`][repyability.NonRepairableRBD.criticality_importance] + gives it. It is ``nan`` where the mission cannot fail. + - ``kind="success"`` gives the success-oriented form, + ``CI_i = (R - R(i failed)) / R``: the probability, given that the + mission succeeds, that it would have failed with component ``i`` + failed from the start (for a diagram, ``B_i * R_i / R_sys``). It + is ``nan`` where the mission cannot succeed. + + Each difference is taken from the probabilities of succeeding or + of failing, whichever are the smaller, so a small share keeps its + precision. Exact, at the mission's end; it assumes independent + components. + + Parameters + ---------- + kind : str, optional + ``"failure"`` (the default) or ``"success"``. + + Returns + ------- + dict[Hashable, float] + ``{component: criticality importance}`` over the mission's + components. + + Raises + ------ + ValueError + If ``kind`` is neither ``"failure"`` nor ``"success"``. + NotImplementedError + If the mission's decision diagram needs more than ``MAX_NODES`` + nodes. + + Examples + -------- + Two engines, both needed to take off and either to cruise: the + mission's failures without either engine's own failure are the + other's in the take-off, so each engine accounts for a share + ``1 - 1 / RRW_i`` of the failures. + + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> ci = mission.criticality_importance() + >>> {k: round(v, 4) for k, v in ci.items()} + {'e1': 0.7714, 'e2': 0.7714} + >>> round(1 - 1 / mission.risk_reduction_worth()["e1"], 4) + 0.7714 + """ + one_of("kind", kind, ("failure", "success")) + success, failure = self._exact(len(self.phases)) + mission, which = (failure, 1) if kind == "failure" else (success, 2) + return { + component: ( + float(measures[which] / mission) if mission > 0.0 else math.nan + ) + for component, measures in self._measures().items() + } + + def fussell_vesely(self, method: str = "exact") -> Dict[Hashable, float]: + """Fussell-Vesely importance of each component over the mission: + the share of the mission's failure probability that involves the + component, through the mission's minimal cut sets (see + [`minimal_cut_sets`][repyability.PhasedMission.minimal_cut_sets]), + sets of events "component ``c`` has failed by the end of phase + ``j``", each with probability ``F_c(T_j)``, that together fail the + mission. A cut set has occurred when every one of its events has, + and a component's are the cut sets naming it in any phase: + + FV_i = P(some minimal cut set naming i has occurred) / F + + with ``F`` the mission's unreliability. ``method="exact"`` (the + default) works out the probability of that union exactly, from a + decision diagram of the cut sets' events over the components' + segments (an event being the component's first ``j + 1`` segments + not all survived), so it is between 0 and 1. + ``method="rare_event"`` gives the rare-event approximation that + many PRA tools report, summing the cut sets' probabilities, each + the product of its events', instead of taking their union: + + FV_i = sum over minimal cut sets C naming i of + prod over (c, j) in C of F_c(T_j), divided by F + + The sum over-estimates the union, so it can exceed 1 when the + failure probabilities are not small. ``F`` and each event's + probability are worked out in their own right (the mission's as + [`unreliability`][repyability.PhasedMission.unreliability] is, a + component's from its model's ``ff``), and so is the union, so + small ones keep their precision. Exact; it assumes independent + components. + + Parameters + ---------- + method : str, optional + ``"exact"`` (the default): the probability of the union of the + cut sets' occurrences; or ``"rare_event"``: the sum of their + probabilities. + + Returns + ------- + dict[Hashable, float] + ``{component: Fussell-Vesely importance}`` over the mission's + components. + + Raises + ------ + ValueError + If ``method`` is not ``"exact"`` or ``"rare_event"``. + NotImplementedError + If the mission's decision diagram, or with ``method="exact"`` + the cut sets', needs more than ``MAX_NODES`` nodes. + + Examples + -------- + Two engines, both needed to take off and either to cruise: an + engine's share is the failures through its own loss in the + take-off or both engines' by the end of the cruise, not through the + other's loss in the take-off alone. The rare-event sum counts a + failure that is both at once twice. + + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> fv = mission.fussell_vesely() + >>> {k: round(v, 4) for k, v in fv.items()} + {'e1': 0.7827, 'e2': 0.7827} + >>> fv = mission.fussell_vesely(method="rare_event") + >>> {k: round(v, 4) for k, v in fv.items()} + {'e1': 0.7941, 'e2': 0.7941} + """ + _, failure = self._exact(len(self.phases)) + numerators = self._fv_numerators(method) + with np.errstate(divide="ignore", invalid="ignore"): + return { + component: float(np.float64(numerator) / failure) + for component, numerator in numerators.items() + } + + def minimal_cut_sets(self) -> List[frozenset]: + """The mission's minimal cut sets: sets of events ``(component, + phase name)``, "the component has failed by the end of that + phase", that together fail the mission, smallest first (then by + name). + + A failure by one phase is one by every later phase, so an event is + implied by the same component's at an earlier phase. A cut set is + minimal when no event can be dropped and none replaced by a weaker + one, the component's failure by a later phase. They are the + phases' own minimal cut sets (each a phase's diagram's, its + components' failures dated to that phase), less those another + implies: a set is dropped when every event of another follows from + one of its own (the other names no component it lacks, each at + that phase or a later one), the usual phased-mission minimisation. + A phase whose diagram can never work has the empty cut set, which + implies every other. They are worked out on first use and kept. + + Returns + ------- + list of frozenset + The minimal cut sets, each a frozenset of ``(component, phase + name)`` events. + + Raises + ------ + NotImplementedError + If a phase's diagram is too meshed for its cut sets. + + Examples + -------- + Two engines, both needed to take off and either to cruise: an + engine failed by the end of the take-off fails the mission, as do + both failed by the end of the cruise. + + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> engine = surv.Exponential.from_params([0.01]) + >>> both = NonRepairableRBD( + ... [("s", "e1"), ("e1", "e2"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> either = NonRepairableRBD( + ... [("s", "e1"), ("s", "e2"), ("e1", "t"), ("e2", "t")], + ... {"e1": engine, "e2": engine}, + ... ) + >>> mission = PhasedMission( + ... [("take-off", 0.1, both), ("cruise", 5.0, either)] + ... ) + >>> for cut in mission.minimal_cut_sets(): + ... print(sorted(cut)) + [('e1', 'take-off')] + [('e2', 'take-off')] + [('e1', 'cruise'), ('e2', 'cruise')] + """ + if self._cut_sets is None: + dated: List[Dict[Hashable, int]] = [] + for j, phase in enumerate(self.phases): + try: + cuts = phase.rbd.get_min_cut_sets() + except ValueError: + cuts = {frozenset()} # no set of working nodes works + dated.extend(dict.fromkeys(cut, j) for cut in cuts) + kept = [ + cut + for cut in dated + if not any(_weaker(other, cut) for other in dated) + ] + self._cut_sets = sorted( + ( + frozenset( + (node, self.phases[j].name) for node, j in cut.items() + ) + for cut in kept + ), + key=lambda cut: (len(cut), sorted(map(str, cut))), + ) + return list(self._cut_sets) + + def analysis_routes(self) -> Dict[str, routes.AnalysisRoute]: + """How each analysis of the mission is computed, found without + running it: exactly, by simulation, or not at all. + + The exact analyses replay the phases' decision diagram over the + components' segments, and refuse when it would need more than + ``MAX_NODES`` nodes (the diagram is built, and kept for the + analyses; the Fussell-Vesely importance's cut sets' diagram too). + They are as exact as the components' reliabilities: a component + whose reliability is numerical makes them numerical, and one with + none (a ``StandbyModel`` only simulations take) makes them refuse, + as the diagrams' routes have it. The refusal is the message the + method would raise. + + Returns + ------- + dict[str, AnalysisRoute] + For each public analysis, by method name: its ``route`` + (``"exact"``, ``"numerical"``, ``"simulated"`` or + ``"refused"``), the ``reason`` and the components that decide + it (see [`AnalysisRoute`][repyability.AnalysisRoute]). + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD, PhasedMission + >>> unit = surv.Exponential.from_params([0.01]) + >>> one = NonRepairableRBD([("s", "a"), ("a", "t")], {"a": unit}) + >>> mission = PhasedMission([("out", 10.0, one), ("back", 10.0, one)]) + >>> routes = mission.analysis_routes() + >>> routes["reliability"].route, routes["reliability_interval"].route + ('exact', 'simulated') + """ + r = routes + # A component whose model has no exact or numerical reliability + # refuses what is built on it, as the diagrams' routes have it. + nodes = {n: r.model_route(m) for n, m in self.components.items()} + refused = r.refusal(lambda: self._plan(len(self.phases))) + diagram = ( + r.refused(refused) + if refused + else r.with_nodes( + r.EXACT, + "The phases' decision diagram over the components' " + "segments (Esary and Ziehms), replayed.", + nodes, + "reliabilities", + ) + ) + out = { + name: diagram + for name in ( + "reliability", + "unreliability", + "sf", + "ff", + "phase_failure_probabilities", + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + ) + } + cuts = refused or r.refusal(self._cut_set_plan) + out["fussell_vesely"] = ( + r.refused(cuts) + if cuts + else r.with_nodes( + r.EXACT, + "The mission's minimal cut sets, their occurrence a " + "decision diagram over the components' segments.", + nodes, + "reliabilities", + ) + ) + out["reliability_interval"] = r.AnalysisRoute( + r.SIMULATED, "Each component's life drawn once per mission." ) + return r.AnalysisRoutes(out) - def _method(self, method: str, mc_samples) -> str: - if method not in ("exact", "simulate"): - raise ValueError( - f"method must be 'exact' or 'simulate', got {method!r}." + def _method(self, method: str, mc_samples, seed, name: str) -> str: + one_of("method", method, ("exact", "simulate")) + if method == "exact": + simulation_options( + f"{name}()", {"mc_samples": mc_samples, "seed": seed} ) - if method == "exact" and mc_samples is not None: - raise ValueError("mc_samples applies only to method='simulate'.") return method - def _outcome(self, method: str, mc_samples, seed) -> Tuple[float, float]: - """The mission's reliability and unreliability.""" - method = self._method(method, mc_samples) + def _outcome( + self, method: str, mc_samples, seed, name: str + ) -> Tuple[float, float]: + """The mission's reliability and unreliability, as ``name`` asks + for them.""" + method = self._method(method, mc_samples, seed, name) if method == "simulate": failed = self._simulated( 10_000 if mc_samples is None else mc_samples, seed diff --git a/repyability/rbd/rbd.py b/repyability/rbd/rbd.py index 0897f0aa..3c3d4858 100644 --- a/repyability/rbd/rbd.py +++ b/repyability/rbd/rbd.py @@ -10,6 +10,7 @@ path sets, and the probability scaling used by reliability allocation. """ +import copy import difflib import functools import inspect @@ -51,13 +52,14 @@ fold, ) from repyability.rbd.rbd_graph import RBDGraph -from repyability.rbd.results import CapacityDistribution +from repyability.rbd.results import CapacityDistribution, _json_key, plain +from repyability.rbd.routes import Refused from repyability.rbd.shannon import ( _evaluate_shannon_plan, _minimal_cut_sets, _shannon_plan, ) -from repyability.utils.checks import is_whole, structure_method +from repyability.utils.checks import is_whole, one_of, structure_method from repyability.utils.deprecation import refuse_removed_names from repyability.utils.wrappers import ( check_probability, @@ -102,6 +104,21 @@ def strip(values: dict) -> dict: return wrapper +def _node_collection(label: str, nodes) -> set: + """The nodes given as ``label`` (``working_nodes`` or + ``broken_nodes``) as a set (none for None), or a TypeError when they + are no collection of node names (a number, say, from an argument given + out of place; text is one name, not a collection of its letters).""" + if nodes is None: + return set() + if isinstance(nodes, (str, bytes)) or not isinstance(nodes, Iterable): + raise TypeError( + f"{label} takes a collection of node names (a list or a set), " + f"got {nodes!r}." + ) + return set(nodes) + + def _set_order(nodes: frozenset) -> tuple: """Path and cut sets' order in a list: smallest first, then by name (as a ``FaultTree`` lists them).""" @@ -515,6 +532,15 @@ def oriented(i: Hashable, j: Hashable) -> Tuple[Hashable, Hashable]: """The pair as it is kept: its names in order as text.""" return (j, i) if str(j) < str(i) else (i, j) + @classmethod + def of(cls, nodes, value: Callable) -> "Pairs": + """Each pair of ``nodes`` once, with ``value(i, j)``.""" + out = cls() + for k, i in enumerate(nodes): + for j in nodes[k + 1 :]: # noqa: E203 + out[cls.oriented(i, j)] = value(i, j) + return out + def _swapped(self, key): if isinstance(key, tuple) and len(key) == 2: return (key[1], key[0]) @@ -536,11 +562,10 @@ def get(self, key, default=None): return self[key] if key in self else default def to_dict(self) -> dict: - """The pairs as plain data, ready for ``json.dumps`` (#235): each + """The pairs as plain data, ready for ``json.dumps``: each value under its pair's first name, then its second, ``{"a": {"b": 0.1}}`` for the pair ``("a", "b")``, the names as JSON holds them (see ``RBD`` results' ``to_dict``).""" - from repyability.rbd.results import _json_key, plain out: dict = {} for (first, second), value in self.items(): @@ -562,6 +587,62 @@ def _take_node_names(cls) -> None: setattr(cls, name, wrapped) +class _Cache: + """What a diagram has worked out and keeps between analyses, each made + when it is first wanted (``kept``): its common-cause plans + (``ccf_plans``), the renewal cycles of age and block replacement by + interval (``age_cycles``, ``block_cycles``), its tested units' lives + and cycles (``tested_lives``, ``tested_units``), its repair crews' + chains (``crew_chains``) and its common-cause groups' long-run tables + (``ccf_tables``). A shallow copy of the diagram shares what is made + when it is taken, but for what it works out again for itself + (``RBD._shallow_copy``).""" + + __slots__ = ( + "ccf_plans", + "age_cycles", + "block_cycles", + "tested_lives", + "tested_units", + "crew_chains", + "ccf_tables", + ) + + def __init__(self) -> None: + for name in self.__slots__: + setattr(self, name, None) + + def kept(self, name: str) -> dict: + """The dictionary ``name`` keeps, made empty if it is not yet.""" + kept = getattr(self, name) + if kept is None: + kept = {} + setattr(self, name, kept) + return kept + + def without(self, *redone: str) -> "_Cache": + """A cache sharing this one's, but for ``redone``.""" + out = _Cache() + for name in self.__slots__: + if name not in redone: + setattr(out, name, getattr(self, name)) + return out + + def copied(self, *names: str) -> "_Cache": + """A cache with copies of this one's dictionaries ``names``, for a + diagram rebuilt with most of this one's models (a lever moved): + what those keep is keyed by the models' identities (and a node's + schedule), so the rebuilt diagram reuses what was worked out for + the models it shares, works out the rest for itself, and keeps + that to itself.""" + out = _Cache() + for name in names: + kept = getattr(self, name) + if kept is not None: + setattr(out, name, dict(kept)) + return out + + class RBD: """Reliability block diagram structure: the base of the RBD classes. @@ -726,6 +807,22 @@ class RBD: {'p1': 0.25, 'p2': 0.25, 'v': 0.75} """ + @property + def _cache(self) -> _Cache: + """What this diagram has worked out and keeps (see ``_Cache``).""" + cache = self.__dict__.get("_kept") + if cache is None: + cache = self.__dict__["_kept"] = _Cache() + return cache + + def _shallow_copy(self, *redone: str): + """A shallow copy of this diagram, sharing what it has worked out + but ``redone`` (names of ``_Cache``), which the copy works out + again for itself.""" + out = copy.copy(self) + out.__dict__["_kept"] = self._cache.without(*redone) + return out + def __init_subclass__(cls, **kwargs): super().__init_subclass__(**kwargs) _take_node_names(cls) @@ -771,6 +868,8 @@ def __getattr__(self, name: str): _FOLDS_JUNCTIONS = False # What simulates a diagram too meshed to work out exactly. _SIMULATE_INSTEAD = "Simulate it instead." + # The kind of diagram, as the messages name it. + _KIND = "RBD" def __init__( self, @@ -972,11 +1071,15 @@ def find_irrelevant_components(self) -> set: A node is irrelevant when it is in no minimal path set, e.g. a node in parallel with a direct edge, which is a connection that never - fails. Whether such a node works never changes whether the system - works, so its Birnbaum and structural importance are zero. They are - found by the reduction to modules, without listing the path sets. - An irrelevant node is still a node of the RBD (it is in - [`node_names`][repyability.RBD.node_names]). The same set is found on + fails, or with a path through junctions alone (a + ``PerfectReliability`` node, such as a vote point, never fails + either). Whether such a node works never changes whether the + system works, so its Birnbaum and structural importance are zero. + They are found by the reduction to modules, with the junctions + folded in as always working, without listing the path sets. An + irrelevant node is still a node of the RBD (it is in + [`node_names`][repyability.RBD.node_names]); a junction is no + component, so it is never listed. The same set is found on construction and stored in ``structure_check["irrelevant_nodes"]``, with ``structure_check["has_irrelevant_nodes"]``. @@ -999,10 +1102,10 @@ def find_irrelevant_components(self) -> set: >>> rbd.find_irrelevant_components() {'b'} """ - decomposition = self._decomposition() - if isinstance(decomposition, GraphStructure): - raise NotImplementedError(decomposition.reason) - return set(self.nodes) - decomposition.nodes + structure = self._set_structure() + if isinstance(structure, GraphStructure): + raise NotImplementedError(structure.reason) + return set(self.nodes) - structure.nodes - self._junctions() def get_all_path_sets(self) -> Iterator[list[Hashable]]: """Iterate over every path from the input node to the output node. @@ -1049,7 +1152,7 @@ def get_min_path_sets( probability, the importance measures and the cut sets are found without listing them. A junction (a node given ``PerfectReliability``, such as a k-out-of-n vote point) always - works, so no path set needs it, and none lists it (#198). + works, so no path set needs it, and none lists it. Parameters ---------- @@ -1289,7 +1392,7 @@ def get_min_cut_sets( module). They are worked out on first use and cached; each call returns a new set. A junction (a node given ``PerfectReliability``, such as a k-out-of-n vote point) never fails, so a cut set with one - never happens, and none is listed (#198). + never happens, and none is listed. Parameters ---------- @@ -1348,22 +1451,24 @@ def path_set_probabilities(self, node_probabilities): Parameters ---------- node_probabilities : dict - The probability that each node works, keyed by node name, as - floats or numpy arrays of equal length (not lists). Every node - in a minimal path set needs an entry; other keys are ignored. + The probability that each node works (not a time: a + reliability at some time, or an availability), keyed by node + name, as floats or numpy arrays of equal length (not lists). + Every node in a minimal path set needs an entry; other keys are + ignored. Returns ------- np.ndarray - One value per minimal path set, in no particular order and with - no labels (to pair values with path sets, take the products over - ``get_min_path_sets(include_in_out_nodes=False)`` directly). - The shape is ``(n_path_sets,)`` for float probabilities and + One value per minimal path set, in the order + [`minimal_path_sets`][repyability.RBD.minimal_path_sets] lists + them (smallest first, then by name), so the two pair up. The + shape is ``(n_path_sets,)`` for float probabilities and ``(n_path_sets, n)`` for arrays of length ``n``. Raises ------ - KeyError + ValueError If a node in a minimal path set has no entry. Examples @@ -1379,7 +1484,23 @@ def path_set_probabilities(self, node_probabilities): >>> sorted(round(float(p), 4) for p in probs) [0.76, 0.855] """ - path_sets = self.get_min_path_sets(include_in_out_nodes=False) + if not isinstance(node_probabilities, Mapping): + raise TypeError( + "path_set_probabilities takes a dict of the probability " + "that each node works, by node name (not a time), got " + f"{node_probabilities!r}." + ) + path_sets = self.minimal_path_sets() + missing = sorted( + {n for path in path_sets for n in path} - set(node_probabilities), + key=str, + ) + if missing: + raise ValueError( + "path_set_probabilities takes the probability that each " + "node works, by node name (not a time); none given for " + f"{_names(missing)}." + ) out = [] for path in path_sets: path_prob = 1 @@ -1686,20 +1807,7 @@ def _meshed_routes( route.route in (r.EXACT, r.NUMERICAL) and name not in free ) or (route.route == r.SIMULATED and name in drawn): out[name] = r.refused(meshed) - return dict(sorted(out.items())) - - def _require_no_ccf_pairs(self) -> None: - """Raise if the diagram has common-cause groups: the joint - importance holds pairs of nodes working and failed, which a group's - members cannot be (the causes they share would still strike the - others).""" - if getattr(self, "ccf_groups", None): - raise NotImplementedError( - "The joint importance holds pairs of nodes working and " - "failed, which a common-cause group's members cannot be " - "(the causes they share would still strike the others): it " - "is not worked out with common-cause groups, as yet." - ) + return r.AnalysisRoutes(sorted(out.items())) def _joint_pairs(self, birnbaum: Callable, working_nodes, broken_nodes): """The joint importance of each pair of nodes (see @@ -1707,10 +1815,9 @@ def _joint_pairs(self, birnbaum: Callable, working_nodes, broken_nodes): broken)``, every node's Birnbaum importance with those nodes held: node ``j``'s with node ``i`` held working, less with it held failed, for each pair once (see ``Pairs``); 0 for a pair with a - node already held.""" - self._require_no_ccf_pairs() - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) + node already held. (With common-cause groups, see + ``_ccf_modules.joint_importance``.)""" + working, broken = self._held_nodes(working_nodes, broken_nodes) base = birnbaum(working, broken) nodes = list(base) held = working | broken @@ -1719,14 +1826,14 @@ def _joint_pairs(self, birnbaum: Callable, working_nodes, broken_nodes): if node not in held: up[node] = birnbaum(working | {node}, broken) down[node] = birnbaum(working, broken | {node}) - out = Pairs() - for k, i in enumerate(nodes): - for j in nodes[k + 1 :]: # noqa: E203 - if i in held or j in held: - out[Pairs.oriented(i, j)] = 0.0 * base[j] - else: - out[Pairs.oriented(i, j)] = up[i][j] - down[i][j] - return out + return Pairs.of( + nodes, + lambda i, j: ( + 0.0 * base[j] + if i in held or j in held + else up[i][j] - down[i][j] + ), + ) def _too_meshed(self) -> Optional[str]: """Why the diagram cannot be worked out exactly, if its core is too @@ -1794,7 +1901,7 @@ def _require_capacity(self) -> None: """Raise unless some node has a capacity, given or from its model.""" if not self._has_capacity(): - raise ValueError( + raise Refused( "No node has a capacity: give each node's throughput with " "capacity={node: capacity} when building the RBD." ) @@ -1892,7 +1999,7 @@ def _require_numbered_capacities(self) -> None: a probability of working does not describe (``system_capacity``).""" own = self._capacity_models() if own: - raise ValueError( + raise Refused( f"Node(s) {sorted(own, key=str)} take their capacity from " "their models, which a probability of working does not " "describe: use capacity_distribution()." @@ -2498,7 +2605,7 @@ def _require_series(self) -> None: alone = works(status) status[node] = True if alone: - raise ValueError( + raise Refused( "the minimum-effort algorithm applies to a series system " "(a single path through every intermediate node); use " "cost_based_allocation for other structures." @@ -2983,10 +3090,18 @@ def from_dict(cls, d: dict) -> "RBD": """ from repyability.rbd.serialisation import rbd_from_dict - if cls is not RBD and cls.__name__ != d.get("type"): + kind = d.get("type") if isinstance(d, Mapping) else None + if kind is None: raise ValueError( - f"{cls.__name__}.from_dict got a {d.get('type')!r} document; " - f"use {d.get('type')}.from_dict or RBD.from_dict." + f"{cls.__name__}.from_dict takes a diagram's to_dict() " + 'document, which names its class under "type" ' + "('NonRepairableRBD' or 'RepairableRBD'); this one has no " + '"type".' + ) + if cls is not RBD and cls.__name__ != kind: + raise ValueError( + f"{cls.__name__}.from_dict got a {kind!r} document; " + f"use {kind}.from_dict or RBD.from_dict." ) return rbd_from_dict(d) @@ -3052,9 +3167,9 @@ def _probabilities_with_overrides( """Returns a copy of ``node_probabilities`` with any forced nodes set to probability 1 (working) or 0 (broken), validating the overrides first. Shared by the importance measures and steady-state metrics.""" - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) + working_nodes, broken_nodes = self._held_nodes( + working_nodes, broken_nodes + ) out = dict(node_probabilities) for node in working_nodes: out[node] = np.ones_like( @@ -3066,16 +3181,28 @@ def _probabilities_with_overrides( ) return out + def _held_nodes(self, working_nodes, broken_nodes) -> Tuple[set, set]: + """The nodes held working and the nodes held failed, as sets (none + for None), checked (see ``_validate_node_overrides``): what every + method that takes ``working_nodes`` and ``broken_nodes`` starts + with.""" + working = _node_collection("working_nodes", working_nodes) + broken = _node_collection("broken_nodes", broken_nodes) + self._validate_node_overrides(working, broken) + return working, broken + def _validate_node_overrides(self, working_nodes, broken_nodes) -> None: """Validate the working/broken node override sets. - Raises a ValueError on invalid input rather than silently ignoring it: - the same node in both sets, the input/output node, or an unknown node - name (e.g. a typo, which would otherwise silently have no effect and + Raises on invalid input rather than silently ignoring it: a + TypeError for a set that is no collection of nodes (a number, say, + from an argument given out of place), and a ValueError for the same + node in both sets, the input/output node, or an unknown node name + (e.g. a typo, which would otherwise silently have no effect and return a plausible-but-wrong result). """ - working_nodes = set(working_nodes) - broken_nodes = set(broken_nodes) + working_nodes = _node_collection("working_nodes", working_nodes) + broken_nodes = _node_collection("broken_nodes", broken_nodes) both = working_nodes & broken_nodes if both: @@ -3239,7 +3366,9 @@ def _risk_achievement_worth( when_failed = self._system_unreliability( {**p, node: zero}, {**q, node: one} ) - node_importance[node] = _averaged(when_failed, weights) / as_is + # 0/0 (nan) where the system cannot fail: no cause for a warning. + with np.errstate(divide="ignore", invalid="ignore"): + node_importance[node] = _averaged(when_failed, weights) / as_is return node_importance def _risk_reduction_worth( @@ -3323,10 +3452,7 @@ def _criticality_importance( ValueError If ``kind`` is neither ``"failure"`` nor ``"success"``. """ - if kind not in ("failure", "success"): - raise ValueError( - f"kind must be 'failure' or 'success', got {kind!r}." - ) + one_of("kind", kind, ("failure", "success")) importance, works, fails, p, q = self._importances( node_probabilities, node_failures ) @@ -3402,19 +3528,18 @@ def _fussell_vesely( "fv_type must be either 'c' (cut-set) or 'p' (path-set), " f"fv_type={fv_type!r} was given." ) - if method not in ("exact", "rare_event"): - raise ValueError( - "method must be 'exact' or 'rare_event', " f"got {method!r}." - ) + one_of("method", method, ("exact", "rare_event")) p, q, size = self._node_pairs(node_probabilities, node_failures) # The system unreliability, the denominator for every node. system_ff = _averaged(self._system_unreliability(p, q), weights) numerators = self._fv_numerators(p, q, size, fv_type, method) - return { - node: _averaged(numerators[node], weights) / system_ff - for node in self.nodes - } + # 0/0 (nan) where the system cannot fail: no cause for a warning. + with np.errstate(divide="ignore", invalid="ignore"): + return { + node: _averaged(numerators[node], weights) / system_ff + for node in self.nodes + } def _fv_numerators( self, p: dict, q: dict, size: int, fv_type: str, method: str diff --git a/repyability/rbd/regression_node.py b/repyability/rbd/regression_node.py index ef6ae6f4..d899333c 100644 --- a/repyability/rbd/regression_node.py +++ b/repyability/rbd/regression_node.py @@ -44,6 +44,59 @@ def _is_semiparametric(model) -> bool: return isinstance(model, semiparametric.SemiParametricRegressionModel) +def _inverse_hazard(hazard, target: np.ndarray) -> np.ndarray: + """The times ``t`` with ``hazard(t) = target``, for a cumulative hazard + rising from 0: bracketed by doubling, then bisected until each bracket + is as narrow as floating point allows. ``inf`` where ``target`` is + (``u = 1``) or the hazard never reaches it; 0 where ``target`` is 0.""" + target = np.asarray(target, dtype=float) + out = np.zeros(target.shape) + live = np.flatnonzero(target > 0.0) + out[target == np.inf] = np.inf + live = live[np.isfinite(target[live])] + if not live.size: + return out + goal = target[live] + lo = np.zeros(live.size) + hi = np.ones(live.size) + short = hazard(hi) < goal + while short.any(): + lo[short] = hi[short] + hi[short] *= 2.0 + too_far = hi > 1e300 + if too_far.any(): + hi[too_far & short] = np.inf + short &= ~too_far + rising = np.flatnonzero(short) + if rising.size: + short[rising] = hazard(hi[rising]) < goal[rising] + open_ = np.flatnonzero(np.isfinite(hi)) + while open_.size: + a, b = lo[open_], hi[open_] + mid = 0.5 * (a + b) + settled = (mid <= a) | (mid >= b) + below = hazard(mid) < goal[open_] + lo[open_] = np.where(below & ~settled, mid, a) + hi[open_] = np.where(~below & ~settled, mid, b) + open_ = open_[~settled] + out[live] = hi + return out + + +def _fitted_width(model) -> Optional[int]: + """How many covariates (stresses) a surpyval regression model was + fitted with, or None if it does not say. Its ``_n_covariates()`` + counts them for every kind; ``phi_param_map`` names a proportional- + hazards or accelerated-failure-time model's coefficients, one each, but + an accelerated-life model's life-stress parameters (an Arrhenius + model's ``a`` and ``b`` for one stress).""" + count = getattr(model, "_n_covariates", None) + if callable(count): + return int(count()) + fitted = getattr(model, "phi_param_map", None) + return len(fitted) if isinstance(fitted, dict) else None + + class RegressionNode: """An RBD node backed by a fitted surpyval regression model. @@ -60,10 +113,12 @@ class RegressionNode: ``sf`` and ``ff`` evaluate that curve directly, so the node takes part in system reliability, importance measures and the condition-based - (``age``) methods with no special handling. ``mean`` and ``random`` - work from the curve tabulated on a grid, which needs a proper - parametric curve: a semiparametric model such as ``surpyval.CoxPH`` - supports ``sf`` but not ``mean`` or ``random``. Do the regression fit + (``age``) methods with no special handling. ``mean`` integrates the + curve to a relative 1e-10, and ``random`` inverts it exactly (the + model's quantile at fixed covariates; the cumulative hazard along a + schedule), which needs a proper parametric curve: a semiparametric + model such as ``surpyval.CoxPH`` supports ``sf`` but not ``mean`` or + ``random``. Do the regression fit in surpyval and pass the fitted model in. Parameters @@ -140,11 +195,11 @@ def __init__( self.schedule = schedule # A covariate vector of another width than the model was fitted # with: say so directly. - fitted = getattr(model, "phi_param_map", None) - if self.covariates is not None and isinstance(fitted, dict): - if len(self.covariates) != len(fitted): + fitted = _fitted_width(model) + if self.covariates is not None and fitted is not None: + if len(self.covariates) != fitted: raise ValueError( - f"The model was fitted with {len(fitted)} covariate(s); " + f"The model was fitted with {fitted} covariate(s); " f"covariates has {len(self.covariates)}." ) # Probe the survival interface so a misuse fails clearly at @@ -164,8 +219,6 @@ def __init__( "models). Probing " f"survival failed: {type(e).__name__}: {e}." ) from e - # Cached (t, sf(t)) grid for mean()/random() (built lazily). - self._grid: Any = None def _Z(self, n: int) -> np.ndarray: """Fixed covariate vector broadcast to ``n`` rows for ``sf(x, Z)``.""" @@ -231,50 +284,79 @@ def ff(self, x: ArrayLike) -> np.ndarray: out = np.asarray(self.model.ff(x, self._Z(len(x))), dtype=float) return out[0] if scalar else out - def _survival_grid(self): - """A cached ``(t, sf(t))`` grid spanning the bulk of the lifetime. + def _require_proper(self) -> None: + """Refuse a lifetime with no proper survival curve (starting at ~1): + a semiparametric baseline (e.g. surpyval's Cox) is defined only on + the observed range, with no tail, so its mean and draws are + undefined there, a clear error rather than a wrong number.""" + if _is_semiparametric(self.model) or ( + # Just after 0, in any unit of time. + float(self._sf_at(np.array([np.finfo(float).tiny]))[0]) + <= 0.99 + ): + raise ValueError( + "mean()/random() need a proper parametric survival curve " + "(sf(0+) ~ 1, decaying to 0), but this model's survival " + "is improper -- e.g. a semiparametric Cox baseline, " + "defined only on the observed range. Its MTTF is " + "undefined; use the sf-based reliability / remaining-life " + "methods instead." + ) - surpyval's regression models expose no working ``random``, so ``mean`` - and ``random`` are obtained from the survival curve directly (a numeric - integral, and inverse-transform sampling). This needs a *proper* - lifetime curve (starting at ~1 and decaying to 0); a semiparametric - baseline (e.g. surpyval's Cox) is defined only on the observed range - and has no proper tail, so MTTF/simulation is undefined there and is - reported as a clear error rather than a wrong number. - """ - if self._grid is None: - if _is_semiparametric(self.model) or ( - float(self._sf_at(np.array([1e-9]))[0]) <= 0.99 - ): - raise ValueError( - "mean()/random() need a proper parametric survival curve " - "(sf(0+) ~ 1, decaying to 0), but this model's survival " - "is improper -- e.g. a semiparametric Cox baseline, " - "defined only on the observed range. Its MTTF is " - "undefined; use the sf-based reliability / remaining-life " - "methods instead." - ) - hi = 1.0 - while self._sf_at(np.array([hi]))[0] > 1e-4: - hi *= 2.0 - if hi > 1e15: - raise ValueError( - "mean()/random(): the survival curve does not decay " - "to 0 (no finite MTTF). Use the sf-based methods." - ) - t = np.linspace(0.0, hi, 4096) - self._grid = (t, self._sf_at(t)) - return self._grid + def _knots(self) -> np.ndarray: + """Times marking the curve's scale, for its integral: at fixed + covariates the model's quantiles, along a schedule none (the + integral finds them from the curve).""" + if self.schedule is not None: + return np.empty(0) + from ._point_availability import _KNOT_PROBABILITIES + + with np.errstate(all="ignore"): + q = self._quantiles(_KNOT_PROBABILITIES) + return q[np.isfinite(q)] + + def _kinks(self) -> np.ndarray: + """Times at which the curve may bend sharply: a step schedule's + change points, where the covariates jump. (A ``CovariatePath`` + moves between its points linearly, so its hazard only bends, which + the integral follows without them.)""" + edges = getattr(self.schedule, "edges", None) + if edges is None: + return np.empty(0) + edges = np.asarray(edges, dtype=float) + return edges[np.isfinite(edges) & (edges > 0.0)] + + def _quantiles(self, p: np.ndarray) -> np.ndarray: + """The model's quantiles ``F^-1(p)`` at the fixed covariates.""" + assert self.covariates is not None # fixed-covariate mode only + return np.asarray( + self.model.qf(p, self.covariates[np.newaxis, :]), dtype=float + ) + + def _draw(self, u: np.ndarray) -> np.ndarray: + """The lifetimes ``F^-1(u)`` of the uniforms ``u``: the model's + quantiles at fixed covariates, and along a schedule the time at + which the cumulative hazard reaches ``-log(1 - u)``.""" + self._require_proper() + u = np.asarray(u, dtype=float) + if self.schedule is None: + return self._quantiles(u) + with np.errstate(divide="ignore"): + target = -np.log1p(-u) + return _inverse_hazard( + lambda t: np.asarray( + self.model.Hf_tvc(t, self.schedule), dtype=float + ), + target, + ) def mean(self) -> float: """Mean time to failure at the stored covariates / along the schedule. For a non-negative lifetime ``E[T] = integral of R(t)``, integrated - numerically over the survival curve: the trapezoidal rule on 4096 - points from 0 to the first ``t`` in 1, 2, 4, 8, ... at which - ``R(t) <= 1e-4``. The tail beyond is left out, so the result is - slightly low (by that tail's integral). The grid is built on first - use and cached. + to a relative accuracy of 1e-10 on pieces split at the model's + quantiles (fixed covariates) or the schedule's change points, as + an RBD's MTTF is: ``inf`` if some lifetimes never end. Returns ------- @@ -285,23 +367,26 @@ def mean(self) -> float: ------ ValueError If the model is semiparametric (a Cox model: its baseline is - defined on the observed range only), if the survival curve is - improper (``R(1e-9) <= 0.99``) or if it does not fall to 1e-4 - by ``t = 1e15``. + defined on the observed range only), or if the survival curve + is improper (``R(0+) <= 0.99``). """ - t, s = self._survival_grid() - return float(np.trapezoid(s, t)) + from ._mean_lifetime import mean_lifetime + + self._require_proper() + return float(mean_lifetime(self._sf_at, self._knots(), self._kinks())) def random(self, size: int) -> np.ndarray: """Draw ``size`` failure times at the stored covariates / schedule. - Inverse-transform sampling on the survival curve tabulated for - ``mean``: each uniform ``u`` maps to the time at which ``R = u`` - (linear interpolation), so draws never exceed the grid's end, - where ``R <= 1e-4``. There is no ``seed`` argument: this uses - numpy's global RNG, so seed it (``np.random.seed``) or wrap the - call in ``repyability.utils.wrappers.numpy_seed`` to reproduce the - draws. An RBD's seeded ``random`` and ``mean`` do this for you. + Inverse-transform sampling, exact: each uniform ``u`` maps to the + time at which the distribution reaches ``u``, the model's + quantile ``qf(u, Z)`` at fixed covariates, and along a schedule the + time at which the cumulative hazard reaches ``-log(1 - u)`` (found + by bisection, to the last bits). There is no ``seed`` argument: + this uses numpy's global RNG, so seed it (``np.random.seed``) or + wrap the call in ``repyability.utils.wrappers.numpy_seed`` to + reproduce the draws. An RBD's seeded ``random`` and ``mean`` do + this for you. Parameters ---------- @@ -316,28 +401,26 @@ def random(self, size: int) -> np.ndarray: Raises ------ ValueError - If the survival curve is improper or does not decay, as for - ``mean``. + If the survival curve is improper, as for ``mean``. """ - t, s = self._survival_grid() - u = np.random.uniform(size=size) - # s decreases in t; np.interp needs an increasing sample-point array. - return np.interp(u, s[::-1], t[::-1]) + return self._draw(np.random.uniform(size=size)) def _row_sampler(self) -> RowSampler: """``random(1)`` as a :class:`~._sampling.RowSampler` (one uniform per draw), so an RBD with this node batches its draws.""" - - def draw(u): - t, s = self._survival_grid() - return np.interp(np.ascontiguousarray(u[:, 0]), s[::-1], t[::-1]) - - return RowSampler(1, draw) + return RowSampler( + 1, lambda u: self._draw(np.ascontiguousarray(u[:, 0])) + ) # -- Serialisation ---------------------------------------------------- @staticmethod def _schedule_to_dict(schedule: Any) -> dict: + if getattr(schedule, "edges", None) is None: + raise NotImplementedError( + f"Serialising a {type(schedule).__name__} is not supported " + "yet; use a StepSchedule." + ) if getattr(schedule, "period", None) is not None: raise NotImplementedError( "Serialising a cyclic StepSchedule is not supported yet; use " diff --git a/repyability/rbd/repairable_rbd.py b/repyability/rbd/repairable_rbd.py index 6ea6b78d..0ffc676d 100644 --- a/repyability/rbd/repairable_rbd.py +++ b/repyability/rbd/repairable_rbd.py @@ -1,32 +1,15 @@ """Repairable reliability block diagrams: the ``RepairableRBD`` class. Besides the class, the module holds the helpers of its availability -simulation: the ``Event`` records of its event queue, the timeline -arithmetic that defines the up/down criticality measures -(``combined_timeline``, ``intersection``, ``union``, ``time_at_status``; -the simulation adds the same times up as it goes) and the failure and +simulation: the ``Event`` records of its event queue and the failure and restoration criticality index ratios. The simulation's random streams are in ``_streams``, and its compiled engine in ``_compiled`` and ``_kernel``. """ -import dataclasses -import functools -import hashlib -import heapq -import itertools -import json -import math -import numbers -import pickle -import warnings -from collections import Counter, defaultdict, deque from collections.abc import Mapping from contextlib import contextmanager from copy import copy -from dataclasses import dataclass, field -from fractions import Fraction -from functools import partial from typing import ( TYPE_CHECKING, Any, @@ -36,8 +19,6 @@ Hashable, Iterable, List, - NamedTuple, - NoReturn, Optional, Sequence, Tuple, @@ -45,4583 +26,147 @@ ) import numpy as np -from scipy.optimize import OptimizeResult, brentq, minimize -from scipy.special import expit, logit, logsumexp, softmax from surpyval import ExactEventTime -from repyability._version import __version__ from repyability.non_repairable import NonRepairable from repyability.rbd import ( - _ccf_chain, _ccf_modules, - _chain_transient, - _conditional, - _crew_chain, _importance_time, -) -from repyability.rbd import _montecarlo as montecarlo -from repyability.rbd import ( - _quadrature, - _rates, _sensitivity, _spares, - _standby_chain, - _streams, - _timeline_runs, -) -from repyability.rbd import capacity as _capacity -from repyability.rbd._block_replacement import ( - BlockHead, - block_availability, - block_cycle, -) -from repyability.rbd._condition_replacement import ( - ConditionHead, - condition_availability, - condition_cycle, ) -from repyability.rbd._exact import ( - ExactSum, - add_columns, - bin_totals, - binned_parts, - compact_parts, -) -from repyability.rbd._hidden_life import ( - TestedLife, - TestedLifeCurve, - TestedLifeSteady, -) -from repyability.rbd._hidden_tests import TestedUnit -from repyability.rbd._hidden_tests import check as check_tested from repyability.rbd._model_utils import ( - distribution_name, - failure_time_scale, - is_fixed_probability, - is_mixture, - lfp_p, - model_mean, refuse_nonparametric, ) -from repyability.rbd._point_availability import ( - Atoms, - BlockCurve, - First, - InspectionCurve, - MinimalRepairCurve, - PartialTestCurve, - ShiftedCurve, - StartedBlockCurve, - SteadyCurve, - SystemCurve, - curve_breaks, -) -from repyability.rbd._point_availability import knots as point_knots -from repyability.rbd._point_availability import unit_curve -from repyability.rbd._sampling import MixtureLife, stream_sampler from repyability.rbd.degrading_node import DegradingNode -from repyability.rbd.helper_classes import PerfectReliability, perfect_class -from repyability.rbd.load_sharing_node import LoadSharingModel -from repyability.rbd.modular import GraphStructure -from repyability.rbd.node_state import NodeState -from repyability.rbd.rbd import RBD, _check_on_infeasible_rbd, _close_name -from repyability.rbd.redundancy_allocation import ( - lowest_total_cost, - redundancy_caps, -) +from repyability.rbd.helper_classes import perfect_class +from repyability.rbd.rbd import RBD, Pairs, _check_on_infeasible_rbd from repyability.rbd.results import ( AvailabilityAllocation, AvailabilityResult, CapacityDistribution, - ConditionalRun, ConfidenceInterval, - ControlVariate, CostResult, - Criticalities, ExpectedCost, ExpectedEvents, - FailureCriticalityIndex, Lever, MaintenancePlan, RateBreakdown, - RestorationCriticalityIndex, SparesDemand, SparesStock, TimelineSimulation, TotalCostAllocation, UncertaintyImportance, UncertaintyResult, - UpDownImportance, ) -from repyability.rbd.standby_node import StandbyModel if TYPE_CHECKING: from repyability.rbd.chunks import SimulationChunk +from repyability.rbd import ( + _ccf_groups, + _costs, + _crews, + _curves, + _event_loop, + _intervals, + _long_run, + _repairable_allocation, + _repairable_capacity, + _repairable_importance, + _repairable_routes, + _repairable_uncertainty, + _runs, + _spec, + _windows, +) +from repyability.rbd._common import ( + _times_first, +) +from repyability.rbd._event_loop import ( + _is_junction, + _never_settles, + _on_duty, + _validate_crews, + _validate_priority, +) +from repyability.rbd._events import ( + _Imperfect, + _Inspection, + _Preventive, + _Standby, + _Unit, +) + +# The criticality indices of a run's counts are importable from here, as +# they were before the run's code moved to _runs. +from repyability.rbd._runs import ( # noqa: E402,F401 + _UNSTREAMED, + failure_criticality_index_per_component_failures, + failure_criticality_index_per_system_failures, + restoration_criticality_index_by_component, + restoration_criticality_index_by_system, +) from repyability.rbd.routes import AnalysisRoute from repyability.utils.checks import ( - is_number, no_distribution, - number_or_nan, - real_array, - structure_method, - unfitted_distribution, + one_of, ) -from repyability.utils.deprecation import renamed -from repyability.utils.wrappers import SIMULATIONS, outside_level - -from ._mean_lifetime import _G7_W, _GK_W, _GK_X - - -class _StreamedRBD: - """Stands in for a nested :class:`RepairableRBD` component during - ``availability()``: runs the nested RBD's own simulation, with its - components drawing from their own streams through their stand-ins (see - ``RepairableRBD._streamed_components``).""" - - def __init__(self, rbd: "RepairableRBD", sources: dict): - self._rbd = rbd - self._sources = sources - - def initialize_event_queue(self, t_simulation, state=None): - """Start the nested RBD's simulation, its components from - ``state`` (checked; see ``RepairableRBD._simulation_states``).""" - self._rbd._start_queue( - t_simulation, set(), set(), "p", self._sources, state or {} - ) - - def next_event(self): - return self._rbd.next_event(sources=self._sources) - - @property - def system_state(self) -> bool: - return self._rbd.system_state - - @property - def last_change_planned(self) -> bool: - return self._rbd.last_change_planned - - -def _planned(source, status: bool) -> bool: - """Whether a nested RBD's change to ``status``, just drawn from - ``source``, starts a planned outage.""" - return not status and getattr(source, "last_change_planned", False) - - -class _EventQueue: - """The simulation's event queue. - - ``queue.PriorityQueue`` is this same heap (``heappush``/``heappop``) plus - thread locking on every operation, which a single-threaded simulation - only pays for; events come out in exactly the same order. - - The heap holds ``(time, event)`` pairs, so that times compare as floats - rather than through the events' own ordering. Two events at the same time - compare equal either way, so ties come out as they would with the events - alone. (A ``(time, counter, event)`` entry would release ties in the - order they were queued instead, which changes the results.) - """ - - __slots__ = ("_heap",) - - def __init__(self): - self._heap: list = [] - - def put(self, event: "Event") -> None: - heapq.heappush(self._heap, (event.time, event)) - - def get(self) -> "Event": - return heapq.heappop(self._heap)[1] - - def empty(self) -> bool: - return not self._heap - - def qsize(self) -> int: - return len(self._heap) - - -class _StreamedComponent: - """Stands in for a :class:`NonRepairable` component during - ``availability()``: alternates failure and repair draws exactly as - ``NonRepairable.next_event`` does, but takes them from the component's - own streams (see ``_streams``). A maintained or inspected component - draws its maintenance or test times from a stream of its own too, or, - when they cannot be streamed, from ``model.random``.""" - - __slots__ = ( - "_failure", - "_repair", - "_fails_next", - "_duration", - "_model", - "_start", - "_test", - ) - - def __init__( - self, - failure, - repair, - duration=None, - model=None, - start=None, - test=None, - ): - self._failure = failure - self._repair = repair - self._fails_next = True - self._duration = duration - self._model = model - self._start = start - self._test = test - - def reset(self): - self._fails_next = True - - def maintenance_time(self) -> float: - if self._duration is not None: - return self._duration.draw() - return self._model.random(1).item() - - def start_uniform(self) -> float: - """The uniform a start from a state draws what is left of the - unit's life, repair or maintenance from (see - ``RepairableRBD._started``): from its ``START`` stream, or numpy's - global RNG.""" - if self._start is not None: - return self._start.draw() - return float(np.random.random()) - - def test_uniform(self) -> float: - """The uniform that decides whether a test that can miss a failure - finds it: from its ``TEST`` stream, or numpy's global RNG.""" - if self._test is not None: - return self._test.draw() - return float(np.random.random()) - - def life_drawn(self) -> None: - """The unit's life has been drawn (given its age, at a start from a - state): its next draw is its repair.""" - self._fails_next = False - - def next_event(self): - if self._fails_next: - self._fails_next = False - return self._failure.draw(), False - self._fails_next = True - return self._repair.draw(), True - - -class _ModelDraws: - """A model's draws, one at a time, from numpy's global RNG: as a - stream gives them, and as ``NonRepairable.next_event`` draws them (one - ``random(1)`` call each).""" - - __slots__ = ("_model",) - - def __init__(self, model): - self._model = model - - def draw(self) -> float: - return self._model.random(1).item() - - -def _exponential_sampler(rate: float): - """The quantile function of an exponential time at ``rate``: ``inf`` - for a rate of 0 (a common-cause group member whose every cause is - shared never fails on its own, see ``RepairableRBD._own_rate``).""" - if rate <= 0.0: - return lambda u: np.full(np.shape(u), np.inf) - return lambda u: -np.log1p(-np.asarray(u, dtype=float)) / rate - - -class _ExponentialDraws: - """Exponential times at ``rate``, one at a time, from numpy's global RNG - (``inf`` for a rate of 0).""" - - __slots__ = ("_rate",) - - def __init__(self, rate: float): - self._rate = rate - - def draw(self) -> float: - if self._rate <= 0.0: - return math.inf - return float(np.random.exponential(1.0 / self._rate)) - - -def _aged_life(model, age: float, u: float) -> float: - """The life left to a unit of lifetime ``model`` at virtual age - ``age``, from the uniform ``u``: the ``x`` with ``H(age + x) = H(age) - - log(u)`` (``H`` the cumulative hazard), as surpyval's virtual-age - renewal models draw it (``conditional_gaps``). A plain Exponential or - Weibull is worked in closed form, any other model by surpyval (a - mixture with its quantile worked out, ``MixtureLife``).""" - if is_mixture(model): - model = MixtureLife.of(model) - dist = getattr(model, "dist", None) - name = getattr(dist, "name", None) - if ( - name in ("Exponential", "Weibull") - and lfp_p(model) == 1 - and getattr(model, "f0", None) == 0 - and not getattr(model, "gamma", 0) - ): - exposure = 0.0 - math.log(u) if u > 0.0 else math.inf - if name == "Exponential": - return exposure / float(model.params[0]) - alpha, beta = (float(p) for p in model.params) - hazard = (age / alpha) ** beta if age > 0.0 else 0.0 - if hazard == 0.0: - # As new (or too young for its hazard to register). - return max(alpha * exposure ** (1.0 / beta) - age, 0.0) - # alpha * (H(age) + exposure) ** (1 / beta) - age, without the - # cancellation. - return age * math.expm1(math.log1p(exposure / hazard) / beta) - from surpyval.recurrent.renewal.renewal_model import conditional_gaps - - return float(conditional_gaps(model, np.array([age]), np.array([u]))[0]) - - -class _ImperfectComponent(_StreamedComponent): - """Stands in for a component repaired imperfectly (a spec's - ``"repair"``; see ``RepairableRBD``) during a simulation, keeping the - unit's virtual age (Kijima): a repair after an operating time ``x`` - takes it from ``v`` to ``v + q * x`` (Kijima I) or ``q * (v + x)`` - (Kijima II), and its next life is drawn given it (``_aged_life``), - from a stream of its own. A unit as new draws from its life stream, as - any component does. At the ``replace_after``-th failure since it was - renewed it is replaced instead, as new, and a scheduled replacement - renews it too (``reset``). ``operated`` is its operating time since it - was renewed, which age replacement counts. Without streams, it draws - from its models and numpy's global RNG.""" - - __slots__ = ( - "_unit", - "_aged", - "_q", - "_second", - "_limit", - "_x", - "age", - "operated", - "count", - ) - - def __init__( - self, - unit, - imperfect: "_Imperfect", - failure=None, - repair=None, - aged=None, - duration=None, - model=None, - test=None, - ): - super().__init__(failure, repair, duration, model, test=test) - self._unit = unit - self._aged = aged - self._q = imperfect.q - self._second = imperfect.kijima == "kijima2" - self._limit = imperfect.replace_after - self._x = 0.0 - self.age = self.operated = 0.0 - self.count = 0 - - def reset(self): - self._fails_next = True - self.age = self.operated = 0.0 - self.count = 0 - - @property - def replacing(self) -> bool: - """Whether the unit's failure (the last drawn) is the - ``replace_after``-th since it was renewed, which replaces it.""" - return self._limit is not None and self.count + 1 >= self._limit - - def next_event(self): - if self._fails_next: - self._fails_next = False - if self.age > 0.0: - u = ( - self._aged.draw() - if self._aged is not None - else float(np.random.random()) - ) - x = _aged_life(self._unit.reliability, self.age, u) - elif self._failure is not None: - x = self._failure.draw() - else: - x = self._unit.reliability.random(1).item() - self._x = x - return x, False - self._fails_next = True - if self.replacing: - self.age = self.operated = 0.0 - self.count = 0 - else: - x = self._x - self.count += 1 - self.operated += x - self.age = ( - self._q * (self.age + x) - if self._second - else self.age + self._q * x - ) - if self._repair is not None: - return self._repair.draw(), True - return self._unit.time_to_replace.random(1).item(), True - - -def _stand_in( - component, - run, - made: dict, - path: tuple, - duration=None, - imperfect: Optional["_Imperfect"] = None, -): - """What a component draws its events from during ``availability()`` - (see ``RepairableRBD._streamed_components``): a stand-in taking its - draws from its own streams, or the component itself when they cannot be - streamed (a model other than a surpyval parametric one, or a subclass, - which may draw its events its own way), which then draws from numpy's - global RNG. ``path`` is the node's place, nested RBDs included, which - names its streams; ``duration`` its maintenance or test time model, and - ``imperfect`` its imperfect repair (see ``_ImperfectComponent``).""" - if type(component) is RepairableRBD: - return _StreamedRBD( - component, component._streamed_components(run, made, path) - ) - if type(component) is not NonRepairable: - return component - failure = run.stream(path, _streams.FAILURE) - repair = run.stream(path, _streams.REPAIR) - if imperfect is not None: - if failure is None or repair is None: - return _ImperfectComponent(component, imperfect, model=duration) - return _ImperfectComponent( - component, - imperfect, - failure, - repair, - run.stream(path, _streams.AGED), - run.stream(path, _streams.DURATION), - duration, - run.stream(path, _streams.TEST), - ) - if failure is None or repair is None: - return component - return _StreamedComponent( - failure, - repair, - run.stream(path, _streams.DURATION), - duration, - run.stream(path, _streams.START), - run.stream(path, _streams.TEST), - ) - - -@dataclass(frozen=True) -class _Unit: - """A unit of a standby group: its repair is a job for the repair crews - under this key (a dataclass, so equal to no node name).""" - - node: Any - unit: int - - -@dataclass(frozen=True) -class _Cause: - """A shared cause of a common-cause group, as the simulation strikes - it (#158; a dataclass, so equal to no node name): the members it fails - (``struck``, those up when it strikes), its rate, and whether a test - that can miss its failures decides for all of them at once - (``coins``).""" - - struck: tuple - rate: float - coins: bool - - -class _CauseDraws: - """What a shared common cause draws in a simulation (#158): the times - between its strikes, exponential at its rate, and at each strike the - uniform that decides whether the tests that can miss its failures find - them (found, or missed, alike); from its own streams, or numpy's global - RNG.""" - - __slots__ = ("_rate", "_gaps", "_coins") - - def __init__(self, rate: float, gaps=None, coins=None): - self._rate = rate - self._gaps = gaps - self._coins = coins - - def gap(self) -> float: - if self._gaps is not None: - return self._gaps.draw() - return float(np.random.exponential(1.0 / self._rate)) - - def coin(self) -> float: - if self._coins is not None: - return self._coins.draw() - return float(np.random.random()) - - -class _StandbyDraws: - """What a standby group's units draw during a simulation: each unit's - lives and repair times from streams of its own, and the group's switches - from one (see ``_streams``); or, when they cannot be streamed, from the - unit models' ``random`` and numpy's global RNG.""" - - __slots__ = ("_models", "_lives", "_repairs", "_switches") - - def __init__(self, models, lives=None, repairs=None, switches=None): - self._models = models - self._lives = lives - self._repairs = repairs - self._switches = switches - - def life(self, unit: int) -> float: - if self._lives is None: - return self._models.reliability.random(1).item() - return self._lives[unit].draw() - - def repair(self, unit: int) -> float: - if self._repairs is None: - return self._models.time_to_replace.random(1).item() - return self._repairs[unit].draw() - - def switch(self) -> float: - if self._switches is None: - return float(np.random.random()) - return self._switches.draw() - - -def _uniforms(u: np.ndarray) -> np.ndarray: - """The draws of a stream of uniforms: the uniforms themselves.""" - return u - - -def _state_key(states: dict) -> Optional[str]: - """The components' ``states`` at the start of a run (checked; see - ``RepairableRBD._simulation_states``) as JSON, the same in every - process, for a chunk's settings (see ``simulate_chunk``); None for - every component new.""" - - def plain(states: dict) -> list: - out = [] - for node, start in states.items(): - if isinstance(start, NodeState): - if start.new: - continue - value: Any = dataclasses.asdict(start) - else: - value = plain(start) - if not value: - continue - out.append([repr(node), value]) - return sorted(out, key=lambda item: item[0]) - - entries = plain(states) - return json.dumps(entries, sort_keys=True) if entries else None - - -def _hot_units(spec: dict) -> "RepairableRBD": - """A standby group's twin (see ``RepairableRBD._twin``): its units - operating together, each failing and repaired on its own, ``k`` of them - needed, as a nested RBD whose components are named ``0, 1, ...``, as - the group's units are, so that their streams are the units'.""" - standby = spec["standby"] - units, k = int(standby.get("units", 2)), int(standby.get("k", 1)) - keys = ("reliability", "repairability") + RepairableRBD.COST_KEYS - unit = {key: spec[key] for key in keys if key in spec} - return RepairableRBD( - [("s", u) for u in range(units)] + [(u, "t") for u in range(units)], - {u: dict(unit) for u in range(units)}, - k={"t": k} if k > 1 else None, - ) - - -def _check_shard_map(shard_map, shard_size, n_jobs) -> None: - """Check ``availability``'s ``shard_map`` and ``shard_size``.""" - if shard_map is None: - if shard_size is not None: - raise ValueError( - "shard_size is the size of the shards shard_map runs: give " - "shard_map too." - ) - return - if not callable(shard_map): - raise ValueError( - "shard_map must be a map: called as shard_map(run_shard, " - f"shards), like map; got {shard_map!r}." - ) - if n_jobs is not None: - raise ValueError( - "shard_map runs the shards wherever it sends them: give its " - "workers the CPUs, and leave out n_jobs." - ) - - -def _module_shards( - shard_map, template: dict, step: int, first: int, count: int, modules: int -): - """A conditional run's simulations ``first`` to ``first + count - 1`` - of its modules (#189) as shards, cut at the multiples of ``step``, - which ``shard_map(run_shard, shards)`` runs wherever it sends them: - the ``modules`` histories joined in order, each simulation's own cost, - the costs by category, and by module (None for a module not costed).""" - from repyability.rbd.shards import run_shard - - stop = first + count - cuts = [first, *range((first // step + 1) * step, stop, step), stop] - ranges = list(zip(cuts, cuts[1:])) - parts = [ - _conditional.from_partial(saved, modules) - for saved in shard_map( - run_shard, [_shard_bytes(template, a, b) for a, b in ranges] - ) - ] - parts.sort(key=lambda part: part[0]) - if [part[0] for part in parts] != ranges: - raise ValueError( - "shard_map gave back other partials than its shards': it must " - "return run_shard's result for each shard it is given." - ) - categories: Dict[str, float] = {} - for part in parts: - for key, value in part[3].items(): - categories[key] = categories.get(key, 0.0) + value - by_module = np.sum([part[4] for part in parts], axis=0) - costed = np.any([part[5] for part in parts], axis=0) - return ( - _conditional.joined([part[1] for part in parts]), - np.concatenate([part[2] for part in parts]), - categories, - [float(v) if c else None for v, c in zip(by_module, costed)], - ) - - -#: A window's expected cost split by category and by component (see -#: ``CostResult.by_category`` and ``by_component``). -_Breakdown = Tuple[Dict[str, float], Dict[Hashable, float]] - - -class _Exacts(NamedTuple): - """A system's expected values over a window, worked out exactly (see - ``RepairableRBD._twin_exact``): its mean availability and, if it is - priced, its expected cost, with that cost's split (see - ``ExpectedCost``).""" - - availability: float - cost: Optional[float] = None - breakdown: Optional[_Breakdown] = None - - -class _ModuleRun: - """A system's dependent modules simulated alone, and every other node - taken exactly given their joint states (#189, see ``_conditional``): - the machinery of a conditional run (``RepairableRBD._conditional_run``) - and of a plain run's means taken given its modules - (``RepairableRBD._conditioned_run``). - - The modules (by default ``_conditional_modules``) are simulated as a - diagram of their own (``_modules_rbd``), from their states, whose - streams are named as in the system, so that its simulations are the - system's modules'. ``prepare`` works out the other nodes' own expected - values over the window; ``simulate`` runs a range of simulations, - working out the system given each joint state its modules meet once, - on a grid; and ``result`` and ``means`` give what they make. The - components' curves are built once for the whole run (see - ``_curves``).""" - - def __init__( - self, - rbd: "RepairableRBD", - t_simulation: float, - working: set, - broken: set, - method: str, - entropy: int, - *, - antithetic: bool, - n_jobs: Optional[int], - engine: str, - curve_points: Optional[int], - state=None, - shard_map: Optional[Callable] = None, - shard_size: Optional[int] = None, - demand: Optional[float] = None, - capacities: bool = False, - control_variate: bool = False, - modules: Optional[list] = None, - ) -> None: - T = _check_window(t_simulation) - self.rbd = rbd - self.T = T - self.working, self.broken, self.method = working, broken, method - self.antithetic, self.n_jobs = antithetic, n_jobs - states = rbd._simulation_states(state, working | broken) - if modules is None: - modules = rbd._conditional_modules(working, broken) - self.modules = modules - self.module_states = { - node: states[node] for node in modules if node in states - } - self.entropy = entropy - self.engine = engine - self.sub = rbd._modules_rbd(modules) if modules else None - self.recorder = _CapacityRecorder(rbd, demand) if capacities else None - self.capacity_given: Dict[int, _conditional.CapacityGiven] = {} - self.capacity_parts: List[_conditional.CapacityValues] = [] - self.shards = None - if self.sub is not None and shard_map is not None: - plan, _ = self.sub._stream_plan( - T, entropy, antithetic, self.module_states - ) - template = { - **self.sub._shard_template( - T, - entropy, - set(), - set(), - "p", - antithetic, - None, - self.engine, - self.module_states, - None, - ), - "histories": True, - } - self.shards = ( - shard_map, - template, - self.sub._shard_size(plan, shard_size), - ) - # The exact twin's stand-ins for the modules, with common random - # numbers, and its exact expected values. - self.twin_sub: Optional["RepairableRBD"] = None - self.twin_module_states: dict = {} - self.exacts = _Exacts(0.0) - if control_variate and self.sub is not None: - if self.shards is not None: - raise ValueError( - "A run with control_variate simulates the system's twin " - "alongside it, here: leave out shard_map." - ) - twin, _ = rbd._twin() - self.exacts = rbd._twin_exact( - twin, T, working, broken, method, state - ) - self.twin_sub = twin._modules_rbd(modules) - twin_states = twin._simulation_states(state, working | broken) - self.twin_module_states = { - node: twin_states[node] - for node in modules - if node in twin_states - } - self.twin_uptimes: List[np.ndarray] = [] - self.twin_costs: List[np.ndarray] = [] - self.curve_x = _conditional.steps(curve_points, T) - self.given: Dict[int, _conditional.Given] = {} - self.parts: List[Tuple[int, _conditional.Values]] = [] - self.module_up = np.zeros(len(modules)) - self.priced = rbd.has_costs - self.own_costs: List[np.ndarray] = [] - self.own_categories = dict.fromkeys(_CATEGORIES, 0.0) - self.own_components: Dict[Hashable, float] = {} - self.fixed = set(modules) - # The other nodes' states, as given (the modules are held). - self.other_states = ( - None - if state is None - else { - node: value - for node, value in state.items() - if node not in self.fixed - } - ) - # The rest given the modules, every node the crews serve among them. - self.exact = rbd._crew_free() - self._memo: dict = {} - self.n = 0 - - @contextmanager - def _curves(self): - """Within it, the exact part's curves are shared, as within - ``RepairableRBD._sharing_curves``, and kept from one round of - simulations to the next; out of it, the system holds none, so that - a plain run of it between rounds can go to other processes.""" - exact = self.exact - if exact._curve_memo is not None: - yield - return - exact._curve_memo = self._memo - try: - yield - finally: - exact._curve_memo = None - - def prepare(self) -> None: - """The other nodes' own expected down times and costs over the - window, which do not depend on the modules, and the grid the - system given each joint state is worked out on.""" - exact, T = self.exact, self.T - with self._curves(): - _, _, curves, self.totals, _, _ = exact._window( - np.array([T]), - self.working | self.fixed, - self.broken, - self.method, - nodes=True, - state=self.other_states, - ) - self.exact_cost = ( - exact.expected_cost( - T, - self.working | self.fixed, - self.broken, - self.method, - state=self.other_states, - ) - if self.priced - else None - ) - self.others = ( - 0.0 - if self.exact_cost is None - else math.fsum( - float(np.ravel(value)[0]) - for key, value in self.exact_cost.by_category.items() - if key != "system_downtime" - ) - ) - self.x = _conditional.grid( - T, [curve_breaks(curve, 0.0, T) for curve in curves.values()] - ) - - def _given(self, paths: "_conditional.Paths") -> None: - """The system given each joint state ``paths`` meet, not worked - out yet.""" - new = [ - state for state in paths.states.tolist() if state not in self.given - ] - if not new: - return - with self._curves(): - for state in new: - self.given[state] = self.exact._conditional_given( - self.modules, - state, - self.working, - self.broken, - self.method, - self.x, - self.other_states, - ) - if self.recorder is not None: - self.capacity_given[state] = self.exact._capacity_given( - self.modules, - state, - self.working, - self.broken, - self.x, - self.other_states, - self.recorder.demand, - ) - - def simulate(self, first: int, count: int) -> None: - """Simulations ``first`` to ``first + count - 1``: the modules' - histories and own costs, and each simulation's expected values - given them (and its twin's, if controlled).""" - T, modules = self.T, self.modules - histories: List[Any] = [] - simulated = np.zeros(count) - if self.shards is not None: - histories, simulated, spent_on, by_module = _module_shards( - *self.shards, first, count, len(modules) - ) - for key, spent in spent_on.items(): - self.own_categories[key] += spent - for node, spent in zip(modules, by_module): - if spent is not None: - self.own_components[node] = ( - self.own_components.get(node, 0.0) + spent - ) - elif self.sub is not None: - # The modules' histories, and their own costs in the same - # simulations. - data, tally = _timeline_runs.with_costs( - self.sub, - T, - count, - None, - self.antithetic, - self.engine, - self.n_jobs, - first, - self.module_states, - entropy=self.entropy, - common=self.twin_sub is not None, - ) - histories = [data[node] for node in modules] - if self.sub.has_costs: - tally._fold() - simulated = np.asarray(tally.cost_samples, dtype=float) - for key, spent in tally.cost_by_category.items(): - self.own_categories[key] += float(spent) - for node, spent in tally.cost_by_component.items(): - self.own_components[node] = self.own_components.get( - node, 0.0 - ) + float(spent) - paths = _conditional.paths(histories, count, T) - self._given(paths) - self.parts.append( - (count, _conditional.values(paths, self.given, self.curve_x)) - ) - if self.recorder is not None: - self.capacity_parts.append( - _conditional.capacity_values( - paths, self.capacity_given, self.curve_x - ) - ) - if self.twin_sub is not None: - # The twin's modules in the same simulations: the other nodes - # are the same, and so is the system given them. - data, tally = _timeline_runs.with_costs( - self.twin_sub, - T, - count, - None, - self.antithetic, - self.engine, - self.n_jobs, - first, - self.twin_module_states, - entropy=self.entropy, - common=self.twin_sub is not None, - ) - twin_paths = _conditional.paths( - [data[node] for node in modules], count, T - ) - self._given(twin_paths) - twin_uptime = _conditional.values( - twin_paths, self.given, self.curve_x - ).uptime - self.twin_uptimes.append(twin_uptime) - spent = np.zeros(count) - if self.twin_sub.has_costs: - tally._fold() - spent = np.asarray(tally.cost_samples, dtype=float) - self.twin_costs.append( - spent - + self.others - + self.rbd.downtime_cost_rate * (T - twin_uptime) - ) - self.module_up += _conditional.module_totals(histories, count, T)[0] - self.own_costs.append(simulated) - self.n = first + count - - def uptimes(self) -> np.ndarray: - """Each simulation's expected up time given its modules.""" - return np.concatenate([values.uptime for _, values in self.parts]) - - def costs(self) -> np.ndarray: - """Each simulation's expected cost given its modules: their own - costs, the other nodes' expected costs, and the system's expected - down time's.""" - return ( - np.concatenate(self.own_costs) - + self.others - + self.rbd.downtime_cost_rate * (self.T - self.uptimes()) - ) - - def breakdown(self) -> _Breakdown: - """The expected cost of a window given the modules' histories (the - mean of ``costs``), by category and by component: the modules' own - costs in the simulations, the other nodes' expected costs, and the - system's expected down time's.""" - n, exact_cost = self.n, self.exact_cost - assert exact_cost is not None - by_category = { - key: self.own_categories[key] / n - + ( - 0.0 - if key == "system_downtime" - else float(np.ravel(exact_cost.by_category.get(key, 0.0))[0]) - ) - for key in _CATEGORIES - } - by_category["system_downtime"] = float( - np.mean(self.rbd.downtime_cost_rate * (self.T - self.uptimes())) - ) - by_component = { - node: float(np.ravel(value)[0]) - for node, value in exact_cost.by_component.items() - if node not in self.fixed - } - for node, amount in self.own_components.items(): - by_component[node] = amount / n - return by_category, by_component - - def controls( - self, - ) -> Tuple[Optional[ControlVariate], Optional[ControlVariate]]: - """The controls of the fractions up and of the costs, if the run is - controlled (the latter if priced).""" - if self.twin_sub is None: - return None, None - control = ControlVariate.of( - self.uptimes() / self.T, - np.concatenate(self.twin_uptimes) / self.T, - self.exacts[0], - self.antithetic, - ) - cost_control = None - if self.priced and self.exacts[1] is not None: - cost_control = ControlVariate.of( - self.costs(), - np.concatenate(self.twin_costs), - self.exacts[1], - self.antithetic, - ) - return control, cost_control - - def judged(self, target: str) -> np.ndarray: - """The values a run to a tolerance judges: of the ``target``'s mean - (see ``_stopping_rule``), controlled if the run is.""" - control, cost_control = self.controls() - if target == "cost": - values = self.costs() - return ( - values - if cost_control is None - else cost_control.controlled(values) - ) - values = self.uptimes() / self.T - return values if control is None else control.controlled(values) - - def unjudged(self) -> Optional[str]: - """Why the spread of the simulations' expected values given the - modules does not show the error (#215): they never changed state, - so their outages were not sampled, and every simulation's values - are the same. None once they have, or with no modules, when the - values are exact.""" - if len(self.given) > 1 or not self.modules: - return None - return ( - f"The modules {list(self.modules)} never changed state in the " - f"{self.n} simulations, so their outages were not sampled: run " - "more simulations to sample them." - ) - - def _warn_unjudged(self, instead: str) -> None: - reason = self.unjudged() - if reason is not None: - warnings.warn( - f"{reason} {instead}", - RuntimeWarning, - stacklevel=outside_level(), - ) - - def split(self, record: ConditionalRun) -> Optional[_Breakdown]: - """The expected cost's split a plain run takes with ``record``, its - means given the modules (see ``means``): ``breakdown``, where the - cost's mean is taken given them (priced, and the modules changed - state, see ``ConditionalRun.informative``); else None, and the - split is the simulations' own.""" - if not self.priced or not record.informative: - return None - return self.breakdown() - - def means(self) -> ConditionalRun: - """The record of a plain run whose means are taken given its - modules (see ``RepairableRBD._conditioned_run``).""" - self._warn_unjudged( - "The means and their intervals are the simulations' own " - "(method='simulated')." - ) - return ConditionalRun( - tuple(self.modules), - len(self.given), - whole=True, - uptimes=self.uptimes(), - costs=self.costs() if self.priced else None, - ) - - def result(self) -> AvailabilityResult: - """The conditional run's result (see - ``RepairableRBD._conditional_run``).""" - self._warn_unjudged( - "The mean intervals have no error to give (nan): a plain run " - "(conditional=False) gives one." - ) - rbd, T, n = self.rbd, self.T, self.n - parts, curve_x = self.parts, self.curve_x - uptimes = self.uptimes() - failures = np.concatenate([v.failures for _, v in parts]) - planned = np.concatenate([v.planned for _, v in parts]) - restorations = np.concatenate([v.restorations for _, v in parts]) - curve = np.zeros(curve_x.size) - square = np.zeros(curve_x.size) - for size, part in parts: - curve += size * part.curve / n - square += size * part.curve_square / n - down = self.totals["downtime"] - node_downtime: Dict[Hashable, float] = {} - for node in rbd.components: - if node in self.working: - node_downtime[node] = 0.0 - elif node in self.broken: - node_downtime[node] = n * T - elif node in self.fixed: - j = self.modules.index(node) - node_downtime[node] = n * T - float(self.module_up[j]) - else: - node_downtime[node] = n * float(np.ravel(down[node])[0]) - node_uptime = { - node: n * T - value for node, value in node_downtime.items() - } - record = ConditionalRun(tuple(self.modules), len(self.given), square) - control, cost_control = self.controls() - cost_result = _acquisition_only(rbd, n, T, self.antithetic) - if self.priced: - by_category, by_component = self.breakdown() - cost_result = CostResult( - samples=self.costs(), - t_simulation=T, - n_simulations=n, - acquisition_cost=rbd.acquisition_cost, - by_category=by_category, - by_component=by_component, - antithetic=self.antithetic, - conditional=record, - control_variate=cost_control, - ) - capacity_fields: dict = {} - if self.recorder is not None: - time_at: Dict[float, float] = {} - for piece in self.capacity_parts: - for level, spent in piece.time_at.items(): - time_at[level] = time_at.get(level, 0.0) + spent - mean = np.sum( - [piece.curve for piece in self.capacity_parts], axis=0 - ) - unlimited = np.sum( - [piece.unlimited for piece in self.capacity_parts], axis=0 - ) - demand = self.recorder.demand - capacity_fields = dict( - capacity_timeline=curve_x.copy(), - capacity=np.where(unlimited > 0, np.inf, mean / n), - capacity_time={ - level: time_at[level] for level in sorted(time_at) - }, - demand=demand, - delivered=( - None - if demand is None - else np.concatenate( - [piece.delivered for piece in self.capacity_parts] - ) - / (demand * T) - ), - ) - return AvailabilityResult( - timeline=curve_x, - availability=curve, - system_uptime=float(math.fsum(uptimes)), - time_simulated_to=T, - criticalities=None, - node_uptime=node_uptime, - node_downtime=node_downtime, - system_downtime=float(n * T - math.fsum(uptimes)), - system_failures=float(math.fsum(failures)), - system_restorations=float(math.fsum(restorations)), - n_simulations=n, - cost=cost_result, - system_planned_outages=float(math.fsum(planned)), - uptimes=uptimes, - antithetic=self.antithetic, - conditional=record, - control_variate=control, - **capacity_fields, - ) - - -def _exact_controls( - tally: "_Tally", - t_simulation: float, - exacts: _Exacts, - antithetic: bool, -) -> Tuple[ControlVariate, Optional[ControlVariate]]: - """The controls of a run's fractions up and costs by the system itself, - whose expected values the exact methods work out (#187): ``exacts``, - its mean availability over the window and its expected cost (None - unpriced). Its intervals are then those values, with no error (see - ``ControlVariate.itself``).""" - exact, exact_cost = exacts.availability, exacts.cost - fractions = np.asarray(tally.uptimes, dtype=float) / t_simulation - control = ControlVariate.of( - fractions, fractions, exact, antithetic, itself=True - ) - cost_control = None - if exact_cost is not None: - costs = np.asarray(tally.cost_samples, dtype=float) - cost_control = ControlVariate.of( - costs, costs, exact_cost, antithetic, itself=True - ) - return control, cost_control - - -def _chunk_means(control_variate, conditional) -> Dict[str, Optional[bool]]: - """A chunk's (or a merge's) ``control_variate`` and ``conditional``, - checked (#236): None, the means ``availability`` takes by default, or - False, the simulations' own; a chunk holds the system's simulations - alone, not a twin's or its modules' alone, which True would take.""" - out: Dict[str, Optional[bool]] = {} - for name, value, alone in ( - ("control_variate", control_variate, "its exact twin's"), - ("conditional", conditional, "its modules' alone"), - ): - if isinstance(value, (bool, np.bool_)) and not value: - value = False - elif value is not None: - raise ValueError( - f"{name} must be None or False for chunks, got {value!r}: a " - f"chunk holds the system's simulations, not {alone}, which " - f"{name}=True simulates. Run the whole run with " - "availability() for that." - ) - out[name] = value - return out - - -def _acquisition_only( - rbd: "RepairableRBD", N: int, t_simulation: float, antithetic: bool -) -> Optional[CostResult]: - """The cost result of ``N`` simulations of ``rbd`` when nothing but its - ``acquisition_cost`` is priced (#234): no running cost, exactly (its - mean interval ``"exact"``), with the acquisition beside it; None when - nothing at all is.""" - if not rbd.acquisition_cost: - return None - zeros = np.zeros(N) - return CostResult( - samples=zeros, - t_simulation=t_simulation, - n_simulations=N, - acquisition_cost=rbd.acquisition_cost, - by_category={category: 0.0 for category in _CATEGORIES}, - by_component={}, - antithetic=antithetic, - control_variate=ControlVariate( - twin=zeros, - exact=0.0, - coefficient=0.0, - correlation=0.0, - itself=True, - ), - ) - - -def _shard_bytes(template: dict, start: int, stop: int) -> bytes: - """A shard (see ``RepairableRBD.shards``): the run's ``template`` with - the range of its simulations, as JSON.""" - return json.dumps({**template, "start": start, "stop": stop}).encode() - - -def _states_from_key(rbd, key: Optional[str]) -> dict: - """The components' states ``_state_key`` saved, for ``rbd`` (a copy of - the system the key was made for, as a shard rebuilds it): each node by - its ``repr``, a nested RBD's own states in turn.""" - - def states(rbd, entries: list) -> dict: - nodes = {repr(node): node for node in rbd.components} - out: dict = {} - for name, value in entries: - node = nodes[name] - if isinstance(value, dict): - out[node] = NodeState(**value) - else: - out[node] = states(rbd.components[node], value) - return out - - return {} if key is None else states(rbd, json.loads(key)) - - -def _draws_start(start) -> bool: - """Whether a component started from ``start`` (a ``NodeState``, or - None for new) draws what is left of its life, repair or maintenance: - when it is down, or up at an age. Up at age 0, it is new, whatever its - phase.""" - return isinstance(start, NodeState) and (not start.alive or start.age > 0) - - -def _standby_draws(component, run, path: tuple, arrangement: "_Standby"): - """A standby group's draws in a run (see ``_StandbyDraws``): from - streams if every unit's lives and repairs have one.""" - units = range(arrangement.units) - lives = [run.stream(path + (u,), _streams.FAILURE) for u in units] - repairs = [run.stream(path + (u,), _streams.REPAIR) for u in units] - if any(stream is None for stream in lives + repairs): - return _StandbyDraws(component) - return _StandbyDraws( - component, lives, repairs, run.stream(path, _streams.SWITCH) - ) -# The kinds of a standby group's own events. -_UNIT_FAILS, _SPARE_FAILS, _UNIT_REPAIRED = 0, 1, 2 +class RepairableRBD(RBD): + """A reliability block diagram of repairable components. + Each component alternates between working and failed: it fails after a + time drawn from its reliability model, and is then repaired, as good + as new, after a time drawn from its repairability model. Components fail + and are repaired independently of one another and of the system state + (unless ``repair_crews`` makes them wait for a crew), and a component + keeps running (and can fail) while the system is down. The system is up + whenever its working components connect the input node to the output + node. -class _StandbyGroup: - """A standby group's units during one simulation (a component spec's - ``"standby"``; see ``RepairableRBD``). + Two kinds of analysis are offered: - Each unit put into service as new draws a life, used up at rate 1 while - it operates and at ``dormancy_factor`` while it waits as a spare (the - cumulative-exposure model of ``StandbyModel``): a spare that uses it up - fails in standby, and is found at once. A failed unit's repair is a job - for the RBD's repair crews, under the key ``_Unit(node, unit)`` (see - ``_Crews``), and its repair time is drawn when it fails. When an - operating unit fails and a spare is waiting, the one that has waited - longest is switched in, which works with ``switching_probability``; a - failed switch leaves the position empty until a repaired unit fills it, - and the spare waits on. A repaired unit fills an empty position, or - joins the spares. The group is up while ``k`` units operate. + - Exact long-run (steady-state) metrics, built from each component's + availability ``MTTF / (MTTF + MTTR)`` and failure frequency + ``1 / (MTTF + MTTR)``: ``mean_availability``, ``node_availability``, + ``system_failure_frequency``, ``mean_up_time``, ``mean_down_time``, + ``mean_time_between_failures``, ``expected_cost_rate``, + ``capacity_distribution`` (with node capacities) and the importance + measures. A component under age or block replacement enters through + its renewal cycle instead (see ``node_availability``). With fewer + ``repair_crews`` than components, the long-run values come from the + Markov chain of the components and the repair queue, for exponential + components (see ``mean_availability``). + - Monte-Carlo simulation of a finite window ``[0, t_simulation]`` that + starts with every component working: ``availability`` (availability + over time, criticality measures and, with node capacities, the + capacity over time and the fraction of a demand delivered) and + ``cost`` (the distribution of the window's cost). - The group keeps its units' events in a queue of its own and has one - event in the RBD's queue, ``entry``: its next. The RBD passes that - event to ``advance``, which returns the group's state after it. - """ + Times are in the time unit of the component models, and costs in the + currency they are given in. - __slots__ = ( - "_rbd", - "_node", - "_k", - "_dormancy", - "_switching", - "_draws", - "_end", - "_operating", - "_spares", - "_life", - "_since", - "_version", - "_events", - "_order", - "entry", - "up", - ) - - def __init__( - self, - rbd: "RepairableRBD", - node, - arrangement: "_Standby", - draws, - t_simulation: float, - ): - self._rbd = rbd - self._node = node - self._k = arrangement.k - self._dormancy = arrangement.dormancy_factor - self._switching = arrangement.switching_probability - self._draws = ( - draws if isinstance(draws, _StandbyDraws) else _StandbyDraws(draws) - ) - self._end = t_simulation - self._operating: set = set() - self._spares: deque = deque() - # Each unit's life left, as of ``_since``; and a count that a - # queued event of the unit's must match to be current. - self._life = [0.0] * arrangement.units - self._since = [0.0] * arrangement.units - self._version = [0] * arrangement.units - self._events: list = [] - self._order = 0 - self.entry: Optional["Event"] = None - self.up = True - for unit in range(arrangement.units): - self._life[unit] = self._draws.life(unit) - if unit < self._k: - self._operate(unit, 0.0) - else: - self._wait(unit, 0.0) - self._arm() - - def _queue(self, time: float, kind: int, unit: int) -> None: - heapq.heappush( - self._events, - (time, self._order, kind, unit, self._version[unit]), - ) - self._order += 1 - - def _operate(self, unit: int, t: float) -> None: - self._operating.add(unit) - self._since[unit] = t - self._queue(t + self._life[unit], _UNIT_FAILS, unit) - - def _wait(self, unit: int, t: float) -> None: - self._spares.append(unit) - self._since[unit] = t - if self._dormancy > 0.0: - self._queue( - t + self._life[unit] / self._dormancy, _SPARE_FAILS, unit - ) - - def _repair(self, unit: int, t: float) -> None: - """Unit ``unit`` has failed at ``t``: its repair, drawn now, is a - job for a crew.""" - self._version[unit] += 1 - ends = t + self._draws.repair(unit) - crews = self._rbd._crews - if crews is None or crews.request( - _Unit(self._node, unit), t, Event(ends, None, True) - ): - self._queue(ends, _UNIT_REPAIRED, unit) - - def crew_started(self, unit: int, ends: float) -> None: - """A crew has started unit ``unit``'s repair late: it ends at - ``ends``.""" - self._queue(ends, _UNIT_REPAIRED, unit) - if self.entry is None or ends < self.entry.time: - self._arm() - - def _arm(self) -> None: - """Queue the group's next event in the RBD's queue (an earlier one - already there is left, and ignored).""" - events, version = self._events, self._version - while events and events[0][4] != version[events[0][3]]: - heapq.heappop(events) - if not events or events[0][0] >= self._end: - self.entry = None - return - self.entry = Event(events[0][0], self._node, self.up) - self._rbd._event_queue.put(self.entry) - - def holds(self, event: "Event") -> bool: - """Whether ``event``, from the RBD's queue, is the group's current - one (not superseded by an earlier).""" - return event is self.entry - - def advance(self, t: float) -> Tuple[bool, int]: - """Take the group's event at ``t`` (its ``entry``): whether it is up - after it, and how many of its units failed (each is repaired).""" - self.entry = None - events, version = self._events, self._version - while events[0][4] != version[events[0][3]]: - heapq.heappop(events) - _, _, kind, unit, _ = heapq.heappop(events) - failures = 0 - if kind == _UNIT_REPAIRED: - crews = self._rbd._crews - if crews is not None: - started = crews.release(_Unit(self._node, unit), t) - if started is not None: - self._rbd._crew_started(started) - self._life[unit] = self._draws.life(unit) - if len(self._operating) < self._k: - self._operate(unit, t) - else: - self._wait(unit, t) - elif kind == _SPARE_FAILS: - self._spares.remove(unit) - failures = 1 - self._repair(unit, t) - else: - self._operating.discard(unit) - failures = 1 - self._repair(unit, t) - if self._spares: - spare = self._spares.popleft() - switching = self._switching - if switching >= 1.0 or ( - switching > 0.0 and self._draws.switch() < switching - ): - # It has used up its life at the dormant rate so far (all - # of it, at most: a spare due to fail in standby now). - self._version[spare] += 1 - used = self._dormancy * (t - self._since[spare]) - self._life[spare] = max(self._life[spare] - used, 0.0) - self._operate(spare, t) - else: - self._spares.appendleft(spare) - self.up = len(self._operating) == self._k - self._arm() - return self.up, failures - - -class _Crews: - """The repair crews of one simulation (``RepairableRBD``'s - ``repair_crews``): how many are free, the components they are working - on, and the jobs waiting for one. - - A job is the work that brings a component back up: a repair or - replacement, maintenance that takes time, or a test that takes time. - The component is down from when the job fell due. A job that finds no - crew free waits, and the next crew to finish starts the waiting job of - the highest priority, and of those the one that fell due first. The - job's length was drawn when it fell due, so waiting shifts the - component's restoration without changing its draws. - """ - - __slots__ = ("free", "served", "holding", "waiting", "_order", "_rank") - - def __init__(self, crews: int, served, priority: dict): - self.free = crews - self.served = frozenset(served) - self.holding: set = set() - self.waiting: list = [] - self._order = 0 - # Higher priority first; a standby group's units have the group's. - self._rank = { - key: -priority.get( - key.node if isinstance(key, _Unit) else key, 0.0 - ) - for key in served - } - - def request(self, node, due: float, done: "Event") -> Optional["Event"]: - """``node``'s job, due at ``due`` and ending with ``done`` if a crew - starts it at once: ``done``, or None while the job waits.""" - if self.free: - self.free -= 1 - self.holding.add(node) - return done - entry = (self._rank[node], due, self._order, node, done) - heapq.heappush(self.waiting, entry) - self._order += 1 - return None - - def release(self, node, t: float) -> Optional[Tuple[Any, float, "Event"]]: - """``node``'s job is done at ``t``: its crew starts the next waiting - job, if any. That job's component, how long it waited, and the event - that now ends it; or None.""" - self.holding.discard(node) - if not self.waiting: - self.free += 1 - return None - _, due, _, waiting, done = heapq.heappop(self.waiting) - self.holding.add(waiting) - wait = t - due - ends = Event( - done.time + wait, - waiting, - done.status, - done.preventive, - done.inspection, - ) - return waiting, wait, ends - - -class _Fixed: - """A cost charged at the same amount every time.""" - - __slots__ = ("amount",) - - def __init__(self, amount: float): - self.amount = amount - - def draw(self) -> float: - return self.amount - - -#: Simulations per task of a parallel run of the Python engine (see -#: ``RepairableRBD``'s ``availability``). -PARALLEL_BLOCK = 250 - -#: The cost categories, in the order ``CostResult.by_category`` lists them. -_CATEGORIES = ( - "repair", - "replace", - "preventive", - "inspection", - "component_downtime", - "system_downtime", - "setup", -) -# The category of a repair's own cost (all an imperfect repair is charged). -_REPAIR = _CATEGORIES.index("repair") - - -class _Replication: - """One simulation's results, which a :class:`_Tally` adds to its totals - in the order of the simulations. - - ``node_up``, ``both_up`` and ``both_down`` hold each component's up - time, and the time it and the system were both up and both down; - ``counts`` each component's failures, the system failures they caused, - its restorations and the system restorations they caused; and - ``changes`` the times the system changed state, with ``deltas`` +1 for - a restoration and -1 for a failure or planned outage. ``cost`` is the - simulation's total cost (None when nothing is priced), ``by_category`` - and ``by_node`` its breakdown, and the ``capacity`` fields what the - capacity trace recorded (see ``_CapacityTrace``). ``replacements`` - counts each component's replacements (its failures, found ones for a - component with hidden failures, its preventive replacements, and a - standby group's units' failures): the spares it used. - """ - - __slots__ = ( - "uptime", - "node_up", - "both_up", - "both_down", - "counts", - "failures", - "restorations", - "planned", - "changes", - "deltas", - "cost", - "by_category", - "by_node", - "capacity_changes", - "capacity_time", - "delivered", - "replacements", - "opportunistic", - "history", - ) - uptime: float - node_up: List[float] - both_up: List[float] - both_down: List[float] - counts: Tuple[List[int], List[int], List[int], List[int]] - failures: int - restorations: int - planned: int - changes: List[float] - deltas: List[int] - cost: Optional[float] - by_category: List[float] - by_node: Dict[Any, float] - capacity_changes: Optional[list] - capacity_time: Optional[list] - delivered: Optional[float] - replacements: List[int] - opportunistic: List[int] - history: Optional[tuple] - - -def _net_by_time( - times: np.ndarray, steps: np.ndarray -) -> Tuple[np.ndarray, np.ndarray]: - """Changes of +1 or -1 (``steps``) at ``times`` (none negative), netted - by time: the distinct times, in order, and the net change at each. - - A time is never negative, so its sign bit is free: each change is one - integer, its time's bits above whether it is +1, and one sort of the - integers puts the changes in time order (``-0.0`` sorts as ``0.0``).""" - keys = times.view(np.uint64) << np.uint64(1) - keys |= steps > 0 - keys.sort() - if not keys.size: - return np.zeros(0), np.zeros(0, dtype=np.int64) - at = keys >> np.uint64(1) - new = np.empty(keys.size, dtype=bool) - new[0] = True - np.not_equal(at[1:], at[:-1], out=new[1:]) - firsts = np.flatnonzero(new) - ups = np.add.reduceat((keys & np.uint64(1)).astype(np.int64), firsts) - sizes = np.diff(np.append(firsts, keys.size)) - return at[firsts].view(np.float64), 2 * ups - sizes - - -def _working_over_time( - changed_at: np.ndarray, - deltas: np.ndarray, - t_end: float, - start: int, -) -> Tuple[np.ndarray, np.ndarray]: - """How many simulated systems work after each time at which one changed - state (and at 0 and ``t_end`` whether or not any did): the times, in - order, and the counts, ``start`` working at 0. Each change is +1 or -1 - (``deltas``); the counts are whole numbers added exactly.""" - times, net = _net_by_time(changed_at, deltas) - time, weights = _with_ends(times, net.astype(float), t_end) - weights[0] += start - return time, weights.cumsum() - - -def _with_ends( - times: np.ndarray, values: np.ndarray, t_end: float -) -> Tuple[np.ndarray, np.ndarray]: - """``times`` (in order, each once) with 0 and ``t_end`` among them, and - ``values`` at their times (0 at an end that had none).""" - for end in (0.0, t_end): - i = int(np.searchsorted(times, end)) - if i == times.size or times[i] != end: - times = np.insert(times, i, end) - values = np.insert(values, i, 0) - return times, values - - -def _add_at(totals: dict, key, value) -> None: - """Add ``value`` (a float or an ``ExactSum``) to ``totals[key]``, - exactly: a float while only one value has come, an ``ExactSum`` once - another does.""" - held = totals.get(key) - if held is None: - totals[key] = ExactSum(value) if isinstance(value, ExactSum) else value - elif isinstance(held, ExactSum): - held.add(value) - else: - totals[key] = ExactSum((held,)).add(value) - - -def _partials(value) -> list: - """The floats whose exact sum is ``value`` (see ``_add_at``).""" - return value.partials if isinstance(value, ExactSum) else [value] - - -def _by_time( - times: np.ndarray, values: np.ndarray -) -> Tuple[np.ndarray, np.ndarray, np.ndarray]: - """``values`` grouped by their ``times``: the distinct times, in order; - the values, in the order of their times; and where each time's values - start. A time's values are in no particular order: what is made of - them (``_group_totals``, ``_group_partials``) is exact. - - Each (time, value) pair is a complex number, which numpy sorts by its - real part, then its imaginary one: one sort of the pairs.""" - pairs = np.empty(times.size, dtype=complex) - pairs.real = times - pairs.imag = values - pairs.sort() - times, values = pairs.real, pairs.imag - new = np.ones(times.size, dtype=bool) - new[1:] = times[1:] != times[:-1] - starts = np.flatnonzero(new) - return times[starts], values, starts - - -def _zeroed(times: np.ndarray) -> np.ndarray: - """``times``, a new array, with ``-0.0`` made ``0.0`` (``+ 0.0``, in - place).""" - return np.add(times, 0.0, out=times) - - -def _capacity_totals( - at: np.ndarray, - change: np.ndarray, - free_at: np.ndarray, - counts: np.ndarray, - t_end: float, -) -> Tuple[np.ndarray, np.ndarray, np.ndarray]: - """The times at which a simulated system's expected capacity (by - ``change``, at the times ``at``) or whether it can carry an unlimited - amount (by the net ``counts`` of systems that can, at ``free_at``) - changed, and 0 and ``t_end``: in order; the systems' total expected - capacity after each (the running total of the changes, as - ``np.cumsum`` adds them); and whether any can carry an unlimited amount - then. ``at`` and ``free_at`` are in order, each time once (as - ``_by_time`` gives them). - - The two are merged as ``_net_by_time`` merges its changes, each time's - bits above whether it is ``free_at``'s: the merged times of each keep - their order, so their changes follow them in turn.""" - keys = np.concatenate( - ( - at.view(np.uint64) << np.uint64(1), - (free_at.view(np.uint64) << np.uint64(1)) | np.uint64(1), - ) - ) - keys.sort() - bits = keys >> np.uint64(1) - new = np.ones(keys.size, dtype=bool) - new[1:] = bits[1:] != bits[:-1] - group = np.cumsum(new) - 1 - free = (keys & np.uint64(1)).astype(bool) - times = bits[new].view(np.float64) - step = np.zeros(times.size) - step[group[~free]] = change - freed = np.zeros(times.size, dtype=np.int64) - freed[group[free]] = counts - times, step = _with_ends(times, step, t_end) - _, freed = _with_ends(bits[new].view(np.float64), freed, t_end) - return times, np.cumsum(step), np.cumsum(freed) > 0 - - -def _group_stops(values: np.ndarray, starts: np.ndarray) -> np.ndarray: - """Where each group of ``values`` that starts at one of ``starts`` - stops.""" - stops = np.empty_like(starts) - stops[:-1] = starts[1:] - stops[-1:] = values.size - return stops - - -def _group_totals(values: np.ndarray, starts: np.ndarray) -> np.ndarray: - """The exact sum of each group of ``values`` (from each of ``starts`` - to the next), rounded once: a group of one value, the value.""" - totals = values[starts] - several = np.flatnonzero(_group_stops(values, starts) - starts > 1) - for i in several.tolist(): - stop = starts[i + 1] if i + 1 < starts.size else values.size - totals[i] = float(ExactSum(values[starts[i] : stop])) # noqa: E203 - return totals - - -def _group_partials(values: np.ndarray, starts: np.ndarray) -> list: - """For each group of ``values`` (from each of ``starts`` to the next), - the floats whose exact sum is the group's (``ExactSum.partials``: none - for a sum of 0).""" - stops = _group_stops(values, starts) - out = [] - for a, b, first in zip( - starts.tolist(), stops.tolist(), values[starts].tolist() - ): - if b - a == 1: - out.append([first] if first else []) - else: - out.append(ExactSum(values[a:b]).partials) - return out - - -def _exact_total(data) -> ExactSum: - """An exact total from ``_Tally.to_dict``'s data: the floats whose - exact sum it is (or a number).""" - if isinstance(data, (int, float)): - return ExactSum(float(data)) - return ExactSum([float(v) for v in data]) - - -class _Tally: - """The running totals of an availability simulation. - - Every total that is a sum of floats is kept exactly, as an ``ExactSum`` - (see ``_exact``), and rounded once when the result is built (#151): so - the totals are the same to the last bit however the simulations ran, - one after another in this process, in blocks in other processes, - compiled, or in chunks merged later. Per-node totals are lists in the - order of ``nodes``; the per-simulation values (``uptimes``, - ``cost_samples``, ``delivered``) are kept in the simulations' order. - """ - - #: Simulations whose per-node values wait to be folded into the - #: per-node totals (see ``_fold``). - _ROWS = 256 - - def __init__( - self, - nodes: list, - costs: dict, - t_simulation: float, - curve_points: Optional[int] = None, - ): - n = len(nodes) - self.nodes = nodes - self.t_simulation = t_simulation - self.n = 0 - # The times the simulated systems changed state, and +1 or -1; and - # the same, as arrays, from the compiled engine. - self.changes: list = [] - self.deltas: list = [] - self.change_arrays: list = [] - # With curve_points (#153), the changes are counted on a grid of - # times instead of kept: the net change in how many systems work - # in each bin up to each grid time (from just after the one - # before), the grid's first time 0. - self.curve_points = curve_points - self.edges: Optional[np.ndarray] = None - self.binned: Optional[np.ndarray] = None - if curve_points is not None: - steps = np.arange(curve_points + 1) - self.edges = t_simulation * steps / curve_points - self.binned = np.zeros(curve_points + 1, dtype=np.int64) - # Per node: its failures, the system failures they caused, its - # restorations and the system restorations they caused. - self.counts = [[0] * n for _ in range(4)] - self.system_restorations = 0 - self.system_failures = 0 - self.system_planned_outages = 0 - self.system_uptime = ExactSum() - self.system_downtime = ExactSum() - self.node_uptime = [ExactSum() for _ in range(n)] - self.node_downtime = [ExactSum() for _ in range(n)] - self.intersection_uptime = [ExactSum() for _ in range(n)] - self.intersection_downtime = [ExactSum() for _ in range(n)] - self.union_uptime = [ExactSum() for _ in range(n)] - self.union_downtime = [ExactSum() for _ in range(n)] - # Each simulation's up time, its nodes' up times and times up and - # down with the system, and its costs, until they are folded into - # the totals (see _fold). - self._rows: list = [] - self._cost_rows: list = [] - # One system uptime (and, when priced, one total cost) per - # replication, in order. - self.uptimes: List[float] = [] - self.cost_samples: List[float] = [] - self.cost_by_category = {key: ExactSum() for key in _CATEGORIES} - self.cost_by_component = {node: ExactSum() for node in costs} - # With capacities (see _CapacityRecorder): the times a simulated - # system's expected capacity changed and by how much, and the times - # whether it can carry an unlimited amount did (+1 or -1), as lists - # from the Python engine and arrays from the compiled one (see - # capacity_records); the time spent at each capacity (exactly, see - # _add_at); and each replication's delivered fraction of the - # demand, in order. - self.capacity_times: list = [] - self.capacity_steps: list = [] - self.unlimited_times: list = [] - self.unlimited_steps: list = [] - self.capacity_arrays: list = [] - # With curve_points (#190), the changes of the expected capacity are - # counted on the grid instead, as the changes of state are: in each - # bin, exactly (arrays whose exact sum, bin by bin, is the bin's - # total change), and the net change in how many can carry an - # unlimited amount. - self.capacity_binned: Optional[list] = None - self.unlimited_binned: Optional[np.ndarray] = None - if self.edges is not None: - self.capacity_binned = [] - self.unlimited_binned = np.zeros(len(self.edges), dtype=np.int64) - self.capacity_time: dict = {} - self.delivered: List[float] = [] - # Each replication's replacements of each node, when asked for - # (see RepairableRBD.spares_demand). - self.replacements: Optional[List[List[int]]] = None - # Per node: its early renewals at stops of its maintenance group. - self.opportunistic = [0] * n - # The simulations' histories, when asked for (see - # RepairableRBD.simulate_timelines): kept, in order, in place of - # every total. - self.histories: Optional[_timeline_runs.Records] = None - - def add(self, rec: _Replication) -> None: - """Add the next simulation's results.""" - if self.histories is not None: - assert rec.history is not None # recorded (see _replicate) - self.histories.add(rec.history) - self.n += 1 - # Each simulation's cost is kept beside its histories (for a - # conditional run's modules, see _conditional_run). - if rec.cost is not None: - self.cost_samples.append(rec.cost) - self._cost_rows.append( - [*rec.by_category, *rec.by_node.values()] - ) - return - self.n += 1 - self.uptimes.append(rec.uptime) - self.system_failures += rec.failures - self.system_restorations += rec.restorations - self.system_planned_outages += rec.planned - self.changes.extend(rec.changes) - self.deltas.extend(rec.deltas) - self._rows.append( - [rec.uptime, *rec.node_up, *rec.both_up, *rec.both_down] - ) - for totals, counts in zip(self.counts, rec.counts): - for c, count in enumerate(counts): - if count: - totals[c] += count - if rec.cost is not None: - self.cost_samples.append(rec.cost) - self._cost_rows.append([*rec.by_category, *rec.by_node.values()]) - if len(self._rows) >= self._ROWS: - self._fold() - if rec.capacity_changes is not None: - for time, mean, limitless in rec.capacity_changes: - if mean: - self.capacity_times.append(time) - self.capacity_steps.append(mean) - if limitless: - self.unlimited_times.append(time) - self.unlimited_steps.append(limitless) - spent = self.capacity_time - for level, time in rec.capacity_time or (): - _add_at(spent, level, time) - if rec.delivered is not None: - self.delivered.append(rec.delivered) - if self.replacements is not None: - self.replacements.append(rec.replacements) - for c, count in enumerate(rec.opportunistic): - if count: - self.opportunistic[c] += count - - def add_changes(self, times: np.ndarray, deltas: np.ndarray) -> None: - """Add simulations' changes of state (times, and +1 or -1), as an - engine gives them: kept, or counted on the grid.""" - self.change_arrays.append((times, deltas)) - if self.binned is not None: - self._fold_changes() - - def add_capacity(self, arrays: tuple) -> None: - """Add simulations' changes of capacity, as an engine gives them - (see ``capacity_records``): kept, or counted on the grid.""" - self.capacity_arrays.append(arrays) - if self.capacity_binned is not None: - self._fold_capacity() - - def _grid_bins(self, times: np.ndarray) -> np.ndarray: - """The grid's bin of each of ``times``: that of the first grid time - at or after it (``-0.0`` taken as ``0.0``).""" - assert self.edges is not None - return np.minimum( - np.searchsorted(self.edges, times + 0.0, side="left"), - len(self.edges) - 1, - ) - - def _fold_capacity(self) -> None: - """Count the changes of capacity kept so far in the grid's bins, - exactly, and drop them.""" - assert self.capacity_binned is not None - assert self.unlimited_binned is not None - size = len(self.unlimited_binned) - for times, steps, free_times, free_steps in self.capacity_arrays: - self.capacity_binned.extend( - binned_parts(self._grid_bins(times), steps, size) - ) - if len(free_times): - self.unlimited_binned += np.bincount( - self._grid_bins(free_times), - weights=free_steps, - minlength=size, - ).astype(np.int64) - self.capacity_arrays = [] - if len(self.capacity_binned) > 64: - self.capacity_binned = compact_parts(self.capacity_binned, size) - - def _fold_changes(self) -> None: - """Count the changes kept so far in the grid's bins (each time in - the bin of the first grid time at or after it), and drop them.""" - assert self.binned is not None and self.edges is not None - if not (self.changes or self.change_arrays): - return - times, deltas = self.state_changes() - self.changes, self.deltas, self.change_arrays = [], [], [] - bins = np.minimum( - np.searchsorted(self.edges, times, side="left"), - len(self.edges) - 1, - ) - counted = np.bincount(bins, weights=deltas, minlength=len(self.edges)) - self.binned += counted.astype(np.int64) - - def _fold(self) -> None: - """Fold the values of the simulations added since the last fold - into the totals, all at once (see ``fold_columns``), and the - changes into the grid's bins, with ``curve_points``.""" - if self.binned is not None: - self._fold_changes() - if self._rows: - table = np.asarray(self._rows, dtype=float) - self._rows = [] - n = len(self.nodes) - self.fold_columns( - table[:, 0], - table[:, 1 : 1 + n], # noqa: E203 - table[:, 1 + n : 1 + 2 * n], # noqa: E203 - table[:, 1 + 2 * n :], # noqa: E203 - ) - if self._cost_rows: - self.fold_costs(np.asarray(self._cost_rows, dtype=float)) - self._cost_rows = [] - if self.capacity_times or self.unlimited_times: - self.capacity_arrays.append( - ( - np.asarray(self.capacity_times, dtype=float), - np.asarray(self.capacity_steps, dtype=float), - np.asarray(self.unlimited_times, dtype=float), - np.asarray(self.unlimited_steps, dtype=np.int64), - ) - ) - self.capacity_times, self.capacity_steps = [], [] - self.unlimited_times, self.unlimited_steps = [], [] - if self.capacity_binned is not None and self.capacity_arrays: - self._fold_capacity() - - def compact(self) -> "_Tally": - """This tally with its simulations' values folded into the totals - and its changes of state (and of capacity) in arrays: small to send - from a worker process to the parent (see ``_simulate_block``).""" - if self.histories is not None: - self.histories.compact() - if self._cost_rows: - self.fold_costs(np.asarray(self._cost_rows, dtype=float)) - self._cost_rows = [] - return self - self._fold() - if self.changes: - times, deltas = self.state_changes() - self.changes, self.deltas = [], [] - self.change_arrays = [(times, deltas)] - return self - - def fold_columns(self, uptime, up, both_up, both_down) -> None: - """Add simulations' values to the totals, exactly (one row per - simulation): the system's up time (and so its down time), each - node's up time (and down time), and its times up and down with the - system (and so either up or either down).""" - t_end = self.t_simulation - uptime = np.asarray(uptime, dtype=float).reshape(-1, 1) - add_columns( - [ - self.system_uptime, - self.system_downtime, - *self.node_uptime, - *self.node_downtime, - *self.intersection_uptime, - *self.intersection_downtime, - *self.union_uptime, - *self.union_downtime, - ], - np.hstack( - [ - uptime, - t_end - uptime, - up, - t_end - up, - both_up, - both_down, - t_end - both_down, - t_end - both_up, - ] - ), - ) - - def fold_costs(self, table) -> None: - """Add simulations' costs to the totals, exactly: one row per - simulation, its costs by category and then by component (in the - order of ``cost_by_component``).""" - add_columns( - [ - *self.cost_by_category.values(), - *self.cost_by_component.values(), - ], - table, - ) - - def state_changes(self) -> Tuple[np.ndarray, np.ndarray]: - """Every time a simulated system changed state (``-0.0`` made - ``0.0``), and +1 for a restoration or -1 for a failure or planned - outage: new arrays, which the caller may overwrite.""" - times = [np.asarray(self.changes, dtype=float)] - deltas = [np.asarray(self.deltas, dtype=np.int64)] - for chunk_times, chunk_deltas in self.change_arrays: - times.append(chunk_times) - deltas.append(chunk_deltas) - return _zeroed(np.concatenate(times)), np.concatenate(deltas) - - def capacity_records(self) -> Tuple[np.ndarray, ...]: - """Every change of a simulated system's expected capacity, as - times (``-0.0`` made ``0.0``) and changes, whose exact sum at a - time is the total change then; and every change of whether it can - carry an unlimited amount, as times and changes in how many systems - can (see ``capacity_changes``): new arrays, which the caller may - overwrite.""" - times = [np.asarray(self.capacity_times, dtype=float)] - steps = [np.asarray(self.capacity_steps, dtype=float)] - free_times = [np.asarray(self.unlimited_times, dtype=float)] - free_steps = [np.asarray(self.unlimited_steps, dtype=np.int64)] - for arrays in self.capacity_arrays: - for kept, array in zip( - (times, steps, free_times, free_steps), arrays - ): - kept.append(array) - return ( - _zeroed(np.concatenate(times)), - np.concatenate(steps), - _zeroed(np.concatenate(free_times)), - np.concatenate(free_steps), - ) - - def capacity_changes(self) -> Tuple[np.ndarray, ...]: - """The times a simulated system's expected capacity changed, in - order, the changes then, in the order of their times, and where - each time's changes start (their exact sum is the total change - then, see ``_group_totals``); and the times whether it can carry an - unlimited amount changed, in order, and the net change then in how - many can.""" - times, steps, free_times, free_steps = self.capacity_records() - at, steps, starts = _by_time(times, steps) - # Whole numbers, each time's added exactly (as floats, below 2**53). - free_at, free_steps, firsts = _by_time(free_times, free_steps) - counts = ( - np.add.reduceat(free_steps, firsts) if firsts.size else free_steps - ) - return at, steps, starts, free_at, counts.astype(np.int64) - - # The totals that are lists of floats, one per node, and the counts. - _NODE_SUMS = ( - "node_uptime", - "node_downtime", - "intersection_uptime", - "intersection_downtime", - "union_uptime", - "union_downtime", - ) - _COUNTS = ( - "system_restorations", - "system_failures", - "system_planned_outages", - ) - - def merge(self, other: "_Tally") -> None: - """Add ``other``'s simulations, which come after this tally's (see - ``SimulationChunk``). The per-simulation values follow on in order, - and the totals, kept exactly, are those of one run of them all, to - the last bit.""" - if self.histories is not None: - assert other.histories is not None # blocks of one run - self.histories.merge(other.histories) - self.n += other.n - self.cost_samples.extend(other.cost_samples) - for rows in (self, other): - if rows._cost_rows: - rows.fold_costs(np.asarray(rows._cost_rows, dtype=float)) - rows._cost_rows = [] - for key, amount in other.cost_by_category.items(): - self.cost_by_category[key].add(amount) - for node, amount in other.cost_by_component.items(): - self.cost_by_component[node].add(amount) - return - self._fold() - other._fold() - self.n += other.n - self.system_uptime.add(other.system_uptime) - self.system_downtime.add(other.system_downtime) - for name in self._COUNTS: - setattr(self, name, getattr(self, name) + getattr(other, name)) - for name in self._NODE_SUMS: - for mine, theirs in zip(getattr(self, name), getattr(other, name)): - mine.add(theirs) - for totals, counts in zip(self.counts, other.counts): - for c, count in enumerate(counts): - totals[c] += count - for c, count in enumerate(other.opportunistic): - self.opportunistic[c] += count - if self.binned is not None: - assert other.binned is not None # chunks of one run - self.binned += other.binned - else: - times, deltas = other.state_changes() - self.change_arrays.append((times, deltas)) - self.uptimes.extend(other.uptimes) - self.cost_samples.extend(other.cost_samples) - for key, amount in other.cost_by_category.items(): - self.cost_by_category[key].add(amount) - for node, amount in other.cost_by_component.items(): - self.cost_by_component[node].add(amount) - if self.capacity_binned is not None: - # Chunks of one run, on one grid. - assert other.capacity_binned is not None - assert self.unlimited_binned is not None - self.capacity_binned.extend(other.capacity_binned) - self.unlimited_binned += other.unlimited_binned - else: - self.capacity_arrays.append(other.capacity_records()) - for level, time in other.capacity_time.items(): - _add_at(self.capacity_time, level, time) - self.delivered.extend(other.delivered) - if self.replacements is not None: - assert other.replacements is not None # blocks of one run - self.replacements.extend(other.replacements) - - def to_dict(self) -> dict: - """The totals, as JSON data (see ``SimulationChunk.to_dict``): each - exact total as the floats whose exact sum it is.""" - self._fold() - times, deltas = self.state_changes() - at, steps, starts, free_at, counts = self.capacity_changes() - return { - "nodes": list(self.nodes), - "t_simulation": self.t_simulation, - "n": self.n, - "changes": times.tolist(), - "deltas": deltas.tolist(), - "curve_points": self.curve_points, - "binned": None if self.binned is None else self.binned.tolist(), - "counts": [list(counts) for counts in self.counts], - **{name: getattr(self, name) for name in self._COUNTS}, - "system_uptime": self.system_uptime.partials, - "system_downtime": self.system_downtime.partials, - **{ - name: [total.partials for total in getattr(self, name)] - for name in self._NODE_SUMS - }, - "uptimes": list(self.uptimes), - "cost_samples": list(self.cost_samples), - "cost_by_category": { - key: amount.partials - for key, amount in self.cost_by_category.items() - }, - "cost_by_component": [ - [node, amount.partials] - for node, amount in self.cost_by_component.items() - ], - "capacity_changes": list( - zip(at.tolist(), _group_partials(steps, starts)) - ), - "unlimited_changes": list(zip(free_at.tolist(), counts.tolist())), - "capacity_binned": ( - None - if self.capacity_binned is None - or self.unlimited_binned is None - else [ - part.tolist() - for part in compact_parts( - self.capacity_binned, len(self.unlimited_binned) - ) - ] - ), - "unlimited_binned": ( - None - if self.unlimited_binned is None - else self.unlimited_binned.tolist() - ), - "capacity_time": sorted( - (level, _partials(time)) - for level, time in self.capacity_time.items() - ), - "delivered": list(self.delivered), - "opportunistic": list(self.opportunistic), - } - - @classmethod - def from_dict(cls, data: dict) -> "_Tally": - """The totals ``to_dict`` gave (node names as JSON gives them - back).""" - from repyability.rbd.serialisation import _node_name - - nodes = [_node_name(node) for node in data["nodes"]] - components = [ - _node_name(node) for node, _ in data["cost_by_component"] - ] - tally = cls( - nodes, - dict.fromkeys(components), - data["t_simulation"], - data.get("curve_points"), - ) - tally.n = int(data["n"]) - tally.changes = [float(t) for t in data["changes"]] - tally.deltas = [int(d) for d in data["deltas"]] - if data.get("binned") is not None: - tally.binned = np.array(data["binned"], dtype=np.int64) - tally.counts = [[int(c) for c in counts] for counts in data["counts"]] - for name in cls._COUNTS: - setattr(tally, name, int(data[name])) - tally.system_uptime = _exact_total(data["system_uptime"]) - tally.system_downtime = _exact_total(data["system_downtime"]) - for name in cls._NODE_SUMS: - setattr(tally, name, [_exact_total(v) for v in data[name]]) - tally.uptimes = [float(v) for v in data["uptimes"]] - tally.cost_samples = [float(v) for v in data["cost_samples"]] - tally.cost_by_category.update( - { - key: _exact_total(v) - for key, v in data["cost_by_category"].items() - } - ) - tally.cost_by_component = { - _node_name(node): _exact_total(amount) - for node, amount in data["cost_by_component"] - } - for level, value in data["capacity_time"]: - tally.capacity_time[float(level)] = _exact_total(value) - for time, value in data["capacity_changes"]: - # A total change of 0 still marks its time (as one 0.0). - for part in _exact_total(value).partials or [0.0]: - tally.capacity_times.append(float(time)) - tally.capacity_steps.append(part) - for time, change in data["unlimited_changes"]: - tally.unlimited_times.append(float(time)) - tally.unlimited_steps.append(int(change)) - if data.get("capacity_binned") is not None: - tally.capacity_binned = [ - np.array(part, dtype=float) for part in data["capacity_binned"] - ] - if data.get("unlimited_binned") is not None: - tally.unlimited_binned = np.array( - data["unlimited_binned"], dtype=np.int64 - ) - tally.delivered = [float(v) for v in data["delivered"]] - tally.opportunistic = [int(c) for c in data["opportunistic"]] - return tally - - def criticality_counts(self) -> Tuple[dict, dict]: - """The failure and restoration counts behind the criticality - indices: node -> Counter, for each node that failed (FCI) or was - restored (RCI), in the order of the nodes.""" - failed, caused_down, restored, caused_up = self.counts - FCI = { - node: Counter( - component_failures=failed[c], system_failures=caused_down[c] - ) - for c, node in enumerate(self.nodes) - if failed[c] - } - RCI = { - node: Counter( - component_restorations=restored[c], - system_restorations=caused_up[c], - ) - for c, node in enumerate(self.nodes) - if restored[c] - } - return FCI, RCI - - -class _CapacityState(NamedTuple): - """The system's capacity in one state of the simulation (see - ``_CapacityRecorder``).""" - - levels: tuple - probabilities: tuple - # The expected capacity when it is finite (0 otherwise), and whether it - # can be unlimited. - finite_mean: float - unlimited: int - # The expected fraction of the demand met, if there is a demand. - delivered: float - - -class _CapacityRecorder: - """The system's capacity in each state of an availability simulation. - - Given which components are down, the capacity has an exact - distribution: one level, unless some nodes work at several (a node's - capacity levels, given as a dict), which the simulation, drawing only - whether each component is up, then weighs by their probabilities. It is - worked out once per state and kept.""" - - def __init__(self, rbd: "RepairableRBD", demand: Optional[float]): - rbd._require_capacities_given() - self._rbd = rbd - self._nodes = list(rbd.nodes) - self._states: Dict[frozenset, _CapacityState] = {} - # Whether each node works at one level: a state's capacity is then - # one level for sure (see evaluate). - self._single = all( - not isinstance(levels, dict) or len(levels) == 1 - for levels in rbd.capacity.values() - ) - if demand is None: - # Everything up at its highest level: the design capacity. - top = float(self._distribution(frozenset())[0][-1]) - self.demand: Optional[float] = top if np.isfinite(top) else None - else: - demand = float(demand) - if not (np.isfinite(demand) and demand > 0.0): - raise ValueError( - f"demand must be a positive, finite number, got " - f"{demand!r}." - ) - self.demand = demand - - def _distribution(self, down: frozenset) -> Tuple[np.ndarray, np.ndarray]: - arrays = { - node: np.zeros(1) if node in down else np.ones(1) - for node in self._nodes - } - levels, rows = self._rbd._capacity_arrays(arrays, 1) - return levels, rows[:, 0] - - def __call__(self, down: frozenset) -> _CapacityState: - state = self._states.get(down) - if state is None: - state = self._states[down] = self._state(*self._distribution(down)) - return state - - def _state( - self, levels: np.ndarray, probabilities: np.ndarray - ) -> _CapacityState: - """The state whose capacity has ``levels`` with ``probabilities`` - (those that can be reached).""" - finite = np.isfinite(levels) - unlimited = int(np.any(probabilities[~finite] > 0.0)) - delivered = 0.0 - if self.demand is not None: - delivered = ( - float(np.minimum(levels, self.demand) @ probabilities) - / self.demand - ) - return _CapacityState( - tuple(levels.tolist()), - tuple(probabilities.tolist()), - ( - 0.0 - if unlimited - else float(levels[finite] @ probabilities[finite]) - ), - unlimited, - delivered, - ) - - def evaluate(self, components: list, up: np.ndarray) -> list: - """The states with ``components`` up as each row of ``up`` (1 or 0 - for each) says, the others up. When each node works at one level, - they are worked out together, one column of probabilities each: - each probability of each is then a product of 0s and 1s, or a sum - with one 1 at most, so exact, and the same as in the state's own - distribution (``__call__``), whatever else is worked out with it. A - state that does not come out as one level for sure is worked out - on its own all the same.""" - states: list = [None] * len(up) - if self._single and len(up) > 1: - ones = np.ones(len(up)) - given = dict(zip(components, np.asarray(up, dtype=float).T)) - levels, rows = self._rbd._capacity_arrays( - {node: given.get(node, ones) for node in self._nodes}, - len(up), - ) - top = rows.argmax(axis=0) - sure = (np.count_nonzero(rows, axis=0) == 1) & ( - rows[top, np.arange(len(up))] == 1.0 - ) - # A state of one level for sure is its level's. - alone: dict = {} - for i in np.flatnonzero(sure).tolist(): - level = int(top[i]) - state = alone.get(level) - if state is None: - state = alone[level] = self._state( - levels[level : level + 1], np.ones(1) # noqa: E203 - ) - states[i] = state - for i, state in enumerate(states): - if state is None: - states[i] = self( - frozenset( - c for c, works in zip(components, up[i]) if not works - ) - ) - return states - - def states(self, downs: list) -> list: - """The state of each of ``downs`` (the components down), those not - met yet worked out together (see ``evaluate``).""" - known = self._states - new = [down for down in dict.fromkeys(downs) if down not in known] - if len(new) > 1 and self._single: - up = [[node not in down for node in self._nodes] for down in new] - for down, state in zip( - new, self.evaluate(self._nodes, np.array(up)) - ): - known[down] = state - return [self(down) for down in downs] - - def trace(self, status: dict) -> "_CapacityTrace": - """The capacity of one simulation, which starts with ``status``.""" - return _CapacityTrace(self, status) - - -class _CapacityTrace: - """One simulation's capacity over time (for the tally to add up in - order): each change of the expected capacity, and the time spent at - each level and the fraction of the demand delivered. The components - down after each change are recorded as it goes, and the states they - are in worked out at the end, those met first together (see - ``_CapacityRecorder.states``).""" - - def __init__(self, recorder: _CapacityRecorder, status): - self.recorder = recorder - self.down = {node for node, up in status.items() if not up} - # The time of each change and the components down after it, from - # the start. - self.records: list = [(0.0, frozenset(self.down))] - - def change(self, time: float, node, up: bool) -> None: - """Component ``node`` went up (or down) at ``time``.""" - if up: - self.down.discard(node) - else: - self.down.add(node) - self.records.append((time, frozenset(self.down))) - - def finish(self, t_simulation: float, rec: _Replication) -> None: - """The trace, closed at the end of the window, into ``rec``: (time, - change of the expected capacity, change of whether it can be - unlimited), from the start's, and (level, time spent at it), in - order.""" - records = self.records + [(t_simulation, None)] - states = self.recorder.states([down for _, down in self.records]) - state = states[0] - since = delivered = 0.0 - changes = [(0.0, state.finite_mean, state.unlimited)] - spent = [] - for (time, _), new in zip(records[1:], states[1:] + [None]): - span = time - since - if span > 0.0: - for level, p in zip(state.levels, state.probabilities): - spent.append((level, p * span)) - delivered += state.delivered * span - since = time - if new is not None: - mean = new.finite_mean - state.finite_mean - unlimited = new.unlimited - state.unlimited - if mean or unlimited: - changes.append((time, mean, unlimited)) - state = new - rec.capacity_changes = changes - rec.capacity_time = spent - rec.delivered = ( - None if self.recorder.demand is None else delivered / t_simulation - ) - - -def _stopping_rule( - N: int, - tolerance: Optional[float], - confidence: float, - max_N: Optional[int], - antithetic: bool, - target: str, - t_simulation: float, -) -> Optional[Callable[..., int]]: - """``None`` for a fixed number of replications; otherwise a function - of the tally so far giving how many more replications to run (0 once - the confidence interval of the mean availability over the window, or of - the mean cost, is at most ``tolerance`` either side, or ``max_N`` have - run: then with a warning). Given the simulations' ``values`` too (a - controlled run's, see ``_controlled_run``), it judges those; given why - they cannot be judged yet (``unjudged``), it runs on to the limit.""" - montecarlo.check_confidence(confidence) - limit = montecarlo.sample_limit( - N, tolerance, max_N, antithetic, ("mc_samples", "max_samples") - ) - if limit is None: - return None - - def stop( - tally: "_Tally", values=None, unjudged: Optional[str] = None - ) -> int: - if values is not None: - # The controlled values of a run with an exact twin. - values = np.asarray(values, dtype=float) - elif target == "cost": - values = np.asarray(tally.cost_samples, dtype=float) - else: - values = np.asarray(tally.uptimes, dtype=float) / t_simulation - return montecarlo.more_samples( - values, - N, - tolerance, # type: ignore[arg-type] - confidence, - limit, - antithetic, - target, - "max_samples", - unjudged, - ) - - return stop - - -#: A parallel run's worker process's run (see ``_start_worker``). -_WORKER: Dict[str, Any] = {} - - -def _start_worker(run: bytes) -> None: - """Start a worker process of a parallel run (see ``_PythonRunner``): - unpickle the run, the system and what its simulations share, once, for - every block the worker runs.""" - rbd, args, curve_points, replacements, histories = pickle.loads(run) - _WORKER["run"] = ( - rbd, - rbd._context(*args), - curve_points, - replacements, - histories, - ) - - -def _simulate_block(span: Tuple[int, int]) -> "_Tally": - """Simulations ``start`` to ``stop - 1`` of a parallel run, in a worker - process (see ``_start_worker``): their totals, compact.""" - rbd, context, curve_points, replacements, histories = _WORKER["run"] - tally = _Tally( - list(rbd.components), - rbd.costs, - context.t_simulation, - curve_points, - ) - if replacements: - tally.replacements = [] - if histories: - tally.histories = _timeline_runs.Records(len(rbd.components)) - for replication in range(*span): - tally.add(rbd._replicate(context, replication)) - return tally.compact() - - -class _Context(NamedTuple): - """What a run's simulations share (see ``RepairableRBD._context``).""" - - t_simulation: float - working: set - broken: set - method: str - capacity: Optional[_CapacityRecorder] - run: _streams.Run - sources: dict - charges: tuple - downtime_cost_rates: dict - has_costs: bool - position: dict - plain: set - works: Callable - #: The components' states at the start (see ``_simulation_states``), - #: and whether the system is up at the start with every component up - #: but those held broken. - states: dict - initial_up: bool - #: Whether each simulation records its components' changes (see - #: ``simulate_timelines``). - history: bool = False - - -class _PythonRunner: - """Runs a run's simulations in Python: one after another in this - process, or, with ``jobs`` above 1, in blocks of ``PARALLEL_BLOCK`` in - that many processes. Each worker process is given the run once, when it - starts (see ``_start_worker``), and sends back each block's totals, - compact (#152), which are merged into the tally in order: the same - totals, kept exactly, as adding the simulations one by one. ``args`` - are ``RepairableRBD._context``'s.""" - - def __init__(self, rbd, tally, progress, args: tuple, jobs): - self._rbd = rbd - self._tally = tally - self._progress = progress - self._context: Optional[_Context] = None - self._executor: Any = None - if jobs is not None and jobs > 1: - run = montecarlo.dumps( - ( - rbd, - args, - tally.curve_points, - tally.replacements is not None, - tally.histories is not None, - ) - ) - self._executor = montecarlo.process_pool( - jobs, initializer=_start_worker, initargs=(run,) - ) - else: - self._context = rbd._context(*args) - - def __call__(self, start: int, stop: int) -> None: - """Simulations ``start`` to ``stop - 1``.""" - if self._executor is None: - for replication in range(start, stop): - self._tally.add( - self._rbd._replicate(self._context, replication) - ) - self._progress.update() - return - spans = [ - (first, min(first + PARALLEL_BLOCK, stop)) - for first in range(start, stop, PARALLEL_BLOCK) - ] - for block in self._executor.map(_simulate_block, spans): - self._tally.merge(block) - self._progress.update(block.n) - - def close(self) -> None: - if self._executor is not None: - self._executor.shutdown() - - -class _ShardRunner: - """Runs a run's simulations as shards (see ``repyability.rbd.shards``): - each range of simulations asked for is cut into shards at the multiples - of ``step``, which ``shard_map(run_shard, shards)`` runs wherever it - sends them, and the partials it gives back, checked to be those - shards', are merged into the tally in order. ``template`` is what - every shard of the run holds (see ``RepairableRBD._shard_template``).""" - - def __init__(self, rbd, tally, progress, shard_map, template, step): - from repyability.rbd.chunks import _run_key - - self._tally = tally - self._progress = progress - self._map = shard_map - self._template = template - self._step = step - self._key = _run_key({**template, "fingerprint": rbd._fingerprint()}) - - def __call__(self, start: int, stop: int) -> None: - """Simulations ``start`` to ``stop - 1``.""" - from repyability.rbd.chunks import SimulationChunk, _run_key - from repyability.rbd.shards import run_shard - - if stop <= start: - return - step = self._step - cuts = [start, *range((start // step + 1) * step, stop, step), stop] - ranges = list(zip(cuts, cuts[1:])) - shards = [_shard_bytes(self._template, a, b) for a, b in ranges] - chunks = [] - for saved in self._map(run_shard, shards): - chunk = SimulationChunk._load(saved) - chunks.append(chunk) - self._progress.update(chunk.n_simulations) - chunks.sort(key=lambda chunk: chunk.ranges[0]) - if [chunk.ranges for chunk in chunks] != [[r] for r in ranges] or any( - _run_key(chunk.settings) != self._key for chunk in chunks - ): - raise ValueError( - "shard_map gave back other partials than those of the " - "shards it was given: it must give back run_shard's result " - "for each shard, as map(run_shard, shards) does." - ) - for chunk in chunks: - self._tally.merge(chunk._tally) - - def close(self) -> None: - pass - - -#: The per-action costs: charged at each preventive action or inspection. -_ACTION_COST_KEYS = ("preventive_cost", "inspection_cost") - -#: Why a simulation does not start in the long-run state. -_STATIONARY_SIMULATION = ( - "A simulation starts from given states, not from the long-run one: give " - "each component's NodeState (its age, or how long it has been down). " - "The exact methods (point_availability, mission_availability, " - "expected_failures, expected_events, expected_cost, point_capacity and " - "mission_capacity) take state='stationary'." -) - - -def _validate_crews(crews) -> Optional[int]: - """``repair_crews``, validated: a whole number of crews, at least 1, or - None for as many as are needed.""" - if crews is None: - return None - if isinstance(crews, bool) or int(crews) != crews or crews < 1: - raise ValueError( - f"repair_crews must be a whole number, 1 or more, or None; got " - f"{crews!r}." - ) - return int(crews) - - -def _validate_priority(node, priority) -> float: - """A component's ``"priority"`` for a crew, validated: a finite - number.""" - if isinstance(priority, bool) or not isinstance( - priority, (int, float, np.integer, np.floating) - ): - raise ValueError( - f"Component {node!r}: priority must be a number, got " - f"{priority!r}." - ) - if not np.isfinite(priority): - raise ValueError( - f"Component {node!r}: priority must be finite, got {priority!r}." - ) - return float(priority) - - -def _safe_mean(model) -> float: - """A model's mean (see ``model_mean``), or NaN if it has none.""" - try: - return model_mean(model) - except Exception: - return float("nan") - - -@dataclass(order=True, slots=True) -class Event: - """A component's change of state in the availability simulation. - - Events compare by ``time`` alone (the other fields take no part in - comparisons), so the simulation's event queue releases them in time - order. - - Attributes - ---------- - time : float - When the change happens, on the simulation clock. - component : Hashable - The node whose state changes. - status : bool - The node's state after the event: False for a failure, True for a - restoration. - preventive : bool - True for scheduled preventive maintenance, by default False. With - ``status`` False it starts a planned outage (maintenance that takes - time, or a nested RBD's planned outage); with ``status`` True it - ends one or, if the node is up, renews it in place (maintenance in - zero time). - inspection : bool - True for an inspection of a node whose failures are hidden, by - default False. With ``status`` True it is a test in zero time of a - working node, or the end of a test that took time; with ``status`` - False it starts such a test (a planned outage) or, if the node has - failed, finds the failure, and the repair starts. - missed : bool - True for an inspection of a failed node that does not find the - failure (a test whose coverage is below 1), by default False: the - node stays down until a full test finds it. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import Event - >>> Event(12.0, "pump", True) > Event(10.0, "valve", False) - True - """ - - time: float - component: Hashable = field(compare=False) - status: bool = field(compare=False) - preventive: bool = field(default=False, compare=False) - inspection: bool = field(default=False, compare=False) - missed: bool = field(default=False, compare=False) - - -class _Preventive(NamedTuple): - """A node's scheduled preventive maintenance: a replacement every - ``interval`` under the ``"age"`` or ``"block"`` policy, or under - ``"condition"`` an inspection at every multiple of ``interval`` that - replaces the unit if it is more likely than ``threshold`` to fail - before the next; a replacement takes a time drawn from ``duration`` - (None: no time).""" - - interval: float - policy: str - duration: Any - threshold: float = 0.0 - #: Under ``"age"``, the age from which the unit is renewed early at a - #: stop of its maintenance group (#108); ``inf`` for never. - opportunity: float = math.inf - - def due(self, renewed: float) -> float: - """When the next preventive action (under ``"condition"``, the next - inspection) falls, for a unit put into service as new at - ``renewed``, or inspected then.""" - if self.policy == "age": - return renewed + self.interval - # Block, or inspections: the next multiple of the interval after - # ``renewed`` (a unit renewed on the schedule is not renewed again - # there). - due = float(np.floor(renewed / self.interval) + 1.0) * self.interval - return due if due > renewed else due + self.interval - - -class _Imperfect(NamedTuple): - """A component's imperfect repair (#109): ``kijima`` (``"kijima1"`` or - ``"kijima2"``) and its restoration factor ``q`` (0 renews, 1 is minimal - repair), and the failure at which it is replaced instead, counting - from its last renewal (None for never).""" - - kijima: str - q: float - replace_after: Optional[int] - - -class _MaintenanceGroup(NamedTuple): - """A maintenance group (#108): its members, the set-up cost charged - once at each of its stops, and whether every system outage is a stop - too.""" - - members: Tuple[Hashable, ...] - setup_cost: float - system_down: bool - - -def _failures_between(model, ages: np.ndarray) -> np.ndarray: - """The probability that a unit of ``model`` that has survived to each - of ``ages`` but the last fails before the next, ``1 - sf(next) / - sf(age)``: from its cumulative hazard where the model gives one, which - keeps a small probability precise.""" - ages = np.asarray(ages, dtype=float) - out = np.full(len(ages) - 1, np.nan) - Hf = getattr(model, "Hf", None) - if Hf is not None: - with np.errstate(all="ignore"): - hazard = np.asarray(Hf(ages), dtype=float) - out = -np.expm1(-np.maximum(np.diff(hazard), 0.0)) - missing = np.flatnonzero(np.isnan(out)) - if missing.size: - with np.errstate(all="ignore"): - survival = np.asarray(model.sf(ages), dtype=float) - now, later = survival[missing], survival[missing + 1] - fails = np.where(now > 0.0, 1.0 - later / now, 1.0) - out[missing] = np.clip(np.nan_to_num(fails, nan=1.0), 0.0, 1.0) - return out - - -class _Inspection(NamedTuple): - """A node's periodic inspection, which is what finds its (hidden) - failures: at ``offset`` and every ``interval`` after it (at every - multiple of ``interval`` with no offset), taking a time drawn from - ``duration`` (None: no time). A test finds a failure with probability - ``coverage``; the tests at ``offset`` and every ``full_test`` after it - (a whole multiple of ``interval``) find every failure, and with a - ``coverage`` of 1 every test does.""" - - interval: float - duration: Any - offset: float = 0.0 - coverage: float = 1.0 - full_test: Optional[float] = None - - @property - def partial(self) -> bool: - """Whether its tests can miss a failure (a coverage below 1).""" - return self.coverage < 1.0 - - @property - def period(self) -> float: - """The time after which its tests repeat: its full tests' interval - when its tests can miss a failure, else its interval.""" - if self.partial and self.full_test is not None: - return self.full_test - return self.interval - - @property - def per_full_test(self) -> int: - """Its tests from one full test to the next.""" - return int(round(self.period / self.interval)) - - def due(self, t: float) -> float: - """The first inspection after ``t``.""" - if not self.offset: - k = np.floor(t / self.interval) + 1.0 - due = float(k * self.interval) - return due if due > t else float((k + 1.0) * self.interval) - k = np.floor((t - self.offset) / self.interval) + 1.0 - due = float(self.offset + k * self.interval) - if due > t: - return due - return float(self.offset + (k + 1.0) * self.interval) - - def finds(self, t: float) -> float: - """The inspection that finds a failure at ``t``: the first at or - after it.""" - if not self.offset: - k = np.ceil(t / self.interval) - due = float(k * self.interval) - return due if due >= t else float((k + 1.0) * self.interval) - k = np.ceil((t - self.offset) / self.interval) - due = float(self.offset + k * self.interval) - if due >= t: - return due - return float(self.offset + (k + 1.0) * self.interval) - - def is_full(self, t: float) -> bool: - """Whether the test at ``t`` (one of its tests) finds every - failure.""" - if not self.partial: - return True - k = int(round((t - self.offset) / self.interval)) - return k % self.per_full_test == 0 - - def without_offset(self) -> "_Inspection": - """The schedule on a calendar from 0 (a start from a state, whose - phase places it).""" - return self._replace(offset=0.0) if self.offset else self - - -class _Standby(NamedTuple): - """A standby group (a component spec's ``"standby"``): ``units`` - identical units, ``k`` of which must operate, the rest waiting as - spares that age at ``dormancy_factor`` of the operating rate, switched - in with ``switching_probability``.""" - - units: int - k: int - dormancy_factor: float - switching_probability: float - - -def _constant_rate(model) -> Optional[float]: - """The failure rate of a model whose rate is constant (an exponential - life, or a Weibull of shape 1), or None.""" - try: - t = model_mean(model) * np.array([0.01, 0.5, 1.0, 3.0]) - hazard = np.asarray(model.hf(t), dtype=float).ravel() - survival = np.asarray(model.sf(t), dtype=float).ravel() - except Exception: - return None - rate = float(hazard[0]) - if not (np.isfinite(rate) and rate > 0.0): - return None - constant = np.allclose(hazard, rate, rtol=1e-9, atol=0.0) - exponential = np.allclose(survival, np.exp(-rate * t), rtol=1e-9) - return rate if constant and exponential else None - - -def _times_like(rbd, value) -> bool: - """Whether ``value``, given where node names go, can only be times: a - number, or numbers none of which is a node of ``rbd`` (#224).""" - if value is None or isinstance(value, (str, bytes, bool, np.bool_)): - return False - if isinstance(value, numbers.Real): - return True - if isinstance(value, np.ndarray): - if value.dtype.kind not in "iuf": - return False - items = value.ravel().tolist() - elif isinstance(value, (list, tuple)): - items = list(value) - if not items or not all( - isinstance(v, numbers.Real) and not isinstance(v, (bool, np.bool_)) - for v in items - ): - return False - else: - return False - nodes = set(rbd.nodes) - return not any(v in nodes for v in items) - - -def _times_first(target: str = "x"): - """Let a repairable diagram's measure take its times first, as a - non-repairable diagram's does (#224): a number, or numbers none of - which is a node, given where the first argument goes (node names, or - Fussell-Vesely's ``fv_type``) is ``target`` (its times ``x``, or a - window's length).""" - - def decorate(method): - @functools.wraps(method) - def wrapper(self, *args, **kwargs): - if args and _times_like(self, args[0]): - if kwargs.get(target) is not None: - raise TypeError( - f"{method.__name__}() was given times both first " - f"and as {target}=: give them once, as {target}=." - ) - kwargs[target] = args[0] - args = args[1:] - return method(self, *args, **kwargs) - - return wrapper - - return decorate - - -def _fixed_length(model) -> Optional[float]: - """The length of a time that is always the same (an - ``ExactEventTime``, say), or None.""" - try: - mean = float(model_mean(model)) - if not (np.isfinite(mean) and mean > 0.0): - return None - below, above = np.asarray( - model.ff(np.array([mean * (1.0 - 1e-9), mean * (1.0 + 1e-9)])), - dtype=float, - ).ravel() - except Exception: - return None - return mean if below <= 1e-12 and above >= 1.0 - 1e-12 else None - - -def _repair_rate(model) -> Optional[float]: - """The rate of an exponential repair time, ``inf`` for an instant one - (in no time), or None.""" - if _safe_mean(model) == 0.0: - return math.inf - return _constant_rate(model) - - -def _cdf(model) -> Callable[[np.ndarray], np.ndarray]: - """A model's CDF, for arrays of times (0 where it gives none).""" - - def cdf(x: np.ndarray) -> np.ndarray: - x = np.asarray(x, dtype=float) - with np.errstate(all="ignore"): - values = np.asarray(model.ff(x), dtype=float).reshape(x.shape) - return np.nan_to_num(values, nan=0.0) - - return cdf - - -def _product(a: np.ndarray, b: np.ndarray) -> np.ndarray: - """The distribution of the sum of two independent counts, with ``a`` - and ``b`` for ``0, 1, 2, ...``, its negligible tail cut.""" - from scipy.signal import fftconvolve - - out = np.maximum(fftconvolve(a, b), 0.0) - keep = np.nonzero(out > 1e-16)[0] - return out[: keep[-1] + 1] if keep.size else np.ones(1) - - -def _fleet(probabilities: np.ndarray, fleet: int) -> np.ndarray: - """The distribution of the sum of ``fleet`` independent counts, each - with ``probabilities`` for ``0, 1, 2, ...``.""" - total, power = np.ones(1), np.asarray(probabilities, dtype=float) - while fleet: - if fleet & 1: - total = _product(total, power) - fleet >>= 1 - if fleet: - power = _product(power, power) - return total / total.sum() - - -def _summed(counts) -> np.ndarray: - """The distribution of the sum of independent counts, each given by its - probabilities for ``0, 1, 2, ...``.""" - total = np.ones(1) - for count in counts: - total = _product(total, np.asarray(count, dtype=float)) - return total / total.sum() - - -def _fractions(counts: np.ndarray) -> np.ndarray: - """The fractions of simulations with each count ``0, 1, 2, ...``.""" - counts = np.asarray(counts, dtype=np.int64) - return np.bincount(counts) / len(counts) - - -def _whole(name: str, value, least: int = 1) -> int: - """``value`` as a whole number of at least ``least``, or a - ValueError naming it.""" - if ( - isinstance(value, bool) - or not isinstance(value, (int, float, np.integer, np.floating)) - or int(value) != value - or value < least - ): - raise ValueError( - f"{name} must be a whole number, {least} or more; got {value!r}." - ) - return int(value) - - -#: The share of its interval below which a test offset is taken as 0: the -#: first test at the interval, not one at the start of a new unit (#237). -_TINY_OFFSET = 1e-9 - -#: What the spares counts say when a component can wait for a repair crew. -_SPARES_CREWS = ( - "the spares counts assume", - "Count them by simulation: spares_demand(method='simulate').", -) -#: What the stock levels say then: they have no simulation of their own to -#: point to (#232). -_STOCK_CREWS = ( - "the stock levels assume", - "spares_stock has no simulation to fall back on; the diagram without " - "the limit (repair_crews=None) gives the stock when a crew is always " - "free.", -) - -#: The most nested RBDs the crews' chain is followed over time with: it -#: is solved for each pattern of them up and down. -_MAX_CREW_NESTED = 10 -#: What the interval choices say when a component can wait for a crew -#: (#184): see ``_require_interval_crews``. -_INTERVAL_CREWS = ( - "the exact long-run values the intervals are chosen by assume (the " - "Markov chain of the repair queue has no place for scheduled " - "maintenance or tests)", - "Choose them as if every repair started at once, with " - "assume_unlimited_crews=True, then simulate the plan with the crews: " - "with_intervals(plan).cost() or .availability(), or compare() it with " - "the schedules you have.", -) - -_ALLOCATION_CREWS = ( - "the allocations assume", - "Compare designs by their long-run values, or simulate them with " - "compare().", -) - - -class _Train(NamedTuple): - """A train ``allocate_redundancy`` may copy (see its ``trains``): its - name, its nodes in order, the node its last one feeds, where its - copies join, and how many of its inputs that node needs.""" - - name: Hashable - members: Tuple[Hashable, ...] - exit: Hashable - k: int - - -@dataclass(frozen=True) -class _TrainCopy: - """A node of a copy of a train, drawn by ``allocate_redundancy``: the - train's ``index``-th copy of its ``node``. Equal only to itself, so - that no node of the diagram can share its name.""" - - train: Hashable - index: int - node: Hashable - - -def _curve_points(value) -> Optional[int]: - """``curve_points`` checked: None, or a whole number of at least 1.""" - if value is None: - return None - if isinstance(value, bool) or not isinstance(value, (int, np.integer)): - raise ValueError( - f"curve_points must be a whole number of grid steps, got " - f"{value!r}." - ) - if value < 1: - raise ValueError(f"curve_points must be at least 1, got {value!r}.") - return int(value) - - -def _is_junction(name, component) -> bool: - """Whether ``component`` makes node ``name`` a junction, which never - fails: ``PerfectReliability`` itself, or a spec whose life it is (#175, - #182). Such a spec may give a repair model, never used, but nothing - that only a part that fails or is maintained has.""" - if component is PerfectReliability: - return True - if not ( - isinstance(component, dict) - and component.get("reliability") is PerfectReliability - ): - return False - other = sorted( - str(key) - for key, value in component.items() - if key not in ("reliability", "repairability") and value is not None - ) - if other: - raise ValueError( - f"Component {name!r} never fails (its reliability is " - f"PerfectReliability), so it takes no {', '.join(other)}: give " - "it as PerfectReliability alone, a junction that always works." - ) - return True - - -def _horizon(horizon) -> float: - """``horizon`` as a finite, non-negative float, or a ValueError (text - that reads as a number too, #233).""" - value = number_or_nan(horizon) - if not (np.isfinite(value) and value >= 0.0): - raise ValueError( - f"horizon must be a finite, non-negative number, got {horizon!r}." - ) - return value - - -def _discount_rate(discount_rate) -> float: - """``discount_rate`` as a float: a ValueError unless it is a finite, - non-negative number (#184).""" - number = isinstance( - discount_rate, (int, float, np.integer, np.floating) - ) and not isinstance(discount_rate, bool) - rate = float(discount_rate) if number else float("nan") - if not (np.isfinite(rate) and rate >= 0.0): - raise ValueError( - "discount_rate must be a finite, non-negative number (a " - "continuous rate per unit time of the component models), got " - f"{discount_rate!r}." - ) - return rate - - -def _horizons(horizon, rate: float) -> np.ndarray: - """``horizon``, a number or an array of them, as an array of - non-negative floats: finite, unless costs are discounted at ``rate`` - (#231), where an endless horizon is worth ``1 / rate`` of a steady - cost rate. Else a ValueError.""" - try: - values = real_array(horizon, "horizon") - except (TypeError, ValueError): - values = np.array(np.nan) - if values.size and np.all(values >= 0.0) and np.all(values < np.inf): - return values - if values.size and np.all(values >= 0.0): - if rate > 0.0: - return values - raise ValueError( - "horizon must be finite without a discount_rate: an undiscounted " - f"cost over an endless horizon is endless; got {horizon!r}." - ) - raise ValueError( - "horizon must be a finite, non-negative number (or an array of " - f"them; inf with a discount_rate), got {horizon!r}." - ) - - -#: A discount rate times the shortest of the components' mean lives -#: beyond which their failures' costs are discounted to almost nothing -#: (``exp(-20)``, 2e-9), and times a horizon beyond which the whole time -#: scale is: a rate given per year with the models in hours, say (#231). -#: A long horizon is no mistake in itself. -_DISCOUNTED_AWAY = 20.0 -_HORIZON_AWAY = 1000.0 - - -def _present_horizon( - horizons: np.ndarray, - rate: float, - lives: Callable[[], List[float]] = list, -) -> np.ndarray: - """What a unit cost rate over ``[0, horizon)`` is worth at the start, - discounted continuously at ``rate`` per unit time (#184): ``(1 - - exp(-rate * horizon)) / rate``, the horizon itself undiscounted, and - ``1 / rate`` for an endless one; for each of ``horizons``. Warns where - the rate discounts the costs away (#231): times the shortest of the - components' mean ``lives`` more than ``_DISCOUNTED_AWAY``, or times a - horizon more than ``_HORIZON_AWAY``.""" - if rate == 0.0: - return horizons - known = [life for life in lives() if 0.0 < life < math.inf] - finite = horizons[np.isfinite(horizons)] - span, what = 0.0, "" - if known and rate * min(known) > _DISCOUNTED_AWAY: - span, what = min(known), "the components' shortest mean life" - elif finite.size and rate * float(finite.max()) > _HORIZON_AWAY: - span, what = float(finite.max()), "the horizon" - if what: - warnings.warn( - f"discount_rate {rate:g} discounts the costs at {what} " - f"({span:g}) by a factor of exp(-{rate * span:.3g}), to " - "almost nothing: it is a continuous rate per unit time of the " - "component models. For an annual rate i with models in hours, " - "give math.log(1 + i) / 8760 (7% a year: " - f"{math.log(1.07) / 8760:.3g} an hour).", - stacklevel=outside_level(), - ) - return -np.expm1(-rate * horizons) / rate - - -#: How closely a discounted expected cost from new is worked out (#231): -#: its integral by parts, to this share of itself. -_DISCOUNT_TOLERANCE = 1e-8 -#: The most rounds of halving its pieces: a jump of the expected cost (a -#: scheduled event's) is closed in on by a half each round. -_DISCOUNT_ROUNDS = 60 - - -def _discount_pieces(ends: np.ndarray, horizon: float): - """The first pieces of a discounted cost's integral (see - ``RepairableRBD._discounted_cost``): from 0 to ``horizon`` in 32, - and split at each window's end, as ``(starts, stops)``.""" - edges = np.unique( - np.concatenate(([0.0], ends, np.linspace(0.0, horizon, 33))) - ) - return edges[:-1], edges[1:] - - -def _objective(cost: float, availability: float, max_cost_rate) -> float: - """What an interval choice minimises: the cost rate, or the - unavailability when the cost rate is capped.""" - return 1.0 - availability if max_cost_rate is not None else cost - - -def _meets(cost, availability, min_availability, max_cost_rate) -> bool: - """Whether long-run values meet an interval choice's target.""" - if min_availability is not None and availability < min_availability: - return False - if max_cost_rate is not None and cost > max_cost_rate: - return False - return True - - -def _allocation_target(target) -> float: - """An availability allocation's target, checked to be in [0, 1].""" - value = _as_float(target) - if not 0.0 <= value <= 1.0: - raise ValueError(f"target must be a number in [0, 1], got {target!r}.") - return value - - -def _as_float(value) -> float: - """``value`` as a float, or NaN if it is not a number (text that reads - as one included, #233).""" - return number_or_nan(value) - - -def _choose_intervals( - nodes: list, - evaluate, - starts: list, - bounds: Tuple[np.ndarray, np.ndarray], - min_availability: Optional[float], - max_cost_rate: Optional[float], -) -> dict: - """The intervals (by node) that minimise the cost rate, or, with a cost - cap, the unavailability, subject to the target: SLSQP over the - logarithms of the intervals from each of ``starts``, keeping the best - that meets the target. ``evaluate(intervals)`` gives their exact - ``(cost rate, availability)``. A ValueError if no intervals in - ``bounds`` meet the target, giving the best they can do.""" - low, high = bounds - box = list(zip(low, high)) - options = {"ftol": 1e-10, "maxiter": 200, "eps": 1e-6} - cache: dict = {} - - def point(x) -> tuple: - """``x`` rounded: the intervals evaluated, and returned.""" - return tuple(np.round(np.asarray(x, dtype=float), 12)) - - def values(x) -> Tuple[float, float]: - key = point(x) - if key not in cache: - cache[key] = evaluate( - {node: math.exp(xi) for node, xi in zip(nodes, key)} - ) - return cache[key] - - # The objective and the constraint, scaled to be about 1. - cost0, availability0 = values(high) - down0 = max(1.0 - availability0, 1e-300) - rate0 = max(abs(cost0), 1e-300) - - def unavailable(x): - return (1.0 - values(x)[1]) / down0 - - def cost(x): - return values(x)[0] / rate0 - - if max_cost_rate is not None: - objective = unavailable - - def slack(x): - return (max_cost_rate * (1.0 - 1e-12) - values(x)[0]) / ( - max_cost_rate - ) - - else: - objective = cost - - def slack(x): - return (values(x)[1] - min_availability - 1e-12) / ( - 1.0 - min_availability - ) - - constrained = min_availability is not None or max_cost_rate is not None - if constrained and not any( - _meets(*values(start), min_availability, max_cost_rate) - for start in starts - ): - # Can the target be met at all? The most available intervals, or - # the cheapest, from the starts nearest to it first. - extreme = unavailable if min_availability is not None else cost - found = [] - for start in sorted(starts, key=extreme): - reach = minimize( - extreme, start, method="SLSQP", bounds=box, options=options - ).x - found.append(reach) - if _meets(*values(reach), min_availability, max_cost_rate): - break - else: - best_cost, best_availability = values( - min(found + list(starts), key=extreme) - ) - if min_availability is not None: - raise ValueError( - f"min_availability {min_availability} cannot be met: the " - f"most any intervals give is {best_availability:.6g}." - ) - raise ValueError( - f"max_cost_rate {max_cost_rate} cannot be met: the least any " - f"intervals cost is {best_cost:.6g} per unit time." - ) - starts = [reach] + list(starts) - best, best_value = None, math.inf - for start in starts: - found = minimize( - objective, - start, - method="SLSQP", - bounds=box, - constraints=( - [{"type": "ineq", "fun": slack}] if constrained else [] - ), - options=options, - ) - for x in (found.x, start): - rate, availability = values(x) - if not _meets(rate, availability, min_availability, max_cost_rate): - continue - value = _objective(rate, availability, max_cost_rate) - if value < best_value: - best, best_value = np.asarray(x, dtype=float), value - assert best is not None # the start that meets the target is kept - return {node: math.exp(xi) for node, xi in zip(nodes, point(best))} - - -#: Allowed intervals are chosen by trying every combination, up to this many; -#: by a local search beyond. -_MAX_COMBINATIONS = 2000 - -#: Plans whose costs are within this share of each other, or whose -#: availabilities are within this of each other, are as good: which is -#: chosen does not hang on the last bits of their arithmetic (#229), as it -#: did for identical members' intervals in turn, and of plans that cost the -#: same the most available is chosen however their costs' last bits fall. -_AS_GOOD = 1e-10 - - -def _lower(a: tuple, b: tuple) -> bool: - """Whether merit ``a`` is lower than merit ``b``: their ``(value, - size)`` pairs compared in turn, values within ``_AS_GOOD`` of their - size taken as equal.""" - for (x, size_x), (y, size_y) in zip(a, b): - if x == y: - continue - if abs(x - y) > _AS_GOOD * max(size_x, size_y) or math.isinf(x - y): - return x < y - return False - - -def _divides(interval: float, full: float) -> bool: - """Whether ``interval`` divides ``full`` a whole number of times.""" - count = full / interval - return round(count) >= 1 and abs(count - round(count)) <= 1e-9 * count - - -def _require_chosen(per_node, chosen: list, name: str) -> None: - """Raise if a dict of per-node options names a node not chosen - (#222): a typo would otherwise be dropped.""" - if isinstance(per_node, dict): - others = [node for node in per_node if node not in chosen] - if others: - raise ValueError( - f"{name} names {others}, which are not among the components " - f"whose intervals are chosen: {chosen}." - ) - - -def _choose_divisor( - node, - full: float, - bounds: Tuple[float, float], - evaluate, - min_availability: Optional[float], - max_cost_rate: Optional[float], -) -> dict: - """The interval of a component whose tests can miss a failure (#221), - from those that divide its full tests' interval ``full`` (``full / - k``) within ``bounds``, as ``_choose_from`` would choose from them all: - from some 50 spread evenly in their logarithms, then every one between - the best's neighbours among those.""" - low, high = bounds - first = max(1, math.ceil(full / high)) - last = max(first, math.floor(full / low)) - - def best_of(ks) -> int: - options = {node: tuple(full / k for k in ks)} - chosen = _choose_from( - [node], options, evaluate, min_availability, max_cost_rate - ) - return int(round(full / chosen[node])) - - if last - first < 64: - return {node: full / best_of(range(first, last + 1))} - spread = sorted({int(k) for k in np.round(np.geomspace(first, last, 50))}) - k = best_of(spread) - i = spread.index(k) - around = range( - spread[max(i - 1, 0)], spread[min(i + 1, len(spread) - 1)] + 1 - ) - return {node: full / best_of(around)} - - -def _choose_from( - nodes: list, - options: dict, - evaluate, - min_availability: Optional[float], - max_cost_rate: Optional[float], -) -> dict: - """The intervals, each from its node's ``options``, that minimise the - cost rate, or with a cost cap the unavailability, subject to the target: - every combination when there are at most 2000, a local search (one - interval changed at a time, from several starts) otherwise. Of plans - that cost the same, the most available is chosen (and of those as - available, the cheapest): tests' offsets change no cost. Of plans as - good (``_AS_GOOD``), the first tried is kept. A ValueError if none - meets the target, giving the best any does.""" - - def merit(intervals: dict) -> tuple: - """(How far from the target, the objective, the other one), each - with the size its differences are measured against: lower is - better.""" - cost, availability = evaluate(intervals) - priced = abs(cost) - if min_availability is not None: - short = (max(0.0, min_availability - availability), 1.0) - elif max_cost_rate is not None: - short = (max(0.0, cost - max_cost_rate), priced) - else: - short = (0.0, 1.0) - down = (1.0 - availability, 1.0) - if max_cost_rate is None: - return short, (cost, priced), down - return short, down, (cost, priced) - - count = math.prod(len(options[node]) for node in nodes) - best: Optional[dict] = None - best_value: tuple = () - if count <= _MAX_COMBINATIONS: - for combination in itertools.product( - *(options[node] for node in nodes) - ): - candidate = dict(zip(nodes, combination)) - candidate_value = merit(candidate) - if best is None or _lower(candidate_value, best_value): - best, best_value = candidate, candidate_value - else: - starts = [ - {node: options[node][0] for node in nodes}, - {node: options[node][-1] for node in nodes}, - {node: options[node][len(options[node]) // 2] for node in nodes}, - ] - for current in starts: - value = merit(current) - improved = True - while improved: - improved = False - for node in nodes: - for option in options[node]: - candidate = {**current, node: option} - candidate_value = merit(candidate) - if _lower(candidate_value, value): - current, value = candidate, candidate_value - improved = True - if best is None or _lower(value, best_value): - best, best_value = current, value - assert best is not None - cost, availability = evaluate(best) - if not _meets(cost, availability, min_availability, max_cost_rate): - if min_availability is not None: - raise ValueError( - f"min_availability {min_availability} cannot be met: the " - f"most the allowed intervals give is {availability:.6g}." - ) - raise ValueError( - f"max_cost_rate {max_cost_rate} cannot be met: the least the " - f"allowed intervals cost is {cost:.6g} per unit time." - ) - return best - - -#: How a system's values over time take in its common-cause groups (#158). -_GROUPS_OVER_TIME = ( - " Each common-cause group's members' joint states come from its Markov " - "chain followed from every member up at 0 (by uniformization, or through " - "its tests' first period, after which they repeat the long run's), and " - "the system is worked out over the combinations of its members up and " - "down, the other nodes independent of them; the members' failures by " - "each cause, at its rate where it takes the system down." -) - -#: How the values of a component with hidden failures whose tests or -#: repairs take time, or whose tests can miss a failure of a life that is -#: not exponential, are found (#159). -_TESTED_CYCLE = ( - "hidden failures with tests or repairs that take time, or tests that " - "miss: the cycle from one test that finds a failure to the next, " - "followed test by test on a grid" -) - -#: Why antithetic pairs and common random numbers (``compare``) are refused -#: when some component's draws do not come from a stream. -_UNSTREAMED = ( - "Antithetic and common random numbers need every component's draws to " - "be replayable (surpyval parametric distributions and the composite " - "models built from them)." -) - - -def _failed_by(component: NonRepairable, age: float) -> float: - """The probability that a component's unit fails before ``age``: from - its model's own ``ff``, which keeps a small one's precision.""" - return float(np.ravel(component.reliability.ff(age))[0]) - - -def _log_kept(rate: float, inspection: "_Inspection") -> float: - """``log(rho)``, ``rho = 1 - (1 - c) (1 - exp(-rate * interval))``: the - chance that a unit with hidden failures, up after a test, is up after - the next one or had its failure found by it (a coverage ``c``).""" - missed = (1.0 - inspection.coverage) * -math.expm1( - -rate * inspection.interval - ) - return math.log1p(-missed) - - -def _tests_in(inspection: "_Inspection", period: float) -> list: - """The times of an inspection's tests in ``[0, period)`` that are not - multiples of its interval (those of a schedule with an offset), on a - calendar that repeats every ``period``.""" - if not inspection.offset: - return [] - count = int(round(period / inspection.interval)) - times = inspection.offset + inspection.interval * np.arange(count) - return list(np.where(times >= period, times - period, times)) - - -def _test_finds(source, coverage: float) -> bool: - """Whether a test that finds a failure with probability ``coverage`` - finds one: from the component's detection stream, or numpy's global - RNG.""" - if isinstance(source, _StreamedComponent): - return source.test_uniform() < coverage - return float(np.random.random()) < coverage - - -def _number_or_nan(value) -> float: - """``value`` as a float, or nan if it is not a number (text that reads - as one included, #233).""" - return number_or_nan(value) - - -def _common_period(intervals: Iterable[float]) -> float: - """The least common multiple of the inspection intervals: the period - after which the schedules repeat together.""" - intervals = sorted(set(intervals)) - if len(intervals) == 1: - return intervals[0] - fractions = [Fraction(x).limit_denominator(10**6) for x in intervals] - if any( - abs(float(f) - x) > 1e-12 * x for f, x in zip(fractions, intervals) - ): - raise NotImplementedError( - f"The inspection intervals {intervals} have no common period, " - "so the long-run values cannot be averaged over one: estimate " - "them by simulation, with availability() or cost()." - ) - numerator = math.lcm(*(f.numerator for f in fractions)) - denominator = math.gcd(*(f.denominator for f in fractions)) - return numerator / denominator - - -def combined_timeline( - timeline_1: List[Tuple[float, int]], timeline_2: List[Tuple[float, int]] -): - """Merge two up/down timelines and count how many are up over time. - - Each timeline is a list of ``(time, change)`` pairs, as the - availability simulation records them for a node and for the system: a - first entry ``(0.0, 1)`` if it starts up or ``(0.0, 0)`` if it starts - down, then ``(t, 1)`` for each restoration and ``(t, -1)`` for each - failure, and a last entry ``(t_end, 0)`` closing the window. The - changes of the two timelines are summed at equal times and accumulated - in time order. - - Parameters - ---------- - timeline_1 : list[tuple[float, int]] - The first timeline, e.g. a node's. - timeline_2 : list[tuple[float, int]] - The second timeline, e.g. the system's. - - Returns - ------- - tuple[numpy.ndarray, numpy.ndarray] - The distinct times in increasing order, and at each time the number - of the two (0, 1 or 2) that are up from that time until the next. - - Examples - -------- - A node down on ``[2, 3)`` and a system down on ``[2, 5)``, over a - window of length 10: - - >>> from repyability.rbd.repairable_rbd import combined_timeline - >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] - >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] - >>> times, n_up = combined_timeline(node, system) - >>> times.tolist(), n_up.tolist() - ([0.0, 2.0, 3.0, 5.0, 10.0], [2, 0, 1, 2, 2]) - """ - joint_timeline: defaultdict = defaultdict(lambda: 0) - for t, e in timeline_1 + timeline_2: - joint_timeline[t] += e - events: np.ndarray = np.fromiter(joint_timeline.values(), dtype=np.int8) - timeline: np.ndarray = np.fromiter(joint_timeline.keys(), dtype=np.float64) - idx = np.argsort(timeline) - timeline = timeline[idx] - events = events[idx] - events = events.cumsum() - return timeline, events - - -def intersection(timeline, event_cumsum): - """Total time during which both of two timelines are up. - - Sums the lengths of the intervals ``[timeline[j], timeline[j + 1])`` - on which ``event_cumsum[j] == 2``. Pass ``2 - event_cumsum`` instead to - get the total time during which both are down. - - Parameters - ---------- - timeline : numpy.ndarray - Increasing times, as returned by ``combined_timeline``. - event_cumsum : numpy.ndarray - How many of the two are up from each time, as returned by - ``combined_timeline``. - - Returns - ------- - float - The total length of those intervals. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import ( - ... combined_timeline, - ... intersection, - ... ) - >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] - >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] - >>> times, n_up = combined_timeline(node, system) - >>> float(intersection(times, n_up)) # both up - 7.0 - >>> float(intersection(times, 2 - n_up)) # both down - 1.0 - """ - from_idx = np.where(event_cumsum[:-1] == 2)[0] - to_idx = from_idx + 1 - intersection = timeline[to_idx] - timeline[from_idx] - return intersection.sum() - - -def union(timeline, event_cumsum): - """Total time during which at least one of two timelines is up. - - Sums the lengths of the intervals ``[timeline[j], timeline[j + 1])`` - on which ``event_cumsum[j] > 0``. Pass ``2 - event_cumsum`` instead to - get the total time during which at least one is down. - - Parameters - ---------- - timeline : numpy.ndarray - Increasing times, as returned by ``combined_timeline``. - event_cumsum : numpy.ndarray - How many of the two are up from each time, as returned by - ``combined_timeline``. - - Returns - ------- - float - The total length of those intervals. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import combined_timeline, union - >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] - >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] - >>> times, n_up = combined_timeline(node, system) - >>> float(union(times, n_up)) # either up - 9.0 - >>> float(union(times, 2 - n_up)) # either down - 3.0 - """ - from_idx = np.where(event_cumsum[:-1] > 0)[0] - to_idx = from_idx + 1 - union = timeline[to_idx] - timeline[from_idx] - return union.sum() - - -def intersection_over_union( - node_timeline: List[Tuple[float, int]], - system_timeline: List[Tuple[float, int]], -): - """Intersection over union of a node's and the system's up time. - - The time both are up divided by the time at least one is up, for one - pair of timelines. (``RepairableRBD.availability`` computes its ``iou`` - measure from totals over all its simulations instead.) If neither is - ever up the result is nan, with a numpy warning. - - Parameters - ---------- - node_timeline : list[tuple[float, int]] - The node's timeline, in the format described in - ``combined_timeline``. - system_timeline : list[tuple[float, int]] - The system's timeline, in the same format. - - Returns - ------- - float - The ratio, in ``[0, 1]``: 1 when the two are up at exactly the same - times. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import intersection_over_union - >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] - >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] - >>> round(float(intersection_over_union(node, system)), 4) # 7 / 9 - 0.7778 - """ - timeline, event_cumsum = combined_timeline(node_timeline, system_timeline) - return intersection(timeline, event_cumsum) / union(timeline, event_cumsum) - - -def time_at_status(timeline, status): - """Sum the gaps that follow a timeline's entries of a given value. - - Sums ``t[j + 1] - t[j]`` over every entry ``(t[j], value)`` of - ``timeline`` whose value equals ``status``. For a single node's or the - system's timeline, in the format described in ``combined_timeline``, - ``status=1`` gives its total up time: every up period starts at the - initial ``(0.0, 1)`` entry or at a restoration. (``status=-1`` would - miss a down period at the start, whose entry is ``(0.0, 0)``; the - simulation only uses ``status=1``.) - - Parameters - ---------- - timeline : list[tuple[float, int]] - One node's or the system's timeline. - status : int - The entry value whose following gaps are summed. - - Returns - ------- - float - The total time. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import time_at_status - >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] - >>> float(time_at_status(node, 1)) - 9.0 - """ - t = np.array([a for a, _ in timeline]) - events = np.array([b for _, b in timeline]) - from_idx = np.where(events[:-1] == status)[0] - to_idx = from_idx + 1 - union = t[to_idx] - t[from_idx] - return union.sum() - - -#: Grid steps over a component's typical up time, for its point -#: availability (see ``_point_availability``): the error falls as the -#: square of the step, to about 4e-8 here in a mission average. -_POINT_STEPS = 1000 -#: The most grid points for one component's point availability. -_POINT_MAX = 2**22 - - -#: The most pieces ``mission_availability`` integrates over. -_MISSION_POINTS = 5_000_000 - - -def _settling_end(curve, long_run: Optional[float], cycle: float, end: float): - """How far to follow a component's curve that has not settled at its - long-run value ``long_run`` by ``end`` (see ``_unit_curve``). Its - distance from that value falls by about the same factor each - ``cycle`` (an up and a down time), so the largest distances in its - last two cycles say when it will be within 1e-10: the curve is followed - until that point is three quarters of the way along, as the check - wants, and a cycle more. Four times as far where they do not say (a - distance that does not fall, or too few cycles followed).""" - grow = 4.0 * end - if long_run is None or not ( - np.isfinite(cycle) and 0.0 < 2.0 * cycle < end - ): - return grow - t = curve.times - off = np.abs(curve.at(t) - long_run) - last = t[-1] - recent = float(off[t > last - cycle].max()) - before = float(off[(t > last - 2.0 * cycle) & (t <= last - cycle)].max()) - if not 0.0 < recent < before: - return grow - cycles = max(math.log(1e-10 / recent) / math.log(recent / before), 0.0) - settled = last + cycles * cycle - return min(grow, max(1.25 * end, settled / 0.75 + cycle)) - - -def _check_window(t_simulation) -> float: - """A simulation's window, ``t_simulation``, as a float: a positive and - finite time.""" - if ( - isinstance(t_simulation, bool) - or not isinstance(t_simulation, (int, float, np.integer, np.floating)) - or not (math.isfinite(t_simulation) and t_simulation > 0.0) - ): - raise ValueError( - f"t_simulation must be a positive, finite time, got " - f"{t_simulation!r}." - ) - return float(t_simulation) - - -def _check_times(x) -> np.ndarray: - """``x`` as a 1-d float array of times, all finite and non-negative: - numbers, not text that reads as them (#233).""" - times = np.atleast_1d(real_array(x, "The times")) - if not np.all(np.isfinite(times)) or np.any(times < 0.0): - raise ValueError(f"Times must be finite and non-negative, got {x!r}.") - return times - - -def _sf_values(sf, x: np.ndarray) -> np.ndarray: - """A survival function's values at ``x``, as floats in ``x``'s shape.""" - with np.errstate(divide="ignore", invalid="ignore"): - return np.asarray(sf(x), dtype=float).reshape(np.shape(x)) - - -def _up_scale(model, age: Optional[float]) -> float: - """A typical up time for a unit with lifetime ``model`` (replaced at - ``age``, if given), to size its point-availability grid by: the smaller - of its median and the width of its middle 80%; else its typical failure - time (NaN if it has none, e.g. it never fails).""" - scale = float("nan") - qf = getattr(model, "qf", None) - if qf is not None: - try: - with np.errstate(all="ignore"): - q10, q50, q90 = np.ravel( - np.asarray(qf(np.array([0.1, 0.5, 0.9])), dtype=float) - ) - widths = [w for w in (q50, q90 - q10) if np.isfinite(w) and w > 0] - if widths: - scale = min(widths) - except Exception: # a model whose qf cannot take these - pass - if not np.isfinite(scale): - try: - scale = failure_time_scale(model) - except Exception: # no mean either (e.g. never fails) - scale = float("nan") - if age is not None: - scale = age if not np.isfinite(scale) else min(scale, age) - return float(scale) - - -def _settling(curves) -> Tuple[float, Optional[float]]: - """When the point availabilities ``curves`` (see - ``_point_availability``) all settle: the time after which they are - constant (a period of None) or repeat together with the period returned - (a time of inf if they never do, as far as is known).""" - curves = list(curves) - settle = max((float(curve.settle) for curve in curves), default=0.0) - periods = {curve.period for curve in curves if curve.period is not None} - if not periods: - return settle, None - try: - return settle, _common_period(periods) - except NotImplementedError: # no common period - return np.inf, None - - -#: The most patterns of nested RBDs' capacity levels the capacity over time -#: with limited repair crews follows the crews' chain for (#162). -_MAX_CREW_LEVELS = 4096 - - -class _CrewCapacity: - """The capacity over time of a system whose components wait for repair - crews, around nested RBDs (#162). The nested RBDs have crews of their - own, so they are independent of the chain and of each other: at a time - the system is at a level with probability ``sum_c w_c(t) p(t) v_c``, - over each combination ``c`` of the nested RBDs' levels (up or down, or - their own capacities), ``w_c(t)`` its probability from their own - curves, and ``v_c`` whether the system is at the level in each of the - chain's states with the nested RBDs at ``c``. The chain is followed - over time (see ``_chain_transient.Uniformized``) for each combination - as it first has a probability, and kept.""" - - def __init__(self, rbd, chain, start, nested: dict, working, broken): - self.rbd, self.chain, self.start = rbd, chain, start - self.nested = nested - size = len(chain.probabilities) - own = { - node: chain.up[:, k].astype(float) - for k, node in enumerate(chain.nodes) - } - # The nested RBDs' places, which each combination of their levels - # takes (see ``_chain_for``). - own.update({node: np.ones(size) for node in nested}) - self.arrays, _ = rbd._node_arrays( - rbd._filled(own, size, working, broken) - ) - self.size = size - self.followed: Dict[tuple, tuple] = {} - timing = rbd._uniformized( - "The repair crews'", - chain.generator, - start, - chain.probabilities, - np.ones((size, 1)), - ) - #: For the pieces of the integrals: the chain's and the nested - #: RBDs' curves. - self.curves = [_chain_transient.ChainCurve(timing), *nested.values()] - - def _chain_for(self, levels: tuple): - """The system's levels with the nested RBDs at ``levels``, and the - chain followed over time with whether it is at each, in each - state.""" - if levels not in self.followed: - own = { - node: (np.array([level]), np.ones((1, self.size))) - for node, level in zip(self.nested, levels) - } - values, rows = self.rbd._capacity_arrays( - self.arrays, self.size, own - ) - self.followed[levels] = ( - values, - self.rbd._uniformized( - "The repair crews'", - self.chain.generator, - self.start, - self.chain.probabilities, - rows.T, - ), - ) - return self.followed[levels] - - def _distributions(self, x: np.ndarray) -> list: - """Each nested RBD's capacity distribution at the times ``x``: its - own, if it has capacities, else up (at its capacity here) or down, - with its point availability.""" - out = [] - models = self.rbd._capacity_models() - for node, curve in self.nested.items(): - if node in models: - out.append(self.rbd.components[node]._capacity_at(curve, x)) - else: - up = np.clip(np.asarray(curve.at(x), dtype=float), 0.0, 1.0) - out.append( - _capacity.node_distribution( - self.rbd.capacity.get(node, np.inf), up, 1.0 - up - ) - ) - return out - - def rows(self, x) -> Tuple[np.ndarray, np.ndarray]: - """The system's capacity levels and their probabilities at the times - ``x`` (one row per level).""" - x = np.asarray(x, dtype=float).ravel() - distributions = self._distributions(x) - count = int(np.prod([len(levels) for levels, _ in distributions])) - if count > _MAX_CREW_LEVELS: - raise NotImplementedError( - f"With {self.rbd.repair_crews} repair crew(s), the capacity " - "over time is worked out for each combination of the nested " - f"RBDs' levels: {count} are more than the {_MAX_CREW_LEVELS} " - "it takes. Simulate it with availability(demand=...)." - ) - totals: Dict[float, np.ndarray] = {} - for choice in itertools.product( - *[range(len(levels)) for levels, _ in distributions] - ): - weight = np.ones(len(x)) - for (_, chances), i in zip(distributions, choice): - weight = weight * chances[i] - if not weight.any(): - continue - key = tuple( - float(levels[i]) - for (levels, _), i in zip(distributions, choice) - ) - values, chain = self._chain_for(key) - at = np.clip(chain.values(x), 0.0, 1.0) - for level, column in zip(values, at.T): - level = float(level) - totals[level] = totals.get(level, 0.0) + weight * column - levels = np.array(sorted(totals), dtype=float) - rows = np.array([totals[level] for level in levels]).reshape( - len(levels), len(x) - ) - return levels, rows - - -class _Jumps(NamedTuple): - """What the system does at the times its nodes fail or are taken down - at exact times (see ``RepairableRBD._atom_groups``): the times, the - probability at each that the system fails there, and that a planned - outage takes it down there, and how much its point availability there - falls with them (``drop``); if asked for, each node's part in the - failures (``caused``).""" - - times: np.ndarray - failures: np.ndarray - planned: np.ndarray - drop: np.ndarray - caused: Optional[dict] = None - - -def _matched(times: np.ndarray, at: np.ndarray, values: np.ndarray): - """``values``, given at the times ``at``, summed at each of ``times`` - (sorted) that they fall on exactly; 0 at the others.""" - index = np.searchsorted(times, at) - hit = index < len(times) - hit[hit] = times[index[hit]] == at[hit] - return np.bincount(index[hit], values[hit], len(times)) - - -def _shaped(values: np.ndarray, t, times: np.ndarray): - """``values``, one per time, as a float for a scalar ``t``, else in the - shape of ``times``.""" - values = np.asarray(values, dtype=float) - if np.ndim(t) == 0: - return float(values[0]) - return values.reshape(times.shape) - - -def _squeeze_values(d: dict) -> dict: - """Convert a dict of 1-element arrays (as produced by the base RBD - importance helpers) to a dict of plain floats. Steady-state availability - has no time dimension, so the repairable importances return floats.""" - return {k: float(np.asarray(v).reshape(-1)[0]) for k, v in d.items()} - - -def _safe_ratio(numerator, denominator): - """Ratio that is 0 when the denominator is 0. - - Several criticality/importance measures divide by a total (system uptime, - downtime, failures, restorations, or a union of intervals) that can - legitimately be 0 -- e.g. when a redundant component is forced working so - the system never fails, or a component is forced broken. With nothing to - attribute, the measure is 0 rather than a NaN/inf (or a crash). - """ - return numerator / denominator if denominator else 0.0 - - -def _mean_cost(cost) -> float: - """A declared cost's expected value: the number itself, or the mean of a - cost distribution.""" - if isinstance(cost, float): - return cost - return float(np.ravel(cost.mean())[0]) - - -def failure_criticality_index_per_system_failures(FCI, system_failures): - """Share of all system failures that each node caused. - - A node causes a system failure when its own failure takes the system - from up to down. For each node this is the number of system failures - it caused divided by ``system_failures``; every node gets 0 when there - were no system failures. - - Parameters - ---------- - FCI : dict - Node -> counts, with the number of system failures the node caused - under ``"system_failures"`` (as accumulated by - ``RepairableRBD.availability``). - system_failures : int - The total number of system failures. - - Returns - ------- - dict - Node -> share of the system failures, in ``[0, 1]``. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import ( - ... failure_criticality_index_per_system_failures as per_system, - ... ) - >>> counts = {"a": {"system_failures": 3}, "b": {"system_failures": 1}} - >>> per_system(counts, 4) - {'a': 0.75, 'b': 0.25} - """ - fci = {} - for node in FCI.keys(): - if system_failures == 0: - # No system failures occurred (e.g. a redundant node was forced - # working, or the system was highly reliable over the simulated - # window), so no node can be credited with causing one. - fci[node] = 0 - else: - fci[node] = FCI[node]["system_failures"] / system_failures - return fci - - -def failure_criticality_index_per_component_failures(FCI): - """Share of each node's own failures that caused a system failure. - - For each node: the number of system failures it caused (its failures - that took the system from up to down) divided by its number of - failures, or 0 if it never failed. - - Parameters - ---------- - FCI : dict - Node -> counts, with the system failures the node caused under - ``"system_failures"`` and its own failures under - ``"component_failures"`` (as accumulated by - ``RepairableRBD.availability``). - - Returns - ------- - dict - Node -> share of the node's failures, in ``[0, 1]``. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import ( - ... failure_criticality_index_per_component_failures as per_component, - ... ) - >>> per_component({"a": {"system_failures": 3, "component_failures": 4}}) - {'a': 0.75} - """ - fci = {} - for node in FCI.keys(): - try: - fci[node] = ( - FCI[node]["system_failures"] / FCI[node]["component_failures"] - ) - except ZeroDivisionError: - # If there were no component failures then there were no - # system failures caused by that node - fci[node] = 0 - return fci - - -def restoration_criticality_index_by_system(RCI, system_restorations): - """Share of all system restorations that each node caused. - - A node causes a system restoration when its own restoration takes the - system from down to up. For each node this is the number of system - restorations it caused divided by ``system_restorations``; every node - gets 0 when there were no system restorations. - - Parameters - ---------- - RCI : dict - Node -> counts, with the number of system restorations the node - caused under ``"system_restorations"`` (as accumulated by - ``RepairableRBD.availability``). - system_restorations : int - The total number of system restorations. - - Returns - ------- - dict - Node -> share of the system restorations, in ``[0, 1]``. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import ( - ... restoration_criticality_index_by_system as by_system, - ... ) - >>> by_system({"a": {"system_restorations": 1}}, 2) - {'a': 0.5} - """ - rci = {} - for node in RCI.keys(): - if system_restorations == 0: - # No system restorations occurred, so no node can be credited with - # causing one. - rci[node] = 0 - else: - rci[node] = RCI[node]["system_restorations"] / system_restorations - return rci - - -def restoration_criticality_index_by_component(RCI): - """Share of each node's own restorations that restored the system. - - For each node: the number of system restorations it caused (its - restorations that took the system from down to up) divided by its - number of restorations, or 0 if it was never restored. - - Parameters - ---------- - RCI : dict - Node -> counts, with the system restorations the node caused under - ``"system_restorations"`` and its own restorations under - ``"component_restorations"`` (as accumulated by - ``RepairableRBD.availability``). - - Returns - ------- - dict - Node -> share of the node's restorations, in ``[0, 1]``. - - Examples - -------- - >>> from repyability.rbd.repairable_rbd import ( - ... restoration_criticality_index_by_component as by_component, - ... ) - >>> counts = {"system_restorations": 1, "component_restorations": 4} - >>> by_component({"a": counts}) - {'a': 0.25} - """ - rci = {} - for node in RCI.keys(): - try: - rci[node] = ( - RCI[node]["system_restorations"] - / RCI[node]["component_restorations"] - ) - except ZeroDivisionError: - # If there were no component restorations then there were no - # times the system was restored by restoring this node. - rci[node] = 0 - return rci - - -def _at_zero(model) -> bool: - """Whether ``model`` is a time of exactly 0 (an instant repair is).""" - return distribution_name(model) == "ExactEventTime" and not np.any( - np.ravel(model.params) - ) - - -def _never_settles(component) -> bool: - """Whether a component fails at once and is repaired at once, so that - the event loop would change its state without end.""" - return _at_zero(getattr(component, "reliability", None)) and _at_zero( - getattr(component, "time_to_replace", None) - ) - - -class RepairableRBD(RBD): - """A reliability block diagram of repairable components. - - Each component alternates between working and failed: it fails after a - time drawn from its reliability model, and is then repaired, as good - as new, after a time drawn from its repairability model. Components fail - and are repaired independently of one another and of the system state - (unless ``repair_crews`` makes them wait for a crew), and a component - keeps running (and can fail) while the system is down. The system is up - whenever its working components connect the input node to the output - node. - - Two kinds of analysis are offered: - - - Exact long-run (steady-state) metrics, built from each component's - availability ``MTTF / (MTTF + MTTR)`` and failure frequency - ``1 / (MTTF + MTTR)``: ``mean_availability``, ``node_availability``, - ``system_failure_frequency``, ``mean_up_time``, ``mean_down_time``, - ``mean_time_between_failures``, ``expected_cost_rate``, - ``capacity_distribution`` (with node capacities) and the importance - measures. A component under age or block replacement enters through - its renewal cycle instead (see ``node_availability``). With fewer - ``repair_crews`` than components, the long-run values come from the - Markov chain of the components and the repair queue, for exponential - components (see ``mean_availability``). - - Monte-Carlo simulation of a finite window ``[0, t_simulation]`` that - starts with every component working: ``availability`` (availability - over time, criticality measures and, with node capacities, the - capacity over time and the fraction of a demand delivered) and - ``cost`` (the distribution of the window's cost). - - Times are in the time unit of the component models, and costs in the - currency they are given in. - - Parameters - ---------- - edges : Iterable[tuple[Hashable, Hashable]] - The directed edges of the diagram, e.g. - ``[("s", "a"), ("a", "b"), ("b", "t")]`` for ``"a"`` and ``"b"`` in - series between the input node ``"s"`` and the output node ``"t"``. - components : dict - One entry per node other than the input and output nodes, keyed by - node name (the input and output nodes never fail: a component given - for one is refused, as its edge to the output node, or from the - input node, is most likely missing). Each is one of: + Parameters + ---------- + edges : Iterable[tuple[Hashable, Hashable]] + The directed edges of the diagram, e.g. + ``[("s", "a"), ("a", "b"), ("b", "t")]`` for ``"a"`` and ``"b"`` in + series between the input node ``"s"`` and the output node ``"t"``. + components : dict + One entry per node other than the input and output nodes, keyed by + node name (the input and output nodes never fail: a component given + for one is refused, as its edge to the output node, or from the + input node, is most likely missing). Each is one of: - A spec dict with ``"reliability"`` (a time-to-failure model, such - as a fitted surpyval distribution) and ``"repairability"`` (a + as a fitted surpyval distribution or ``MixtureModel``, a fitted + Wiener or gamma degradation process (its time to the threshold), + or a model surpyval fits to a repairable unit's failures, + ``CrowAMSAA``, ``Duane``, ``HPP`` or ``GeneralizedRenewal``, + which is the life and ``"repair"`` it is: + see the guide's imperfect repair) and ``"repairability"`` (a time-to-repair model, or ``"instant"`` for repair in zero time: the component still fails, and any repair or replace cost is charged, but it is never down), plus optional costs. @@ -4640,9 +185,13 @@ class RepairableRBD(RBD): multiple of ``interval``, whatever its age, unless it is down then. Under ``"condition"`` it is inspected at every multiple of ``interval`` while it is up, in no time, and replaced if it is - then more likely than ``"threshold"`` (a probability, required) - to fail before the next inspection, given its age ``a``: - ``1 - R(a + interval) / R(a)``. Each inspection is charged + then more likely than ``"threshold"`` (a probability) to fail + before the next inspection, given its age ``a``: + ``1 - R(a + interval) / R(a)``; or, for a life that is a + degradation process, with ``"level"`` in place of + ``"threshold"``, if its measured level is then at or past + ``"level"`` (simulated only; see the guide's replacement on + condition). Each inspection is charged ``"inspection_cost"``, a number or a distribution drawn afresh each time. The replacement takes a time drawn from ``"duration"``, a time-to-maintain model, during which the unit is down (a planned @@ -4665,8 +214,8 @@ class RepairableRBD(RBD): the interval: tests of redundant components staggered). An offset of 0, the default, is no offset, with no test at 0: a positive one puts a test there, near the start, so an offset just above 0 adds - one test (one within a billionth of the interval is taken as 0, - #237). A test finds a failure with + one test (one within a billionth of the interval is taken as + 0). A test finds a failure with probability ``"coverage"`` (by default 1); a failure it misses stays hidden until a full test, every ``"full_test"`` (required with a coverage below 1, a whole multiple of the interval, from @@ -4704,11 +253,21 @@ class RepairableRBD(RBD): A preventive replacement renews the unit, and age replacement counts its operating time since it was renewed. The exact methods refuse such a component, and the simulations follow it, - in Python; but minimally repaired (``q = 1``) in no time, with no - ``"replace_after"``, maintenance or tests, it is up throughout - and fails ``H(t)`` times by ``t`` on average (``H`` its life's - cumulative hazard), which the values over a window from new - (``expected_failures`` and the others) take in exactly. + in Python; but minimally repaired (``q = 1``), with no + ``"replace_after"``, maintenance or tests, it fails at its + life's hazard at its operating age, the time it has been up: + in no time it is up throughout and fails ``H(t)`` times by + ``t`` on average (``H`` its life's cumulative hazard), which + the values over a window from new (``expected_failures`` and + the others) take in exactly, and with repairs that take time + its operating age is followed on a grid, numerically. + ``"duty"``, a fraction ``d`` in (0, 1], is the share of the time + the component operates, for a ``"reliability"`` fitted in + operating time: it ages only while it operates, so its life on + the diagram's clock is its operating life over ``d`` + (``R(d t)``), which every method then takes; its repairs, + maintenance and tests stay on the clock. The life must be a + surpyval parametric distribution of a time. ``"standby"`` makes the node a standby group of identical units, each failing and repaired as ``"reliability"`` and ``"repairability"`` say: a dict of ``"units"`` (by default 2), @@ -4742,7 +301,7 @@ class RepairableRBD(RBD): - [`PerfectReliability`][repyability.PerfectReliability] itself (or a spec whose ``"reliability"`` it is, with no costs or maintenance), for a junction: a node that never fails, such as a - k-out-of-n vote point (#182). It is no component: the analyses + k-out-of-n vote point. It is no component: the analyses leave it out, the simulations draw nothing for it, and it passes whatever reaches it, up to a ``capacity`` if it is given one. k : dict[Hashable, int], optional @@ -4820,8 +379,9 @@ class RepairableRBD(RBD): ccf_groups : list of CCFGroup, optional Common-cause groups (see [`CCFGroup`][repyability.CCFGroup]), by default none: identical components (the same life and repair - models) that fail together. A group's model (a ``BetaFactor`` or - ``MGL``) splits the members' failure rate between causes, each + models) that fail together. A group's model (a ``BetaFactor``, + ``MGL`` or ``AlphaFactor``) splits the members' failure rate + between causes, each member's own and shared ones, and each cause fails the members it names that are up, at once (whatever the model's ``basis``: a repairable component's failures are a rate). Each member alone @@ -4841,7 +401,8 @@ class RepairableRBD(RBD): members' availability) and the values over time from new, each group's chain followed from every member up. The simulations draw each cause as a Poisson process, failing the members it names that - are up; they need exponential lives alone (#158). + are up; they need exponential lives alone. + Attributes ---------- @@ -5013,63 +574,19 @@ class RepairableRBD(RBD): False """ - #: Optional per-component cost fields accepted in a component spec dict. - COST_KEYS = ("repair_cost", "replace_cost", "downtime_cost") - #: The costs charged per failure, which may also be given as a - #: distribution of the cost (drawn afresh at each failure). - PER_FAILURE_COST_KEYS = ("repair_cost", "replace_cost") - #: Every key a component spec dict may carry. - COMPONENT_SPEC_KEYS = ( - ("reliability", "repairability") - + COST_KEYS - + ( - "preventive", - "inspection", - "acquisition_cost", - "priority", - "standby", - "group", - "repair", - "replace_after", - ) - ) - #: The models of imperfect repair a spec's ``"repair"`` can name. - REPAIR_MODELS = ("kijima1", "kijima2") + COST_KEYS = _spec.COST_KEYS + PER_FAILURE_COST_KEYS = _spec.PER_FAILURE_COST_KEYS + COMPONENT_SPEC_KEYS = _spec.COMPONENT_SPEC_KEYS + REPAIR_MODELS = _spec.REPAIR_MODELS #: The curves kept while an analysis shares them (see #: ``_sharing_curves``); None otherwise. - _curve_memo: Optional[dict] = None - #: The keys of a component's ``"preventive"`` spec. - PREVENTIVE_KEYS = ( - "interval", - "policy", - "duration", - "cost", - "threshold", - "inspection_cost", - "opportunity", - ) - #: The policies a ``"preventive"`` spec can name: replacement at an - #: age, on a calendar (block), or on the condition an inspection finds. - PREVENTIVE_POLICIES = ("age", "block", "condition") - #: The keys of a maintenance group's options. - GROUP_KEYS = ("setup_cost", "system_down") - #: The keys of a component's ``"inspection"`` spec. - INSPECTION_KEYS = ( - "interval", - "duration", - "cost", - "offset", - "coverage", - "full_test", - ) - #: The keys of a component's ``"standby"`` spec. - STANDBY_KEYS = ("units", "k", "dormancy_factor", "switching_probability") - #: The costs charged per action (per failure, per preventive - #: replacement or per inspection), which may be distributions. - _PER_ACTION_COST_KEYS = PER_FAILURE_COST_KEYS + ( - "preventive_cost", - "inspection_cost", - ) + _curve_memo: Optional[dict] = None + PREVENTIVE_KEYS = _spec.PREVENTIVE_KEYS + PREVENTIVE_POLICIES = _spec.PREVENTIVE_POLICIES + GROUP_KEYS = _spec.GROUP_KEYS + INSPECTION_KEYS = _spec.INSPECTION_KEYS + STANDBY_KEYS = _spec.STANDBY_KEYS + _PER_ACTION_COST_KEYS = _spec._PER_ACTION_COST_KEYS def __init__( self, @@ -5089,16 +606,21 @@ def __init__( # Capture the constructor inputs verbatim (before any mutation) so the # RBD can be faithfully serialised via to_dict()/to_json(). edges = list(edges) + from repyability.rbd._processes import as_spec from repyability.rbd.ccf import as_groups # PerfectReliability() stands for the class (#232), given alone or - # as a spec's life. + # as a spec's life; a fitted process as a spec's life is the life + # and repair it is (#269). components = { name: ( - { - **component, - "reliability": perfect_class(component["reliability"]), - } + as_spec( + name, + { + **component, + "reliability": perfect_class(component["reliability"]), + }, + ) if isinstance(component, dict) and "reliability" in component else perfect_class(component) ) @@ -5120,10 +642,14 @@ def __init__( ), "ccf_groups": list(ccf_groups) if ccf_groups else None, } + # A component that operates part of the time takes its life on + # the calendar from here on; the spec as given keeps its life in + # operating time, for its levers, draws and saving. + components = _on_duty(components) self.repair_crews = _validate_crews(repair_crews) # Each component's place in the queue for a crew (higher first). self._priority: dict[Any, float] = {} - self.downtime_cost_rate = self._validate_cost( + self.downtime_cost_rate = _spec._validate_cost( "", "downtime_cost_rate", downtime_cost_rate ) # Per-node cost fields, pulled out of the component specs: a number, @@ -5171,24 +697,27 @@ def __init__( repairability = {} for name, component in components.items(): if isinstance(component, dict): - self._validate_component_spec(name, component) + _spec._validate_component_spec(name, component) node_costs = {} for key in self.COST_KEYS: if component.get(key) is None: continue - cost = self._validate_component_cost( + cost = _spec._validate_component_cost( name, key, component[key] ) # A cost of 0 prices nothing, so it is left out. if not (isinstance(cost, float) and cost == 0.0): node_costs[key] = cost if component.get("preventive") is not None: - schedule, cost, inspecting = self._validate_preventive( - name, component["preventive"] + schedule, cost, inspecting = _spec._validate_preventive( + name, component["preventive"], component["reliability"] ) if cost is not None: node_costs["preventive_cost"] = cost - if np.isfinite(schedule.interval): + if ( + np.isfinite(schedule.interval) + or schedule.policy == "calendar" + ): self._preventive[name] = schedule if inspecting is not None: node_costs["inspection_cost"] = inspecting @@ -5199,7 +728,7 @@ def __init__( "schedule and an inspection; give it one or the " "other." ) - inspection, cost = self._validate_inspection( + inspection, cost = _spec._validate_inspection( name, component["inspection"] ) if cost is not None: @@ -5210,21 +739,26 @@ def __init__( name, component["priority"] ) if component.get("standby") is not None: - self._standby[name] = self._validate_standby( + self._standby[name] = _spec._validate_standby( name, component ) if component.get("group") is not None: self._member_group[name] = component["group"] - imperfect = self._validate_imperfect(name, component) + imperfect = _spec._validate_imperfect(self, name, component) if imperfect is not None: self._imperfect[name] = imperfect if component.get("acquisition_cost") is not None: - acquisition = self._validate_cost( + acquisition = _spec._validate_cost( name, "acquisition_cost", component["acquisition_cost"] ) if acquisition: self.acquisition_costs[name] = acquisition - if node_costs: + calendar = self._preventive.get(name) + if node_costs or ( + calendar is not None and calendar.policy == "calendar" + ): + # A component on a calendar policy is charged its + # calendar's prices, whatever else it declares. self.costs[name] = node_costs repair_model = component["repairability"] if isinstance(repair_model, str): @@ -5271,7 +805,7 @@ def __init__( reliability[name] = component.reliability repairability[name] = component.time_to_replace else: - raise TypeError(self._unknown_component(name, component)) + raise TypeError(_spec._unknown_component(name, component)) refuse_nonparametric( { name: spec @@ -5279,6 +813,7 @@ def __init__( if not isinstance(spec, RepairableRBD) } ) + _spec._refuse_shared_nested(components) super().__init__( edges, @@ -5306,9 +841,13 @@ def __init__( ) self.components = components self.repairability = copy(repairability) - self._maintenance = self._validate_groups(maintenance_groups) - self.ccf_groups = self._validate_ccf_groups(ccf_groups) + self._maintenance = _spec._validate_groups(self, maintenance_groups) + self.ccf_groups = _spec._validate_ccf_groups(self, ccf_groups) + _spec._warn_nested_costs(components) + #: Whether a diagram this one is nested in has warned that it carries + #: costs the parent does not count (see ``_spec._warn_nested_costs``). + _costs_reported = False #: The junctions are folded out of the structure (see ``RBD``). _FOLDS_JUNCTIONS = True _SIMULATE_INSTEAD = ( @@ -5344,1435 +883,114 @@ def _repr_details(self) -> List[str]: out.append(f"{self.repair_crews} repair crew(s)") junctions = self._junctions() if junctions: - out.append( - "junction(s) " - + ", ".join(repr(n) for n in sorted(junctions, key=str)) - ) - return out - - def _validate_ccf_groups(self, ccf_groups) -> list: - """The common-cause groups, checked: each a ``CCFGroup`` of - components of this RBD (not a nested RBD or a standby group), each - component in one group at most, the members of a group identical - (the same life and repair models, compared through their saved - form).""" - if not ccf_groups: - return [] - from repyability.rbd.ccf import CCFGroup - from repyability.rbd.serialisation import serialise_model - - seen: set = set() - for group in ccf_groups: - if not isinstance(group, CCFGroup): - raise ValueError( - "ccf_groups must contain CCFGroup instances, got " - f"{type(group).__name__}." - ) - for member in group.members: - if member not in self.components: - raise self._not_a_component( - member, f"common-cause group {list(group.members)}" - ) - if ( - isinstance(self.components[member], RepairableRBD) - or member in self._standby - ): - raise ValueError( - f"CCF group member {member!r} is a nested RBD or a " - "standby group: a group's members are single " - "components." - ) - if member in seen: - raise ValueError( - f"Node {member!r} appears in more than one CCF group." - ) - seen.add(member) - try: - specs = [ - ( - serialise_model(self.components[m].reliability), - serialise_model(self.components[m].time_to_replace), - ) - for m in group.members - ] - except Exception: # models that cannot be saved are not compared - continue - if any(spec != specs[0] for spec in specs[1:]): - raise ValueError( - f"The members of a CCF group, {list(group.members)}, " - "must be identical components, with the same life and " - "repair models." - ) - return list(ccf_groups) - - def _has_ccf(self) -> bool: - """Whether this RBD, or one nested in it, has common-cause - groups.""" - return bool(getattr(self, "ccf_groups", ())) or any( - isinstance(c, RepairableRBD) and c._has_ccf() - for c in self.components.values() - ) - - def _ccf_rates(self, group) -> Tuple[float, Optional[float]]: - """A common-cause group's members' failure rate, and their repair - rate (None for hidden failures, found by tests), for its chain - (see ``_ccf_chain``); raise if the chain does not cover them.""" - where = f"Common-cause group {list(group.members)}" - if self._crews_couple(): - raise NotImplementedError( - f"{where}: with limited repair crews, the crews' Markov " - "chain does not take common causes in, as yet. Estimate the " - "values by simulation, with availability() or cost(), which " - "take both." - ) - rates = set() - for member in group.members: - for kinds, what in ( - (self._preventive, "scheduled maintenance"), - (self._imperfect, "imperfect repair"), - (self._member_group, "a maintenance group"), - ): - if member in kinds: - raise NotImplementedError( - f"{where}: member {member!r} has {what}, which the " - "group's chain does not take in." - ) - life = _constant_rate(self.components[member].reliability) - if life is None: - raise NotImplementedError( - f"{where}: its chain needs exponential lives (a constant " - f"failure rate), and the life of member {member!r} is " - "not. The simulations need one too, as the rate is " - "split between the causes." - ) - rates.add(life) - tested = [member in self._inspection for member in group.members] - if any(tested) and not all(tested): - raise NotImplementedError( - f"{where}: some members' failures are hidden and others' " - "revealed; the group's chain takes one kind." - ) - life = rates.pop() - if all(tested): - # Tests and repairs of no time, a fixed one or an exponential - # one (#220). - self._ccf_timings(group) - for member in group.members: - self._require_tested_exact(member) - if len({self._inspection[m].coverage for m in group.members}) > 1: - raise NotImplementedError( - f"{where}: its members' tests have different coverages; " - "the group's chain takes one, which a cause's failures " - "are found by alike." - ) - return life, None - component = self.components[group.members[0]] - repair = _constant_rate(component.time_to_replace) - if repair is None: - raise NotImplementedError( - f"{where}: the chain of members whose failures are revealed " - "needs exponential repairs, and theirs are not. Estimate the " - "values by simulation, with availability() or cost(), which " - "take repairs of any length." - ) - return life, repair - - def _ccf_timings(self, group) -> Optional[List["_ccf_chain.Timing"]]: - """How long the tests and repairs of a common-cause group's members - (whose failures are hidden) take, for its chain (#220): each a - ``_ccf_chain.Duration`` of no time, a fixed time or an exponential - one; None where none takes time. Raise for what the chain does not - take, saying what to do: another distribution, a test of an - exponential length followed by a repair of a fixed one (whose end - is then no fixed time after the test), and fixed lengths that run - into the member's next test.""" - if not all(member in self._inspection for member in group.members): - return None # revealed failures (see _ccf_rates) - where = f"Common-cause group {list(group.members)}" - out = [] - for member in group.members: - inspection = self._inspection[member] - parts = [] - for what, model in ( - ("tests", inspection.duration), - ("repairs", self.components[member].time_to_replace), - ): - if model is None or _safe_mean(model) == 0.0: - parts.append(_ccf_chain.Duration()) - continue - fixed = _fixed_length(model) - if fixed is not None: - parts.append(_ccf_chain.Duration(fixed=fixed)) - continue - rate = _constant_rate(model) - if rate is None: - raise NotImplementedError( - f"{where}: its chain takes tests and repairs that " - "take no time, a fixed time or an exponential one, " - f"and the {what} of member {member!r} take none of " - f"those. Estimate the values by simulation, with " - "availability() or cost(), which take any; or, where " - f"the {what} are short next to the test interval, " - "give them a fixed length." - ) - parts.append(_ccf_chain.Duration(rate=rate)) - test, repair = parts - if test.rate is not None and repair.fixed > 0.0: - raise NotImplementedError( - f"{where}: the tests of member {member!r} take an " - "exponential time, and its repairs a fixed one, which " - "its chain does not take (the repair would end no fixed " - "time after the test). Give both a fixed length, or both " - "an exponential one; or estimate the values by " - "simulation, with availability() or cost()." - ) - length = test.fixed + repair.fixed - if length >= inspection.interval: - raise NotImplementedError( - f"{where}: the test and repair of member {member!r} take " - f"{length:g} together, as long as its test interval " - f"({inspection.interval:g}) or longer, which its chain " - "does not take. Estimate the values by simulation, with " - "availability() or cost()." - ) - out.append(_ccf_chain.Timing(test, repair)) - if not any(timing.takes_time for timing in out): - return None - return out - - def _shared_causes(self) -> List[_Cause]: - """The common-cause groups' shared causes, as the simulation strikes - them (see ``_Cause``): each a Poisson process at its share of the - members' failure rate. A member's own share is its own life (see - ``_own_rate``).""" - out = [] - for group in self.ccf_groups: - rate = _constant_rate( - self.components[group.members[0]].reliability - ) - if rate is None: - continue # refused (see _require_groups_simulated) - coins = any( - member in self._inspection and self._inspection[member].partial - for member in group.members - ) - for struck, cause in _ccf_chain.causes( - group.model, group.members, rate - ): - if len(struck) > 1: - out.append( - _Cause( - tuple(group.members[i] for i in struck), - cause, - coins, - ) - ) - return out - - def _own_rate(self, node) -> Optional[float]: - """A common-cause group member's own cause's rate, at which the - simulation draws its life, the shared causes striking it too (see - ``_shared_causes``): 0 if every cause it has is shared; None for - any other node (and for a member whose life is not exponential, - which the simulation refuses, see ``_require_groups_simulated``).""" - for group in self.ccf_groups: - if node in group.members: - rate = _constant_rate(self.components[node].reliability) - if rate is None: - return None - for struck, cause in _ccf_chain.causes( - group.model, group.members, rate - ): - if struck == (group.members.index(node),): - return float(cause) - return None - - def _require_groups_simulated(self, states: Optional[dict] = None): - """Raise unless the simulation takes the common-cause groups in - (#158): each member's life exponential, as its rate is split between - the causes; no member maintained on a schedule, repaired - imperfectly, in a maintenance group, or started from a state.""" - for group in self.ccf_groups: - where = f"Common-cause group {list(group.members)}" - for member in group.members: - life = self.components[member].reliability - if _constant_rate(life) is None: - raise NotImplementedError( - f"{where}: its members' failure rate is split " - "between the causes, which needs exponential lives, " - f"and the life of member {member!r} is not." - ) - for kinds, what in ( - (self._preventive, "scheduled maintenance"), - (self._imperfect, "imperfect repair"), - (self._member_group, "a maintenance group"), - ): - if member in kinds: - raise NotImplementedError( - f"{where}: member {member!r} has {what}, which " - "the simulation of its group does not take in, " - "as yet." - ) - start = (states or {}).get(member) - if start is not None and not start.new: - raise NotImplementedError( - f"{where}: member {member!r} starts from a state, " - "which the simulation of its group does not take in, " - "as yet: its members start new." - ) - for component in self.components.values(): - if isinstance(component, RepairableRBD): - component._require_groups_simulated() - - def _require_groups_allocated(self, chosen, drawn) -> dict: - """Raise unless the redundancy allocation takes in the common-cause - groups (#158): their chains cover them, a member given copies is in - a ``BetaFactor`` group (its copies join the group, and an MGL - model's letters are for its group's size), and no member is in a - train. Returns each member given copies' group.""" - from repyability.rbd.ccf import BetaFactor - - self._require_ccf_long_run() - group_of = {m: g for g in self.ccf_groups for m in g.members} - for node in chosen: - group = group_of.get(node) - if group is not None and self._ccf_timings(group) is not None: - raise NotImplementedError( - f"Node {node!r} is in a common-cause group whose " - "members' tests or repairs take time: its copies would " - "join the group, which its chain does not take in, as " - "yet. Leave the group's members out of nodes, or " - "estimate the design by simulation, with cost()." - ) - if group is not None and not isinstance(group.model, BetaFactor): - raise NotImplementedError( - f"Node {node!r} is in a common-cause group whose model, " - f"{group.model!r}, is for a group of " - f"{len(group.members)}: its copies would join the group, " - "and an MGL model's letters for a larger group are not " - "known. Give the group a BetaFactor model, whose beta " - "holds at any size, or leave its members out of nodes." - ) - for train in drawn: - inside = [m for m in train.members if m in group_of] - if inside: - raise NotImplementedError( - f"Train {train.name!r} holds {inside}, in a common-cause " - "group, whose copies the allocation of trains does not " - "take in, as yet: give the members copies as nodes." - ) - return {node: group_of[node] for node in chosen if node in group_of} - - def _require_ccf_long_run(self) -> None: - """Raise if a common-cause group's chain does not cover it.""" - for group in self.ccf_groups: - self._ccf_rates(group) - - def _calendar_period(self) -> float: - """The period of the schedules the long-run values average over - (see ``_long_run_grid``): of the tests and block replacements.""" - intervals = { - self._preventive[node].interval for node in self._block_nodes() - } - for schedule in self._inspection.values(): - intervals |= {schedule.interval, schedule.period} - return _common_period(intervals) if intervals else 1.0 - - def _group_states(self, group, times: np.ndarray, counts=None): - """A common-cause group's members' joint states, at each of - ``times`` (of ``_long_run_grid``), in the long run (see - ``_ccf_chain``). With ``counts``, each member's number of copies, - which join the group (#158, a ``BetaFactor`` model's beta holding - at any size, see ``_ccf_chain._Counted``), tested with it: the - states of the members' nodes, each down while all its copies - are.""" - life, repair = self._ccf_rates(group) - counts = [1] * len(group.members) if counts is None else list(counts) - # A member with no copies counted (one made perfect, by the - # allocation's bounds) is left out: the others' states are as with - # it. - kept = [i for i, n in enumerate(counts) if n > 0] - if not kept: - return _ccf_chain.GroupStates( - (), np.zeros((1, 0), dtype=bool), np.ones((len(times), 1)) - ) - members = [group.members[i] for i in kept] - copies = [int(counts[i]) for i in kept] - if repair is not None: - states = _ccf_chain.revealed( - group.model, members, life, repair, copies - ) - return states._replace( - probabilities=np.repeat( - states.probabilities, len(times), axis=0 - ) - ) - tests, period = self._group_tests(group) - place = {old: new for new, old in enumerate(kept)} - tests = [ - test._replace(member=place[test.member]) - for test in tests - if test.member in place - ] - coverage = self._inspection[group.members[0]].coverage - timings = self._ccf_timings(group) - if timings is not None and ( - len(kept) < len(counts) or max(copies) > 1 - ): - raise NotImplementedError( - f"Common-cause group {list(group.members)}: its members' " - "tests or repairs take time, and copies of them, which the " - "allocation's chains count, are not taken in, as yet. Leave " - "its members out of the nodes to allocate." - ) - return _ccf_chain.hidden( - group.model, - members, - life, - coverage, - tests, - period, - times, - copies, - timings, - ) - - def _group_tests(self, group) -> Tuple[list, float]: - """A common-cause group's members' tests in one period of the - calendar (``(0, period]``, see ``_calendar_period``), and the - period.""" - period = self._calendar_period() - tests = [] - for position, member in enumerate(group.members): - schedule = self._inspection[member] - count = int(round(period / schedule.interval)) - first = 0 if schedule.offset else 1 - for k in range(first, first + count): - time = schedule.offset + k * schedule.interval - tests.append( - _ccf_chain.ProofTest( - time, position, schedule.is_full(time) - ) - ) - return tests, period - - def _groups_over_time(self) -> list: - """Each common-cause group's members' joint states over time from - every member up at 0 (see ``_ccf_chain.OverTime``).""" - out = [] - for group in self.ccf_groups: - life, repair = self._ccf_rates(group) - if repair is not None: - where = f"Common-cause group {list(group.members)}'s" - out.append( - _ccf_chain.OverTime.revealed( - group.model, - group.members, - life, - repair, - partial(self._uniformized, where), - ) - ) - continue - tests, period = self._group_tests(group) - out.append( - _ccf_chain.OverTime.hidden( - group.model, - group.members, - life, - self._inspection[group.members[0]].coverage, - tests, - period, - self._ccf_timings(group), - ) - ) - return out - - def _require_groups_over_time(self, states: dict) -> None: - """Raise unless the common-cause groups can be followed over time - from new (see ``_groups_curve``): their chains cover them, and no - member starts from a state.""" - self._require_ccf_long_run() - started = sorted( - (m for g in self.ccf_groups for m in g.members if m in states), - key=str, - ) - if started: - raise NotImplementedError( - f"Node(s) {started} are in a common-cause group: its " - "values over time are worked out from every member new at " - "0, and a member's current state is not taken in, as yet." - ) - - def _groups_system(self, working, broken, method: str): - """The system with its common-cause groups at given times (see - ``_ccf_chain.GroupsSystem``).""" - causes = [ - _ccf_chain.causes( - group.model, group.members, self._ccf_rates(group)[0] - ) - for group in self.ccf_groups - ] - return _ccf_chain.GroupsSystem( - self, - self._groups_over_time(), - causes, - working, - broken, - structure_method(method), - ) - - def _groups_curve( - self, - horizon: float, - working_nodes=frozenset(), - broken_nodes=frozenset(), - method: str = "p", - states: Optional[dict] = None, - counts: bool = False, - stages: bool = False, - ) -> "_ccf_chain.GroupsCurve": - """The system over time with common-cause groups (#158, see - ``_ccf_chain.GroupsCurve``): its nodes outside the groups as their - own curves (each from its state in ``states``) to ``horizon``, - counting their events with ``counts`` and following their stages - with ``stages``, and the groups' members' joint states from their - chains, from every member up at 0.""" - states = states or {} - working, broken = set(working_nodes), set(broken_nodes) - self._require_free_members(working, broken) - self._require_groups_over_time(states) - members = {m for group in self.ccf_groups for m in group.members} - curves = self._availability_curves( - horizon, - working | broken | members, - counts=counts, - stages=stages, - state=states, - groups=True, - ) - system = self._groups_system(working, broken, method) - return _ccf_chain.GroupsCurve(self, curves, system) - - def _require_free_members(self, working_nodes, broken_nodes) -> None: - """Raise if a common-cause group's member is held working or - broken: the cause it shares would still strike the others.""" - held = set(working_nodes or ()) | set(broken_nodes or ()) - for group in self.ccf_groups: - caught = held & set(group.members) - if caught: - raise NotImplementedError( - f"Node(s) {sorted(caught, key=str)} are in a " - "common-cause group, whose shared causes would still " - "strike the others: they cannot be held working or " - "broken." - ) - - def _with_ccf_groups( - self, - times: np.ndarray, - probabilities: dict, - failures: Optional[dict], - weights: np.ndarray, - states_by_group: Optional[list] = None, - what: str = "This analysis", - ) -> Tuple[dict, Optional[dict], np.ndarray, np.ndarray]: - """The long-run points (the times of ``_long_run_grid``, and their - weights) split by the common-cause groups' joint states: each - time into one point for each combination of each group's members - up or down, weighted by its probability then, with those members - up or down for certain and the other nodes as at the time. The - long-run values are averages over these points as over the times. - Also each point's time's position in ``times``. ``states_by_group`` - gives the groups' states (see ``_group_states``), in their order, - when they are not the groups' own (an allocation's design). - - Every combination of every group's states is a point of its own, - so the points multiply with the groups: ``what`` (the analysis that - splits them) refuses, before anything is built, where they would - take more than ``_ccf_chain.SPLIT_VALUES`` values (#218). The - long-run values, the importance measures and the values over time - condition on each group within its module instead (see - ``_ccf_tabled``).""" - tables = [ - ( - self._group_states(group, times) - if states_by_group is None - else states_by_group[number] - ) - for number, group in enumerate(self.ccf_groups) - ] - _ccf_chain.check_split( - len(times), - [len(states.down) for states in tables], - len(probabilities) * (1 if failures is None else 2), - what, - self.ccf_groups, - ) - index = np.arange(len(times)) - for states in tables: - combinations = len(states.down) - mass = (weights[:, None] * states.probabilities[index, :]).ravel() - keep = mass > 0.0 - weights = mass[keep] - - def spread(values, combinations=combinations, keep=keep): - points = len(keep) // combinations - values = np.broadcast_to( - np.asarray(values, dtype=float), (points,) - ) - return np.repeat(values, combinations)[keep] - - probabilities = {n: spread(v) for n, v in probabilities.items()} - if failures is not None: - failures = {n: spread(v) for n, v in failures.items()} - for k, member in enumerate(states.members): - down = np.tile(states.down[:, k], len(index))[keep] - probabilities[member] = np.where(down, 0.0, 1.0) - if failures is not None: - failures[member] = np.where(down, 1.0, 0.0) - index = np.repeat(index, combinations)[keep] - return probabilities, failures, weights, index - - def _ccf_plan(self, structure) -> "_ccf_modules.Plan": - """Each common-cause group's owner in ``structure`` (the - decomposition, or its dual), the smallest module holding its - members (see ``_ccf_modules.Plan``): once for each structure.""" - cache = self.__dict__.setdefault("_ccf_plans", {}) - entry = cache.get(id(structure)) - if entry is None or entry[0] is not structure: - entry = cache[id(structure)] = ( - structure, - _ccf_modules.Plan(structure, self.ccf_groups), - ) - return entry[1] - - def _ccf_tabled( - self, p: dict, q: dict, tables: list, structure=None - ) -> "_ccf_modules.Tabled": - """The system with its common-cause groups at given points (#218): - each node outside the groups up with the probability ``p[node]`` - and down with ``q[node]`` (1-d arrays of one length), and each - group's members in each combination of their states with the - probabilities ``tables`` give (each group's - ``_ccf_chain.GroupStates`` at the points). Given a group's - combination its members are independent of everything else, so - each group is conditioned on only within the smallest module - holding its members (see ``_ccf_modules.Tabled``): groups in - separate modules cost a sum, not a product. Over ``structure`` - (the dual, for Fussell-Vesely over path sets) if given, else the - decomposition.""" - structure = self._decomposition() if structure is None else structure - if isinstance(structure, GraphStructure): - raise NotImplementedError(structure.reason) - p, q, size = self._node_pairs(p, q) - return _ccf_modules.Tabled( - self._ccf_plan(structure), self.ccf_groups, p, q, size, tables - ) - - def _ccf_long_run(self, working_nodes, broken_nodes) -> tuple: - """The long-run grid's times and weights (see ``_long_run_grid``), - and what ``_ccf_tabled`` takes there: the nodes' availabilities - and unavailabilities at the times (the forced nodes held at 1 or - 0), and the groups' members' joint states then, from their chains - (see ``_group_states``).""" - # The groups' chains first, as the routes report them. - self._require_ccf_long_run() - self._require_free_members(working_nodes, broken_nodes) - times, weights = self._long_run_grid() - p = self._probabilities_with_overrides( - self._availabilities_at(times), working_nodes, broken_nodes - ) - q = self._failures_with_overrides( - self._unavailabilities_at(times), working_nodes, broken_nodes - ) - # The groups' chains, kept for the other long-run values (#229): - # a choice of intervals asks for the cost rate and the - # availability of each plan. - tables = self.__dict__.get("_ccf_tables") - if tables is None: - tables = self.__dict__["_ccf_tables"] = [ - self._group_states(group, times) for group in self.ccf_groups - ] - return times, weights, (p, q, tables) - - def _ccf_long_run_measure( - self, - measure: str, - working_nodes, - broken_nodes, - kind: str = "failure", - fv_type: str = "c", - method: str = "exact", - ) -> dict: - """An importance measure of every node in the long run, with the - common-cause groups (see ``_ccf_measure``).""" - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) - self._validate_node_overrides(working, broken) - _, weights, inputs = self._ccf_long_run(working, broken) - return _squeeze_values( - self._ccf_measure(measure, inputs, weights, kind, fv_type, method) - ) - - def _ccf_measure( - self, - measure: str, - inputs: tuple, - weights: Optional[np.ndarray], - kind: str = "failure", - fv_type: str = "c", - method: str = "exact", - ) -> dict: - """An importance measure of every node with the common-cause groups - (``measure``, one of ``_importance_time``'s names), at the points - ``inputs`` gives (see ``_ccf_tabled``): averaged with ``weights``, - each ratio a ratio of the averages, or at each point without. - - A node outside the groups is independent of them, so its measures - come from the system's probabilities with it held working and - failed, as without groups. A member's state says something of its - group's, so it is conditioned on: ``A(1_i)`` and ``A(0_i)`` at each - point are the system's availability given the member up and given - it down then, from the joint probabilities summed over its group's - combinations (see ``_ccf_modules.Tabled.joints``). Birnbaum's - measure is the difference between the two, taken from whichever - end keeps its precision, the system's unavailability or its - availability. Fussell-Vesely sums each combination's numerators, - the nodes being independent given it.""" - if kind not in ("failure", "success"): - raise ValueError( - f"kind must be 'failure' or 'success', got {kind!r}." - ) - if fv_type not in ("c", "p"): - raise ValueError( - "fv_type must be either 'c' (cut-set) or 'p' (path-set), " - f"fv_type={fv_type!r} was given." - ) - if method not in ("exact", "rare_event"): - raise ValueError( - f"method must be 'exact' or 'rare_event', got {method!r}." - ) - evaluation = self._ccf_tabled(*inputs) - R_t, Q_t = evaluation.system() - - def mean(value): - if weights is None: - return np.broadcast_to( - np.asarray(value, dtype=float), R_t.shape - ) - return np.atleast_1d(weights @ np.asarray(value, dtype=float)) - - R, Q = mean(R_t), mean(Q_t) - out: dict = {} - with np.errstate(divide="ignore", invalid="ignore"): - if measure == _importance_time.FUSSELL_VESELY: - shares = self._ccf_fv_shares( - evaluation, inputs, fv_type, method - ) - return {node: mean(shares[node]) / Q for node in self.nodes} - for node in self.nodes: - values, joint = evaluation.joints(node) - works, fails = values["works"], values["fails"] - if joint: - r1, q1 = values["up_ok"] / works, values["down_ok"] / works - r0, q0 = ( - values["up_bad"] / fails, - values["down_bad"] / fails, - ) - else: - r1, q1 = values["up_ok"], values["down_ok"] - r0, q0 = values["up_bad"], values["down_bad"] - change = np.where(Q_t <= R_t, q0 - q1, r1 - r0) - if measure == _importance_time.BIRNBAUM: - value = mean(change) - elif measure == _importance_time.IMPROVEMENT: - value = mean(fails * change) - elif measure == _importance_time.RAW: - value = mean(q0) / Q - elif measure == _importance_time.RRW: - value = Q / mean(q1) - elif kind == "failure": - value = mean(fails * change) / Q - else: - value = mean(works * change) / R - out[node] = value - return out - - def _ccf_fv_shares( - self, evaluation, inputs: tuple, fv_type: str, method: str - ) -> dict: - """The numerators of the Fussell-Vesely importances (see - ``_fv_numerators``) with the common-cause groups, at each point: - exactly, each group conditioned on within the smallest module - holding its members (see ``_ccf_modules.Evaluation - .failed_cut_sets``); as rare events, each set's probability of - having failed summed over its groups' combinations (see - ``_ccf_modules.expected_product``).""" - size = evaluation.shape - zero = np.zeros(size) - if method == "exact": - structure = self._set_structure() - if structure.always_works: - failed: dict = {} - else: - if fv_type == "p": - p, q, tables = inputs - evaluation = self._ccf_tabled( - p, q, tables, structure=structure.dual() - ) - failed = evaluation.failed_cut_sets() - return { - node: np.broadcast_to( - np.asarray(failed.get(node, zero), dtype=float), (size,) - ) - for node in self.nodes - } - outcomes = evaluation.outcomes_of_tables() - if fv_type == "c": - node_sets = self.get_min_cut_sets() - else: - node_sets = { - frozenset(path_set) - for path_set in self.get_min_path_sets( - include_in_out_nodes=False - ) - } - out = {node: np.zeros(size) for node in self.nodes} - for node_set in node_sets: - value = np.broadcast_to( - np.asarray( - _ccf_modules.expected_product( - node_set, evaluation.q, outcomes - ), - dtype=float, - ), - (size,), - ) - for node in node_set: - out[node] = out[node] + value - return out - - def _require_ccf_frequencies(self) -> None: - """Raise if the failure frequency with common-cause groups is not - worked out: with block replacements that take time (planned - outages at block times, which the groups' states would change).""" - if not self.ccf_groups: - return - self._require_ccf_long_run() - timed = [ - node - for node in self._block_nodes() - if self._preventive[node].duration is not None - ] - if timed: - raise NotImplementedError( - "The system's planned outages at block replacements that " - f"take time (of {sorted(timed, key=str)}) are not worked " - "out with common-cause groups, as yet." - ) - tested = [ - node - for node in self._inspection - if self._inspection[node].duration is not None - ] - if tested: - raise NotImplementedError( - "The system's planned outages at tests that take time (of " - f"{sorted(tested, key=str)}) are not worked out with " - "common-cause groups, as yet." - ) - - def _ccf_outage_terms( - self, working_nodes, broken_nodes - ) -> Tuple[List[Tuple[Any, float]], float]: - """``_outage_terms`` with common-cause groups, over the long-run - grid's times (see ``_ccf_long_run``): a node outside the groups - fails at its rate and takes the system down where it is critical - (its Birnbaum importance then, over the groups' states); each cause - strikes at its rate and takes the system down by failing the - members it names that are up (the rise in the system's - unavailability with them down).""" - self._require_ccf_frequencies() - self._require_free_members(working_nodes, broken_nodes) - times, weights, inputs = self._ccf_long_run( - working_nodes, broken_nodes - ) - availability = inputs[0] - evaluation = self._ccf_tabled(*inputs) - R_t, Q_t = evaluation.system() - forced = set(working_nodes or ()) | set(broken_nodes or ()) - members = {m for group in self.ccf_groups for m in group.members} - terms: List[Tuple[Any, float]] = [] - planned = 0.0 - blocks = set(self._block_nodes()) - for node in self.components: - if node in forced or node in members: - continue - up_1, down_1 = evaluation.system(hold={node: True}) - up_0, down_0 = evaluation.system(hold={node: False}) - importance = np.where(Q_t <= R_t, down_0 - down_1, up_1 - up_0) - if node in self._inspection: - node_failures: Any = self._tested_intensity( - node, times, availability[node] - ) - node_planned: Any = 0.0 - elif node in blocks: - node_failures = self._block_profile(node, times, rates=True) - node_planned = 0.0 - else: - node_failures, _, node_planned = self._node_frequencies(node) - terms.append((node, float(weights @ (importance * node_failures)))) - planned += float(weights @ (importance * node_planned)) - for group in self.ccf_groups: - life, _ = self._ccf_rates(group) - for struck, rate in _ccf_chain.causes( - group.model, group.members, life - ): - hit = {group.members[position]: False for position in struck} - rise = evaluation.system(hold=hit)[1] - Q_t - cause = ( - group.members[struck[0]] - if len(struck) == 1 - else tuple(group.members) - ) - terms.append((cause, rate * float(weights @ rise))) - return terms, planned - - def _validate_imperfect(self, node, spec: dict) -> Optional[_Imperfect]: - """A component spec's imperfect repair (``"repair"`` and - ``"replace_after"``), validated; None if it is renewed by every - repair, as by default (a restoration factor of 0).""" - repair, limit = spec.get("repair"), spec.get("replace_after") - if repair is None and limit is None: - return None - q, model = 0.0, "" - if repair is not None: - if ( - not isinstance(repair, dict) - or set(repair) != {"model", "q"} - or repair["model"] not in self.REPAIR_MODELS - ): - raise ValueError( - f"Component {node!r}: repair must be a dict of a " - f"'model' ({' or '.join(map(repr, self.REPAIR_MODELS))}) " - f"and its restoration factor 'q', got {repair!r}." - ) - q = repair["q"] - if ( - isinstance(q, bool) - or not isinstance(q, (int, float, np.number)) - or not 0.0 <= float(q) <= 1.0 - ): - raise ValueError( - f"Component {node!r}: the restoration factor q must be " - "a number in [0, 1] (0 renews the unit, 1 is minimal " - f"repair), got {q!r}." - ) - q, model = float(q), repair["model"] - if limit is not None: - if ( - isinstance(limit, bool) - or not isinstance(limit, (int, np.integer)) - or limit < 1 - ): - raise ValueError( - f"Component {node!r}: replace_after must be a whole " - f"number of failures of at least 1, got {limit!r}." - ) - if q == 0.0: - raise ValueError( - f"Component {node!r}: replace_after needs imperfect " - "repair (a 'repair' with q above 0): renewed by every " - "repair, the unit is as new after each failure anyway." - ) - limit = int(limit) - if q == 0.0: - return None - if spec.get("standby") is not None: - raise ValueError( - f"Component {node!r} is a standby group, whose units are " - "renewed by their repairs, so it cannot be repaired " - "imperfectly." - ) - preventive = spec.get("preventive") - if isinstance(preventive, dict) and preventive.get("policy") == ( - "condition" - ): - raise ValueError( - f"Component {node!r} is replaced on condition, judged by its " - "age as new, so it cannot be repaired imperfectly." - ) - life = spec["reliability"] - if not ( - is_mixture(life) or (hasattr(life, "Hf") and hasattr(life, "qf")) - ): - raise ValueError( - f"Component {node!r}: an imperfectly repaired unit's lives " - "are drawn given its virtual age, which needs a lifetime " - "model with a cumulative hazard (Hf) and quantiles (qf), " - "such as a surpyval distribution." - ) - return _Imperfect(model, q, limit) - - def _validate_groups( - self, options: Optional[dict] - ) -> Dict[Hashable, _MaintenanceGroup]: - """The maintenance groups (#108), from the components' ``"group"`` - and the options given for each, checked; and the members that can - be renewed early (``_early_members``).""" - members: Dict[Hashable, list] = {} - for node, group in self._member_group.items(): - try: - hash(group) - except TypeError: - raise ValueError( - f"Component {node!r}: a group is named by a hashable " - f"value, got {group!r}." - ) from None - for spec_key, what in ( - ("_inspection", "hidden failures"), - ("_standby", "a standby group"), - ): - if node in getattr(self, spec_key): - raise ValueError( - f"Component {node!r} has {what}, so it cannot be in " - "a maintenance group." - ) - members.setdefault(group, []).append(node) - self._early_members = { - node - for node, schedule in self._preventive.items() - if schedule.opportunity < schedule.interval - } - for node in self._early_members: - if node not in self._member_group: - raise ValueError( - f"Component {node!r} has an opportunity, but no " - "maintenance group to give it one: add a 'group'." - ) - options = dict(options or {}) - unknown = set(options) - set(members) - if unknown: - raise ValueError( - f"maintenance_groups names group(s) " - f"{sorted(map(str, unknown))} that no component is in." - ) - out: Dict[Hashable, _MaintenanceGroup] = {} - for group, nodes in members.items(): - given = options.get(group) or {} - if not isinstance(given, dict) or set(given) - set( - self.GROUP_KEYS - ): - raise ValueError( - f"Maintenance group {group!r}: its options must be a " - f"dict of {', '.join(self.GROUP_KEYS)}, got {given!r}." - ) - setup = given.get("setup_cost") - setup = ( - 0.0 - if setup is None - else self._validate_cost(group, "setup_cost", setup) + out.append( + "junction(s) " + + ", ".join(repr(n) for n in sorted(junctions, key=str)) ) - system_down = given.get("system_down", False) - if not isinstance(system_down, bool): - raise ValueError( - f"Maintenance group {group!r}: system_down must be True " - f"or False, got {system_down!r}." - ) - out[group] = _MaintenanceGroup(tuple(nodes), setup, system_down) return out - @staticmethod - def _unknown_component(node, component) -> str: - """Why ``component`` cannot be node ``node``: it is none of the - kinds a node can be.""" - from repyability.repairable import Repairable - - kinds = ( - "Give a spec dict with 'reliability' and 'repairability', a " - "NonRepairable, or a nested RepairableRBD (or, for a junction " - "that never fails, such as a k-out-of-n vote point, " - "PerfectReliability)." - ) - if isinstance(component, Repairable): - return ( - f"Component {node!r} is a Repairable, which models the " - "minimal or imperfect repair of a single unit: it cannot be " - "a node, whose repairs renew it as new. " + kinds - ) - spec = ( - "{'reliability': , 'repairability': }" - ) - distribution = unfitted_distribution(component) - if distribution is not None: - return ( - f"Component {node!r} is surpyval's {distribution} " - f"distribution itself: fit it to data " - f"(surv.{distribution}.fit(times)) or give its parameters " - f"(surv.{distribution}.from_params([...])), and give it " - f"with its repairs, as {spec}." - ) - if callable(getattr(component, "sf", None)): - return ( - f"Component {node!r} is a model of a life alone (a " - f"{type(component).__name__}); a repairable component needs " - f"its repairs too: give {spec}." + def _ccf_plan(self, structure) -> "_ccf_modules.Plan": + """Each common-cause group's owner in ``structure`` (the + decomposition, or its dual), the smallest module holding its + members (see ``_ccf_modules.Plan``): once for each structure.""" + cache = self._cache.kept("ccf_plans") + entry = cache.get(id(structure)) + if entry is None or entry[0] is not structure: + entry = cache[id(structure)] = ( + structure, + _ccf_modules.Plan(structure, self.ccf_groups), ) - return f"Component {node!r} is a {type(component).__name__}. " + kinds + return entry[1] - @classmethod - def _validate_component_spec(cls, node, spec: dict) -> None: - """Reject unknown keys in a component spec. + def _ccf_measure( + self, + measure: str, + inputs: tuple, + weights: Optional[np.ndarray], + kind: str = "failure", + fv_type: str = "c", + method: str = "exact", + held: Collection[Hashable] = (), + ) -> dict: + """An importance measure of every node with the common-cause groups + (``measure``, one of ``_importance_time``'s names), at the points + ``inputs`` gives (see ``_ccf_tabled``): averaged with ``weights``, + each ratio a ratio of the averages, or at each point without. - A mistyped cost key (``repair_costs``) would otherwise be silently - ignored and priced at zero, which is a quiet way to get the money - wrong; surface it at construction instead. - """ - unknown = set(spec) - set(cls.COMPONENT_SPEC_KEYS) - if unknown: + A node outside the groups is independent of them, so its measures + come from the system's probabilities with it held working and + failed, as without groups. A member's state says something of its + group's, so it is conditioned on: ``A(1_i)`` and ``A(0_i)`` at each + point are the system's availability given the member up and given + it down then, from the joint probabilities summed over its group's + combinations (see ``_ccf_modules.Evaluation.given``); where the + state has no chance at a point (a member off line for its test), + the system with it held so. Birnbaum's measure is the difference + between the two, taken from whichever end keeps its precision, the + system's unavailability or its availability. The joint importance + (``_importance_time.JOINT``) conditions on each pair's states the + same way (see ``_ccf_modules.joint_importance``), 0 for a pair + with a node in ``held`` (those held up or down). Fussell-Vesely + sums each combination's numerators, the nodes being independent + given it.""" + one_of("kind", kind, ("failure", "success")) + if fv_type not in ("c", "p"): raise ValueError( - f"Component {node!r} has unknown key(s) " - f"{sorted(map(str, unknown))}. A component spec takes " - f"{', '.join(cls.COMPONENT_SPEC_KEYS)}." + "fv_type must be either 'c' (cut-set) or 'p' (path-set), " + f"fv_type={fv_type!r} was given." ) - for key, what in ( - ("reliability", "its lives (time to failure)"), - ("repairability", "its repair times"), - ): - if key not in spec: - raise ValueError( - f"Component {node!r} needs a {key!r}: the " - f"distribution of {what}." - ) - value = spec[key] - if isinstance(value, (int, float)) and not isinstance(value, bool): - mean = "MTTF" if key == "reliability" else "MTTR" - instant = ( - ", or 'instant' for a repair in zero time" - if key == "repairability" - else "" - ) - raise TypeError( - f"Component {node!r}: {key} is the number {value!r}, " - f"not a distribution of {what}. For an {mean} of " - f"{value:g}, give e.g. " - f"surpyval.Exponential.from_params([1 / {value:g}]); " - "for a fixed time, " - f"surpyval.ExactEventTime.from_params([{value:g}])" - f"{instant}." - ) + one_of("method", method, ("exact", "rare_event")) + evaluation = _ccf_groups._ccf_tabled(self, *inputs) + R_t, Q_t = evaluation.system() - @classmethod - def _validate_preventive(cls, node, spec) -> Tuple[_Preventive, Any, Any]: - """A component's ``"preventive"`` spec, validated: its schedule, - and the cost of each replacement and of each inspection (None if - it prices nothing).""" - if not isinstance(spec, dict): - raise ValueError( - f"Component {node!r}: preventive must be a dict with " - f"{', '.join(cls.PREVENTIVE_KEYS)}, got {spec!r}." - ) - unknown = set(spec) - set(cls.PREVENTIVE_KEYS) - if unknown: - raise ValueError( - f"Component {node!r} has unknown preventive key(s) " - f"{sorted(map(str, unknown))}. A preventive spec takes " - f"{', '.join(cls.PREVENTIVE_KEYS)}." - ) - try: - interval = number_or_nan(spec["interval"]) - except KeyError: - raise ValueError( - f"Component {node!r}: a preventive spec needs an interval." - ) from None - if not interval > 0.0: - raise ValueError( - f"Component {node!r}: the preventive interval must be a " - f"positive number (inf for none), got {spec['interval']!r}." - ) - policy = spec.get("policy", "age") - if policy not in cls.PREVENTIVE_POLICIES: - raise ValueError( - f"Component {node!r}: the preventive policy must be 'age', " - f"'block' or 'condition', got {policy!r}." - ) - threshold = 0.0 - if policy == "condition": - if spec.get("threshold") is None: - raise ValueError( - f"Component {node!r}: the 'condition' policy needs a " - "threshold: the probability of failing before the next " - "inspection above which the unit is replaced." - ) - threshold = spec["threshold"] - if ( - isinstance(threshold, bool) - or not isinstance(threshold, (int, float, np.number)) - or not 0.0 <= float(threshold) <= 1.0 - ): - raise ValueError( - f"Component {node!r}: the threshold must be a " - f"probability, in [0, 1], got {threshold!r}." - ) - threshold = float(threshold) - else: - for key in ("threshold", "inspection_cost"): - if spec.get(key) is not None: - raise ValueError( - f"Component {node!r}: {key} applies only to the " - "'condition' policy, which inspects the unit." - ) - opportunity = math.inf - if spec.get("opportunity") is not None: - if policy != "age": - raise ValueError( - f"Component {node!r}: an opportunity applies only to " - "the 'age' policy (a unit renewed early at a stop of " - "its maintenance group)." - ) - opportunity = spec["opportunity"] - if ( - isinstance(opportunity, bool) - or not isinstance(opportunity, (int, float, np.number)) - or not 0.0 <= float(opportunity) <= interval - ): - raise ValueError( - f"Component {node!r}: the opportunity must be an age " - f"from 0 to the interval ({interval:g}), got " - f"{opportunity!r}." - ) - opportunity = float(opportunity) - duration = spec.get("duration", "instant") - if isinstance(duration, str) and duration == "instant": - duration = None - elif not hasattr(duration, "random"): - raise ValueError( - f"Component {node!r}: the preventive duration must be a " - "time-to-maintain model (such as a fitted surpyval " - f"distribution) or 'instant', got {duration!r}." - ) - costs = [] - for key, name in ( - ("cost", "preventive_cost"), - ("inspection_cost", "inspection_cost"), - ): - cost = spec.get(key) - if cost is not None: - cost = cls._validate_component_cost(node, name, cost) - if isinstance(cost, float) and cost == 0.0: - cost = None - costs.append(cost) - schedule = _Preventive( - interval, policy, duration, threshold, opportunity - ) - return schedule, costs[0], costs[1] - - @classmethod - def _validate_standby(cls, node, spec: dict) -> _Standby: - """A component spec's ``"standby"``, validated: the group of - units it makes the component (see ``_StandbyGroup``).""" - standby = spec["standby"] - if not isinstance(standby, dict): - raise ValueError( - f"Component {node!r}: standby must be a dict of " - f"{', '.join(cls.STANDBY_KEYS)}, got {standby!r}." - ) - unknown = set(standby) - set(cls.STANDBY_KEYS) - if unknown: - raise ValueError( - f"Component {node!r}: unknown standby key(s) " - f"{sorted(map(str, unknown))}; it takes " - f"{', '.join(cls.STANDBY_KEYS)}." - ) - for key in ("preventive", "inspection"): - if spec.get(key) is not None: - raise ValueError( - f"Component {node!r} is a standby group, which takes no " - f"{key!r} schedule." + def mean(value): + if weights is None: + return np.broadcast_to( + np.asarray(value, dtype=float), R_t.shape ) - if isinstance(spec["repairability"], str): - raise ValueError( - f"Component {node!r} is a standby group, whose units need a " - "repair time: repaired instantly, a spare would never be " - "needed." - ) - counts = {"units": standby.get("units", 2), "k": standby.get("k", 1)} - for key, value in counts.items(): - if ( - isinstance(value, bool) - or not isinstance(value, (int, float, np.integer, np.floating)) - or int(value) != value - or value < 1 - ): - raise ValueError( - f"Component {node!r}: standby {key} must be a whole " - f"number, 1 or more; got {value!r}." + return np.atleast_1d(weights @ np.asarray(value, dtype=float)) + + R, Q = mean(R_t), mean(Q_t) + out: dict = {} + with np.errstate(divide="ignore", invalid="ignore"): + if measure == _importance_time.FUSSELL_VESELY: + shares = _ccf_groups._ccf_fv_shares( + self, evaluation, inputs, fv_type, method ) - units, k = int(counts["units"]), int(counts["k"]) - if k >= units: - raise ValueError( - f"Component {node!r}: a standby group needs a spare, so k " - f"must be less than units; got k={k} and units={units}." - ) - fractions = { - "dormancy_factor": standby.get("dormancy_factor", 0.0), - "switching_probability": standby.get("switching_probability", 1.0), - } - for key, value in fractions.items(): - if ( - isinstance(value, bool) - or not isinstance(value, (int, float, np.integer, np.floating)) - or not 0.0 <= value <= 1.0 - ): - raise ValueError( - f"Component {node!r}: standby {key} must be a number in " - f"[0, 1]; got {value!r}." + return {node: mean(shares[node]) / Q for node in self.nodes} + if measure == _importance_time.JOINT: + held = set(held) + return Pairs.of( + list(self.nodes), + lambda i, j: mean( + 0.0 * R_t + if i in held or j in held + else _ccf_modules.joint_importance(evaluation, i, j) + ), ) - return _Standby( - units, - k, - float(fractions["dormancy_factor"]), - float(fractions["switching_probability"]), - ) - - @classmethod - def _validate_inspection(cls, node, spec) -> Tuple[_Inspection, Any]: - """A component's ``"inspection"`` spec, validated: its schedule, and - its cost (None if it prices nothing).""" - if not isinstance(spec, dict): - raise ValueError( - f"Component {node!r}: inspection must be a dict with " - f"{', '.join(cls.INSPECTION_KEYS)}, got {spec!r}." - ) - unknown = set(spec) - set(cls.INSPECTION_KEYS) - if unknown: - raise ValueError( - f"Component {node!r} has unknown inspection key(s) " - f"{sorted(map(str, unknown))}. An inspection spec takes " - f"{', '.join(cls.INSPECTION_KEYS)}." - ) - try: - interval = number_or_nan(spec["interval"]) - except KeyError: - raise ValueError( - f"Component {node!r}: an inspection spec needs an interval." - ) from None - if not (interval > 0.0 and np.isfinite(interval)): - raise ValueError( - f"Component {node!r}: the inspection interval must be a " - f"positive, finite number, got {spec['interval']!r}. (A " - "hidden failure is found only by an inspection.)" - ) - duration = spec.get("duration", "instant") - if isinstance(duration, str) and duration == "instant": - duration = None - elif not hasattr(duration, "random"): - raise ValueError( - f"Component {node!r}: the inspection duration must be a " - "time-to-test model (such as a fitted surpyval distribution) " - f"or 'instant', got {duration!r}." - ) - cost = spec.get("cost") - if cost is not None: - cost = cls._validate_component_cost(node, "inspection_cost", cost) - if isinstance(cost, float) and cost == 0.0: - cost = None - offset = _number_or_nan(spec.get("offset", 0.0)) - if not 0.0 <= offset < interval: - raise ValueError( - f"Component {node!r}: the inspection offset, the time of its " - "first test, must be at least 0 and less than its interval, " - f"{interval:g}, got {spec['offset']!r}." - ) - if offset <= _TINY_OFFSET * interval: - # An offset of 0 puts the first test at the interval; one this - # close to 0 (a share of the interval worked out as nearly 0, - # say) is taken as 0, not as a test at the start (#237). - offset = 0.0 - coverage = _number_or_nan(spec.get("coverage", 1.0)) - if not 0.0 <= coverage <= 1.0: - raise ValueError( - f"Component {node!r}: the inspection coverage, the chance " - "that a test finds a failure, must be from 0 to 1, got " - f"{spec['coverage']!r}." - ) - full_test = spec.get("full_test") - if full_test is not None: - given = full_test - full_test = _number_or_nan(full_test) - count = full_test / interval - if not ( - np.isfinite(count) - and round(count) >= 1 - and abs(count - round(count)) <= 1e-9 * count - ): - raise ValueError( - f"Component {node!r}: the full_test interval must be a " - f"whole multiple of its inspection interval, {interval:g} " - f"(every so many tests is a full one), got {given!r}." + for node in self.nodes: + given = evaluation.given((node,)) + (works, r1, q1), (fails, r0, q0) = ( + given[True,], + given[False,], ) - full_test = round(count) * interval - elif coverage < 1.0: - raise ValueError( - f"Component {node!r}: with a coverage below 1, the failures " - "its tests miss are found only by a full test: give its " - "full_test interval, a whole multiple of its inspection " - "interval (the mission time, say, after which it is " - "renewed)." - ) - schedule = _Inspection(interval, duration, offset, coverage, full_test) - return schedule, cost - - @classmethod - def _validate_cost(cls, node, key: str, value) -> float: - """Coerce a cost to a finite, non-negative float.""" - if hasattr(value, "qf"): - raise ValueError( - f"{node!r}: {key} must be a number, not a distribution. Only " - "the costs charged per action (repair_cost, replace_cost and " - "the preventive and inspection costs) may be distributions." - ) - if not is_number(value): - # Text that reads as a number, and a bool, are refused (#233). - raise ValueError( - f"{node!r}: {key} must be a number, got {value!r}." - ) - cost = float(value) - if not np.isfinite(cost) or cost < 0.0: - raise ValueError( - f"{node!r}: {key} must be finite and non-negative, got " - f"{value!r}." - ) - return cost - - @classmethod - def _validate_component_cost(cls, node, key: str, value): - """Validate a per-component cost: a number (coerced to float) or, for - the per-action costs, a distribution of the cost -- anything with - ``qf`` and ``mean``, such as a fitted surpyval model. - - A distribution must have a finite mean and put no more than 1e-12 - probability on a negative cost. - """ - if key not in cls._PER_ACTION_COST_KEYS or not hasattr(value, "qf"): - return cls._validate_cost(node, key, value) - mean = _mean_cost(value) - if not np.isfinite(mean): - raise ValueError( - f"{node!r}: the {key} distribution must have a finite mean, " - f"got {mean:g}." - ) - lowest = float(np.ravel(value.qf(np.array([1e-12])))[0]) - if not lowest >= 0.0: - raise ValueError( - f"{node!r}: the {key} distribution puts appreciable " - f"probability on a negative cost (its 1e-12 quantile is " - f"{lowest:g}). Use one on [0, inf), such as a LogNormal, " - "Gamma or Weibull." - ) - return value + if node not in evaluation.group_of: + works, fails = evaluation.p[node], evaluation.q[node] + change = np.where(Q_t <= R_t, q0 - q1, r1 - r0) + if measure == _importance_time.BIRNBAUM: + value = mean(change) + elif measure == _importance_time.IMPROVEMENT: + value = mean(fails * change) + elif measure == _importance_time.RAW: + value = mean(q0) / Q + elif measure == _importance_time.RRW: + value = Q / mean(q1) + elif kind == "failure": + value = mean(fails * change) / Q + else: + value = mean(works * change) / R + out[node] = value + return out @property def has_costs(self) -> bool: @@ -6780,18 +998,21 @@ def has_costs(self) -> bool: True if some component declares a non-zero ``"repair_cost"``, ``"replace_cost"``, ``"downtime_cost"``, or preventive-maintenance - or inspection ``"cost"`` (a cost distribution always counts), a + or inspection ``"cost"`` (a cost distribution always counts), is + on a ``"calendar"`` policy (charged its calendar's prices), a maintenance group has a set-up cost, or ``downtime_cost_rate`` is non-zero: whether running the system costs anything. Costs of 0 price nothing, costs declared inside a nested ``RepairableRBD`` do - not count, and neither does an + not count (a nested diagram that carries any is warned of when + this one is built), and neither does an ``"acquisition_cost"`` (a one-off cost, see ``total_cost``). When - nothing is priced there is no cost model to evaluate, so the cost - methods short-circuit rather than doing the work: - ``expected_cost_rate`` returns 0.0, and ``cost`` and the ``cost`` of - ``availability``'s result have no running cost, exactly, with the - acquisition cost beside it (#234), or are None if that is not - given either. + nothing is priced, not even an acquisition cost, there is no cost + model to evaluate: the cost analyses (``expected_cost_rate``, + ``total_cost``, ``expected_cost`` and ``cost``) refuse, as + ``routes.Refused``, and the ``cost`` of ``availability``'s result + is None. With an acquisition cost alone, ``expected_cost_rate`` + is 0.0, and ``cost`` and that result's ``cost`` have no running + cost, exactly, with the acquisition cost beside it. Returns ------- @@ -6807,352 +1028,6 @@ def has_costs(self) -> bool: ) ) - def _stream_specs( - self, - t_simulation: float, - prefix: tuple = (), - specs: Optional[dict] = None, - states: Optional[dict] = None, - ) -> Tuple[dict, bool]: - """The streams this RBD's simulations draw from (see ``_streams``), - by name, nested RBDs' included; and whether every draw comes from - one. A component whose draws cannot be streamed (a model other than - a surpyval parametric one, or a subclass of the component classes) - has none, nor has a maintenance or test time that cannot: they draw - from numpy's global RNG. ``prefix`` is this RBD's place in the RBD - simulated: only that RBD's own costs are drawn. A component started - from a state in ``states`` (see ``_simulation_states``) that draws - what is left of its life or repair has a stream for that too.""" - specs = {} if specs is None else specs - states = {} if states is None else states - complete = True - - def add(path: tuple, kind: int, sampler, expected: float) -> None: - rows = _streams.first_rows(expected) - specs[(path, kind)] = _streams.Spec( - path, kind, sampler, rows, _streams.block_width(rows) - ) - - for name, component in self.components.items(): - path = prefix + (name,) - if type(component) is RepairableRBD: - nested = component._stream_specs( - t_simulation, path, specs, states.get(name) - ) - complete = complete and nested[1] - continue - failure = repair = None - if type(component) is NonRepairable: - # A common-cause group's member draws its own cause's life. - own = self._own_rate(name) - failure = ( - stream_sampler(component.reliability) - if own is None - else _exponential_sampler(own) - ) - repair = stream_sampler(component.time_to_replace) - if failure is None or repair is None: - complete = False - continue - arrangement = self._standby.get(name) - if arrangement is not None: - # Each unit's lives and repairs, and the group's switches. - lives, switches = self._standby_draw_counts(name, t_simulation) - for unit in range(arrangement.units): - add(path + (unit,), _streams.FAILURE, failure, lives) - add(path + (unit,), _streams.REPAIR, repair, lives) - if 0.0 < arrangement.switching_probability < 1.0: - add(path, _streams.SWITCH, _uniforms, switches) - continue - expected = self._expected_draws(name, t_simulation) - add(path, _streams.FAILURE, failure, expected["failure"]) - add(path, _streams.REPAIR, repair, expected["repair"]) - if _draws_start(states.get(name)): - add(path, _streams.START, _uniforms, 1.0) - if name in self._imperfect: - # A life after each repair, given the unit's virtual age. - add(path, _streams.AGED, _uniforms, expected["repair"]) - inspection = self._inspection.get(name) - if inspection is not None and inspection.partial: - # Whether a test finds each failure: one for each failure - # a test can miss. - add(path, _streams.TEST, _uniforms, expected["failure"]) - duration = self._duration_model(name) - if duration is not None: - sampler = stream_sampler(duration) - if sampler is None: - complete = False - else: - add(path, _streams.DURATION, sampler, expected["action"]) - for cause in self._shared_causes(): - # A shared common cause's strikes (#158), and at each the - # uniform its failures' tests decide by. - strikes = cause.rate * t_simulation + 1.0 - add( - prefix + cause.struck, - _streams.CAUSE, - _exponential_sampler(cause.rate), - strikes, - ) - if cause.coins: - add( - prefix + cause.struck, - _streams.CAUSE_TEST, - _uniforms, - strikes, - ) - if not prefix: - for node, node_costs in self.costs.items(): - expected = self._expected_draws(node, t_simulation) - for key, kind in _streams.COST_KINDS.items(): - cost = node_costs.get(key) - if cost is None or isinstance(cost, float): - continue - count = expected[ - "action" if key in _ACTION_COST_KEYS else "repair" - ] - add((node,), kind, _streams.cost_sampler(cost), count) - return specs, complete - - def _duration_model(self, node): - """The model of ``node``'s maintenance or test time, or None.""" - schedule = self._preventive.get(node) or self._inspection.get(node) - return None if schedule is None else schedule.duration - - def _standby_draw_counts( - self, node, t_simulation: float - ) -> Tuple[float, float]: - """Roughly how many lives (and repairs) a simulation draws for each - unit of a standby group, and how many switches: its operating units - fail at their rate and its spares at the dormant one (see - ``_expected_draws``).""" - arrangement = self._standby[node] - life = _safe_mean(self.components[node].reliability) - if not life > 0.0: - return float("nan"), float("nan") - active = ( - arrangement.k - + (arrangement.units - arrangement.k) * arrangement.dormancy_factor - ) - failures = t_simulation * active / life - return failures / arrangement.units + 1.0, failures + 1.0 - - def _expected_draws(self, node, t_simulation: float) -> Dict[str, float]: - """Roughly how many times a simulation draws ``node``'s time to - failure (``"failure"``: one per unit put into service), its repair - time and per-failure costs (``"repair"``), and its maintenance or - test time and their costs (``"action"``). NaN where it is not known. - Only how the draws are computed depends on these (see - ``_streams``), never what they are.""" - component = self.components[node] - if node in self._standby: - # Its units' failures, each a repair charged for. - _, failures = self._standby_draw_counts(node, t_simulation) - return {"failure": failures, "repair": failures, "action": 0.0} - life = repair = float("nan") - if isinstance(component, NonRepairable): - life = _safe_mean(component.reliability) - repair = _safe_mean(component.time_to_replace) - if life == np.inf: - failures = 0.0 - elif life + repair > 0.0: - failures = t_simulation / (life + repair) - else: - failures = float("nan") - # A member renewed early at its group's stops (#108) has more lives - # and actions than these: its streams then take more chunks, which - # changes only how its draws are computed. - lives, actions = failures + 1.0, 0.0 - schedule = self._preventive.get(node) - inspection = self._inspection.get(node) - if schedule is not None: - actions = t_simulation / schedule.interval + 1.0 - if schedule.policy == "age": - lives = t_simulation / np.fmin(life, schedule.interval) + 1.0 - else: - lives = failures + actions - elif inspection is not None: - actions = t_simulation / inspection.interval + 1.0 - return { - "failure": float(lives), - "repair": failures + 1.0, - "action": actions, - } - - def _stream_plan( - self, - t_simulation: float, - entropy, - antithetic: bool, - states: Optional[dict] = None, - ) -> Tuple[_streams.Plan, bool]: - """A run's streams, and whether every draw comes from one; the - components start from ``states``.""" - specs, complete = self._stream_specs(t_simulation, states=states) - return _streams.Plan(entropy, antithetic, specs), complete - - def _streamed_components( - self, - run: _streams.Run, - made: Optional[dict] = None, - prefix: tuple = (), - ) -> dict[Any, Any]: - """What each component draws its events from in a run: a stand-in - replaying its draws from its own streams, nested RBDs' components - included, or the component itself when they cannot be streamed - (see ``_stand_in``). The stand-ins are called at exactly the points - the components would be. - - A component object used for several nodes has one simulation state, - so it gets one stand-in (``made``, by object). Since each node has - its own copy of a ``NonRepairable``, only a nested RBD object can be. - """ - made = {} if made is None else made - streamed: dict[Any, Any] = {} - for name, component in self.components.items(): - if id(component) not in made: - arrangement = self._standby.get(name) - made[id(component)] = ( - _stand_in( - component, - run, - made, - prefix + (name,), - self._duration_model(name), - self._imperfect.get(name), - ) - if arrangement is None - else _standby_draws( - component, run, prefix + (name,), arrangement - ) - ) - own = self._own_rate(name) - if own is not None and made[id(component)] is component: - # Not streamed: its own cause's life all the same. - made[id(component)] = self._own_draws(name, own) - streamed[name] = made[id(component)] - for cause in self._shared_causes(): - path = prefix + cause.struck - streamed[cause] = _CauseDraws( - cause.rate, - run.stream(path, _streams.CAUSE), - run.stream(path, _streams.CAUSE_TEST), - ) - return streamed - - def _own_draws(self, node, own: float) -> "_StreamedComponent": - """A common-cause group member's draws from numpy's global RNG: its - own cause's life (at the rate ``own``), and its repair, maintenance - and test times as it would draw them.""" - component = self.components[node] - return _StreamedComponent( - _ExponentialDraws(own), - _ModelDraws(component.time_to_replace), - model=self._duration_model(node), - ) - - def _keyed_charges(self, run: _streams.Run) -> Tuple[ - dict[Any, list[tuple[int, Any]]], - dict[Any, Any], - dict[Any, Any], - ]: - """For each costed node, its ``(category, charges)`` pairs: what is - charged at each of its failures (the categories are positions in - ``_CATEGORIES``); and, for each node with a preventive-maintenance - or an inspection cost, what is charged at each preventive action or - inspection. ``charges.draw()`` gives the next amount: a fixed cost, - or the next draw from the cost's own stream (see ``_streams``).""" - - def charges(node, key: str): - cost = self.costs[node][key] - if isinstance(cost, float): - return _Fixed(cost) - return run.stream((node,), _streams.COST_KINDS[key]) - - failures = { - node: [ - ( - _CATEGORIES.index(key.removesuffix("_cost")), - charges(node, key), - ) - for key in self.PER_FAILURE_COST_KEYS - if key in node_costs - ] - for node, node_costs in self.costs.items() - } - preventive = { - node: charges(node, "preventive_cost") - for node, node_costs in self.costs.items() - if "preventive_cost" in node_costs - } - inspection = { - node: charges(node, "inspection_cost") - for node, node_costs in self.costs.items() - if "inspection_cost" in node_costs - } - return failures, preventive, inspection - - def _context( - self, - t_simulation: float, - working_nodes, - broken_nodes, - method: str, - capacity: Optional[_CapacityRecorder], - entropy, - antithetic: bool, - states: Optional[dict] = None, - history: bool = False, - ) -> "_Context": - """Everything a run's simulations share: the streams, what each - component draws from, the charges, the structure function, and the - components' ``states`` at the start (checked, as - ``_simulation_states`` gives them; None for new). With - ``history``, each simulation records its histories (see - ``_replicate``).""" - states = {} if states is None else states - plan, complete = self._stream_plan( - t_simulation, entropy, antithetic, states - ) - run = _streams.Run(plan, reseed=not complete) - has_costs = self.has_costs - nodes = list(self.components) - return _Context( - t_simulation=t_simulation, - working=set(working_nodes), - broken=set(broken_nodes), - method=method, - capacity=capacity, - run=run, - sources=self._streamed_components(run), - charges=self._keyed_charges(run) if has_costs else ({}, {}, {}), - downtime_cost_rates={ - node: c["downtime_cost"] - for node, c in self.costs.items() - if "downtime_cost" in c - }, - has_costs=has_costs, - position={node: i for i, node in enumerate(nodes)}, - # Components with no maintenance or inspection that are not - # nested RBDs: their next event is simply their next draw. - plain={ - node - for node, component in self.components.items() - if node not in self._preventive - and node not in self._inspection - and node not in self._standby - and not isinstance(component, RepairableRBD) - }, - works=self._decomposition().structure_function(method), - states=states, - initial_up=bool( - self.is_system_working( - {c: c not in broken_nodes for c in self.components}, method - ) - ), - history=history, - ) - def expected_cost_rate( self, working_nodes: Optional[Collection[Hashable]] = None, @@ -7211,11 +1086,14 @@ def expected_cost_rate( and it fails at its constant rate while it is up. Every cost is optional and defaults to 0, so any subset can be - priced; with nothing priced (see ``has_costs``) this is 0.0. Costs + priced; with nothing priced, not even an acquisition cost (see + ``has_costs``), there is no cost model to evaluate, and this + refuses (with an acquisition cost alone it is 0.0). Costs are undiscounted, and only this RBD's own costs count: costs - declared inside a nested ``RepairableRBD`` are left out. The - simulated counterpart is ``cost``, whose ``cost_rate`` converges to - this value as the window grows. + declared inside a nested ``RepairableRBD`` are left out (a nested + diagram that carries any is warned of when this one is built). + The simulated counterpart is ``cost``, whose ``cost_rate`` + converges to this value as the window grows. Parameters ---------- @@ -7235,16 +1113,17 @@ def expected_cost_rate( Raises ------ ValueError - If something is priced and a working/broken node is unknown, is - the input or output node, or is in both sets. (With nothing - priced this returns 0.0 without any checks.) + If nothing is priced, not even an acquisition cost (see + ``has_costs``; as ``routes.Refused``), or a working/broken + node is unknown, is the input or output node, or is in both + sets. NotImplementedError - If something is priced and a component is under block - replacement with models its exact values do not cover (see - ``node_availability``), or has hidden failures its numerical - values do not cover (see ``mean_availability``); or, while a - component can wait for a repair crew, as for - ``mean_availability``: simulate it with ``cost``. + If a component is under block replacement with models its + exact values do not cover (see ``node_availability``), or has + hidden failures its numerical values do not cover (see + ``mean_availability``); or, while a component can wait for a + repair crew, as for ``mean_availability``: simulate it with + ``cost``. Examples -------- @@ -7268,128 +1147,16 @@ def expected_cost_rate( >>> round(rbd.expected_cost_rate(), 4) 13.6364 """ - if not self.has_costs: - return 0.0 - - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - forced = working_nodes | broken_nodes - setups = [ - group for group in self._maintenance.values() if group.setup_cost - ] - if setups: - self._require_separate_setups() - - rate = 0.0 - - # Production lost while the *system* is down. - if self.downtime_cost_rate: - unavailability = self.mean_unavailability( - working_nodes, broken_nodes - ) - rate += self.downtime_cost_rate * unavailability - - if not self.costs and not setups: - return rate - - if self._crews_couple(): - # Held working or broken, a node needs no crew, and the others - # have more of them: from the chain without it. - probabilities, weights = self._chain_probabilities( - working_nodes, broken_nodes - ) - node_availability = { - node: float(weights @ probabilities[node]) - for node in self.nodes - } - else: - node_availability = _squeeze_values( - self._probabilities_with_overrides( - self.node_availability(), working_nodes, broken_nodes - ) - ) - for node in self.costs: - rate += self._node_cost_rate( - node, node_availability[node], node in forced - ) - # A maintenance group's set-up, at each failure and each preventive - # replacement of a member (none is renewed early: that would have - # been refused), a forced member making none. - for group in setups: - rate += group.setup_cost * sum( - sum(self._node_actions(node, node_availability[node])) - for node in group.members - if node not in forced - ) - return rate - - def _node_cost_rate( - self, node, availability: float, forced: bool = False - ) -> float: - """A component's own running cost per unit time, in the long run: - its corrective, preventive and inspection actions, and its own - downtime (``availability`` its long-run availability). A forced node - never changes state, so it incurs no actions.""" - node_costs = self.costs.get(node, {}) - rate = 0.0 - # Corrective actions, charged per failure, and preventive ones. - per_action = sum( - _mean_cost(node_costs[key]) - for key in self.PER_FAILURE_COST_KEYS - if key in node_costs + return _costs.expected_cost_rate( + self, working_nodes=working_nodes, broken_nodes=broken_nodes ) - preventive = node_costs.get("preventive_cost") - if (per_action or preventive is not None) and not forced: - failures, maintained = self._node_actions(node, availability) - rate += per_action * failures - if preventive is not None: - rate += _mean_cost(preventive) * maintained - inspection = node_costs.get("inspection_cost") - if inspection is not None and not forced: - schedule = self._preventive.get(node) - if schedule is not None and schedule.policy == "condition": - # Inspected at each multiple of the interval at which it is - # up (one in a repair or replacement is not). - up = self._block_cycle(node).before - rate += _mean_cost(inspection) * up / schedule.interval - else: - # One test per interval, but those that fall in a repair, - # which are not done. - unit = self._tested_unit(node) - tests = ( - 1.0 / self._inspection[node].interval - if unit is None - else unit.long_run.inspections - ) - rate += _mean_cost(inspection) * tests - # Optional cost of *this component* being down, whether or not the - # system as a whole is. - downtime_cost = node_costs.get("downtime_cost", 0.0) - if downtime_cost: - rate += downtime_cost * (1.0 - availability) - return rate - - def _node_actions(self, node, availability: float) -> Tuple[float, float]: - """A component's corrective and preventive actions per unit time, - in the long run (``availability`` its long-run availability).""" - if self._crews_couple(): - # Waiting for a crew, as while repaired, it cannot fail: it - # fails at its constant rate while it is up. - life = self._crew_chain_rates()[node][0] - return life * availability, 0.0 - if node in self._standby: - # Each of its units' failures is a repair. - return self._standby_long_run(node).unit_failure_frequency, 0.0 - failures, maintained, _ = self._node_frequencies(node) - return failures, maintained @property def acquisition_cost(self) -> float: """The one-off cost of buying the components: the sum of their ``"acquisition_cost"`` (0.0 if none is given). Only this RBD's own components count, not those inside a nested ``RepairableRBD``.""" - return float(sum(self.acquisition_costs.values())) + return _costs.acquisition_cost(self) def total_cost( self, @@ -7410,7 +1177,7 @@ def total_cost( with ``acquisition_cost`` the sum of the components' ``"acquisition_cost"``. With a ``discount_rate`` ``r`` it is the - present value (#184): the components are bought at the start, and + present value: the components are bought at the start, and the running costs, spent at a steady rate, are discounted continuously, so the horizon counts as ``(1 - exp(-r * horizon)) / r``. The running cost is the long-run rate, @@ -7425,7 +1192,7 @@ def total_cost( horizon : float or array-like How long the system is owned, in the time unit of the component models: non-negative, and finite unless the costs are discounted - (``inf``, owned for ever, #231). An array of horizons gives a + (``inf``, owned for ever). An array of horizons gives a total for each. working_nodes : Collection[Hashable], optional As for ``expected_cost_rate``, by default None. @@ -7436,7 +1203,7 @@ def total_cost( models, by default 0 (undiscounted): for an annual rate of 7% with models in hours, ``math.log(1.07) / 8760``. A rate that discounts the costs away (one per year with models in hours, - say) is warned of (#231). + say) is warned of. Returns ------- @@ -7446,9 +1213,10 @@ def total_cost( Raises ------ ValueError - If ``horizon`` is negative, or infinite without a - ``discount_rate``; if ``discount_rate`` is not a finite, - non-negative number; or as for ``expected_cost_rate``. + As for ``expected_cost_rate`` (nothing priced, as + ``routes.Refused``, first); if ``horizon`` is negative, or + infinite without a ``discount_rate``; or if ``discount_rate`` + is not a finite, non-negative number. NotImplementedError As for ``expected_cost_rate``. @@ -7485,298 +1253,13 @@ def total_cost( >>> round(rbd.total_cost(87600.0, discount_rate=r)) 114538 """ - rate = _discount_rate(discount_rate) - present = _present_horizon( - _horizons(horizon, rate), rate, self._mean_lives - ) - total = self.acquisition_cost + present * self.expected_cost_rate( - working_nodes, broken_nodes - ) - return float(total) if np.ndim(total) == 0 else total - - def _mean_lives(self) -> List[float]: - """The components' mean lives (their failure-time scales), against - which an endless horizon's discount rate is judged (#231).""" - return [ - failure_time_scale(component.reliability) - for component in self.components.values() - if isinstance(component, NonRepairable) - ] - - def _spares_nodes(self, nodes, given: str = "nodes") -> list: - """The components whose spares are counted: ``nodes`` (given in - ``given``) checked, or every component that is not a nested RBD.""" - if nodes is None: - return [ - node - for node, component in self.components.items() - if not isinstance(component, RepairableRBD) - ] - chosen = list(dict.fromkeys(nodes)) - for node in chosen: - self._require_component(node, given) - if isinstance(self.components[node], RepairableRBD): - raise ValueError( - f"Node {node!r} is a nested RBD: its spares are its " - "components', which its own spares_demand counts." - ) - return chosen - - def _spares_parts(self, parts) -> Dict[Hashable, Tuple[Hashable, ...]]: - """The parts whose spares are pooled (#183), ``{part: members}``, - checked: each a collection of components (no nested RBD), a node - in one part at most, a part's name no component's, and no two - members in one common-cause group, whose shared causes replace - them together.""" - if parts is None: - return {} - if not isinstance(parts, Mapping) or not parts: - raise ValueError( - "parts must be a dict of part name: the components that use " - f"it, e.g. {{'seal': ['seal1', 'seal2']}}; got {parts!r}." - ) - out: Dict[Hashable, Tuple[Hashable, ...]] = {} - owner: Dict[Hashable, Hashable] = {} - for part, members in parts.items(): - if part in self.components: - raise ValueError( - f"Part {part!r} has the name of a component: name the " - "part apart, as both can be counted." - ) - if isinstance(members, (str, bytes)) or not isinstance( - members, Iterable - ): - raise ValueError( - f"Part {part!r}: give the components that use it as a " - f"list, got {members!r}." - ) - listed = list(members) - twice = sorted( - {repr(n) for n in listed if listed.count(n) > 1}, key=str - ) - if twice: - raise ValueError( - f"Part {part!r} lists {', '.join(twice)} more than once: " - "name each component that uses it once." - ) - chosen = self._spares_nodes(listed, f"part {part!r}") - if not chosen: - raise ValueError(f"Part {part!r} has no components.") - for node in chosen: - if node in owner: - raise ValueError( - f"Node {node!r} is in parts {owner[node]!r} and " - f"{part!r}: its spares come from one shelf." - ) - owner[node] = part - for group in self.ccf_groups: - shared = [m for m in group.members if m in chosen] - if len(shared) > 1: - raise NotImplementedError( - f"Part {part!r}: {shared} are a common-cause group, " - "whose shared causes replace them together, so " - "their pooled demand is not the sum of independent " - "ones: count them apart." - ) - out[part] = tuple(chosen) - return out - - def _spares_counted(self, nodes, parts) -> Tuple[list, dict, list]: - """The components counted on their own (``nodes``, or every one - but a nested RBD; none when only ``parts`` are asked for), the - parts (see ``_spares_parts``), and every component either counts, - in order.""" - pools = self._spares_parts(parts) - chosen = [] if (nodes is None and pools) else self._spares_nodes(nodes) - if nodes is not None: - for part, members in pools.items(): - both = [node for node in chosen if node in members] - if both: - raise ValueError( - f"Node {both[0]!r} is in nodes and in part {part!r}: " - "its spares come from one shelf, the part's or its " - "own." - ) - counted = list( - dict.fromkeys( - [*chosen, *(m for members in pools.values() for m in members)] - ) - ) - return chosen, pools, counted - - def _replacements(self, node, long_run: bool = False): - """What a component's replacements follow, for counting them - exactly (see ``_spares``): a renewal process, a block schedule's - (``_spares.Block``) or a test lattice's (``_spares.Tested``); raise - if they are not counted exactly. ``long_run``: for the counts in a - lead time, which need a demand that goes on.""" - simulate = ( - # The stock in a lead time has no simulation of its own to - # point to (#232). - "spares_stock has no simulation to count them by; " - "spares_demand(method='simulate') counts the spares used over a " - "window from new." - if long_run - else "count its spares by simulation: " - "spares_demand(method='simulate')." - ) - self._require_reliabilities(node) - if node in self._standby: - raise NotImplementedError( - f"Component {node!r} is a standby group, whose units' " - f"failures depend on each other: {simulate}" - ) - if node in self._inspection: - return self._tested_replacements(node, simulate) - schedule = self._preventive.get(node) - if schedule is not None and schedule.policy == "block": - blocker = self._block_lead_time_blocker(node) if long_run else None - if blocker is not None: - raise NotImplementedError( - f"Component {node!r} is replaced on a block schedule, " - f"and {blocker}: replacements can then come several at " - "one instant (a new unit failing as it is put in), " - "which its demand in a lead time does not take. " - "spares_demand counts its spares over a horizon, and " - "spares_demand(method='simulate') in the simulation." - ) - component = self.components[node] - duration = schedule.duration - return _spares.Block( - _cdf(component.reliability), - _cdf(component.time_to_replace), - None if duration is None else _cdf(duration), - float(schedule.interval), - ) - if schedule is not None and schedule.policy == "condition": - raise NotImplementedError( - f"Component {node!r} is replaced on condition, at " - f"inspections on a calendar that do not renew it: {simulate}" - ) - if node in self._early_members: - raise NotImplementedError( - f"Component {node!r} is renewed early at the stops of its " - f"maintenance group, which depend on the other members: " - f"{simulate}" - ) - if node in self._imperfect: - raise NotImplementedError( - f"Component {node!r} is repaired imperfectly " - f"({self._imperfect_phrase(node)}), so its replacements are " - f"not a renewal process of lives as new: {simulate}" - ) - component = self.components[node] - life, repair = component.reliability, component.time_to_replace - age = math.inf if schedule is None else float(schedule.interval) - maintenance = None if schedule is None else schedule.duration - if math.isinf(age): - up = _safe_mean(life) - fails = 1.0 - else: - up = float(component.avg_replacement_time(age)) - fails = float(np.ravel(_cdf(life)(np.array([age])))[0]) - down = fails * _safe_mean(repair) + (1.0 - fails) * ( - 0.0 if maintenance is None else _safe_mean(maintenance) - ) - mean_cycle = up + down - if long_run and not (0.0 < mean_cycle < math.inf): - raise NotImplementedError( - f"The life of component {node!r} may never end, or has no " - "mean, " - "so its demand has no long run: it has no stock level." - ) - return _spares.Replacements( - _cdf(life), - _cdf(repair), - None if maintenance is None else _cdf(maintenance), - age, - mean_cycle, - ) - - def _block_lead_time_blocker(self, node) -> Optional[str]: - """Why a block-replaced component's demand in a lead time is not - worked out (None if it is): a unit may be dead on arrival while its - repairs or block replacements may take no time, so that - replacements can come several at one instant (#160; see - ``_spares``).""" - component = self.components[node] - - def at_zero(model) -> float: - return 1.0 - float(np.ravel(_sf_values(model.sf, np.zeros(1)))[0]) - - if at_zero(component.reliability) <= 0.0: - return None - duration = self._preventive[node].duration - if at_zero(component.time_to_replace) > 0.0: - return ( - "its life may end at 0 (dead on arrival) while its repairs " - "may take no time" - ) - if duration is None or at_zero(duration) > 0.0: - return ( - "its life may end at 0 (dead on arrival) while its block " - "replacements may take no time" - ) - return None - - def _tested_replacements(self, node, simulate: str) -> "_spares.Tested": - """A component with hidden failures: its replacements fall on its - tests (see ``_spares.Tested``): exactly, tested and repaired in no - time by tests that find every failure, or from its cycle on a grid - when its tests or repairs take time or its tests can miss a failure - (#159); raise if they cannot be counted so.""" - inspection = self._inspection[node] - component = self.components[node] - if inspection.partial or self._tested_kind(node) == "unit": - # The replacements still fall on the tests that find failures, - # whatever the tests and repairs take; with tests that can miss - # a failure, a cycle depends on the place in the full tests' - # period of the test that starts it (#159). - instead = simulate[0].upper() + simulate[1:] - check_tested( - component.reliability, - component.time_to_replace, - inspection.duration, - inspection.interval, - node, - instead, - ) - unit = self._tested_unit(node, any_kind=True) - assert unit is not None - first = float(inspection.offset or inspection.interval) - # The full tests are at the offset and every full test's - # interval from it: the first test is one, or the first after - # 0, an interval on, is the one after it. - position = 0 if inspection.offset else 1 % unit.per - found_at, cycle = unit.renewals(first, position, instead) - return _spares.Tested( - first, float(inspection.interval), found_at, cycle, position - ) - life = self._tested_life(node) or TestedLife( - component.reliability, inspection.interval - ) - cycle = np.where( - life.down[:-1] <= 0.5, - np.diff(life.down), - life.up[:-1] - life.up[1:], + return _costs.total_cost( + self, + horizon=horizon, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + discount_rate=discount_rate, ) - model = component.reliability - first = float(inspection.offset or inspection.interval) - interval = float(inspection.interval) - - def found(tests: int) -> np.ndarray: - # The unit new at 0 fails before the first test, or between two. - times = first + interval * np.arange(tests) - failed = np.clip(_cdf(model)(times), 0.0, 1.0) - survive = np.clip(_sf_values(model.sf, times), 0.0, 1.0) - before = np.concatenate([[0.0], failed[:-1]]) - alive = np.concatenate([[1.0], survive[:-1]]) - return np.maximum( - np.where(before <= 0.5, failed - before, alive - survive), - 0.0, - ) - - return _spares.Tested(first, interval, found, np.maximum(cycle, 0.0)) def spares_demand( self, @@ -7821,19 +1304,27 @@ def spares_demand( Identical parts often share one shelf: the seals of a station's three pumps come from one bin. ``parts={part: [nodes]}`` pools the - spares of each part's components (#183): their counts are + spares of each part's components: their counts are independent, so the part's is their sum (by simulation, the sum in each simulation). Its components may differ, one under age replacement and the others not. + The exact count convolves a component's replacement times on a + grid, once for each replacement likely in the horizon, so it + grows with the horizon over the component's life (a few seconds + for some hundreds of replacements), and under block replacement + with the number of block intervals too: count a long horizon by + simulation. + Parameters ---------- horizon : float The time counted over, from new. nodes : Collection[Hashable], optional - The components to count for, by default every component but a - nested RBD (whose own ``spares_demand`` counts its components'), - or none when ``parts`` are given. + The components to count for, by default every component, or + none when ``parts`` are given. A nested RBD's components' + spares are not counted here (its own ``spares_demand`` counts + them): with one, give the components counted. fleet : int, optional How many systems, by default 1. method : str, optional @@ -7861,7 +1352,9 @@ def spares_demand( If ``horizon`` is negative or not finite, ``fleet`` is not a whole number of at least 1, ``method`` is unknown, ``nodes`` or a part names a node that is not a component, or a nested RBD, - or ``parts`` is not as above. + or ``parts`` is not as above; and, as ``routes.Refused``, when + no ``nodes`` are given and a component is a nested RBD, whose + components' spares are not counted here. NotImplementedError For a part with two members in one common-cause group; and with ``method="exact"``, for a standby group, a component with @@ -7892,71 +1385,16 @@ def spares_demand( >>> demand.stock(0.95) 62 """ - end = _horizon(horizon) - fleet = _whole("fleet", fleet) - if method not in ("exact", "simulate"): - raise ValueError( - f"method must be 'exact' or 'simulate', got {method!r}." - ) - chosen, pools, counted = self._spares_counted(nodes, parts) - if method == "simulate": - runs = self._simulated_replacements( - end, - counted, - 10_000 if mc_samples is None else mc_samples, - seed, - ) - counts = {node: _fractions(runs[node]) for node in chosen} - pooled = { - part: _fractions(np.sum([runs[m] for m in members], axis=0)) - for part, members in pools.items() - } - else: - self._require_unlimited_crews(*_SPARES_CREWS) - models = {node: self._replacements(node) for node in counted} - each = { - node: _spares.count(model, end, "new") - for node, model in models.items() - } - counts = {node: each[node] for node in chosen} - pooled = { - part: _summed(each[m] for m in members) - for part, members in pools.items() - } - out = { - node: SparesDemand(_fleet(count, fleet), end, fleet, method) - for node, count in counts.items() - } - for part, count in pooled.items(): - out[part] = SparesDemand( - _fleet(count, fleet), end, fleet, method, pools[part] - ) - return out - - def _simulated_replacements( - self, horizon: float, nodes: list, mc_samples: int, seed - ) -> Dict[Any, np.ndarray]: - """How many times each of ``nodes`` was replaced in each of - ``mc_samples`` simulations of ``[0, horizon)``.""" - montecarlo.check_count(mc_samples, False, "mc_samples") - if horizon == 0.0: - # Nothing is replaced in no time (as the exact count says). - return { - node: np.zeros(mc_samples, dtype=np.int64) for node in nodes - } - tally = self._run( - horizon, - set(), - set(), - "p", - mc_samples, - False, - seed, - replacements=True, + return _spares.spares_demand( + self, + horizon=horizon, + nodes=nodes, + fleet=fleet, + method=method, + mc_samples=mc_samples, + seed=seed, + parts=parts, ) - counts = np.array(tally.replacements, dtype=np.int64) - column = {node: c for c, node in enumerate(self.components)} - return {node: counts[:, column[node]] for node in nodes} def spares_stock( self, @@ -7993,7 +1431,7 @@ def spares_stock( tests that can miss a failure, over where each cycle starts between the full tests, see ``_spares``). - Under block replacement every ``T`` (#160), with repairs and block + Under block replacement every ``T``, with repairs and block replacements in no time, each block interval starts with a new unit, so the demand repeats every interval, and a lead time's depends on where in the interval it starts: it is averaged over @@ -8007,19 +1445,21 @@ def spares_stock( interval or a block replacement, each followed block interval by block interval, the replacements before one as those after it, and from a random time by Campbell's formula; on grids of the - interval, extrapolated, to about 1e-6. A unit dead on arrival is - refused while its repairs or block replacements may take no time, - as replacements could then come several at one instant. A fleet's + interval, extrapolated, to about 1e-6. A unit dead on arrival + fails as it is put in: with repairs and block replacements in no + time, each replacement brings a geometric number more at its + instant, which the lasting units' count is compounded with + (#322); taking time, they are followed on the grid. A fleet's systems are taken as on block schedules of their own, out of step with each other. - A part's components draw on one shelf (#183), which needs fewer + A part's components draw on one shelf, which needs fewer spares than a shelf each: its spares on order are the sum of its components' independent ones, and a demand comes from component ``i`` with the share ``rate_i / sum(rates)`` of their long-run replacement rates, finding ``i``'s as its own demands do and the others' as at a random time (see the spares - [guide](guide/spares.md#one-shelf-for-interchangeable-parts)). + [guide](../guide/spares.md#one-shelf-for-interchangeable-parts)). Parameters ---------- @@ -8032,8 +1472,10 @@ def spares_stock( At least one of the two targets must be given; the stock meets both. nodes : Collection[Hashable], optional - The components, by default every component but a nested RBD - (none when ``parts`` are given). + The components, by default every component (none when + ``parts`` are given); with a nested RBD, whose components' + spares are not counted here, give them, as for + ``spares_demand``. fleet : int, optional How many systems share the stock, by default 1. parts : dict, optional @@ -8053,12 +1495,13 @@ def spares_stock( given or one is not in ``(0, 1)``, ``fleet`` is not a whole number of at least 1, ``nodes`` or a part names a node that is not a component, or a nested RBD, or ``parts`` is not as for - ``spares_demand``. + ``spares_demand``; and, as ``routes.Refused``, when no + ``nodes`` are given and a component is a nested RBD. NotImplementedError For a part with two members in one common-cause group, or two under block replacement, a component under block replacement whose repairs or block replacements take time or whose life - may end at 0 (#160), with hidden failures whose tests can last + may end at 0, with hidden failures whose tests can last as long as their interval, a standby group or a life that may never end, or while a component can wait for a repair crew, or if more than 2,000 replacements are likely in a lead time. @@ -8086,105 +1529,15 @@ def spares_stock( >>> round(stock.fill_rate, 4) 0.9665 """ - tau = _horizon(lead_time) - fleet = _whole("fleet", fleet) - if fill_rate is None and stockout_probability is None: - raise ValueError( - "Give a fill_rate, a stockout_probability, or both." - ) - for name, target in ( - ("fill_rate", fill_rate), - ("stockout_probability", stockout_probability), - ): - if target is not None and not 0.0 < target < 1.0: - raise ValueError(f"{name} must be in (0, 1), got {target!r}.") - chosen, pools, counted = self._spares_counted(nodes, parts) - self._require_unlimited_crews(*_STOCK_CREWS) - models = {node: self._replacements(node, True) for node in counted} - for part, members in pools.items(): - blocks = [ - m for m in members if isinstance(models[m], _spares.Block) - ] - if len(blocks) > 1: - raise NotImplementedError( - f"Part {part!r}: {blocks} are replaced on block " - "schedules, which keep step with each other, so their " - "demands in a lead time are not independent: count " - "them apart." - ) - random = { - node: _spares.count(model, tau, "random") - for node, model in models.items() - } - arrival = { - node: _spares.count(model, tau, "arrival") - for node, model in models.items() - } - - def stocked(members: Tuple[Hashable, ...]) -> SparesStock: - if len(members) == 1: - node = members[0] - on_order = _fleet(random[node], fleet) - at_demand = ( - arrival[node] - if fleet == 1 - else _fleet( - np.convolve( - arrival[node], _fleet(random[node], fleet - 1) - ), - 1, - ) - ) - else: - # A demand finds its own component's replacements as they - # come (``arrival``), and the others' as at a random time, - # each component's share of the demands its share of the - # long-run rates. - one = _summed(random[m] for m in members) - others = _fleet(one, fleet - 1) - rates = np.array([_spares.rate(models[m]) for m in members]) - on_order = _fleet(one, fleet) - at_demand = np.zeros(1) - for i, member in enumerate(members): - seen = _summed( - [arrival[member], others] - + [random[m] for j, m in enumerate(members) if j != i] - ) - size = max(len(at_demand), len(seen)) - at_demand = np.pad(at_demand, (0, size - len(at_demand))) - at_demand += ( - rates[i] - / rates.sum() - * np.pad(seen, (0, size - len(seen))) - ) - at_demand = at_demand / at_demand.sum() - result = SparesStock(0, 0.0, 1.0, tau, fleet, on_order, at_demand) - stock = 0 - while not ( - ( - fill_rate is None - or result.fill_rate_for(stock) >= fill_rate - 1e-12 - ) - and ( - stockout_probability is None - or result.stockout_probability_for(stock) - <= stockout_probability + 1e-12 - ) - ): - stock += 1 - return dataclasses.replace( - result, - stock=stock, - fill_rate=result.fill_rate_for(stock), - stockout_probability=result.stockout_probability_for(stock), - ) - - out: Dict[Hashable, SparesStock] = { - node: stocked((node,)) for node in chosen - } - for part, members in pools.items(): - out[part] = dataclasses.replace(stocked(members), members=members) - return out + return _spares.spares_stock( + self, + lead_time=lead_time, + fill_rate=fill_rate, + stockout_probability=stockout_probability, + nodes=nodes, + fleet=fleet, + parts=parts, + ) def allocate_redundancy( self, @@ -8253,7 +1606,7 @@ def allocate_redundancy( their costs are counted once. Nested ``RepairableRBD`` nodes cannot be given copies. Given ``trains``, by default none. trains : dict, optional - Trains of components that may be given copies (#184), ``{name: + Trains of components that may be given copies, ``{name: [nodes]}``: each a chain in series, its first node fed by any nodes, each of the others by the one before it alone, and its last feeding one node alone (a pump train of seal, bearing and @@ -8310,8 +1663,9 @@ def allocate_redundancy( a component can wait for a repair crew (see ``repair_crews``): the search assumes that none does. Or if a common-cause group's chain does not cover it (see ``ccf_groups``), a member given - copies is in an ``MGL`` group (its letters are for its group's - size), or a train holds a member. + copies is in an ``MGL`` or ``AlphaFactor`` group (its + parameters are for its group's size), or a train holds a + member. Examples -------- @@ -8380,378 +1734,17 @@ def allocate_redundancy( >>> best.units, round(best.total_cost) ({'train 1': 2}, 295306) """ - rate = _discount_rate(discount_rate) - if np.ndim(horizon) != 0: - raise ValueError( - f"horizon must be a number for allocate_redundancy, got " - f"{horizon!r}." - ) - horizon = float(_horizons(horizon, rate)) - # The running costs count over the horizon's present value. - present = float( - _present_horizon(np.array(horizon), rate, self._mean_lives) - ) - if method not in ("exact", "greedy"): - raise ValueError( - f"method must be 'exact' or 'greedy', got {method!r}." - ) - if nodes is None and trains is not None: - chosen: list = [] - elif nodes is None: - chosen = [ - n for n in self.components if n in self.acquisition_costs - ] - if not chosen: - raise ValueError( - "No component has an acquisition_cost, so there is " - "nothing to buy copies of: give the components one, or " - "name the nodes that may be given copies with `nodes`." - ) - else: - chosen = list(dict.fromkeys(nodes)) - if not chosen: - raise ValueError("nodes must name at least one component.") - for node in chosen: - self._require_component(node, "nodes") - if isinstance(self.components[node], RepairableRBD): - raise ValueError( - f"Node {node!r} is a nested RepairableRBD, which " - "cannot be given copies here: its costs are not " - "this RBD's." - ) - drawn = self._allocation_trains(trains, chosen) - items = chosen + [train.name for train in drawn] - caps = redundancy_caps( - items, max_units, named="nodes or trains" if drawn else "nodes" - ) - max_unavailability = None - if min_availability is not None: - try: - target = float(min_availability) - except (TypeError, ValueError): - target = float("nan") - if not 0.0 < target < 1.0: - raise ValueError( - "min_availability must be a number in (0, 1), got " - f"{min_availability!r}." - ) - max_unavailability = 1.0 - target - - # Adding copies adds jobs for the crews: the search assumes none waits. - self._require_unlimited_crews(*_ALLOCATION_CREWS) - grouped = self._require_groups_allocated(chosen, drawn) - # Every node's availability (and unavailability) over the times the - # long-run values average over, and each component's own cost. - times, weights = self._long_run_grid() - up = { - node: np.atleast_1d(np.asarray(a, dtype=float)) - for node, a in self._availabilities_at(times).items() - } - down = { - node: np.atleast_1d(np.asarray(u, dtype=float)) - for node, u in self._unavailabilities_at(times).items() - } - copy_cost = {} - for node in self.components: - rate = self._node_cost_rate(node, self._node_availability(node)) - copy_cost[node] = ( - self.acquisition_costs.get(node, 0.0), - rate, - self.acquisition_costs.get(node, 0.0) + present * rate, - ) - # A train's copy costs its nodes'. - train_cost = [ - math.fsum(copy_cost[node][2] for node in train.members) - for train in drawn - ] - item_cost = [copy_cost[node][2] for node in chosen] + train_cost - for index, (name, cost, cap) in enumerate(zip(items, item_cost, caps)): - if cost <= 0.0 and cap == math.inf: - what = "train " if index >= len(chosen) else "" - raise ValueError( - f"A copy of {what}{name!r} costs nothing over the horizon " - "(it has no acquisition or running cost), so copies " - "could be added without end: give it an " - "acquisition_cost or a max_units." - ) - size = len(times) - structures: Dict[tuple, Any] = {(): self._decomposition()} - - def structure(extra: tuple): - # The diagram with extra[i] copies of train i drawn alongside it. - if not any(extra): - return structures[()] - if extra not in structures: - structures[extra] = self._decompose_graph( - self._with_train_copies(drawn, extra) - ) - return structures[extra] - - # With common-cause groups (#158), a member's copies join its group: - # each design's groups' states, kept. - designs: Dict[tuple, list] = {} - - def group_states(copies: dict) -> list: - key = tuple( - copies.get(m, 1) for g in self.ccf_groups for m in g.members - ) - if key not in designs: - designs[key] = [ - self._group_states( - group, - times, - [ - ( - 0 - if copies.get(m, 1) == math.inf - else copies.get(m, 1) - ) - for m in group.members - ], - ) - for group in self.ccf_groups - ] - return designs[key] - - def unavailability(counts) -> float: - p, q = dict(up), dict(down) - joining: dict = {} - for node, n in zip(chosen, counts): - if node in grouped: - joining[node] = n - if n == math.inf: - p[node], q[node] = np.ones(size), np.zeros(size) - elif n != 1: - q[node] = down[node] ** n - p[node] = 1.0 - q[node] - extra = [] - for train, n in zip(drawn, counts[len(chosen) :]): - # Made perfect (the search's bound), it is as many perfect - # copies as its vote needs. - copies = train.k if n == math.inf else int(n) - 1 - extra.append(copies) - for index in range(1, copies + 1): - for node in train.members: - name = _TrainCopy(train.name, index, node) - if n == math.inf: - p[name], q[name] = np.ones(size), np.zeros(size) - else: - p[name], q[name] = up[node], down[node] - points, at = size, weights - if self.ccf_groups: - p, split, at, _ = self._with_ccf_groups( - times, - p, - q, - weights, - group_states(joining), - "The redundancy allocation", - ) - assert split is not None - q, points = split, len(at) - _, fails = structure(tuple(extra)).probabilities( - p, q, shape=points, works=False, fails=True - ) - fails = np.broadcast_to(np.asarray(fails, dtype=float), (points,)) - return float(at @ fails) - - copies_down: Dict[tuple, np.ndarray] = {} - - def all_down(node, k: int) -> np.ndarray: - # The chance that all k copies of a common-cause group member - # are down, at each time: from its group's chain of them alone, - # as the others' copies do not change their law. - if (node, k) not in copies_down: - group = grouped[node] - counts = [k if m == node else 0 for m in group.members] - states = self._group_states(group, times, counts) - copies_down[(node, k)] = states.probabilities[:, -1] - return copies_down[(node, k)] - - def gain(node): - # The most the k + 1-th copy of the node can lower the system - # unavailability: the node's own fall in unavailability (a - # common-cause group member's, all its copies down, from its - # group's chain). - if node in grouped: - return lambda k: float( - weights @ (all_down(node, k) - all_down(node, k + 1)) - ) - return lambda k: float(weights @ (down[node] ** k * up[node])) - - def in_series(node) -> bool: - # Down alone, it takes the system down: it lies on every path. - alone = {n: np.ones(1) for n in self.nodes} - alone[node] = np.zeros(1) - return float(self.system_probability(alone)[0]) == 0.0 - - def train_gain(train): - # The n + 1-th train changes the system only if it brings its - # vote to k working inputs, so at most k - 1 of the n there - # work: it lowers the unavailability by at most its own - # availability times the chance of that. - from scipy.stats import binom - - own = -np.expm1( - np.sum([np.log1p(-down[node]) for node in train.members], 0) - ) - return lambda n: float( - weights - @ ((1.0 - own) * binom.sf(n - train.k, n, np.minimum(own, 1))) - ) - - series = None - varying = set(self._inspection) | set(self._block_nodes()) - if not drawn and all( - node not in varying and node not in grouped and in_series(node) - for node in chosen - ): - # Each such node's availability is constant, and the system's is - # theirs times the rest's: the exact search is then a dynamic - # program over the nodes. - series = [ - (lambda n, q=float(down[node][0]): q**n) for node in chosen - ] - counts, _, u = lowest_total_cost( - unavailability, - item_cost, - present * self.downtime_cost_rate, - [gain(node) for node in chosen] - + [train_gain(train) for train in drawn], - caps, - max_unavailability, - method, - series, - ) - units = {name: int(n) for name, n in zip(items, counts)} - # Every component once, its copies as many times more, and each - # train's copies their nodes' costs as many times. - extra = { - node: units[train.name] - 1 - for train in drawn - for node in train.members - } - acquisition = math.fsum( - (units.get(node, 1) + extra.get(node, 0)) * copy_cost[node][0] - for node in self.components - ) - cost_rate = ( - math.fsum( - (units.get(node, 1) + extra.get(node, 0)) * copy_cost[node][1] - for node in self.components - ) - + self.downtime_cost_rate * u - ) - return TotalCostAllocation( - units=units, - total_cost=acquisition + present * cost_rate, - acquisition_cost=acquisition, - cost_rate=cost_rate, - availability=1.0 - u, + return _repairable_allocation.allocate_redundancy( + self, horizon=horizon, + nodes=nodes, + trains=trains, + min_availability=min_availability, + max_units=max_units, method=method, - discount_rate=float(discount_rate), - trains=( - {train.name: list(train.members) for train in drawn} - if drawn - else None - ), + discount_rate=discount_rate, ) - def _allocation_trains(self, trains, chosen) -> List[_Train]: - """The trains ``allocate_redundancy`` may copy (see its ``trains``), - checked: each a chain of components in series ending at a node that - feeds one node, a node in one train at most and not in ``chosen`` - (the nodes copied alone), and named apart from the nodes.""" - if trains is None: - return [] - if not isinstance(trains, Mapping) or not trains: - raise ValueError( - "trains must be a non-empty dict {name: [nodes in series]}, " - f"got {trains!r}." - ) - graph = self.G - out: List[_Train] = [] - seen: Dict[Hashable, Hashable] = {} - for name, members in trains.items(): - if name in graph.nodes: - raise ValueError( - f"Train {name!r} is named as a node: name a train apart " - "from the nodes, as units holds both." - ) - if isinstance(members, (str, bytes)) or not isinstance( - members, Collection - ): - raise ValueError( - f"Train {name!r} must be a list of nodes, got " - f"{members!r}." - ) - members = list(members) - if not members: - raise ValueError(f"Train {name!r} has no nodes.") - for node in members: - self._require_component(node, f"train {name!r}") - if isinstance(self.components[node], RepairableRBD): - raise ValueError( - f"Node {node!r} of train {name!r} is a nested " - "RepairableRBD, which cannot be given copies here: " - "its costs are not this RBD's." - ) - if node in seen: - raise ValueError( - f"Node {node!r} is in trains {seen[node]!r} and " - f"{name!r}: a node is in one train at most." - ) - if node in chosen: - raise ValueError( - f"Node {node!r} is in nodes and in train {name!r}: " - "copy it on its own or with its train." - ) - seen[node] = name - for a, b in zip(members, members[1:]): - if list(graph.successors(a)) != [b] or list( - graph.predecessors(b) - ) != [a]: - raise ValueError( - f"Train {name!r} must be a chain of nodes in series: " - f"{a!r} feeding {b!r} alone, and {b!r} fed by {a!r} " - "alone." - ) - exits = list(graph.successors(members[-1])) - if len(exits) != 1: - raise ValueError( - f"Train {name!r} must end at a node feeding one node, " - f"which its copies join; {members[-1]!r} feeds " - f"{len(exits)}." - ) - out.append( - _Train( - name, - tuple(members), - exits[0], - int(graph.nodes[exits[0]]["k"]), - ) - ) - return out - - def _with_train_copies(self, trains: List[_Train], extra: tuple): - """The diagram's graph with ``extra[i]`` copies of ``trains[i]`` - drawn alongside it: each fed as its first node is, feeding the node - its last feeds, whose ``k`` is kept (see ``allocate_redundancy``).""" - graph = self.G.copy() - for train, copies in zip(trains, extra): - first = list(self.G.predecessors(train.members[0])) - for index in range(1, copies + 1): - names = [ - _TrainCopy(train.name, index, n) for n in train.members - ] - for name, node in zip(names, train.members): - graph.add_node(name, **self.G.nodes[node]) - graph.add_edges_from((u, names[0]) for u in first) - graph.add_edges_from(zip(names, names[1:])) - graph.add_edge(names[-1], train.exit) - return graph - def availability_allocation( self, target: float, @@ -8875,60 +1868,15 @@ def availability_allocation( >>> round(need.mttf["v"]), round(need.mttr["v"], 2) (128, 0.78) """ - methods = ("cost_based", "improvement", "minimum_effort", "equal") - if method not in methods: - raise ValueError( - "method must be one of " - f"{', '.join(repr(m) for m in methods)}, got {method!r}." - ) - options = { - "weights": (weights, "improvement"), - "max_availabilities": (max_availabilities, "cost_based"), - "feasibility": (feasibility, "cost_based"), - } - for name, (value, owner) in options.items(): - if value is not None and method != owner: - raise ValueError(f"{name} applies to method={owner!r} only.") - target = _allocation_target(target) - current, free, held = self._allocatable(fixed) - for name, (value, _) in options.items(): - self._allocation_option(name, value, held) - view = self._allocation_view(held) - start = {node: current[node] for node in self.nodes} - if method == "cost_based": - maxima = {node: current[node] for node in held} - maxima.update(max_availabilities or {}) - allocated = view.cost_based_allocation( - target, - start, - max_probabilities=maxima, - feasibility=feasibility, - ) - elif method == "improvement": - allocated = view.improvement_allocation( - target, start, fixed=list(held), weights=weights - ) - elif method == "equal": - # From 0.5 for every component allocated, scaled alike, so they - # all end up the same. - allocated = view.improvement_allocation( - target, - { - node: current[node] if node in held else 0.5 - for node in self.nodes - }, - fixed=list(held), - ) - else: - allocated = view._minimum_effort_held(target, current, held) - if view is not self and method != "minimum_effort": - self.res = view.res - mttf, mttr = {}, {} - for node, (up, down) in free.items(): - a = allocated[node] - mttf[node] = down * a / (1.0 - a) if a < 1.0 else math.inf - mttr[node] = up * (1.0 - a) / a if a > 0.0 else math.inf - return self._allocation_result(view, allocated, mttf, mttr) + return _repairable_allocation.availability_allocation( + self, + target=target, + method=method, + fixed=fixed, + weights=weights, + max_availabilities=max_availabilities, + feasibility=feasibility, + ) def mttf_mttr_allocation( self, @@ -9068,337 +2016,15 @@ def mttf_mttr_allocation( >>> {node: round(t, 2) for node, t in repairs.mttr.items()} {'p1': 0.74, 'p2': 0.74, 'v': 0.78} """ - if levers not in ("both", "mttf", "mttr"): - raise ValueError( - f"levers must be 'both', 'mttf' or 'mttr', got {levers!r}." - ) - target = _allocation_target(target) - current, free, held = self._allocatable(fixed) - most = self._allocation_option("max_mttf", max_mttf, held) - least = self._allocation_option("min_mttr", min_mttr, held) - ease = ( - self._allocation_option( - "mttf_feasibility", mttf_feasibility, held - ), - self._allocation_option( - "mttr_feasibility", mttr_feasibility, held - ), - ) - names = ("mttf_feasibility", "mttr_feasibility") - # Each component's failure rate and repair time, as (start, floor): - # each falls from its start towards its floor as its lever is used, - # to floor + (start - floor) * exp(-v) for the lever's v >= 0. - span: Dict[Hashable, Tuple[Tuple[float, float], ...]] = {} - variables: List[Tuple[Hashable, int, float]] = [] - for node, (mttf, mttr) in free.items(): - top = _as_float(most.get(node, math.inf)) - if not top >= mttf * (1.0 - 1e-12): - raise ValueError( - f"max_mttf[{node!r}] must be at least the component's " - f"MTTF, {mttf:.6g}, got {most[node]!r}." - ) - bottom = _as_float(least.get(node, 0.0)) - if not 0.0 <= bottom <= mttr * (1.0 + 1e-12): - raise ValueError( - f"min_mttr[{node!r}] must be between 0 and the " - f"component's MTTR, {mttr:.6g}, got {least[node]!r}." - ) - span[node] = ( - (1.0 / mttf, 1.0 / max(top, mttf)), - (mttr, min(bottom, mttr)), - ) - for lever, name in enumerate(names): - f = _as_float(ease[lever].get(node, 0.5)) - if not 0.0 <= f < 1.0: - raise ValueError( - f"{name}[{node!r}] must be in [0, 1), got " - f"{ease[lever][node]!r}." - ) - start, floor = span[node][lever] - if levers in ("both", ("mttf", "mttr")[lever]) and ( - start > floor - ): - variables.append((node, lever, 1.0 - f)) - if not variables: - raise ValueError( - "No MTTF or MTTR can change: every lever is held (by levers, " - "max_mttf or min_mttr)." - ) - steepness = np.array([s for _, _, s in variables]) - size = len(variables) - - def design(v) -> Dict[Hashable, List[float]]: - """Each component's failure rate and repair time.""" - values = { - node: [ends[0][0], ends[1][0]] for node, ends in span.items() - } - for (node, lever, _), x in zip(variables, v): - if x > 0.0: - start, floor = span[node][lever] - gap = (start - floor) * math.exp(-x) - values[node][lever] = floor + gap - return values - - def probabilities(v) -> Tuple[Dict, Dict]: - p = {node: current[node] for node in self.nodes} - q = {node: 1.0 - current[node] for node in self.nodes} - for node, (rate, repair) in design(v).items(): - odds = rate * repair # of being down - p[node] = 1.0 / (1.0 + odds) - q[node] = odds / (1.0 + odds) - return p, q - - def result(v) -> AvailabilityAllocation: - values = design(v) - mttf = {} - for node, (rate, _) in values.items(): - if rate == span[node][0][0]: - mttf[node] = free[node][0] # unchanged, exactly - else: - mttf[node] = 1.0 / rate if rate > 0.0 else math.inf - return self._allocation_result( - view, - probabilities(v)[0], - mttf, - {node: repair for node, (_, repair) in values.items()}, - ) - - view = self._allocation_view(held) - goal = float(logit(target)) - now = view._log_odds(*probabilities(np.zeros(size)))[0] - if goal <= now: - self.res = OptimizeResult( - x=np.zeros(size), - fun=float(np.log(size)), - success=True, - message="The target is already met.", - ) - return result(np.zeros(size)) - best = view._log_odds(*probabilities(np.full(size, np.inf)))[0] - # The limits are only approached (at an ever-growing cost), so a - # target at them, to within rounding, cannot be met either. - if goal >= best - 1e-9: - raise ValueError( - f"target {target} cannot be reached: from the current " - f"availability ({expit(now):.6g}) the system can only " - f"approach {expit(best):.6g}, with every MTTF and MTTR that " - "may change at its limit." - ) - - def shortfall(v: np.ndarray) -> float: - return view._log_odds(*probabilities(v))[0] - goal - - def shortfall_gradient(v: np.ndarray) -> np.ndarray: - p, q = probabilities(v) - derivative = view._log_odds(p, q)[1] - values = design(v) - gradient = np.empty(size) - for i, ((node, lever, _), x) in enumerate(zip(variables, v)): - start, floor = span[node][lever] - # The lever lowers the odds of being down, rate * repair, at - # the other quantity times its own fall; p = 1 / (1 + odds). - other = values[node][1 - lever] - fall = other * (start - floor) * math.exp(-x) - gradient[i] = derivative[node] * p[node] ** 2 * fall - return gradient - - def log_total_cost(v: np.ndarray) -> Tuple[float, np.ndarray]: - costs = steepness * np.expm1(v) - return float(logsumexp(costs)), softmax( - costs - ) * steepness * np.exp(v) - - def common_shift(start: np.ndarray, along: np.ndarray) -> np.ndarray: - """The start moved up by the least common v, on the levers - ``along`` marks, meeting the target (it exists, as the target is - below what the limits allow).""" - high = 1.0 - while shortfall(start + high * along) < 0.0 and high < 1e6: - high *= 2.0 - return start + along * brentq( - lambda d: shortfall(start + d * along), 0.0, high, xtol=1e-14 - ) - - every = np.ones(size) - res = minimize( - log_total_cost, - common_shift(np.zeros(size), every), - jac=True, - method="SLSQP", - bounds=[(0.0, None)] * size, - constraints=[ - {"type": "ineq", "fun": shortfall, "jac": shortfall_gradient} - ], - options={"ftol": 1e-12, "maxiter": 1000}, - ) - # A lever the solver left within a hair of its bound is unused. - v = np.where(res.x < 1e-9, 0.0, res.x) - if shortfall(v) < 0.0: - # Close the solver's last sliver of constraint tolerance, with - # the levers in use. - used = (v > 0.0).astype(float) - v = common_shift(v, used if used.any() else every) - res.x = v - res.fun = log_total_cost(v)[0] - self.res = res - if not res.success: - warnings.warn( - "the cost minimisation stopped before converging " - f"({res.message}); the design meets the target but may not " - "be the cheapest.", - stacklevel=outside_level(), - ) - return result(v) - - def _allocatable( - self, fixed - ) -> Tuple[ - Dict[Hashable, float], Dict[Hashable, Tuple[float, float]], set - ]: - """For an availability allocation: every component's long-run - availability; the MTTF and MTTR of those allocated one (with - corrective repair alone: they fail and take time to repair, with no - preventive or inspection schedule, are not in a common-cause group, - whose members' MTTF and MTTR are the group's, and are not in - ``fixed``); and the set of the others, which keep theirs.""" - self._require_unlimited_crews(*_ALLOCATION_CREWS) - self._require_ccf_long_run() - held = set() - if fixed is not None: - held = set(fixed) - unknown = [node for node in held if node not in self.components] - if unknown: - raise ValueError( - f"fixed names {sorted(unknown, key=str)}, which are not " - "components of this RBD." - ) - current = { - node: self._node_availability(node) for node in self.components - } - free = self._allocated_levers(held) - for node, (mttf, mttr) in free.items(): - current[node] = mttf / (mttf + mttr) - return current, free, held - - def _allocated_levers( - self, held: set - ) -> Dict[Hashable, Tuple[float, float]]: - """The MTTF and MTTR of each component an availability allocation - allocates one: those with corrective repair alone (they fail and - take time to repair, with no preventive or inspection schedule), - not nested, not in a common-cause group (whose members' MTTF and - MTTR are the group's) and not in ``held``, which gains the others. - Raises if there are none: ``analysis_routes`` asks it, as it follows - from the diagram.""" - members = {m for group in self.ccf_groups for m in group.members} - free: Dict[Hashable, Tuple[float, float]] = {} - for node, component in self.components.items(): - if ( - node in held - or isinstance(component, RepairableRBD) - or node in self._preventive - or node in self._inspection - or node in self._standby - or node in members - ): - held.add(node) - continue - mttf = model_mean(component.reliability) - mttr = model_mean(component.time_to_replace) - if not (0.0 < mttf < math.inf and 0.0 < mttr < math.inf): - # Never down, down for good in the end, or up for good. - held.add(node) - continue - free[node] = (mttf, mttr) - if not free: - raise ValueError( - "No component can be allocated an availability: only " - "components with corrective repair alone (that fail and take " - "time to repair, with no preventive or inspection schedule), " - "not in fixed, can." - ) - return free - - def _allocation_option(self, name: str, mapping, held: set) -> dict: - """A per-component option of an availability allocation, checked to - name only components allocated an availability.""" - values = self._node_overrides(name, mapping) - kept = [node for node in values if node in held] - if kept: - raise ValueError( - f"{name} names {sorted(kept, key=str)}, which keep their " - "availability: only components with corrective repair alone, " - "not in fixed, are allocated one." - ) - return values - - def _allocation_view(self, held: set) -> "RepairableRBD": - """This RBD as an availability allocation scores it. The components - allocated an availability have a constant one, so when every held - component does too the system's availability is the structure - function of theirs, and the RBD itself will do. A held component - that is inspected or block-replaced is up with a probability that - varies over its schedule, together with the others on the calendar: - the system's availability is then averaged over the long-run grid, - as ``mean_availability`` averages it, by a view of the RBD whose - ``_allocation_probability`` and ``_log_odds`` do so. With - common-cause groups, whose members are held, its points are split - by the groups' joint states (see ``_with_ccf_groups``).""" - times, weights = self._long_run_grid() - if len(times) == 1 and not self.ccf_groups: - return self - profiles = self._availabilities_at(times) - if self.ccf_groups: - profiles, _, weights, _ = self._with_ccf_groups( - times, - profiles, - None, - weights, - what="The availability allocation", - ) - view = copy(self) - view.__dict__.pop("_ccf_tables", None) - view.__dict__["_allocation_calendar"] = ( - weights, - {node: profiles[node] for node in held}, - [node for node in self.nodes if node not in held], - ) - return view - - def _calendar_arrays( - self, p: Dict, q: Optional[Dict] - ) -> Tuple[Dict, Dict]: - """Node probabilities ``p`` and their complements ``q`` (by default - ``1 - p``) over the allocation calendar's grid, with the held - components' own availabilities there.""" - weights, profiles, _ = self.__dict__["_allocation_calendar"] - size = len(weights) - works = {node: np.full(size, float(p[node])) for node in p} - fails = { - node: np.full( - size, 1.0 - float(p[node]) if q is None else float(q[node]) - ) - for node in p - } - for node, profile in profiles.items(): - works[node] = profile - fails[node] = 1.0 - profile - for node in self.in_or_out: - works[node] = np.ones(size) - fails[node] = np.zeros(size) - return works, fails - - def _calendar_means(self, works: Dict, fails: Dict) -> Tuple[float, float]: - """The system's probabilities of working and of failing over the - allocation calendar's grid, averaged over it.""" - weights = self.__dict__["_allocation_calendar"][0] - size = len(weights) - up, down = self._decomposition().probabilities( - works, fails, shape=size - ) - return ( - float(weights @ np.broadcast_to(up, (size,))), - float(weights @ np.broadcast_to(down, (size,))), + return _repairable_allocation.mttf_mttr_allocation( + self, + target=target, + levers=levers, + fixed=fixed, + max_mttf=max_mttf, + min_mttr=min_mttr, + mttf_feasibility=mttf_feasibility, + mttr_feasibility=mttr_feasibility, ) def _allocation_probability( @@ -9406,8 +2032,11 @@ def _allocation_probability( ) -> float: if "_allocation_calendar" not in self.__dict__: return super()._allocation_probability(probabilities) - return self._calendar_means( - *self._calendar_arrays(probabilities, None) + return _repairable_allocation._calendar_means( + self, + *_repairable_allocation._calendar_arrays( + self, probabilities, None + ), )[0] def _log_odds( @@ -9416,8 +2045,8 @@ def _log_odds( if "_allocation_calendar" not in self.__dict__: return super()._log_odds(p, q) weights, _, free = self.__dict__["_allocation_calendar"] - works, fails = self._calendar_arrays(p, q) - up, down = self._calendar_means(works, fails) + works, fails = _repairable_allocation._calendar_arrays(self, p, q) + up, down = _repairable_allocation._calendar_means(self, works, fails) with np.errstate(divide="ignore"): log_odds = float(np.log(up) - np.log(down)) scale = up * down @@ -9427,127 +2056,15 @@ def _log_odds( for node in free: # The system's availability is linear in the node's, with # slope P(down | node down) - P(down | node up). - if_up = self._calendar_means( - {**works, node: ones}, {**fails, node: zeros} + if_up = _repairable_allocation._calendar_means( + self, {**works, node: ones}, {**fails, node: zeros} )[1] - if_down = self._calendar_means( - {**works, node: zeros}, {**fails, node: ones} + if_down = _repairable_allocation._calendar_means( + self, {**works, node: zeros}, {**fails, node: ones} )[1] gradient[node] = (if_down - if_up) / scale return log_odds, gradient - def _minimum_effort_held(self, target: float, current: Dict, held: set): - """Albert's minimum-effort allocation to the components not - ``held``: in series, the held components' availabilities (averaged - over their calendars) are a factor of the system's, and the others - make up the rest.""" - self._require_series() - limit = self._allocation_probability( - { - node: current[node] if node in held else 1.0 - for node in self.nodes - } - ) - if target > limit: - raise ValueError( - f"target {target} cannot be reached: the components that " - "keep their availability hold the system to at most " - f"{limit:.6g}." - ) - raised = self.minimum_effort_allocation( - target / limit if limit > 0.0 else 0.0, - { - node: 1.0 if node in held else current[node] - for node in self.nodes - }, - ) - return { - node: current[node] if node in held else raised[node] - for node in self.nodes - } - - def _allocation_result( - self, view: "RepairableRBD", allocated: Dict, mttf: Dict, mttr: Dict - ) -> AvailabilityAllocation: - """An availability allocation's result, scored as ``view`` scores - it.""" - return AvailabilityAllocation( - availability={ - node: float(allocated[node]) for node in self.components - }, - mttf=mttf, - mttr=mttr, - system_availability=view._allocation_probability( - {node: allocated[node] for node in self.nodes} - ), - ) - - def _with_intervals( - self, preventive=None, inspection=None, offsets=None - ) -> "RepairableRBD": - """This RBD with the intervals of some of its preventive or - inspection schedules changed, for the exact long-run values: a - shallow copy, sharing everything else, including the renewal cycles - already worked out (kept by interval). ``offsets`` gives some - inspected components' first tests as shares of their intervals.""" - self.__dict__.setdefault("_age_cycles", {}) - self.__dict__.setdefault("_block_cycles", {}) - self.__dict__.setdefault("_tested_units", {}) - plan = copy(self) - # What its own schedules decide is worked out again. - plan.__dict__.pop("_ccf_tables", None) - if preventive: - plan._preventive = dict(self._preventive) - for node, interval in preventive.items(): - plan._preventive[node] = self._preventive[node]._replace( - interval=float(interval) - ) - if inspection: - plan._inspection = dict(self._inspection) - for node, interval in inspection.items(): - schedule = self._inspection[node] - # An offset keeps its share of the interval (a pair tested - # half an interval apart stays so). - share = schedule.offset / schedule.interval - plan._inspection[node] = schedule._replace( - interval=float(interval), offset=share * float(interval) - ) - if offsets: - plan._inspection = dict(plan._inspection) - for node, share in offsets.items(): - schedule = plan._inspection[node] - plan._inspection[node] = schedule._replace( - offset=float(share) * schedule.interval - ) - return plan - - def _interval_targets(self, min_availability, max_cost_rate): - """The checked targets of an interval choice.""" - if min_availability is not None and max_cost_rate is not None: - raise ValueError( - "Give at most one of min_availability and max_cost_rate." - ) - if not self.has_costs: - raise ValueError( - "Nothing is priced, so no interval costs more than another: " - "give the components costs (or a downtime_cost_rate)." - ) - if min_availability is not None: - if not 0.0 < float(min_availability) < 1.0: - raise ValueError( - "min_availability must be a number in (0, 1), got " - f"{min_availability!r}." - ) - return float(min_availability), None - if max_cost_rate is not None: - if not 0.0 < float(max_cost_rate) < math.inf: - raise ValueError( - "max_cost_rate must be a positive number, got " - f"{max_cost_rate!r}." - ) - return None, float(max_cost_rate) - return None, None - def optimal_replacement_intervals( self, nodes: Optional[Collection[Hashable]] = None, @@ -9585,7 +2102,7 @@ def optimal_replacement_intervals( found cost almost the same. Replacements are often made on a calendar (quarterly, yearly): the - intervals are then chosen from ``allowed`` (#230), as + intervals are then chosen from ``allowed``, as ``optimal_inspection_intervals`` chooses tests', every combination tried when there are at most 2000 of them, which gives the optimum, and a local search (one interval changed at a time, from @@ -9596,11 +2113,11 @@ def optimal_replacement_intervals( where they take the system down together (redundant units whose replacements take time) its first years cost more than the plan's cost rate says: ``expected_cost`` gives the cost from new, and from - units of other ages (``state``) a staggered start (#234). + units of other ages (``state``) a staggered start. With limited ``repair_crews`` a component can wait for a crew, and the exact long-run values do not hold: the choice is refused unless - ``assume_unlimited_crews`` (#184), which chooses the intervals as if + ``assume_unlimited_crews``, which chooses the intervals as if every repair started at once. Simulate that plan with the crews, ``with_intervals(plan).cost()``, to see what the waiting costs. @@ -9680,91 +2197,14 @@ def optimal_replacement_intervals( The pump alone in the line is replaced later than those with a standby: its replacements stop the plant too. """ - chosen = self._maintained(nodes) - min_availability, max_cost_rate = self._interval_targets( - min_availability, max_cost_rate - ) - if self._crews_couple(): - if not assume_unlimited_crews: - self._require_interval_crews() - return self._unlimited_crews().optimal_replacement_intervals( - chosen, - allowed=allowed, - min_availability=min_availability, - max_cost_rate=max_cost_rate, - ) - - def evaluate(intervals: dict) -> Tuple[float, float]: - plan = self._with_intervals(preventive=intervals) - return plan.expected_cost_rate(), plan.mean_availability() - - if allowed is not None: - best = _choose_from( - chosen, - self._allowed_intervals(allowed, chosen, never=True), - evaluate, - min_availability, - max_cost_rate, - ) - else: - best = self._searched_replacements( - chosen, evaluate, min_availability, max_cost_rate - ) - cost, availability = evaluate(best) - return MaintenancePlan( - {node: float(best[node]) for node in chosen}, cost, availability - ) - - def _searched_replacements( - self, - chosen: list, - evaluate: Callable[[dict], Tuple[float, float]], - min_availability: Optional[float], - max_cost_rate: Optional[float], - ) -> dict: - """The age-replacement intervals of ``chosen`` that - ``optimal_replacement_intervals`` gives without ``allowed``: by a - gradient search over their logarithms, each from a thousandth to a - thousand times its component's mean life, one at the top of its - range compared with never replacing it.""" - scale = {} - for node in chosen: - life = failure_time_scale(self.components[node].reliability) - scale[node] = life if np.isfinite(life) and life > 0.0 else 1.0 - low = np.array([math.log(1e-3 * scale[n]) for n in chosen]) - high = np.array([math.log(1e3 * scale[n]) for n in chosen]) - current = [ - math.log(min(self._preventive[n].interval, 1e3 * scale[n])) - for n in chosen - ] - starts = [ - np.clip(current, low, high), - np.array([math.log(scale[n]) for n in chosen]), - np.array([math.log(0.3 * scale[n]) for n in chosen]), - high.copy(), - ] - best = _choose_intervals( - chosen, - evaluate, - starts, - (low, high), - min_availability, - max_cost_rate, + return _intervals.optimal_replacement_intervals( + self, + nodes=nodes, + allowed=allowed, + min_availability=min_availability, + max_cost_rate=max_cost_rate, + assume_unlimited_crews=assume_unlimited_crews, ) - # An interval at the top of its range may be better never used. - for i, node in enumerate(chosen): - if best[node] >= 0.99 * math.exp(high[i]): - never = {**best, node: math.inf} - cost, availability = evaluate(never) - if _meets( - cost, availability, min_availability, max_cost_rate - ) and _objective( - cost, availability, max_cost_rate - ) <= _objective( - *evaluate(best), max_cost_rate - ): - best = never - return best def optimal_inspection_intervals( self, @@ -9773,7 +2213,6 @@ def optimal_inspection_intervals( allowed=None, min_availability: Optional[float] = None, max_cost_rate: Optional[float] = None, - offsets=None, assume_unlimited_crews: bool = False, offset_shares=None, ) -> MaintenancePlan: @@ -9809,13 +2248,13 @@ def optimal_inspection_intervals( at once are down together for as long as a failure of both stays hidden, where tests half an interval apart find a common-cause failure twice as soon. ``offset_shares`` chooses the times of the - first tests with the intervals (#184). + first tests with the intervals. A component whose tests can miss a failure (a ``coverage`` below 1) keeps its full tests' interval (``full_test``), a whole number of its test intervals: its interval is chosen among those that divide it, from ``allowed`` (which must) or, left out, from every - one in range (#221). + one in range. Parameters ---------- @@ -9832,10 +2271,6 @@ def optimal_inspection_intervals( max_cost_rate : float, optional The highest long-run cost rate allowed: the intervals then give the highest availability within it. - offsets : sequence of float, dict or str, optional - Deprecated: ``offset_shares``, which it is renamed, as its - values are shares of the interval where ``with_intervals``' - and the plan's ``offsets`` are times (#222). Refused in 0.14. assume_unlimited_crews : bool, optional With limited ``repair_crews``, choose as if every repair started at once, as ``optimal_replacement_intervals`` does, by default @@ -9941,145 +2376,19 @@ def optimal_inspection_intervals( >>> round(staggered.cost_rate, 4), round(1 - staggered.availability, 7) (0.1142, 0.0004925) """ - if offsets is not None: - if offset_shares is not None: - raise ValueError( - "Give offset_shares alone: offsets is its old name." - ) - renamed( - "optimal_inspection_intervals", - "offsets", - "offset_shares", - "its values are shares of the interval, where " - "with_intervals' and the plan's offsets are times", - ) - offset_shares = offsets - chosen = self._inspected(nodes) - min_availability, max_cost_rate = self._interval_targets( - min_availability, max_cost_rate - ) - if self._crews_couple(): - if not assume_unlimited_crews: - self._require_interval_crews() - return self._unlimited_crews().optimal_inspection_intervals( - chosen, - allowed=allowed, - min_availability=min_availability, - max_cost_rate=max_cost_rate, - offset_shares=offset_shares, - ) - rates = {node: self._tested_scale(node) for node in chosen} - - def evaluate(intervals: dict) -> Tuple[float, float]: - plan = self._with_intervals(inspection=intervals) - return plan.expected_cost_rate(), plan.mean_availability() - - if offset_shares is not None: - if allowed is None: - raise ValueError( - "offset_shares are chosen with intervals from allowed: " - "give allowed (when a lone component is tested changes " - "nothing in the long run)." - ) - return self._choose_tests( - chosen, - self._dividing(self._allowed_intervals(allowed, chosen)), - self._allowed_shares(offset_shares, chosen), - min_availability, - max_cost_rate, - ) - if allowed is None and self._inspection[chosen[0]].partial: - self._require_one_inspected() - (node,) = chosen - rate = rates[node] - best = _choose_divisor( - node, - # Its full tests' interval (see _Inspection.period). - self._inspection[node].period, - (1e-4 / rate, 10.0 / rate), - evaluate, - min_availability, - max_cost_rate, - ) - elif allowed is None: - self._require_one_inspected() - (node,) = chosen - rate = rates[node] - low = np.array([math.log(1e-4 / rate)]) - high = np.array([math.log(10.0 / rate)]) - current = math.log(self._inspection[node].interval) - starts = [ - np.clip([current], low, high), - np.array([math.log(0.01 / rate)]), - np.array([math.log(0.1 / rate)]), - np.array([math.log(1.0 / rate)]), - ] - best = _choose_intervals( - chosen, - evaluate, - starts, - (low, high), - min_availability, - max_cost_rate, - ) - else: - best = _choose_from( - chosen, - self._dividing(self._allowed_intervals(allowed, chosen)), - evaluate, - min_availability, - max_cost_rate, - ) - cost, availability = evaluate(best) - # The offsets the plan's tests keep: each its share of the interval. - offsets_kept = { - node: float(best[node]) - * self._inspection[node].offset - / self._inspection[node].interval - for node in chosen - } - return MaintenancePlan( - {node: float(best[node]) for node in chosen}, - cost, - availability, - offsets=offsets_kept, + return _intervals.optimal_inspection_intervals( + self, + nodes=nodes, + allowed=allowed, + min_availability=min_availability, + max_cost_rate=max_cost_rate, + assume_unlimited_crews=assume_unlimited_crews, + offset_shares=offset_shares, ) - def _dividing(self, options: dict) -> dict: - """``options``, the intervals allowed each chosen component, - checked to divide its full tests' interval where its tests can - miss a failure (#221): every full test is a whole number of tests - on, so an interval that does not divide it cannot be its.""" - for node, intervals in options.items(): - schedule = self._inspection[node] - if not schedule.partial: - continue - full = schedule.period - bad = [i for i in intervals if not _divides(i, full)] - if bad: - some = ", ".join(f"{full / k:g}" for k in range(1, 5)) - raise ValueError( - f"The tests of component {node!r} can miss a failure, so " - f"its full tests' interval, {full:g}, must be a whole " - "number of its test intervals: " - f"{', '.join(map(repr, bad))} " - f"do{'es' if len(bad) == 1 else ''} not divide it. Allow " - f"intervals that do ({some}, ...)." - ) - return options - - def _require_interval_crews(self) -> None: - """Raise if a component can wait for a repair crew, for the interval - choices (see ``_INTERVAL_CREWS``).""" - self._require_unlimited_crews(*_INTERVAL_CREWS) - - def _unlimited_crews(self) -> "RepairableRBD": - """This RBD with as many repair crews as jobs, built as it was.""" - return RepairableRBD(**{**self._init_args, "repair_crews": None}) - def with_intervals(self, intervals, offsets=None) -> "RepairableRBD": """A copy of this RBD with some maintenance or test intervals - changed (#184). + changed. To simulate a plan that ``optimal_replacement_intervals`` or ``optimal_inspection_intervals`` chose, with the repair crews it was @@ -10117,286 +2426,19 @@ def with_intervals(self, intervals, offsets=None) -> "RepairableRBD": >>> import surpyval as surv >>> from repyability import RepairableRBD >>> pump = { - ... "reliability": surv.Weibull.from_params([1000, 2.5]), - ... "repairability": surv.Exponential.from_params([0.05]), - ... "replace_cost": 5000.0, - ... "preventive": {"interval": 1000.0, "cost": 1000.0}, - ... } - >>> rbd = RepairableRBD([("s", "p"), ("p", "t")], {"p": pump}) - >>> plan = rbd.optimal_replacement_intervals() - >>> rbd.with_intervals(plan).expected_cost_rate() == plan.cost_rate - True - """ - if isinstance(intervals, MaintenancePlan): - if offsets is None: - offsets = intervals.offsets - intervals = intervals.intervals - offsets = dict(offsets or {}) - components = dict(self._init_args["components"]) - for node in list(dict(intervals)) + list(offsets): - self._require_component(node, "with_intervals") - for node, interval in dict(intervals).items(): - spec = components.get(node) - # A component has one schedule at most (see the constructor). - kinds = [ - kind - for kind in ("preventive", "inspection") - if isinstance(spec, dict) and spec.get(kind) is not None - ] - if not kinds: - raise ValueError( - f"Node {node!r} has no maintenance or test schedule to " - "give an interval." - ) - (kind,) = kinds - assert isinstance(spec, dict) - if node in offsets and kind != "inspection": - raise ValueError( - f"Node {node!r} has an offset but no test schedule: an " - "offset is the time of a component's first test." - ) - old = dict(spec[kind]) - interval = float(interval) - new = {**old, "interval": interval} - if kind == "inspection": - if node in offsets: - new["offset"] = float(offsets[node]) - elif old.get("offset"): - share = float(old["offset"]) / float(old["interval"]) - new["offset"] = share * interval - spec = {**spec, kind: new} - if kind == "preventive" and math.isinf(interval): - if old.get("policy", "age") != "age": - raise ValueError( - f"Node {node!r}: only age replacement can be left to " - "failures (an interval of inf)." - ) - spec = {k: v for k, v in spec.items() if k != "preventive"} - components[node] = spec - unknown = set(offsets) - set(dict(intervals)) - for node in unknown: - spec = components.get(node) - if not (isinstance(spec, dict) and spec.get("inspection")): - raise ValueError( - f"Node {node!r} has an offset but no test schedule." - ) - components[node] = { - **spec, - "inspection": { - **spec["inspection"], - "offset": float(offsets[node]), - }, - } - return RepairableRBD(**{**self._init_args, "components": components}) - - def _choose_tests( - self, - chosen: list, - intervals: dict, - shares: dict, - min_availability: Optional[float], - max_cost_rate: Optional[float], - ) -> MaintenancePlan: - """``optimal_inspection_intervals`` with the offsets chosen too: each - node's interval from ``intervals`` and its first test's share of it - from ``shares``, searched together as ``_choose_from`` searches - intervals.""" - if set(chosen) == set(self._inspection): - # Shifting every test by one time changes nothing in the long - # run: the first node's tests are kept from 0. - shares = {**shares, chosen[0]: (0.0,)} - options = {("interval", n): intervals[n] for n in chosen} - options.update({("offset", n): shares[n] for n in chosen}) - - def evaluate(choice: dict) -> Tuple[float, float]: - plan = self._with_intervals( - inspection={n: choice[("interval", n)] for n in chosen}, - offsets={n: choice[("offset", n)] for n in chosen}, - ) - return plan.expected_cost_rate(), plan.mean_availability() - - best = _choose_from( - list(options), options, evaluate, min_availability, max_cost_rate - ) - cost, availability = evaluate(best) - return MaintenancePlan( - {n: float(best[("interval", n)]) for n in chosen}, - cost, - availability, - offsets={ - n: float(best[("offset", n)] * best[("interval", n)]) - for n in chosen - }, - ) - - @staticmethod - def _allowed_shares(offsets, chosen: list) -> dict: - """``offsets`` (see ``optimal_inspection_intervals``) as a sorted - tuple of shares of the interval per chosen node.""" - if isinstance(offsets, str): - if offsets != "stagger": - raise ValueError( - "offset_shares must be shares of the interval in [0, " - "1), a dict of them per node, or 'stagger', got " - f"{offsets!r}." - ) - n = len(chosen) - return {node: tuple(k / n for k in range(n)) for node in chosen} - per_node = ( - offsets - if isinstance(offsets, dict) - else {node: offsets for node in chosen} - ) - _require_chosen(per_node, chosen, "offset_shares") - out = {} - for node in chosen: - if node not in per_node: - raise ValueError( - f"offset_shares gives no share of the interval for node " - f"{node!r}: give one, in [0, 1), for each of {chosen}." - ) - given = per_node[node] - if isinstance(given, (str, bytes)) or not isinstance( - given, Collection - ): - given = [given] - values = [] - for value in given: - number = isinstance( - value, (int, float, np.integer, np.floating) - ) and not isinstance(value, bool) - share = float(value) if number else float("nan") - if not 0.0 <= share < 1.0: - raise ValueError( - "offset_shares are shares of the interval, in [0, " - f"1), got {value!r} for node {node!r}." - ) - values.append(share) - if not values: - raise ValueError( - f"offset_shares gives no share of the interval for node " - f"{node!r}." - ) - out[node] = tuple(sorted(set(values))) - return out - - def _inspected(self, nodes) -> list: - """The components with hidden failures named by ``nodes`` (all of - them by default), checked.""" - if nodes is None: - if not self._inspection: - raise ValueError( - "No component has hidden failures: give the components " - "an 'inspection' schedule to have its interval chosen." - ) - chosen = list(self._inspection) - else: - chosen = list(nodes) - if not chosen: - raise ValueError("nodes is empty.") - for node in chosen: - self._require_component(node, "nodes") - if node not in self._inspection: - raise ValueError( - f"Node {node!r} has no hidden failures: give it an " - "'inspection' schedule to have its interval chosen." - ) - return chosen - - def _require_component(self, node, given: str) -> None: - """Raise if ``node``, given in ``given``, is not a component (see - ``_not_a_component``).""" - if node not in self.components: - raise self._not_a_component(node, given) - - def _not_a_component(self, node, given: str) -> ValueError: - """The error for ``node``, given in ``given``, that is not a - component: a junction, which never fails, or an unknown name, with - the closest component's name if one is close (#232), and those - that are (#222).""" - if node in self._junction_nodes: - return ValueError( - f"Node {node!r} given in {given} is a junction " - "(PerfectReliability): it never fails, so it is no component " - "to fail, repair, maintain or stock spares for." - ) - close = _close_name(node, self.components) - hint = "" if close is None else f" Did you mean {close!r}?" - return ValueError( - f"Unknown node {node!r} given in {given}; it is not a component " - f"of the RBD.{hint} Its components are: {list(self.components)}." - ) - - @staticmethod - def _allowed_intervals(allowed, chosen: list, never: bool = False) -> dict: - """``allowed`` as a sorted tuple of intervals per chosen node; with - ``never``, ``inf`` among them (never replacing a component).""" - per_node = ( - allowed - if isinstance(allowed, dict) - else {node: allowed for node in chosen} + ... "reliability": surv.Weibull.from_params([1000, 2.5]), + ... "repairability": surv.Exponential.from_params([0.05]), + ... "replace_cost": 5000.0, + ... "preventive": {"interval": 1000.0, "cost": 1000.0}, + ... } + >>> rbd = RepairableRBD([("s", "p"), ("p", "t")], {"p": pump}) + >>> plan = rbd.optimal_replacement_intervals() + >>> rbd.with_intervals(plan).expected_cost_rate() == plan.cost_rate + True + """ + return _intervals.with_intervals( + self, intervals=intervals, offsets=offsets ) - _require_chosen(per_node, chosen, "allowed") - options = {} - for node in chosen: - if node not in per_node: - raise ValueError( - f"allowed gives no intervals for node {node!r}." - ) - values = [] - for value in per_node[node]: - try: - interval = float(value) - except (TypeError, ValueError): - interval = float("nan") - if not ( - 0.0 < interval < math.inf or (never and interval > 0.0) - ): - kind = ( - "positive numbers (inf for never)" - if never - else "positive, finite numbers" - ) - raise ValueError( - f"allowed intervals must be {kind}, got {value!r} " - f"for node {node!r}." - ) - values.append(interval) - if not values: - raise ValueError( - f"allowed gives no intervals for node {node!r}." - ) - options[node] = tuple(sorted(set(values))) - return options - - def _maintained(self, nodes) -> list: - """The components under age replacement named by ``nodes`` (all of - them by default), checked.""" - if nodes is None: - chosen = [ - node - for node, schedule in self._preventive.items() - if schedule.policy == "age" - ] - if not chosen: - raise ValueError( - "No component is under age replacement: give the " - "components a 'preventive' schedule to have its " - "interval chosen." - ) - return chosen - chosen = list(nodes) - if not chosen: - raise ValueError("nodes is empty.") - for node in chosen: - schedule = self._preventive.get(node) - if schedule is None or schedule.policy != "age": - raise ValueError( - f"Node {node!r} is not a component under age " - "replacement: give it a 'preventive' schedule with " - "'policy': 'age' to have its interval chosen." - ) - return chosen def initialize_event_queue( self, @@ -10460,270 +2502,16 @@ def initialize_event_queue( If ``state`` is ``"stationary"``, or gives a state to a component whose state is not taken (see ``availability``). """ - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - states = self._simulation_states(state, working_nodes | broken_nodes) - self._require_free_members(working_nodes, broken_nodes) - self._require_groups_simulated(states) - self._start_queue( - t_simulation, working_nodes, broken_nodes, method, sources, states - ) - - def _start_queue( - self, - t_simulation, - working_nodes: set, - broken_nodes: set, - method: str, - sources: Optional[dict], - states: dict, - ) -> None: - """``initialize_event_queue``, with ``states`` checked (see - ``_simulation_states``).""" - # What the components draw their events from: themselves (an - # imperfectly repaired one through a stand-in keeping its virtual - # age, and one started from a state through one whose next draw can - # be set; kept for next_event), or stand-ins drawing from their own - # streams (see _streamed_components), which the caller passes to - # next_event too. - if sources is None: - sources = self._step_sources = self._own_sources(states) - else: - self.__dict__.pop("_step_sources", None) - # The window's end, which the first events already look to (see - # _replaced_at). - self.t_simulation = t_simulation - # Each calendar a state's phase shifts: by node, the time since its - # last scheduled replacement or test (see _due). - self._phases: dict = { - node: start.phase - for node, start in states.items() - if isinstance(start, NodeState) - and ( - start.phase - or ( - # On a calendar from 0, not at its offset. - start.phase is not None - and node in self._inspection - and self._inspection[node].offset - ) - ) - } - - # Keep record of component status', initially they're all working - component_status: dict[Any, bool] = { - component: True for component in self.components.keys() - } - - for component in broken_nodes: - component_status[component] = False - - # The queue supplies failure/repair events in chronological order - event_queue = _EventQueue() - # When each working node with hidden failures is due to fail (None - # once it has failed). - self._pending_failure: dict[Any, Optional[float]] = {} - # When each working node replaced on condition is due to fail, and - # to be replaced (see _replaced_at). - self._in_service: dict[Any, Tuple[float, float]] = {} - # Opportunistic maintenance (#108): when each member that can be - # renewed early was put into service as new, and its pending event; - # the pending events early renewals have cancelled, and the early - # renewals queued, by identity, and their members (see _stop). - self._renewed_at: dict[Any, float] = {} - self._pending_event: dict[Any, Event] = {} - self._cancelled: dict[int, Event] = {} - self._early: dict[int, Event] = {} - self._renewing: set = set() - # Common-cause groups (#158): each member's next event, which a - # shared cause that strikes it first supersedes, and the uniform a - # strike has decided its tests by (see _strike). - self._ccf_members = frozenset( - m for group in self.ccf_groups for m in group.members - ) - self._ccf_pending: dict[Any, Optional[Event]] = {} - self._ccf_coins: dict[Any, float] = {} - # The components down at the start in a repair or maintenance going - # on, which holds a repair crew. - in_hand: list = [] - - # For each component add in the initial failure - for component_id in self.components.keys(): - component = self.components[component_id] - if component_id in working_nodes: - continue - elif component_id in broken_nodes: - continue - elif component_id in self._standby: - continue # started below, once the queue and crews exist - source = sources[component_id] - start = states.get(component_id) - if isinstance(component, RepairableRBD): - source.initialize_event_queue(t_simulation, state=start) - component_status[component_id] = source.system_state - t_event, event = source.next_event() - first = Event( - t_event, component_id, event, _planned(source, event) - ) - elif start is not None and not start.new: - up, first = self._started(component_id, source, start) - component_status[component_id] = up - if not up and first.status: - in_hand.append(component_id) - elif component_id in self._preventive: - # Put into service as new at 0: it fails, or is maintained. - source.reset() - first = self._renewal( - component_id, 0.0, source, self._preventive[component_id] - ) - elif component_id in self._inspection: - # Put into service as new at 0: it fails, or is inspected. - source.reset() - first = self._inspected_renewal( - component_id, 0.0, source, self._inspection[component_id] - ) - else: - source.reset() - t_event, event = source.next_event() - first = Event(t_event, component_id, event) - - # Only consider it if it occurs within the simulation window - if first.time < t_simulation: - event_queue.put(first) - if component_id in self._ccf_members: - self._ccf_pending[component_id] = first - for cause in self._shared_causes(): - first = Event(sources[cause].gap(), cause, False) - if first.time < t_simulation: - event_queue.put(first) - self._event_queue = event_queue - # The repair crews, when there are fewer than the components that - # may need one (with enough, no job ever waits). - crews = self.repair_crews - self._crews = ( - _Crews(crews, self._crew_served(), self._priority) - if crews is not None and self._crews_limited() - else None - ) - if self._crews is not None: - # A repair or maintenance going on at the start holds a crew - # (_simulation_states has checked that there are enough). - for node in in_hand: - if node in self._crews.served: - self._crews.free -= 1 - self._crews.holding.add(node) - # Each standby group's units, which queue the group's first event. - self._groups: dict[Any, _StandbyGroup] = { - node: _StandbyGroup( - self, node, arrangement, sources[node], t_simulation - ) - for node, arrangement in self._standby.items() - if node not in working_nodes and node not in broken_nodes - } - self.last_change_planned = False - # The initial system state must reflect any forced-broken components - # (e.g. a broken component in series starts the system down), and - # any component that starts down, rather than assuming everything is - # up. - self.system_state = self.is_system_working(component_status, method) - self.component_status = component_status - - def _started(self, node, source, start: NodeState) -> Tuple[bool, Event]: - """Whether component ``node`` is up at 0, started from ``start`` - (not new), and its first event. - - One uniform, from the component's own stream (see - ``_streamed_components``), draws what is left of its repair or - maintenance given how long it has taken so far, or of its life - given its age (``_aged_life``: ``S(a + x) / S(a)`` is the chance - of more than ``x`` left). Up at age 0 it is new, and draws its life - as a new unit does. Its calendar is shifted by the state's phase - (see ``_due``); under age replacement, its replacement is due when - it reaches the age, at once if it has. A unit with hidden failures - is known to have been up only at its last test (or when put into - service, if since): its life left is drawn from then, and if it - has run out it failed unseen, and the next test finds it. After - the first, its units are new, as from new. - """ - component = self.components[node] - schedule = self._preventive.get(node) - inspection = self._inspection.get(node) - source.reset() - if not start.alive: - # Down: up again, as new, once its repair or maintenance is - # done. - model = ( - schedule.duration # type: ignore[union-attr] - if start.maintenance - else component.time_to_replace - ) - left = _aged_life(model, start.down_for, source.start_uniform()) - if inspection is not None: - self._pending_failure[node] = None - return False, Event(left, node, True, start.maintenance) - if not start.age: - # New at 0, on a calendar at its phase. - if schedule is not None: - return True, self._renewal(node, 0.0, source, schedule) - if inspection is not None: - return True, self._inspected_renewal( - node, 0.0, source, inspection - ) - delay, status = source.next_event() - return True, Event(delay, node, status) - since = 0.0 - if inspection is not None: - # Last known up at its last test, or when put into service. - since = min(start.age, start.phase or 0.0) - life = ( - _aged_life( - component.reliability, - start.age - since, - source.start_uniform(), - ) - - since - ) - source.life_drawn() - if inspection is not None: - if life < 0.0: - # Failed, unseen, since it was last known up. - self._pending_failure[node] = None - found = self._finds(node, inspection, life) - return False, Event(found, node, False, inspection=True) - self._pending_failure[node] = life - return True, self._inspected_next(node, 0.0, inspection) - if schedule is None: - return True, Event(life, node, False) - return True, self._renewal( - node, 0.0, source, schedule, life=life, start=-start.age + return _event_loop.initialize_event_queue( + self, + t_simulation=t_simulation, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + sources=sources, + state=state, ) - def _due(self, node, schedule, t: float) -> float: - """The next scheduled action of ``schedule`` (a ``_Preventive`` or - ``_Inspection``) after ``t``, on ``node``'s calendar: shifted, from - a state, by its phase, so that the last action fell that long - before 0.""" - phase = self._phases.get(node) if self._phases else None - if phase is None: - return schedule.due(t) - if isinstance(schedule, _Inspection): - # The phase places the calendar: no offset. - schedule = schedule.without_offset() - due = schedule.due(t + phase) - phase - return due if due > t else due + schedule.interval - - def _finds(self, node, inspection: _Inspection, t: float) -> float: - """The test that finds a failure at ``t``, on ``node``'s calendar - (see ``_due``): the first at or after it, and not before 0.""" - phase = self._phases.get(node) if self._phases else None - if phase is None: - return inspection.finds(t) - inspection = inspection.without_offset() - found = inspection.finds(t + phase) - phase - if found < t or found < 0.0: - found += inspection.interval - return found - def mean_unavailability( self, working_nodes: Optional[Collection[Hashable]] = None, @@ -10785,21 +2573,12 @@ def mean_unavailability( >>> f"{rbd.mean_unavailability():.6e}" '9.999980e-13' """ - method = structure_method(method) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - if self.ccf_groups: - # Each group conditioned on within its module (#218). - _, weights, inputs = self._ccf_long_run( - working_nodes, broken_nodes - ) - return float(weights @ self._ccf_tabled(*inputs).system()[1]) - availability, unavailability, weights = ( - self._long_run_unavailabilities(working_nodes, broken_nodes) + return _long_run.mean_unavailability( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, ) - system = self._system_unreliability(availability, unavailability) - return float(weights @ system) def analysis_routes(self) -> Dict[str, "AnalysisRoute"]: """How each analysis of this RBD is computed, found without running @@ -10869,1012 +2648,7 @@ def analysis_routes(self) -> Dict[str, "AnalysisRoute"]: >>> routes["availability"].route 'simulated' """ - from repyability.rbd import routes as r - - out: Dict[str, r.AnalysisRoute] = {} - - def give(names, route) -> None: - for name in names: - out[name] = route - - long_run = self._long_run_route() - - long_run_nodes = self._long_run_nodes() - - def from_long_run(route, reason): - """A method of ``route``, explained by ``reason``, built on the - long-run values: refused as they are, and numerical or simulated - as the components' are.""" - if long_run.route == r.REFUSED: - return long_run - return r.with_nodes( - route, reason, long_run_nodes, "long-run values" - ) - - give(("mean_availability", "mean_unavailability"), long_run) - out["node_availability"] = ( - self._long_run_route(groups=False) if self.ccf_groups else long_run - ) - frequencies = ( - None - if long_run.route == r.REFUSED - else r.refusal(self._require_ccf_frequencies) - ) - give( - ( - "system_failure_frequency", - "mean_time_between_failures", - "mean_up_time", - "mean_down_time", - ), - r.refused(frequencies) if frequencies else long_run, - ) - give( - ("barlow_proschan_importance",), - ( - r.refused(frequencies) - if frequencies - else ( - long_run - if long_run.route == r.REFUSED - else dataclasses.replace( - long_run, - reason="Each component's share of the system's " - "long-run failure frequency (the Birnbaum/Vesely " - "formula's terms). " + long_run.reason, - ) - ) - ), - ) - # The allocations assume independent components, which limited - # repair crews make them not; the common-cause groups' chains take - # theirs in. - allocation = r.refusal( - partial(self._require_unlimited_crews, *_ALLOCATION_CREWS) - ) or r.refusal(self._require_ccf_long_run) - conditioned = long_run - if self.ccf_groups and long_run.route != r.REFUSED: - conditioned = dataclasses.replace( - long_run, - reason=long_run.reason - + " A common-cause group's member is conditioned on its " - "state at each time, as the shared causes tie the other " - "members' to it.", - ) - if self._crews_couple() and long_run.route != r.REFUSED: - conditioned = dataclasses.replace( - long_run, - reason=long_run.reason - + " With the crews, Birnbaum's measure, the improvement " - "potential, RAW and RRW hold each node working and failed " - "in the chain, solved without it; the criticality and " - "Fussell-Vesely measures are probabilities over its " - "states.", - ) - give( - ( - "birnbaum_importance", - "improvement_potential", - "risk_achievement_worth", - "risk_reduction_worth", - "criticality_importance", - ), - conditioned, - ) - pairs = r.refusal(self._require_no_ccf_pairs) - give( - ("joint_importance",), - ( - r.refused(pairs) - if pairs - else ( - conditioned - if conditioned.route == r.REFUSED - else dataclasses.replace( - conditioned, - reason="Each component's Birnbaum importance with " - "the other held up, less with it held down. " - + conditioned.reason, - ) - ) - ), - ) - give( - ("differential_importance",), - ( - conditioned - if conditioned.route == r.REFUSED - else dataclasses.replace( - conditioned, - reason="Birnbaum's measure (or, for a proportional " - "change, the criticality importance), shared out; " - "over='parameters', parameter_sensitivity's " - "derivatives. " + conditioned.reason, - ) - ), - ) - give( - ("fussell_vesely",), - from_long_run( - r.EXACT, - "The exact probability that a minimal cut set containing " - "each node is down, over the system's unavailability, from " - "the exact long-run values (method='rare_event' sums the cut " - "sets' probabilities instead).", - ), - ) - give( - ("parameter_sensitivity",), - from_long_run( - r.NUMERICAL, - "Central differences of the long-run availability (or cost " - "rate) in each lever, the diagram rebuilt with the lever " - "moved; at times or over a window, of the values over time.", - ), - ) - if self.has_costs: - setups = r.refusal(self._require_separate_setups) - give( - ("expected_cost_rate", "total_cost"), - ( - r.refused(setups) - if setups - else from_long_run( - r.EXACT, - "The long-run cost rate, from the exact long-run " - "values (total_cost multiplies it by the time).", - ) - ), - ) - else: - give( - ("expected_cost_rate", "total_cost"), - r.AnalysisRoute( - r.EXACT, - "No running cost is priced, so the cost rate is 0 " - "(total_cost adds any acquisition costs).", - ), - ) - capacity = ( - None if long_run.route == r.REFUSED else self._capacity_refusal() - ) - out["capacity_distribution"] = ( - r.refused(*capacity) - if capacity - else from_long_run( - r.EXACT, - "The exact long-run distribution of the system's capacity, " - + ( - "averaged over the states of the repair crews' Markov " - "chain." - if self._crews_couple() - else "from the components' long-run availabilities." - ), - ) - ) - no_capacity = r.refusal(self._require_numbered_capacities) or ( - r.refusal(self._require_capacity) - ) - out["system_capacity"] = ( - r.refused(no_capacity) - if no_capacity - else r.AnalysisRoute( - r.EXACT, - "The exact distribution of the system's capacity from the " - "node probabilities given.", - ) - ) - give( - ( - "point_availability", - "mission_availability", - "point_unavailability", - "mission_unavailability", - ), - self._over_time(), - ) - give( - ("availability_rate",), - ( - self._crew_over_time("rate") - if self._crews_couple() - else self._over_time( - "Each component's renewal equation solved on a grid (to " - "about 1e-7) and differentiated on it, times its Birnbaum " - "importance at the components' availabilities; the jumps " - "at scheduled events split along the path between the " - "values either side." - + ( - " A common-cause group's part is its chain's moves, " - "split by cause and by member repair, at the " - "system's availability with the group in each of " - "its states (exact); the jumps at a hidden group's " - "tests are split by the Shapley value of what jumps." - if self.ccf_groups - else "" - ) - ) - ), - ) - over_time_capacity = self._capacity_refusal() - give( - ("point_capacity", "mission_capacity"), - ( - r.refused(*over_time_capacity) - if over_time_capacity - else self._over_time( - "Each component's renewal equation solved on a grid (to " - "about 1e-7), a degrading component's stages through its " - "renewals too, and the system's capacity distribution " - "exactly at its components' at each time.", - kind="capacity", - ) - ), - ) - window = self._over_time( - "Each component's expected events from its renewal equation, " - "solved on a grid (to about 1e-7), and the system's failures by " - "the time-dependent Birnbaum/Vesely formula.", - kind="window", - ) - give(("expected_failures", "expected_events"), window) - out["expected_cost"] = ( - window - if self.has_costs - else r.AnalysisRoute( - r.EXACT, - "No running cost is priced, so the expected cost is 0 (with " - "any acquisition costs beside it).", - ) - ) - # A group's members' MTTF and MTTR are their group's: the - # availability allocations hold them, and a member's copies join its - # group. - held_members = ( - " A common-cause group's members keep their availability: " - "their MTTF and MTTR are their group's." - if self.ccf_groups - else "" - ) - out["allocate_redundancy"] = ( - r.refused(allocation) - if allocation - else from_long_run( - r.EXACT, - "A search over the exact long-run values of the candidates." - + ( - " A common-cause group member's copies join its group, " - "whose chain takes them in: a BetaFactor group's (an MGL " - "group's member refuses)." - if self.ccf_groups - else "" - ), - ) - ) - out["availability_allocation"] = ( - r.refused(allocation) - if allocation - else from_long_run( - r.EXACT, - "A search over the exact long-run values of the candidates." - + held_members, - ) - ) - out["mttf_mttr_allocation"] = ( - r.refused(allocation) - if allocation - else from_long_run( - r.NUMERICAL, - "A solver for the components' MTTF and MTTR targets, over the " - "exact long-run values." + held_members, - ) - ) - if out["availability_allocation"].route != r.REFUSED: - # Which components are allocated one follows from the diagram - # (and fixed, which only holds more). - none = r.refusal(partial(self._allocated_levers, set())) - if none: - give( - ("availability_allocation", "mttf_mttr_allocation"), - r.refused(none), - ) - # The optimisers, called for every node they can choose for. - targets = r.refusal(partial(self._interval_targets, None, None)) - interval_crews = r.refusal(self._require_interval_crews) - refusal = ( - r.refusal(partial(self._maintained, None)) - or targets - or interval_crews - ) - out["optimal_replacement_intervals"] = ( - r.refused(refusal) - if refusal - else from_long_run( - r.NUMERICAL, - "An optimiser over the age-replacement intervals, with the " - "exact long-run values at each.", - ) - ) - refusal = ( - r.refusal(partial(self._inspected, None)) - or targets - or interval_crews - or next( - ( - message - for message in ( - r.refusal(partial(self._require_tested_exact, node)) - for node in self._inspection - ) - if message - ), - None, - ) - or r.refusal(self._require_one_inspected) - ) - out["optimal_inspection_intervals"] = ( - r.refused(refusal) - if refusal - else from_long_run( - r.NUMERICAL, - "An optimiser over the inspection interval, with the exact " - "long-run values at each.", - ) - ) - # Spares: each component's replacements, counted as a renewal - # process, in the order the methods check them. - for name, long_run_count in ( - ("spares_demand", False), - ("spares_stock", True), - ): - refusal = r.refusal( - partial( - self._require_unlimited_crews, - *(_STOCK_CREWS if long_run_count else _SPARES_CREWS), - ) - ) - which: Tuple[Hashable, ...] = () - for node in self._spares_nodes(None): - if refusal: - break - refusal = r.refusal( - partial(self._replacements, node, long_run_count) - ) - which = (node,) - out[name] = ( - r.refused(refusal, which) - if refusal - else r.AnalysisRoute( - r.NUMERICAL, - "Each component's replacements, a renewal process, " - "counted on a grid (to about 1e-6): under block " - + ( - "replacement, averaged over where the lead time " - "falls in the block interval (with repairs or " - "block replacements that take time, from a typical " - "replacement, a failure at each phase of the " - "interval or a block replacement, in the long run)" - if long_run_count - else "replacement, from one block interval to the " - "next" - ) - + "; with hidden failures, on its tests: exactly, tested " - "and repaired in no time by tests that find every " - "failure, and otherwise from the cycle between the tests " - "that find failures, followed test by test (#159).", - ) - ) - plan, streamed = self._stream_plan(1.0, 0, False) - paired = ( - "" - if streamed - else " Antithetic pairs are refused: some components' draws " - "do not come from a stream." - ) - groups = r.refusal(self._require_groups_simulated) - given = r.refusal(self._require_capacities_given) - engine, why = self._engine_choice(capacity=self._has_capacity()) - causes = ( - " Each common-cause group's shared causes strike as Poisson " - "processes, each failing the members it names that are up." - if self._has_ccf() - else "" - ) - out["availability"] = ( - r.refused(groups) - if groups - else ( - r.refused(given, tuple(self._capacity_models())) - if given - else r.AnalysisRoute( - r.SIMULATED, - "A discrete-event simulation of the components' failures, " - "repairs and maintenance." + causes + paired, - engine=engine, - engine_reason=why, - ) - ) - ) - out["simulate_chunk"] = out["availability"] - unsaved = r.refusal(self._shard_system) - out["shards"] = ( - out["availability"] - if out["availability"].route == r.REFUSED - else ( - r.refused(unsaved) - if unsaved - else r.AnalysisRoute( - r.SIMULATED, - "The run's simulations as shards, to simulate anywhere " - "with run_shard (see shards); availability_from_chunks " - "merges their partials into its result.", - ) - ) - ) - out["availability_from_chunks"] = ( - out["availability"] - if given or groups - else r.AnalysisRoute( - r.SIMULATED, - "Simulated chunks of a run (see simulate_chunk), merged into " - "its result.", - ) - ) - engine, why = _timeline_runs.engine_choice(self, plan) - out["simulate_timelines"] = ( - r.refused(groups) - if groups - else r.AnalysisRoute( - r.SIMULATED, - "The simulations availability runs, their histories kept " - "as timelines: recorded by the event loop as it runs." - + paired, - engine=engine, - engine_reason=why, - ) - ) - engine, why = self._engine_choice(capacity=False) - out["cost"] = ( - r.refused(groups) - if groups and self.has_costs - else r.AnalysisRoute( - r.SIMULATED, - "A discrete-event simulation of the components' failures, " - "repairs and maintenance, and what they cost." + paired, - engine=engine, - engine_reason=why, - ) - ) - twin = self._twin_report() - # The expected values over the window, where the methods over time - # work them out, need no simulation, and a run takes them (#187). - exact_means = "" - if all( - out[name].route in (r.EXACT, r.NUMERICAL) - for name in ("mission_availability", "expected_events") - + (("expected_cost",) if self.has_costs else ()) - ): - exact_means = ( - " Its expected values over the window need none: " - "mission_availability, expected_events and expected_cost " - "work them out, and by default a run's means are theirs, " - "with no error, so a run to a tolerance stops at " - "once. Simulate for their spread: each simulation's values, " - "percentiles, the chance of no failure." - ) - # Or a run's means are taken given its modules' histories (#189). - if not exact_means: - modules = self._conditional_applies() - if modules: - names = ", ".join(map(repr, modules)) - exact_means = ( - " By default a run's means take each " - "simulation's expected values given the histories of " - f"{names}, the rest exact given their states, which " - "vary less than its own; with conditional=True only " - f"{names} are simulated, for those means at a fraction " - "of the work, and no spread." - ) - for name in ("availability", "cost"): - if out[name].route == r.SIMULATED: - out[name] = dataclasses.replace( - out[name], reason=out[name].reason + exact_means, twin=twin - ) - # The difference of exact expected values needs no simulation - # (#236): where _exact_means works them out. - needed = [out["mission_availability"]] + ( - [out["expected_cost"]] if self.has_costs else [] - ) - # A diagram too meshed to work out has no exact expected values, - # so compare simulates it. - if self._too_meshed() is None and all( - route.route in (r.EXACT, r.NUMERICAL) for route in needed - ): - out["compare"] = r.AnalysisRoute( - ( - r.NUMERICAL - if any(route.route == r.NUMERICAL for route in needed) - else r.EXACT - ), - "The difference of the two systems' expected values over " - "the window (mission_availability, and expected_cost their " - "costs), where the other's are worked out too (#236); " - "otherwise, or with control_variate=False, the two systems " - "simulated with common random numbers.", - ) - elif groups: - out["compare"] = r.refused(groups) - elif streamed: - out["compare"] = r.AnalysisRoute( - r.SIMULATED, - "The two systems simulated with common random numbers.", - engine=engine, - engine_reason=why, - ) - else: - out["compare"] = r.refused(_UNSTREAMED) - give( - ("initialize_event_queue", "next_event"), - ( - r.refused(groups) - if groups - else r.AnalysisRoute( - r.SIMULATED, - "One simulation, stepped through event by event, drawing " - "from numpy's global RNG.", - ) - ), - ) - out["structural_importance"] = r.AnalysisRoute( - r.EXACT, "From the structure alone." - ) - out["system_probability"] = r.AnalysisRoute( - r.EXACT, - "The structure function over the node probabilities given.", - ) - out["path_set_probabilities"] = r.AnalysisRoute( - r.EXACT, "From the node probabilities given." - ) - allocation_by_probability = r.AnalysisRoute( - r.NUMERICAL, - "A solver over the exact system probability, from the node " - "probabilities given (not the component models).", - ) - give( - ( - "improvement_allocation", - "equal_allocation", - "simple_allocation", - "cost_based_allocation", - ), - allocation_by_probability, - ) - series = r.refusal(self._require_series) - out["minimum_effort_allocation"] = ( - r.refused(series) if series else allocation_by_probability - ) - # Each component's own values, and the long-run costs but the - # system's downtime, need no structure; nor the expected cost when - # nothing is priced. - free = {"node_availability", "spares_demand", "spares_stock"} - if not self.downtime_cost_rate: - free |= {"expected_cost_rate", "total_cost"} - if not self.has_costs: - free.add("expected_cost") - meshed = self._too_meshed() - if meshed is not None and self._has_capacity(): - # The simulations that follow the system's capacity do so - # through its reduced diagram, which a structure too meshed has - # not. - for name in ( - "availability", - "simulate_chunk", - "availability_from_chunks", - "shards", - ): - out[name] = r.refused(meshed) - out = self._meshed_routes(out, free) - # Parameter uncertainty (#200): each draw's diagram rebuilt and its - # value worked out as the diagram's own, so refused where that is - # (a structure too meshed included). - for name, method in ( - ("mean_availability_uncertainty", "mean_availability"), - ("point_availability_uncertainty", "point_availability"), - ("mission_availability_uncertainty", "mission_availability"), - ("expected_cost_rate_uncertainty", "expected_cost_rate"), - ): - own = out[method] - out[name] = ( - own - if own.route == r.REFUSED - else r.AnalysisRoute( - r.SIMULATED, - "The components' models drawn from their uncertainty, " - f"and each draw's diagram's {method} worked out as its " - f"own ({own.route}).", - ) - ) - sensitivity = out["parameter_sensitivity"] - out["uncertainty_importance"] = ( - sensitivity - if sensitivity.route == r.REFUSED - else r.AnalysisRoute( - r.NUMERICAL, - "The delta method: parameter_sensitivity's derivatives in " - "each uncertain parameter, with their covariance " - "(method='sobol' estimates Sobol indices from draws " - "instead).", - ) - ) - return dict(sorted(out.items())) - - def _capacity_refusal(self) -> Optional[Tuple[str, tuple]]: - """What ``capacity_distribution`` refuses beyond the long-run - values, in the order it checks: a degrading component on a - schedule, here or in a nested RBD, then no capacity at all. The - message and the nodes, or None.""" - from repyability.rbd import routes as r - - for node, model in self._capacity_models().items(): - if isinstance(model, RepairableRBD): - inner = model._capacity_refusal() - if inner: - return inner[0], (node,) - continue - message = r.refusal( - partial(self._require_unscheduled_stages, node) - ) - if message: - return message, (node,) - message = r.refusal(self._require_capacity) - return (message, ()) if message else None - - def _node_long_run(self, node) -> Tuple[str, str]: - """How a component's long-run values are found: the route, and a - phrase saying how (the message it raises, if refused).""" - from repyability.rbd import routes as r - - component = self.components[node] - if isinstance(component, RepairableRBD): - inner = component._long_run_route() - if inner.route == r.REFUSED: - return r.REFUSED, inner.reason - return ( - inner.route, - f"a nested RBD's long-run values, {inner.route}", - ) - message = r.refusal(partial(self._require_perfect_repair, node)) - if message: - return r.REFUSED, message - if node in self._standby: - message = r.refusal(partial(self._standby_rates, node)) - if message: - return r.REFUSED, message - return r.EXACT, "a standby group's Markov chain" - if node in self._inspection: - message = r.refusal(partial(self._require_tested_exact, node)) - if message: - return r.REFUSED, message - kind = self._tested_kind(node) - if kind == "unit": - return r.NUMERICAL, _TESTED_CYCLE - if kind == "life": - return ( - r.NUMERICAL, - "hidden failures, renewed at the tests, summed over the " - "test intervals", - ) - return r.EXACT, "hidden failures at a constant rate" - schedule = self._preventive.get(node) - if schedule is None: - message = r.refusal(component.mean_availability) - if message: - return r.REFUSED, message - life, how = r.mean_route(component.reliability) - if life == r.EXACT: - return r.EXACT, "its mean life and mean repair time" - return life, f"its mean life, {how}" - message = r.refusal(partial(self._require_no_opportunities, node)) - if message: - return r.REFUSED, message - life, how = r.model_route(component.reliability) - if schedule.policy in ("block", "condition"): - message = r.refusal(partial(self._require_block_models, node)) - if message: - return r.REFUSED, message - if schedule.policy == "condition": - return ( - r.NUMERICAL, - "replacement on condition: its renewal cycle, followed " - "from one inspection to the next on a grid", - ) - return ( - r.NUMERICAL, - "block replacement: its renewal cycle, solved on a grid", - ) - if life == r.SIMULATED: - return life, f"age replacement of a life from {how}" - return r.NUMERICAL, "age replacement: its mean up time, by quadrature" - - def _long_run_nodes(self) -> Dict[Any, Tuple[str, str]]: - """Each component's long-run route (see ``_node_long_run``).""" - return {node: self._node_long_run(node) for node in self.components} - - def _long_run_route(self, groups: bool = True) -> "AnalysisRoute": - """How the exact long-run values are found (see - ``analysis_routes``); without ``groups``, the components' own, as - ``node_availability`` gives them (the common-cause groups change - only which are down together).""" - from repyability.rbd import routes as r - - chains = r.refusal(self._require_ccf_long_run) if groups else None - if chains: - # (Before the crews: the groups' chains do not take them in.) - return r.refused(chains) - if self._crews_couple(): - return self._crew_chain_route() - calendars = r.refusal(self._require_calendars) - if calendars: - return r.refused(calendars) - nodes = self._long_run_nodes() - refusals = { - n: how for n, (route, how) in nodes.items() if route == r.REFUSED - } - if refusals: - return r.refused(next(iter(refusals.values())), tuple(refusals)) - return r.with_nodes( - r.EXACT, - "The structure function over the components' long-run " - "availabilities, exactly" - + ( - ", and over each common-cause group's members' joint states " - "(a Markov chain)." - if self.ccf_groups and groups - else "." - ), - nodes, - "long-run values", - ) - - def _crew_chain_route(self) -> "AnalysisRoute": - """How the exact long-run values are found with limited repair - crews: from their Markov chain, and nested RBDs' own values.""" - from repyability.rbd import routes as r - - chain = r.refusal(self._require_crew_chain) - if chain: - return r.refused(chain) - nested = { - node: self._node_long_run(node) - for node, component in self.components.items() - if isinstance(component, RepairableRBD) - } - refusals = { - n: how for n, (route, how) in nested.items() if route == r.REFUSED - } - if refusals: - return r.refused(next(iter(refusals.values())), tuple(refusals)) - return r.with_nodes( - r.EXACT, - "The long-run distribution of the Markov chain of the " - "components' states and the repair queue, solved exactly, and " - "the system's values averaged over its states.", - nested, - "long-run values", - ) - - def _node_over_time( - self, node, kind: str = "availability" - ) -> Tuple[str, str]: - """How a component's availability over time is found, for ``kind`` - (see ``_over_time``): the route, and a phrase saying how (the - message it raises, if refused).""" - from repyability.rbd import routes as r - - component = self.components[node] - if isinstance(component, RepairableRBD): - if ( - kind == "capacity" - and component._crews_couple() - and component._has_capacity() - ): - return r.REFUSED, str( - r.refusal( - partial( - component._no_crew_window, None, "the capacities" - ) - ) - ) - inner = component._over_time(kind=kind) - if inner.route == r.REFUSED: - return r.REFUSED, inner.reason - return inner.route, f"a nested RBD's availability, {inner.route}" - if node in self._standby: - message = r.refusal(partial(self._standby_rates, node)) - if message: - return r.REFUSED, message - return ( - r.NUMERICAL, - "its units' Markov chain, followed over time by " - "uniformization", - ) - message = r.refusal(partial(self._require_time_models, node)) - if not message: - message = r.refusal(partial(self._require_no_opportunities, node)) - if not message and node in self._imperfect: - message = r.refusal(partial(self._require_minimal_repair, node)) - if message: - return r.REFUSED, message - return ( - r.EXACT, - "minimal repair in no time: up throughout, failing as often " - "as its life's cumulative hazard", - ) - if not message and node in self._inspection: - message = r.refusal(partial(self._require_tested_exact, node)) - schedule = self._preventive.get(node) - calendar = schedule is not None and schedule.policy != "age" - if not message and calendar: - message = r.refusal(partial(self._require_block_models, node)) - if message: - return r.REFUSED, message - if calendar and schedule.policy == "condition": # type: ignore - return ( - r.NUMERICAL, - "replacement on condition: followed from one inspection to " - "the next on a grid, the units kept by age", - ) - if node in self._inspection and self._tested_kind(node) == "unit": - return r.NUMERICAL, _TESTED_CYCLE + " over time" - return r.NUMERICAL, "its renewal equation, solved on a grid" - - def _over_time( - self, - reason: str = ( - "Each component's renewal equation solved on a grid (to about " - "1e-7), and the system exactly at its components' availabilities " - "at each time." - ), - kind: str = "availability", - ) -> "AnalysisRoute": - """How the availability over time from new is found (see - ``analysis_routes``), and what is built on it: the method's - ``reason``. ``kind`` is what is asked: the ``"availability"``, the - expected events over a ``"window"``, or the ``"capacity"``, which - with limited repair crews come from their chain (see - ``_crew_over_time``).""" - from repyability.rbd import routes as r - - if self._crews_couple(): - return self._crew_over_time(kind) - if self.ccf_groups: - refusal = r.refusal(partial(self._require_groups_over_time, {})) - if refusal: - return r.refused(refusal) - reason = reason + _GROUPS_OVER_TIME - nodes = { - node: self._node_over_time(node, kind) for node in self.components - } - refusals = { - n: how for n, (route, how) in nodes.items() if route == r.REFUSED - } - if refusals: - return r.refused(next(iter(refusals.values())), tuple(refusals)) - return r.with_nodes(r.NUMERICAL, reason, nodes, "availabilities") - - def _crew_over_time(self, kind: str) -> "AnalysisRoute": - """How ``kind`` over time (see ``_over_time``) is found with limited - repair crews: from their chain, followed over time by - uniformization (see ``_crew_curve``), unless it refuses.""" - from repyability.rbd import routes as r - - refusal = r.refusal( - partial(self._require_crew_over_time, frozenset(), kind) - ) - if refusal: - return r.refused(refusal) - chain = ( - "The Markov chain of the components' states and the repair " - "queue, followed over time from its state at 0 by " - "uniformization (to about 1e-13): " - ) - nested = self._crew_nested(frozenset()) - if kind == "window" and nested: - reason = chain + ( - "the system's expected failures are those of its " - "components, at their rates in each state, and of each " - "nested RBD, independent of the chain, at its importance " - "over the chain and the other nested RBDs' patterns, " - "integrated by quadrature." - ) - elif kind == "window": - reason = chain + ( - "the system's expected failures, and each component's " - "failures and down time, are the integrals of their rates " - "over its states." - ) - elif kind == "rate": - reason = chain + ( - "the system's rate of change is split by the component " - "whose failure or repair makes each transition (a crew " - "taking the next job belongs to the repair that freed it), " - "each part one more vector of the chain, exact" - + ( - "; a nested RBD's part is its importance over the chain " - "and the other nested RBDs' patterns times its own rate" - if nested - else "" - ) - + "." - ) - elif kind == "capacity" and nested: - reason = chain + ( - "the capacity distribution is the probability of the states " - "at each level, for each combination of the nested RBDs' " - "levels, weighted by their own distributions, and over a " - "mission its integral, by quadrature." - ) - elif kind == "capacity": - reason = chain + ( - "the capacity distribution is the probability of the states " - "at each level, and over a mission its integral." - ) - elif nested: - reason = chain + ( - "the system's availability is the probability of the states " - "it is up in, worked out for each pattern of the nested RBDs " - "up and down with their own availabilities, and the mission " - "availability its integral, by quadrature." - ) - else: - reason = chain + ( - "the system's availability is the probability of the states " - "it is up in, and the mission availability its integral, " - "exactly." - ) - inner = "availability" if kind == "rate" else kind - nodes = {node: self._node_over_time(node, inner) for node in nested} - refusals = { - n: how for n, (route, how) in nodes.items() if route == r.REFUSED - } - if refusals: - return r.refused(next(iter(refusals.values())), tuple(refusals)) - return r.with_nodes(r.NUMERICAL, reason, nodes, "availabilities") - - def _require_crew_over_time( - self, forced=frozenset(), kind: str = "availability" - ) -> list: - """Raise what the crews' chain over time refuses, in the order the - methods check it: common-cause groups, what the chain does not - cover (see ``_require_crew_chain``), and too many nested RBDs (see - ``_crew_nested``). Return the nested RBDs, less those in - ``forced``.""" - self._require_ccf_long_run() - self._require_crew_chain() - return self._crew_nested(forced) - - def _engine_choice(self, capacity: bool) -> Tuple[str, str]: - """The engine ``engine="auto"`` runs a long simulation on, and why - (see ``_simulation_engine``).""" - from repyability.rbd import _compiled - - plan = self._stream_plan(1.0, 0, False)[0] - engine, reason = _compiled.choice( - self, plan, object() if capacity else None - ) - if reason is not None: - return "python", f"the compiled engine does not simulate {reason}" - if engine is None: - return ( - "python", - "numba is not installed; pip install 'repyability[fast]' " - "for the compiled engine", - ) - return ( - engine, - "compiled for a run long enough to repay loading it; a short " - "one runs in Python", - ) + return _repairable_routes.analysis_routes(self) def mean_availability( self, @@ -11980,29 +2754,12 @@ def mean_availability( >>> round(rbd.mean_availability(), 4) 0.9091 """ - # Good reference on the Availability of a system - # https://www.diva-portal.org/smash/get/diva2:986067/FULLTEXT01.pdf - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - - if self.ccf_groups: - # Each group conditioned on within its module (#218). - method = structure_method(method) - _, weights, inputs = self._ccf_long_run( - working_nodes, broken_nodes - ) - up, down = self._ccf_tabled(*inputs).system() - return float(weights @ (up if method == "p" else 1.0 - down)) - # Over one period of the inspection schedules, if any (see - # _long_run_grid), or over the states of the repair crews' Markov - # chain; otherwise at one point, as the node availabilities are - # constant. - availability, weights = self._long_run_probabilities( - working_nodes, broken_nodes + return _long_run.mean_availability( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, ) - system = self.system_probability(availability, method=method) - return float(weights @ system) def point_availability( self, @@ -12119,7 +2876,14 @@ def point_availability( >>> rbd.point_availability([0.0, 1.0, 100.0]).round(4).tolist() [1.0, 0.9394, 0.9091] """ - return self._point(x, working_nodes, broken_nodes, method, state) + return _windows.point_availability( + self, + x=x, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, + ) def point_unavailability( self, @@ -12131,8 +2895,8 @@ def point_unavailability( ): """The probability that the system is down at each time ``x``, with every component new at 0: one less ``point_availability``, worked - out in its own right so that a small one keeps its precision - (#237), as ``mean_unavailability`` is in the long run. + out in its own right so that a small one keeps its precision, + as ``mean_unavailability`` is in the long run. Each component's probability of being down enters the structure function's sum of products for the system's failing, so that a @@ -12197,65 +2961,14 @@ def point_unavailability( >>> f"{pair.point_unavailability(1000.0):.3e}" '6.400e-17' """ - return self._point( - x, working_nodes, broken_nodes, method, state, down=True - ) - - def _point( - self, x, working_nodes, broken_nodes, method, state, down=False - ): - """``point_availability``, or with ``down``, ``point_unavailability`` - (#237).""" - times = _check_times(x) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - horizon = float(times.max()) if times.size else 0.0 - states = self._states(state, working_nodes | broken_nodes) - system_at, _, _ = self._system_at( - horizon, working_nodes, broken_nodes, method, states, down + return _windows.point_unavailability( + self, + x=x, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, ) - values = system_at(times.ravel()) - if np.ndim(x) == 0: - return float(values[0]) - return values.reshape(times.shape) - - def _system_at( - self, horizon, working_nodes, broken_nodes, method, states, down=False - ): - """The system's point availability (with ``down``, unavailability) - over ``[0, horizon]``, as a function of the times; the curves whose - bends its integral's pieces follow; and the crews' curve, where a - component can wait for a crew (else None).""" - if self._crews_couple(): - crew = self._crew_curve( - horizon, working_nodes, broken_nodes, method, states - ) - if not down: - return crew.at, [crew], crew - - def crew_down(x): - return np.clip(1.0 - crew.at(x), 0.0, 1.0) - - return crew_down, [crew], crew - if self.ccf_groups: - grouped = self._groups_curve( - horizon, working_nodes, broken_nodes, method, states - ) - return (grouped.down_at if down else grouped.at), [grouped], None - forced = working_nodes | broken_nodes - curves = self._availability_curves(horizon, forced, state=states) - - def system_at(x): - if down: - return self._curves_down_at( - curves, x, working_nodes, broken_nodes, states - ) - return self._curves_at( - curves, x, working_nodes, broken_nodes, method - ) - - return system_at, list(curves.values()), None def mission_availability( self, @@ -12341,7 +3054,14 @@ def mission_availability( >>> round(rbd.mission_availability(10.0), 4) 0.9174 """ - return self._mission(t, working_nodes, broken_nodes, method, state) + return _windows.mission_availability( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, + ) def mission_unavailability( self, @@ -12353,8 +3073,8 @@ def mission_unavailability( ): """The expected fraction of ``[0, t]`` the system is down, with every component new at 0: one less ``mission_availability``, worked - out in its own right so that a small one keeps its precision - (#237): the mean of + out in its own right so that a small one keeps its precision: + the mean of [`point_unavailability`][repyability.RepairableRBD.point_unavailability] over the window, integrated as ``mission_availability`` integrates the availability. @@ -12406,472 +3126,67 @@ def mission_unavailability( >>> f"{pair.mission_unavailability(8760.0):.3e}" '6.391e-17' """ - return self._mission( - t, working_nodes, broken_nodes, method, state, down=True - ) - - def _mission( - self, t, working_nodes, broken_nodes, method, state, down=False - ): - """``mission_availability``, or with ``down``, - ``mission_unavailability`` (#237).""" - windows = _check_times(t) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - ends = windows.ravel() - horizon = float(ends.max()) if ends.size else 0.0 - states = self._states(state, working_nodes | broken_nodes) - system_at, pieces, crew = self._system_at( - horizon, working_nodes, broken_nodes, method, states, down - ) - if crew is not None and not crew.nested: - # The crews' chain integrates it exactly. - totals = crew.integral(ends) - if down: - totals = np.maximum(ends - totals, 0.0) - else: - scale = None - if down: - # To the unavailability's own size, and the time scale of - # the fastest closed form in it (see _closed_form_down); a - # unit repaired at once is never down, and sets none. - scale = ( - min( - [horizon or 1.0] - + [ - 1.0 / rate - for rate in self._closed_form_rates( - states - ).values() - if 0.0 < rate < math.inf - ] - ) - / 4.0 - ) - totals = self._integrated(system_at, pieces, ends, horizon, scale) - averages = np.empty(len(ends)) - positive = ends > 0.0 - averages[positive] = totals[positive] / ends[positive] - if not positive.all(): - averages[~positive] = system_at(np.zeros(1))[0] - if np.ndim(t) == 0: - return float(averages[0]) - return averages.reshape(windows.shape) - - def _integrated( - self, - system_at, - curves: list, - ends: np.ndarray, - horizon: float, - relative: Optional[float] = None, - ) -> np.ndarray: - """The integral from 0 to each of ``ends`` of the system's point - availability, ``system_at``, from its nodes' ``curves`` (over the - pieces of ``_quadrature``, and settling into a constant or a period - it is extended over: see ``mission_availability``). With - ``relative``, a time scale, to a tolerance relative to the integral - itself, pieces being halved down to that scale where the grids are - coarser: for an unavailability, small, and changing in closed form - faster than the grids (#237).""" - # After ``settle`` the system's availability is constant, or repeats - # with ``period``: it is integrated up to ``reach``, and extended. - settle, period = _settling(curves) - reach = min(horizon, settle if period is None else settle + period) - beyond = ends > reach - fixed = [ends[~beyond]] - if period is not None and beyond.any(): - cycles = np.floor((ends[beyond] - settle) / period) - rest = ends[beyond] - settle - cycles * period - rest = np.clip(rest, 0.0, period) - fixed += [np.array([settle]), settle + rest] - - def estimate(a, b): - x, half = _quadrature.points(a, b) - return {"up": _quadrature.summed(system_at(x), half)} - - try: - edges, finest = _quadrature.pieces( - curves, np.concatenate(fixed), reach, _MISSION_POINTS - ) - keys: tuple = () - if relative is not None: - keys, finest = ("up",), np.minimum(finest, relative) - edges, integrals = _quadrature.refined( - estimate, edges, finest, _MISSION_POINTS, keys - ) - except _quadrature.TooMany as error: - raise NotImplementedError( - f"Integrating the availability over [0, {reach}] takes " - f"{error.count} pieces (the components' curves bend that " - "often), more than the limit: estimate it by simulation, " - "with availability()." - ) from None - running = _quadrature.running(integrals.get("up"), len(edges) - 1) - - def uptime(x): - return running[np.searchsorted(edges, x)] - - totals = np.empty(len(ends)) - totals[~beyond] = uptime(ends[~beyond]) - if beyond.any(): - if period is None: - level = system_at(np.array([horizon]))[0] - totals[beyond] = uptime(reach) + (ends[beyond] - reach) * level - else: - base = uptime(settle) - totals[beyond] = ( - base - + cycles * (uptime(reach) - base) - + (uptime(settle + rest) - base) - ) - return totals - - def _window( - self, - t, - working_nodes, - broken_nodes, - method: str, - nodes: bool = False, - setups: bool = False, - state=None, - causes: bool = False, - ): - """The windows ``t`` checked, flat, and the nodes' curves and the - system's expected events over them (see ``_window_counts``): the - forced nodes have no curves. With ``setups``, the stops of each - maintenance group with a set-up cost are counted too; with - ``causes``, the system failures each node (or common cause) - causes.""" - method = structure_method(method) - windows = _check_times(t) - working = set() if working_nodes is None else set(working_nodes) - broken = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working, broken) - forced = working | broken - ends = windows.ravel() - horizon = float(ends.max()) if ends.size else 0.0 - states = self._states(state, forced) - groups = None - if setups: - groups = { - name: [node for node in spec.members if node not in forced] - for name, spec in self._maintenance.items() - if spec.setup_cost - } - if self._crews_couple(): - curves, counts = self._crew_window( - ends, working, broken, method, states, nodes, groups, causes - ) - return windows, ends, curves, counts, working, broken - if self.ccf_groups: - curves, counts = self._groups_window( - ends, working, broken, method, states, nodes, groups, causes - ) - return windows, ends, curves, counts, working, broken - curves = self._availability_curves( - horizon, forced, counts=True, state=states - ) - counts = self._window_counts( - curves, - ends, - working, - broken, - method, - nodes=nodes, - groups=groups, - causes=causes, - ) - return windows, ends, curves, counts, working, broken - - def expected_failures( - self, - t, - working_nodes: Optional[Collection[Hashable]] = None, - broken_nodes: Optional[Collection[Hashable]] = None, - method: str = "p", - state=None, - ): - """The expected number of system failures in ``[0, t)``, with every - component new at 0: exact, with no simulation. - - A component's failure takes the system down if the component is - critical then: if the system is up with it and down without it. - The components fail and are repaired independently, so at a time - ``s`` that is so with probability ``I_B^i(s)``, the component's - Birnbaum importance at the components' point availabilities then, - and the system's expected failures are - - ```text - integral from 0 to t of sum_i I_B^i(s) dM_i(s) - ``` - - ``M_i(s)`` the component's expected failures by ``s``: the - time-dependent form of the Birnbaum/Vesely formula, whose long-run - rate is ``system_failure_frequency``. Each ``M_i`` follows from the - component's renewal equation, solved on the same grid as its point - availability (see ``point_availability``), to about 1e-7; the - integral is summed over the pieces ``mission_availability`` - integrates over, on each with the importance taken as the cubic - through its values at the quadrature points, and extended exactly - past the time the curves have settled. Failures at exact - times (units dead on arrival, an exact lifetime) are taken together - when several fall at once. It is what ``availability`` estimates as - ``system_failures / n_simulations``: every system failure counts, - including the zero-length outages an instantly repaired component - causes; planned outages do not (see ``expected_events``). - - With fewer ``repair_crews`` than components, the components are not - independent: the rate of system failures is then that of the crews' - Markov chain in each of its states (a component up failing at its - rate where it is critical), integrated over the chain's states over - time by uniformization, to about 1e-13 (see ``point_availability``). - The crews' chain takes no nested RBD's events in, as yet (#162). - - Parameters - ---------- - t : float or array-like - The windows' lengths (one count for each). - working_nodes : Collection[Hashable], optional - Nodes that always work, and never fail, by default None. - broken_nodes : Collection[Hashable], optional - Nodes that are always failed, by default None. - method : str, optional - Evaluate the structure function from the minimal path sets - (``"p"`` or ``"paths"``, the default) or the cut sets (``"c"`` - or ``"cuts"``); both give the - same result. - state : dict or str, optional - Start from the components' current states rather than new: - ``{node: NodeState}`` (see - [`NodeState`][repyability.NodeState]), a nested RBD's own such - dict for a nested RBD, or ``"stationary"`` for every component - in its long-run state. A component left out starts new. By - default None: every component new at 0. - - Returns - ------- - float or numpy.ndarray - The expected number of system failures in each window, in - ``t``'s shape. - - Raises - ------ - ValueError, NotImplementedError - As for ``point_availability``. - - Examples - -------- - One component with failure rate 0.1 and repair rate 1 fails at the - rate 0.1 while it is up, ``0.1 * (10 / 1.1 + 0.1 / 1.1 ** 2 * (1 - - exp(-11)))`` times in 10 time units: - - >>> import surpyval as surv - >>> from repyability import RepairableRBD - >>> rbd = RepairableRBD( - ... [("s", "c"), ("c", "t")], - ... { - ... "c": { - ... "reliability": surv.Exponential.from_params([0.1]), - ... "repairability": surv.Exponential.from_params([1.0]), - ... } - ... }, - ... ) - >>> round(rbd.expected_failures(10.0), 4) - 0.9174 - """ - windows, _, _, counts, _, _ = self._window( - t, working_nodes, broken_nodes, method, state=state - ) - return _shaped(counts["failures"], t, windows) - - def expected_events( - self, - t, - working_nodes: Optional[Collection[Hashable]] = None, - broken_nodes: Optional[Collection[Hashable]] = None, - method: str = "p", - state=None, - ) -> ExpectedEvents: - """What the system and each component are expected to do in - ``[0, t)``, with every component new at 0: exact, with no - simulation. - - The system's failures are those of ``expected_failures``, and its - planned outages are counted the same way, from its components' - preventive maintenance that takes time; its down time is ``t * (1 - - mission_availability(t))``. Each component's failures, corrective - and preventive actions and tests follow from its renewal equation, - solved on the grid of its point availability (see - ``point_availability``), to about 1e-7, and its down time is the - integral of its unavailability. The maintenance due at exact times - (at a replacement age, at block times) is counted exactly, and - maintenance of several components due at the same time takes the - system down at most once. It is the mean of what ``availability`` - counts in each simulation of the window, and events at ``t`` itself - fall outside the window, as there. - - A component's corrective actions, at each of which its - ``repair_cost`` and ``replace_cost`` are charged, are its failures, - except for hidden failures: those found by a test in the window. A - nested RBD's failures are its system failures, and its own - maintenance is not this RBD's to count. A forced node does nothing: - held working it is never down, held broken it is down throughout. - - Parameters - ---------- - t : float or array-like - The windows' lengths. - working_nodes : Collection[Hashable], optional - Nodes that always work, by default None. - broken_nodes : Collection[Hashable], optional - Nodes that are always failed, by default None. - method : str, optional - Evaluate the structure function from the minimal path sets - (``"p"`` or ``"paths"``, the default) or the cut sets (``"c"`` - or ``"cuts"``); both give the - same result. - state : dict or str, optional - Start from the components' current states rather than new: - ``{node: NodeState}`` (see - [`NodeState`][repyability.NodeState]), a nested RBD's own such - dict for a nested RBD, or ``"stationary"`` for every component - in its long-run state. A component left out starts new. By - default None: every component new at 0. - - Returns - ------- - ExpectedEvents - The system's expected failures, planned outages and down time, - and each component's expected failures, corrective and - preventive actions, tests and down time: floats for one window, - arrays in ``t``'s shape for several. - - Raises - ------ - ValueError, NotImplementedError - As for ``point_availability``. - - Examples - -------- - A pump that wears out, replaced at 600 hours in about 7 (a planned - outage), over a year: - - >>> import surpyval as surv - >>> from repyability import RepairableRBD - >>> rbd = RepairableRBD( - ... [("s", "p"), ("p", "t")], - ... { - ... "p": { - ... "reliability": surv.Weibull.from_params([1000, 2.5]), - ... "repairability": surv.LogNormal.from_params([3, 0.5]), - ... "preventive": { - ... "interval": 600.0, - ... "duration": surv.Weibull.from_params([8, 3]), - ... }, - ... } - ... }, - ... ) - >>> year = rbd.expected_events(8760.0) - >>> round(year.system_failures, 3), round(year.node_preventive["p"], 3) - (3.706, 11.275) - >>> round(year.system_planned_outages, 3) # each replacement - 11.275 - """ - windows, ends, curves, counts, working, broken = self._window( - t, working_nodes, broken_nodes, method, nodes=True, state=state - ) - node_events = { - node: curve.events(ends) for node, curve in curves.items() - } - zero = np.zeros(len(ends)) - - def shaped(values) -> Any: - return _shaped(values, t, windows) - - def per_node(key: str) -> dict: - return { - node: shaped( - node_events[node][key] if node in curves else zero - ) - for node in self.components - } - - downtime = { - node: shaped( - counts["downtime"][node] - if node in curves - else (ends if node in broken else zero) - ) - for node in self.components - } - return ExpectedEvents( - window=shaped(ends), - system_failures=shaped(counts["failures"]), - system_planned_outages=shaped(counts["planned"]), - system_downtime=shaped(np.maximum(ends - counts["uptime"], 0.0)), - node_failures=per_node("failures"), - node_corrective=per_node("corrective"), - node_preventive=per_node("preventive"), - node_inspections=per_node("inspections"), - node_downtime=downtime, + return _windows.mission_unavailability( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, ) - def expected_cost( + def expected_failures( self, t, working_nodes: Optional[Collection[Hashable]] = None, broken_nodes: Optional[Collection[Hashable]] = None, method: str = "p", state=None, - *, - discount_rate: float = 0.0, - ) -> ExpectedCost: - """The expected cost of running the system for ``t``, from new: - exact, with no simulation. + ): + """The expected number of system failures in ``[0, t)``, with every + component new at 0: exact, with no simulation. - Each category is its events' expected number over ``[0, t)`` (see - ``expected_events``) times its mean cost, as ``cost`` charges them: - ``repair_cost`` and ``replace_cost`` at each corrective action, the - preventive cost at each preventive replacement, the inspection cost - at each test, ``downtime_cost`` over each component's expected down - time, ``downtime_cost_rate`` over the system's, and a maintenance - group's set-up cost once per stop (each failure or replacement of a - member, those at one instant one stop). It is the mean that - ``cost`` estimates by simulation, with its distribution; - ``total_cost`` is the long-run approximation, exact only over a - window long compared with the components' cycles. As ``t`` grows - this approaches ``expected_cost_rate() * t`` plus a constant, of the - early years from new. + A component's failure takes the system down if the component is + critical then: if the system is up with it and down without it. + The components fail and are repaired independently, so at a time + ``s`` that is so with probability ``I_B^i(s)``, the component's + Birnbaum importance at the components' point availabilities then, + and the system's expected failures are - Every cost is optional, and a cost given as a distribution enters - through its mean. Only this RBD's own costs count, not those inside - a nested ``RepairableRBD``. With nothing priced (see - ``has_costs``), every category is 0, without any checks. + ```text + integral from 0 to t of sum_i I_B^i(s) dM_i(s) + ``` - With a ``discount_rate`` ``r`` each value is its present value - (#231), the costs discounted continuously from when they fall: - ``integral from 0 to t of exp(-r s) dC(s)``, ``C`` the expected - cost from new, worked out by parts, ``exp(-r t) C(t) + r * - integral from 0 to t of exp(-r s) C(s) ds``, the integral by - Gauss-Kronrod quadrature on pieces halved until the total is - within 1e-8 of itself. It is the exact form of ``total_cost``'s - discounted value, which spends the long-run rate from the start: - the early years, which count most, are those that differ from it. - The acquisition cost, at the start, is not discounted. + ``M_i(s)`` the component's expected failures by ``s``: the + time-dependent form of the Birnbaum/Vesely formula, whose long-run + rate is ``system_failure_frequency``. Each ``M_i`` follows from the + component's renewal equation, solved on the same grid as its point + availability (see ``point_availability``), to about 1e-7; the + integral is summed over the pieces ``mission_availability`` + integrates over, on each with the importance taken as the cubic + through its values at the quadrature points, and extended exactly + past the time the curves have settled. Failures at exact + times (units dead on arrival, an exact lifetime) are taken together + when several fall at once. It is what ``availability`` estimates as + ``system_failures / n_simulations``: every system failure counts, + including the zero-length outages an instantly repaired component + causes; planned outages do not (see ``expected_events``). + + With fewer ``repair_crews`` than components, the components are not + independent: the rate of system failures is then that of the crews' + Markov chain in each of its states (a component up failing at its + rate where it is critical), integrated over the chain's states over + time by uniformization, to about 1e-13 (see ``point_availability``). + The crews' chain takes no nested RBD's events in, as yet. Parameters ---------- t : float or array-like - The windows' lengths. + The windows' lengths (one count for each). working_nodes : Collection[Hashable], optional - Nodes that always work: they incur no corrective, preventive, - inspection or downtime cost, by default None. + Nodes that always work, and never fail, by default None. broken_nodes : Collection[Hashable], optional - Nodes that are always failed: they incur no corrective cost, but - their own downtime cost throughout, by default None. + Nodes that are always failed, by default None. method : str, optional Evaluate the structure function from the minimal path sets (``"p"`` or ``"paths"``, the default) or the cut sets (``"c"`` @@ -12884,2011 +3199,297 @@ def expected_cost( dict for a nested RBD, or ``"stationary"`` for every component in its long-run state. A component left out starts new. By default None: every component new at 0. - discount_rate : float, optional - The continuous discount rate per unit time of the component - models, as for ``total_cost``, by default 0 (undiscounted). Returns ------- - ExpectedCost - The expected cost of each window, by category and component, - with the acquisition cost beside it: present values with a - ``discount_rate``. + float or numpy.ndarray + The expected number of system failures in each window, in + ``t``'s shape. Raises ------ ValueError, NotImplementedError - When something is priced, as for ``point_availability``. + As for ``point_availability``. Examples -------- - A pump bought for 20,000, failing on average every 1000 hours, - repaired in 10 at 500 per repair, with lost production at 100 per - hour, owned for a year: + One component with failure rate 0.1 and repair rate 1 fails at the + rate 0.1 while it is up, ``0.1 * (10 / 1.1 + 0.1 / 1.1 ** 2 * (1 - + exp(-11)))`` times in 10 time units: >>> import surpyval as surv >>> from repyability import RepairableRBD >>> rbd = RepairableRBD( - ... [("s", "p"), ("p", "t")], + ... [("s", "c"), ("c", "t")], ... { - ... "p": { - ... "reliability": surv.Exponential.from_params([1e-3]), - ... "repairability": surv.Exponential.from_params([0.1]), - ... "repair_cost": 500.0, - ... "acquisition_cost": 20000.0, + ... "c": { + ... "reliability": surv.Exponential.from_params([0.1]), + ... "repairability": surv.Exponential.from_params([1.0]), ... } ... }, - ... downtime_cost_rate=100.0, ... ) - >>> year = rbd.expected_cost(8760.0) - >>> round(year.mean), round(year.total) - (13000, 33000) - >>> round(rbd.total_cost(8760.0)) # at the long-run rate - 33010 - - Owned for ten years at 7% a year, from new and at the long-run rate: - - >>> import math - >>> r = math.log(1.07) / 8760 # per hour - >>> round(rbd.expected_cost(87600.0, discount_rate=r).total) - 114528 - >>> round(rbd.total_cost(87600.0, discount_rate=r)) - 114538 - """ - rate = _discount_rate(discount_rate) - if rate > 0.0 and self.has_costs: - _present_horizon( - _horizons(t, 0.0), rate, self._mean_lives - ) # warns of a rate that discounts the costs away - return self._discounted_cost( - t, rate, working_nodes, broken_nodes, method, state - ) - if not self.has_costs: - windows = _check_times(t) - zero = _shaped(np.zeros(windows.size), t, windows) - return ExpectedCost( - window=_shaped(windows.ravel(), t, windows), - mean=zero, - by_category={category: zero for category in _CATEGORIES}, - by_component={}, - acquisition_cost=self.acquisition_cost, - ) - windows, ends, curves, counts, working, broken = self._window( - t, - working_nodes, - broken_nodes, - method, - nodes=True, - setups=True, - state=state, - ) - node_events = { - node: curve.events(ends) for node, curve in curves.items() - } - categories = { - category: np.zeros(len(ends)) for category in _CATEGORIES - } - by_component: Dict[Hashable, Any] = {} - for node, node_costs in self.costs.items(): - own = np.zeros(len(ends)) - charges = [] - events = node_events.get(node) - if events is not None: - for key, category, kind in ( - ("repair_cost", "repair", "corrective"), - ("replace_cost", "replace", "corrective"), - ("preventive_cost", "preventive", "preventive"), - ("inspection_cost", "inspection", "inspections"), - ): - if category == "replace" and node in self._imperfect: - # Minimally repaired (the only imperfect repair - # with exact values), it is never replaced: its - # repairs are charged only as repairs, as the - # simulation charges them. - continue - if key in node_costs: - charges.append( - ( - category, - _mean_cost(node_costs[key]) * events[kind], - ) - ) - rate = node_costs.get("downtime_cost", 0.0) - if rate: - down = ( - counts["downtime"][node] - if node in curves - else (ends if node in broken else np.zeros(len(ends))) - ) - charges.append(("component_downtime", rate * down)) - for category, charge in charges: - categories[category] = categories[category] + charge - own = own + charge - by_component[node] = _shaped(own, t, windows) - if self.downtime_cost_rate: - categories["system_downtime"] = self.downtime_cost_rate * ( - np.maximum(ends - counts["uptime"], 0.0) - ) - for name, spec in self._maintenance.items(): - if not spec.setup_cost: - continue - # Each failure or replacement of a member is a stop, but those - # at one instant are one. - stops = -counts["overlaps"].get(name, np.zeros(len(ends))) - for node in spec.members: - if node in node_events: - stops = stops + ( - node_events[node]["failures"] - + node_events[node]["preventive"] - ) - categories["setup"] = categories["setup"] + spec.setup_cost * stops - mean = np.zeros(len(ends)) - for values in categories.values(): - mean = mean + values - return ExpectedCost( - window=_shaped(ends, t, windows), - mean=_shaped(mean, t, windows), - by_category={ - category: _shaped(values, t, windows) - for category, values in categories.items() - }, - by_component=by_component, - acquisition_cost=self.acquisition_cost, - ) - - def _discounted_cost( - self, t, rate: float, working_nodes, broken_nodes, method: str, state - ) -> ExpectedCost: - """``expected_cost`` at ``t`` discounted at ``rate`` (#231): each - window's ``exp(-r T) C(T) + r * integral_0^T exp(-r s) C(s) ds``, - ``C`` the expected cost from new, the integral by the Gauss-Kronrod - (7, 15) rule on pieces from 0, with the windows among their ends. - The pieces whose two rules differ most are halved until the - differences add up to ``_DISCOUNT_TOLERANCE`` of the integral: - ``C`` jumps at scheduled events (tests, block replacements), where - a piece's error only halves with it.""" - windows = _check_times(t) - ends = windows.ravel() - horizon = float(ends.max()) if ends.size else 0.0 - keys: list = [] - - def values(times: np.ndarray) -> np.ndarray: - """``C`` at ``times``: its mean, each category's and each - component's, as rows. The longest window is worked out too, so - that every call's curves reach it, and are shared.""" - got = self.expected_cost( - np.append(times, horizon), - working_nodes, - broken_nodes, - method, - state, - ) - if not keys: - keys.extend(got.by_component) - rows = [got.mean] - rows += [got.by_category[category] for category in _CATEGORIES] - rows += [got.by_component[node] for node in keys] - return np.vstack([np.ravel(row)[:-1] for row in rows]) - - def rules(a: np.ndarray, b: np.ndarray): - """The pieces' integrals of ``exp(-r s) C(s)`` by the Kronrod - rule, as rows, and the mean's error, against the Gauss rule.""" - half = 0.5 * (b - a) - x = (0.5 * (a + b))[:, None] + half[:, None] * _GK_X - g = np.exp(-rate * x) * values(x.ravel()).reshape(-1, *x.shape) - kronrod = (g @ _GK_W) * half - return kronrod, np.abs(kronrod[0] - (g[0] @ _G7_W) * half) - - with self._sharing_curves(): - at_ends = values(ends) - a, b = _discount_pieces(ends, horizon) - pieces, error = rules(a, b) - for _ in range(_DISCOUNT_ROUNDS): - budget = _DISCOUNT_TOLERANCE * abs(float(pieces[0].sum())) - if error.sum() <= budget: - break - # Halve the pieces over their share of the error allowed. - split = error > budget / error.size - middle = 0.5 * (a + b)[split] - halves, errors = rules( - np.concatenate([a[split], middle]), - np.concatenate([middle, b[split]]), - ) - a = np.concatenate([a[~split], a[split], middle]) - b = np.concatenate([b[~split], middle, b[split]]) - pieces = np.concatenate([pieces[:, ~split], halves], axis=1) - error = np.concatenate([error[~split], errors]) - order = np.argsort(b, kind="stable") - running = np.concatenate( - [np.zeros((len(at_ends), 1)), np.cumsum(pieces[:, order], axis=1)], - axis=1, - ) - # Every window is a piece's end: the pieces up to it are within it. - upto = np.searchsorted(b[order], ends, side="right") - present = np.exp(-rate * ends) * at_ends + rate * running[:, upto] - rows = iter(present) - mean = next(rows) - categories = {category: next(rows) for category in _CATEGORIES} - components = {node: next(rows) for node in keys} - return ExpectedCost( - window=_shaped(ends, t, windows), - mean=_shaped(mean, t, windows), - by_category={ - category: _shaped(v, t, windows) - for category, v in categories.items() - }, - by_component={ - node: _shaped(v, t, windows) for node, v in components.items() - }, - acquisition_cost=self.acquisition_cost, - discount_rate=rate, - ) - - def _filled( - self, probabilities: dict, size: int, working_nodes, broken_nodes - ) -> dict: - """The nodes' ``probabilities`` (arrays of ``size``), with the input - and output nodes, and the forced nodes held at 1 or 0, added.""" - out = dict(probabilities) - for node in list(self.in_or_out) + list(working_nodes | broken_nodes): - out[node] = np.ones(size) - return self._probabilities_with_overrides( - out, working_nodes, broken_nodes - ) - - def _atom_groups( - self, - curves: dict, - atoms: dict, - working_nodes, - broken_nodes, - crew=None, - causes: bool = False, - ) -> _Jumps: - """The system at the times its nodes fail or are taken down at - exact times (their ``atoms``): a unit dead on arrival at 0, an exact - lifetime, a replacement at its age or at a block time. - - Nodes due at one time go down together, so the system fails there - if it is up just before and down just after: since they only go - down, that is the fall in its availability, worked out as the rise - in its unavailability (a sum of products, which keeps a small one's - precision). A node's value just before is its point availability - there with its own events' fall (``drop``) added back; just after, - less the probabilities of its failures, then of its planned outages - (the failures are taken to come first). The others are at their - point availability there: tests and restorations due then are - done. One node alone fails the system with the probability of its - failure times its Birnbaum importance there. With limited repair - crews (``crew``, see ``_chain_transient.CrewSystem``), the nodes are - the nested RBDs, and the system is worked out over the crews' chain - at those times. With ``causes``, the failures are split among the - nodes failing together as ``_rates.split_jumps`` splits the fall - in availability from just before to just after their failures.""" - down = [ - a.times[(a.failure > 0.0) | (a.planned > 0.0)] - for a in atoms.values() - ] - times = np.unique(np.concatenate(down)) if down else np.empty(0) - if not times.size: - empty = np.empty(0) - return _Jumps( - empty, - empty, - empty, - empty, - {node: empty for node in curves} if causes else None, - ) - at, before, failed, planned = {}, {}, {}, {} - for node, curve in curves.items(): - value = np.asarray(curve.at(times.copy()), dtype=float) - a = atoms[node] - drop = _matched(times, a.times, a.drop) - failing = _matched(times, a.times, a.failure) - stopping = _matched(times, a.times, a.planned) - at[node] = value - before[node] = np.clip(value + drop, 0.0, 1.0) - failed[node] = np.clip(before[node] - failing, 0.0, 1.0) - planned[node] = np.clip(failed[node] - stopping, 0.0, 1.0) - - def unreliability(values: dict) -> np.ndarray: - if crew is not None: - return crew.probabilities(values, times)[1] - return self._system_unreliability( - self._filled(values, len(times), working_nodes, broken_nodes) - ) - - now, first = unreliability(at), unreliability(before) - after, last = unreliability(failed), unreliability(planned) - caused = None - if causes: - - def importances(values: dict) -> dict: - size = len(next(iter(values.values()))) - if crew is not None: - # Each jump's time, for each point of its path. - points = size // len(times) - return crew.importance(values, np.repeat(times, points)) - return self._importances( - self._filled(values, size, working_nodes, broken_nodes) - )[0] - - falls = _rates.split_jumps(importances, before, failed) - caused = {node: -fall for node, fall in falls.items()} - return _Jumps( - times, - np.maximum(after - first, 0.0), - np.maximum(last - after, 0.0), - np.maximum(now - first, 0.0), - caused, - ) - - def _system_atoms(self, curves: dict, stop: float, crew=None) -> Atoms: - """This RBD's own failures and planned outages at exact times before - ``stop``, from its nodes' ``curves`` (and its crews' chain, - ``crew``), as a node of another (see ``Atoms``).""" - atoms = {node: curve.atoms(stop) for node, curve in curves.items()} - jumps = self._atom_groups(curves, atoms, set(), set(), crew) - return Atoms( - jumps.times, - jumps.failures, - jumps.planned, - np.zeros(len(jumps.times)), - jumps.drop, - ) - - def _window_counts( - self, - curves: dict, - ends, - working_nodes, - broken_nodes, - method: str, - nodes: bool = False, - groups: Optional[dict] = None, - crew=None, - causes: bool = False, - ) -> dict: - """The system's expected up time, failures and planned outages in - ``[0, end)`` for each of ``ends``, from its nodes' ``curves`` (which - count their events); with ``nodes``, each node's expected down time - too; with ``causes``, the system failures each node causes (its - terms of the failures' sum, below, under ``"caused"``; with the - crews' chain or common-cause groups, each of their components' or - causes' too, from their rates, #199); and for each of ``groups`` (a - name and members), the - expected number of its members' failures and replacements that fall - at one instant with another's (each stop of a maintenance group is - one). - With limited repair crews (``crew``, a - ``_chain_transient.CrewSystem``, #162), the ``curves`` are the nested - RBDs', independent of the crews' chain: the system is worked out - over the chain at each time, and the chain's own components' - failures, at their rate in each state, add to its failures. - - The up time integrates the system's point availability as - ``mission_availability`` does. A node's failure takes the system - down if the node is critical then, which, the nodes being - independent, it is at ``t`` with probability ``I_B(t)``, its - Birnbaum importance at the nodes' availabilities then: the system's - failures are ``integral of sum_i I_B^i(t) dM_i(t)``, ``M_i(t)`` the - node's expected failures by ``t``, and its planned outages likewise. - It is summed over the pieces of ``_quadrature``, on each with the - importance taken as the cubic through its values at the points the - up time is integrated at (see ``_quadrature.stieltjes``); the events - at exact times are kept apart and taken together (see - ``_atom_groups``), and start pieces. The ends do not: a total at - an end within a piece adds the integral from the piece's start, - by the same quadrature, so that counting at many times (a nested - RBD's, for the RBD it is in) costs no more pieces. Past the time the - nodes have all settled, every count is extended exactly: at its - long-run rate, or a period at a time. - """ - ends = np.asarray(ends, dtype=float).ravel() - horizon = float(ends.max()) if ends.size else 0.0 - followed = list(curves.values()) + ([crew.curve] if crew else []) - settle, period = _settling(followed) - reach = min(horizon, settle if period is None else settle + period) - beyond = ends > reach - atoms = {node: curve.atoms(reach) for node, curve in curves.items()} - jumps = self._atom_groups( - curves, atoms, working_nodes, broken_nodes, crew, causes - ) - # The events at exact times start pieces; the ends need not (see - # ``totals_at``), so that a nested RBD counted at many times costs - # no more pieces. - fixed = [jumps.times] + [own.times for own in atoms.values()] - cycles = rest = np.empty(0) - if period is not None and beyond.any(): - cycles = np.floor((ends[beyond] - settle) / period) - rest = np.clip( - ends[beyond] - settle - cycles * period, 0.0, period - ) - fixed += [np.array([settle]), settle + rest] - - # Each node's events at exact times, kept apart (taken together, - # below): their running totals, by time. - exact = {} - for node, own in atoms.items(): - order = np.argsort(own.times, kind="stable") - exact[node] = ( - own.times[order], - _quadrature.running(own.failure[order], len(order)), - _quadrature.running(own.planned[order], len(order)), - ) - - def spread(node, x: np.ndarray): - """The node's expected failures and planned outages before - each time ``x``, less those at exact times.""" - events = curves[node].events(x) - times, failures, planned = exact[node] - before = np.searchsorted(times, x, side="left") - return ( - events["failures"] - failures[before], - events["planned"] - planned[before], - ) - - def estimate(a: np.ndarray, b: np.ndarray) -> dict: - x, half = _quadrature.points(a, b) - at_points = {node: curve.at(x) for node, curve in curves.items()} - # The system's availability and every node's importance at each - # point, in one pass. - own: dict = {} - if crew is not None: - importance, up, _, rate, own = crew.evaluate(at_points, x) - else: - importance, works, fails, _, _ = self._importances( - self._filled( - at_points, len(x), working_nodes, broken_nodes - ) - ) - up = works if structure_method(method) == "p" else 1.0 - fails - out: Dict[Hashable, np.ndarray] = { - "uptime": _quadrature.summed(up, half) - } - if nodes: - for node, values in [*at_points.items(), *own.items()]: - out[("downtime", node)] = _quadrature.summed( - 1.0 - values, half - ) - n = len(a) - failures, planned = np.zeros(n), np.zeros(n) - if crew is not None: - # The chain's components' failures, at their rate in each - # state, where they take the system down. - failures += _quadrature.summed(rate, half) - if causes: - for key, value in crew.caused(at_points, x).items(): - out[("caused", key)] = _quadrature.summed(value, half) - for node in curves: - counted = spread(node, np.concatenate([a, b, x])) - for total, values, key in zip( - (failures, planned), counted, ("caused", None) - ): - term = _quadrature.stieltjes( - importance[node], - values[:n], - values[n : 2 * n], # noqa: E203 - values[2 * n :], # noqa: E203 - ) - total += term - if causes and key is not None: - out[(key, node)] = term - out["failures"], out["planned"] = failures, planned - return out - - try: - edges, finest = _quadrature.pieces( - curves.values(), np.concatenate(fixed), reach, _MISSION_POINTS - ) - edges, integrals = _quadrature.refined( - estimate, - edges, - finest, - _MISSION_POINTS, - relative={"failures", "planned"}, - ) - except _quadrature.TooMany as error: - raise NotImplementedError( - f"Counting the expected events over [0, {reach}] takes " - f"{error.count} pieces (the components' curves bend that " - "often), more than the limit: estimate them by simulation, " - "with availability() or cost()." - ) from None - pieces = len(edges) - 1 - # The events at exact times, at the start of the piece they fall - # in (each is at an edge). - starting: Dict[Any, np.ndarray] = { - "failures": np.zeros(pieces), - "planned": np.zeros(pieces), - } - if causes: - for node in curves: - starting[("caused", node)] = np.zeros(pieces) - if jumps.times.size: - piece = np.searchsorted(edges, jumps.times, side="right") - 1 - np.add.at(starting["failures"], piece, jumps.failures) - np.add.at(starting["planned"], piece, jumps.planned) - if causes: - assert jumps.caused is not None - for node, values in jumps.caused.items(): - np.add.at(starting[("caused", node)], piece, values) - for name, members in (groups or {}).items(): - starting[("overlaps", name)] = self._overlaps( - atoms, members, edges - ) - keys: List[Any] = ["uptime", "failures", "planned"] - if causes: - keys += [("caused", node) for node in curves] - if crew is not None: - keys += [("caused", key) for key in crew.cause_keys] - if nodes: - keys += [("downtime", node) for node in curves] - if crew is not None: - keys += [("downtime", node) for node in crew.served] - keys += [("overlaps", name) for name in groups or {}] - series: Dict[Any, np.ndarray] = { - key: _quadrature.running( - integrals.get(key, np.zeros(pieces)) - + starting.get(key, np.zeros(pieces)), - pieces, - ) - for key in keys - } - - def totals_at(x: np.ndarray) -> Dict[Any, np.ndarray]: - """Each series before each time ``x`` (within ``[0, reach]``): - at an edge, its running total; within a piece, the running - total at its start, the events there, and its integral from - there, by the quadrature of the piece's start to ``x``.""" - at = np.minimum(np.searchsorted(edges, x), len(edges) - 1) - on_edge = edges[at] == x - out = {key: series[key][at] for key in keys} - if pieces and not on_edge.all(): - inside = ~on_edge - k = np.searchsorted(edges, x[inside], side="right") - 1 - k = np.clip(k, 0, pieces - 1) - partial = _quadrature.in_blocks(estimate, edges[k], x[inside]) - for key in keys: - out[key][inside] = ( - series[key][k] - + starting.get(key, np.zeros(pieces))[k] - + partial.get(key, 0.0) - ) - return out - - # Past ``reach``: constant rates, or a period at a time. - rates: dict = {} - if period is None and beyond.any(): - rates = self._settled_rates( - curves, - reach, - horizon, - working_nodes, - broken_nodes, - method, - crew, - ) - out: dict = {"downtime": {}, "overlaps": {}, "caused": {}} - inside = totals_at(ends[~beyond]) - for key, values in series.items(): - totals = np.empty(len(ends)) - totals[~beyond] = inside[key] - if beyond.any(): - if period is None: - rate = rates.get(key, 0.0) - totals[beyond] = values[-1] + (ends[beyond] - reach) * rate - else: - base = values[np.searchsorted(edges, settle)] - partial = values[np.searchsorted(edges, settle + rest)] - totals[beyond] = ( - base + cycles * (values[-1] - base) + (partial - base) - ) - if isinstance(key, tuple): - out[key[0]][key[1]] = totals - else: - out[key] = totals - return out - - def _settled_rates( - self, - curves: dict, - reach, - horizon, - working_nodes, - broken_nodes, - method, - crew=None, - ) -> dict: - """What ``_window_counts`` counts, per unit time, once every node's - curve is constant: the system's availability, its failures and - planned outages (each node's rates, from its counts' slope then, - times its Birnbaum importance), and each node's unavailability; - with the crews' chain (``crew``), its own components' failures - too.""" - x = np.array([horizon]) - values = {node: curve.at(x) for node, curve in curves.items()} - failures = planned = 0.0 - if crew is not None: - importance, up, _, rate, own = crew.evaluate(values, x) - rates: dict = {"uptime": float(up[0])} - failures += float(rate[0]) - for node, value in own.items(): - rates[("downtime", node)] = 1.0 - float(value[0]) - for key, value in crew.caused(values, x).items(): - rates[("caused", key)] = float(np.ravel(value)[0]) - else: - filled = self._filled(values, 1, working_nodes, broken_nodes) - rates = { - "uptime": float( - np.ravel(self.system_probability(filled, method=method))[0] - ) - } - importance = self._importances(filled)[0] - span = np.array([reach, reach + max(reach, 1.0)]) - for node, curve in curves.items(): - events = curve.events(span) - width = span[1] - span[0] - weight = float(importance[node][0]) - caused = weight * np.diff(events["failures"])[0] / width - failures += caused - planned += weight * np.diff(events["planned"])[0] / width - rates[("downtime", node)] = 1.0 - float(values[node][0]) - rates[("caused", node)] = caused - rates["failures"], rates["planned"] = failures, planned - return rates - - @staticmethod - def _overlaps(atoms: dict, members, edges: np.ndarray) -> np.ndarray: - """In each piece between ``edges``, the expected number of the - ``members``' failures and replacements at exact times that fall at - one instant with another member's: at each such instant the - members' expected number, less the probability of at least one - (they are independent). A maintenance group charges one set-up per - instant.""" - pieces = len(edges) - 1 - own = [atoms[m] for m in members if m in atoms] - acting = [a.times[(a.failure + a.preventive) > 0.0] for a in own] - if len(own) < 2 or not any(t.size for t in acting): - return np.zeros(pieces) - times = np.unique(np.concatenate(acting)) - expected = np.zeros(len(times)) - none = np.ones(len(times)) - for a in own: - chance = np.clip( - _matched(times, a.times, a.failure + a.preventive), 0.0, 1.0 - ) - expected += chance - none *= 1.0 - chance - overlap = np.maximum(expected - (1.0 - none), 0.0) - piece = np.searchsorted(edges, times, side="right") - 1 - keep = (piece >= 0) & (piece < pieces) - return np.bincount(piece[keep], overlap[keep], pieces) - - def _availability_curves( - self, - horizon: float, - skip, - counts: bool = False, - stages: bool = False, - state: Optional[dict] = None, - groups: bool = False, - ) -> dict: - """Each node's point availability over ``[0, horizon]``, from new, - as a curve (see ``_point_availability``), except the nodes in - ``skip`` (held working or failed). With ``counts``, the curves - count the nodes' expected events too, and follow them until they - have settled as well; with ``stages``, a degrading component's - curve follows its stages (see ``_capacity_models``). ``state`` - (checked by ``_states``) starts the nodes it names from their - states rather than new. With limited repair crews the nodes are not - independent: their values over time come from the crews' chain - instead (see ``_crew_curve``). Nor are the members of common-cause - groups, though each member's own curve is as without its group: - ``groups`` says the caller takes them in (see ``_groups_curve``).""" - assert not self._crews_couple(), "the crews' chain, not curves" - assert groups or not self.ccf_groups, "the groups' curve, not curves" - state = state or {} - curves: dict = {} - degrading = self._capacity_models() if stages else {} - memo = self._curve_memo - # The plain units whose curves are built, for their twins. - plain: list = [] - for node, component in self.components.items(): - if node in skip: - continue - start = state.get(node) - key: Optional[tuple] = (node, float(horizon), stages, start) - try: - hash(key) - except TypeError: # a nested RBD's own states, as a dict - key = None - if memo is not None and key is not None: - # A curve that counts its events serves one that need not. - found = memo.get((key, True)) - if found is None and not counts: - found = memo.get((key, False)) - if found is not None: - curves[node] = found - continue - twin = self._curve_twin(node, start, node in degrading, plain) - if twin is not None: - curves[node] = curves[twin] - else: - curves[node] = self._node_curve( - node, - component, - horizon, - counts, - node in degrading, - stages, - start, - ) - if self._plain_unit(node): - plain.append((node, start, node in degrading)) - if memo is not None and key is not None: - memo[(key, counts)] = curves[node] - return curves - - def _plain_unit(self, node) -> bool: - """Whether a component's curve follows from its life and repair - models alone (and its state at 0): a unit with no schedule, tests, - imperfect repair or maintenance group, not a nested RBD or a - standby group. Another with the same models has the same curve.""" - return not ( - isinstance(self.components[node], RepairableRBD) - or node in self._standby - or node in self._inspection - or node in self._preventive - or node in self._imperfect - or node in self._member_group - ) - - def _curve_twin(self, node, start, degrading: bool, plain: list): - """A component among ``plain`` (each ``(node, start, degrading)``, - a plain unit whose curve is built) whose curve is ``node``'s: the - same life and repair models, the same state at 0 and stages followed - alike; or None. Identical components (a bank of pumps) then share - one curve.""" - if not plain or not self._plain_unit(node): - return None - from repyability.rbd.non_repairable_rbd import NonRepairableRBD - - same = NonRepairableRBD._same_model - component = self.components[node] - for other, other_start, other_degrading in plain: - known = self.components[other] - if ( - other_degrading == degrading - and other_start == start - and same(known.reliability, component.reliability) - and same(known.time_to_replace, component.time_to_replace) - ): - return other - return None - - @contextmanager - def _sharing_curves(self): - """Within it, the curves ``_availability_curves`` builds are kept - and shared, so that an analysis asking for several exact values - builds each node's curve once (#185): keyed by the node, the - horizon, the stages followed and its state, a curve that counts its - events serving one that need not. Out of it, every call builds its - own, so that a change to the diagram or its models is taken.""" - if self._curve_memo is not None: - yield - return - self._curve_memo = {} - try: - yield - finally: - self._curve_memo = None - - def _node_curve( - self, - node, - component, - horizon: float, - counts: bool, - degrading: bool, - stages: bool, - start, - ): - """A node's curve for ``_availability_curves``: a nested RBD's, a - standby group's, a tested component's or any other unit's.""" - if isinstance(component, RepairableRBD): - return component._nested_curve(horizon, counts, stages, start) - if node in self._standby: - return self._standby_curve(node, start) - if node not in self._inspection: - return self._unit_curve(node, horizon, counts, degrading, start) - unit = self._tested_unit(node) - if unit is not None: - return self._tested_unit_curve(node, unit, horizon, start) - life = self._tested_life(node) - if life is not None: - return self._tested_life_curve(node, life, start) - rate, interval = self._inspected_rate(node) - inspection = self._inspection[node] - if inspection.partial: - # From new (its state is not taken: see _check_state). - return PartialTestCurve( - rate, - interval, - inspection.offset, - inspection.coverage, - inspection.per_full_test, - ) - if start is None and inspection.offset: - # New at 0, first tested at the offset: on a calendar whose last - # test was interval - offset before 0, last known up at 0. - return InspectionCurve( - rate, interval, interval - inspection.offset, 0.0 - ) - if start is not None and not start.alive: - raise NotImplementedError( - f"Component {node!r} has hidden failures, repaired at once " - "when a test finds them, as the exact values need: it cannot " - "be down. Give the time since its last test as its phase (it " - "was found working), or simulate it: availability(state=...)." - ) - phase = 0.0 if start is None else start.phase or 0.0 - # Last known up at its last test, or, put into service since, when - # it was. - since = ( - phase - if start is None or start.stationary - else min(start.age, phase) - ) - return InspectionCurve(rate, interval, phase, since) - - def _tested_unit_curve( - self, node, unit: TestedUnit, horizon: float, start - ): - """A tested component's point availability and events from 0 when - its tests or repairs take time, or its tests can miss a failure of - a life that is not exponential (see ``_hidden_tests``): new at 0, - first tested at its offset (or after an interval); in its long-run - state; up at the age its state gives, known up at its last test (or - when put into service since); or in a repair, under way for its - ``down_for``. From a state its tests are on the calendar its phase - places (``k T - phase``), as the simulation's.""" - inspection = self._inspection[node] - if start is None or start.new: - if inspection.offset: - return unit.curve(horizon, inspection.offset, 0) - return unit.curve(horizon, 0.0, 1) - phase = start.phase or 0.0 - if start.stationary: - return unit.curve(horizon, -phase, 1, ("stationary",)) - if not start.alive: - return unit.curve(horizon, -phase, 1, ("down", start.down_for)) - since = min(start.age, phase) - return unit.curve(horizon, -phase, 1, ("alive", start.age, since)) - - def _tested_life_curve(self, node, life: TestedLife, start): - """A tested component's point availability and events from 0, for - a life other than exponential (see ``_hidden_life``): new at 0 - (first tested at its offset, or after an interval), in its long-run - state, or known up at its last test (or since put into service) at - the age its state gives.""" - inspection = self._inspection[node] - interval = inspection.interval - if start is None: - return TestedLifeCurve(life, inspection.offset or interval) - # Up: its repairs take no time (see _check_state). - phase = start.phase or 0.0 - if start.stationary: - return TestedLifeSteady(life, phase) - model = life.model - # Last known up at its last test (or when put into service since), - # at age ``known_age``; at 0 it is ``start.age`` old. - since = min(start.age, phase) - known_age = np.array([start.age - since]) - known_up = float(_sf_values(model.sf, known_age)[0]) - known_down = float(np.atleast_1d(model.ff(known_age))[0]) - - def initial(x): - later = start.age + np.asarray(x, dtype=float) - return _sf_values(model.sf, later) / known_up - - def initial_down(x): - later = start.age + np.asarray(x, dtype=float) - if known_down <= 0.5: - failed = np.asarray(model.ff(later), dtype=float) - known_down - else: - failed = known_up - _sf_values(model.sf, later) - return np.clip(failed / known_up, 0.0, 1.0) - - first = interval - phase if phase else interval - return TestedLifeCurve(life, first, initial, initial_down) - - def _states(self, state, forced=frozenset()) -> dict: - """``state`` checked (see ``point_availability``): each component it - names, its ``NodeState``, and each nested RBD, its components' - states, in the same form; for ``"stationary"``, every component in - its long-run state. Nodes in ``forced`` (held working or broken) - take none.""" - if state is None: - return {} - if isinstance(state, str): - if state != "stationary": - raise ValueError( - "state must be a dict of NodeStates by node, or " - f"'stationary', got {state!r}." - ) - return { - node: ( - component._states("stationary") - if isinstance(component, RepairableRBD) - else NodeState(stationary=True) - ) - for node, component in self.components.items() - if node not in forced - } - if not isinstance(state, Mapping): - raise TypeError( - "state must be a dict of NodeStates by node, or " - f"'stationary', got {state!r}." - ) - out: dict = {} - for node, value in state.items(): - if node not in self.components: - raise self._not_a_component(node, "state") - if node in forced: - raise ValueError( - f"Node {node!r} is held working or broken: give it no " - "state." - ) - component = self.components[node] - if isinstance(component, RepairableRBD): - if isinstance(value, NodeState): - if not value.stationary: - raise ValueError( - f"Node {node!r} is a nested RBD: give its " - "components' states, as a dict of NodeStates " - "(or 'stationary')." - ) - value = "stationary" - out[node] = component._states(value) - continue - if value == "stationary": - value = NodeState(stationary=True) - if not isinstance(value, NodeState): - raise TypeError( - f"The state of component {node!r} must be a NodeState, " - f"got {value!r}." - ) - self._check_state(node, value) - out[node] = value - return out - - def _check_state(self, node, state: NodeState) -> None: - """Raise if a component cannot be in ``state``: a phase off a - calendar or past its interval, maintenance it does not have, an age - it cannot be up at or a repair or maintenance that is always over - sooner, or a state of a kind of component whose state is not - taken.""" - schedule = self._preventive.get(node) - inspection = self._inspection.get(node) - if ( - inspection is not None - and inspection.partial - and (state.phase is not None or not state.new) - ): - raise NotImplementedError( - f"The tests of component {node!r} can miss a failure (a " - "coverage " - "below 1): its state (how long since its last full test, " - "and whether a failure its tests missed is waiting) is not " - "taken: leave it out (new)." - ) - interval = None - if inspection is not None: - interval = inspection.interval - elif schedule is not None and schedule.policy != "age": - interval = schedule.interval - if state.phase is not None: - if interval is None: - raise ValueError( - f"Component {node!r} is not on a calendar (block " - "replacement, replacement on condition, or tests): give " - "it no phase." - ) - if state.phase >= interval: - raise ValueError( - f"Component {node!r}: the phase, {state.phase:g}, is " - "the time since its last scheduled replacement or test, " - f"less than its interval, {interval:g}." - ) - if state.maintenance and ( - schedule is None or schedule.duration is None - ): - raise ValueError( - f"Component {node!r} has no preventive maintenance that " - "takes time: it cannot be down for one." - ) - if state.new or (state.stationary and node in self._standby): - # A standby group in its long-run state starts its units' - # chain from its long-run distribution. - return - for kinds, what in ( - (self._standby, "a standby group, whose units' states"), - (self._imperfect, "repaired imperfectly: its virtual age"), - ): - if node in kinds: - raise NotImplementedError( - f"Component {node!r} is {what} are not taken as a " - "state: leave it out (new)." - ) - if state.stationary or (state.alive and not state.age): - return - component = self.components[node] - for what, model in [ - ("reliability", component.reliability), - ("repairability", component.time_to_replace), - ]: - if is_fixed_probability(model): - raise NotImplementedError( - f"Component {node!r}: its {what} model is a probability, " - "not a distribution of times, so it cannot start part " - "way through a life or repair: leave it out (new)." - ) - if state.alive: - # A unit with hidden failures was known to be up only at its - # last test, or when put into service since. - since = 0.0 - if inspection is not None: - since = min(state.age, state.phase or 0.0) - age = state.age - since - survive = _sf_values(component.reliability.sf, np.array([age])) - if not survive[0] > 0.0: - raise ValueError( - f"Component {node!r} cannot be up at age {age:g}: its " - "reliability there is 0." - ) - return - model = ( - schedule.duration # type: ignore[union-attr] - if state.maintenance - else component.time_to_replace + >>> round(rbd.expected_failures(10.0), 4) + 0.9174 + """ + return _windows.expected_failures( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, ) - left = _sf_values(model.sf, np.array([state.down_for])) - if not left[0] > 0.0: - raise ValueError( - f"Component {node!r} cannot have been down for " - f"{state.down_for:g}: its " - f"{'maintenance' if state.maintenance else 'repair'} is " - "always over by then." - ) - - def _simulation_states(self, state, forced=frozenset()) -> dict: - """``state`` checked for a simulation (see ``availability``): as - ``_states`` checks it, none stationary (a simulation starts from - given states), and see ``_check_simulated``.""" - if state is None: - return {} - states = self._states(state, forced) - self._check_simulated(states) - return states - - def _check_simulated(self, states: dict) -> None: - """Raise if a simulation cannot start from ``states`` (checked by - ``_states``): a component in its long-run state, which only the - exact methods take; one started from a state whose draws are not - streamed, being its own subclass or having a model other than a - surpyval parametric one; or more components down in a repair or - maintenance going on than repair crews to work on them.""" - in_hand = 0 - served = set(self._crew_served()) if self._crews_limited() else set() - for node, start in states.items(): - component = self.components[node] - if isinstance(component, RepairableRBD): - component._check_simulated(start) - continue - if start.stationary: - raise NotImplementedError(_STATIONARY_SIMULATION) - if start.new: - continue - if ( - type(component) is not NonRepairable - or stream_sampler(component.reliability) is None - or stream_sampler(component.time_to_replace) is None - ): - raise NotImplementedError( - f"Component {node!r} draws its events its own way (it is " - f"a {type(component).__name__}, or its models are not " - "surpyval parametric ones), so it cannot start part way " - "through a life or repair: leave it out (new)." - ) - if not start.alive and node in served: - in_hand += 1 - if served and in_hand > self.repair_crews: # type: ignore[operator] - raise ValueError( - f"{in_hand} components are down at the start, in repairs " - "or maintenance going on, which takes more than the " - f"{self.repair_crews} repair crew(s): a state does not say " - "which jobs wait." - ) - def _curves_at( - self, curves: dict, x: np.ndarray, working_nodes, broken_nodes, method - ) -> np.ndarray: - """The system's point availability at the times ``x``, from its - nodes' curves (the forced nodes held at 1 or 0).""" - probabilities = {node: curve.at(x) for node, curve in curves.items()} - for node in list(self.in_or_out) + list(working_nodes | broken_nodes): - probabilities[node] = np.ones(len(x)) - probabilities = self._probabilities_with_overrides( - probabilities, working_nodes, broken_nodes - ) - return np.asarray( - self.system_probability(probabilities, method=method), dtype=float - ) + def expected_events( + self, + t, + working_nodes: Optional[Collection[Hashable]] = None, + broken_nodes: Optional[Collection[Hashable]] = None, + method: str = "p", + state=None, + ) -> ExpectedEvents: + """What the system and each component are expected to do in + ``[0, t)``, with every component new at 0: exact, with no + simulation. - def _curves_down_at( - self, curves: dict, x: np.ndarray, working_nodes, broken_nodes, states - ) -> np.ndarray: - """The system's point unavailability at the times ``x``, from its - nodes' curves (the forced nodes held at 1 or 0), and the system's - failing worked out in its own right, so that a small value keeps - its precision (#237): each node down with one less its - availability, or in closed form where it has one (see - ``_closed_form_down``).""" - failures = {} - for node, curve in curves.items(): - down = self._closed_form_down(node, x, states) - failures[node] = 1.0 - curve.at(x) if down is None else down - probabilities = {node: 1.0 - q for node, q in failures.items()} - for node in list(self.in_or_out) + list(working_nodes | broken_nodes): - probabilities[node] = np.ones(len(x)) - failures[node] = np.zeros(len(x)) - probabilities = self._probabilities_with_overrides( - probabilities, working_nodes, broken_nodes - ) - failures = self._failures_with_overrides( - failures, working_nodes, broken_nodes - ) - return np.clip( - np.asarray( - self._system_unreliability(probabilities, failures), - dtype=float, - ), - 0.0, - 1.0, - ) + The system's failures are those of ``expected_failures``, and its + planned outages are counted the same way, from its components' + preventive maintenance that takes time; its down time is ``t * (1 + - mission_availability(t))``. Each component's failures, corrective + and preventive actions and tests follow from its renewal equation, + solved on the grid of its point availability (see + ``point_availability``), to about 1e-7, and its down time is the + integral of its unavailability. The maintenance due at exact times + (at a replacement age, at block times) is counted exactly, and + maintenance of several components due at the same time takes the + system down at most once. It is the mean of what ``availability`` + counts in each simulation of the window, and events at ``t`` itself + fall outside the window, as there. - def _closed_form_down(self, node, x, states) -> Optional[np.ndarray]: - """A component's probability of being down at the times ``x``, in - closed form where it has one: new at 0, with an exponential life - and an exponential or instant repair and nothing else, - ``lambda / (lambda + mu) (1 - exp(-(lambda + mu) t))``, which keeps - its precision however small, as the curve's grid does not within - its first step (#237). Else None: its curve gives it.""" - rates = self._closed_form_rates(states, (node,)).get(node) - if rates is None: - return None - life, total = self._closed_form_pair(node) - if math.isinf(total): - return np.zeros(len(x)) - return -(life / total) * np.expm1(-total * np.asarray(x, float)) - - def _closed_form_pair(self, node) -> Tuple[float, float]: - """A component's failure rate and the sum of its failure and - repair rates (``inf`` for an instant repair), or ``(nan, nan)`` - where they are not constant.""" - component = self.components.get(node) - if type(component) is not NonRepairable: - return math.nan, math.nan - life = _constant_rate(component.reliability) - repair = _repair_rate(component.time_to_replace) - if life is None or repair is None: - return math.nan, math.nan - return life, life + repair - - def _closed_form_rates(self, states, nodes=None) -> Dict[Hashable, float]: - """Of ``nodes`` (by default every component), those whose - probability of being down over time has a closed form (see - ``_closed_form_down``), with the rate it settles at: the sum of - their failure and repair rates (``inf`` for an instant repair).""" - out: Dict[Hashable, float] = {} - for node in self.components if nodes is None else nodes: - if ( - (states or {}).get(node) is not None - or node in self._preventive - or node in self._inspection - or node in self._imperfect - or node in self._standby - ): - continue - life, total = self._closed_form_pair(node) - if not math.isnan(total): - out[node] = total - return out + A component's corrective actions, at each of which its + ``repair_cost`` and ``replace_cost`` are charged, are its failures, + except for hidden failures: those found by a test in the window. A + nested RBD's failures are its system failures, and its own + maintenance is not this RBD's to count. A forced node does nothing: + held working it is never down, held broken it is down throughout. - @staticmethod - def _uniformized(what: str, generator, start, steady, vectors): - """A Markov chain followed over time from ``start`` (see - ``_chain_transient.Uniformized``), or a NotImplementedError saying - whose (``what``) cannot be.""" - try: - return _chain_transient.Uniformized( - generator, start, steady, vectors - ) - except NotImplementedError as error: - raise NotImplementedError( - f"{what} Markov chain cannot be followed over time here: " - f"{error}. Simulate it with availability()." - ) from None - - def _standby_curve(self, node, start: Optional[NodeState] = None): - """A standby group over time (see ``_chain_transient.ChainCurve``): - its units' Markov chain from every unit ready, or, for a - ``start`` that is stationary, from its long-run distribution.""" - life, repair = self._standby_rates(node) - arrangement = self._standby[node] - group = _standby_chain.chain( - arrangement.units, - arrangement.k, - life, - repair, - arrangement.dormancy_factor, - arrangement.switching_probability, - self.repair_crews if self._crews_limited() else None, - ) - if start is not None and start.stationary: - initial = group.probabilities - else: - initial = np.zeros(len(group.states)) - initial[0] = 1.0 - chain = self._uniformized( - f"Component {node!r} is a standby group, whose", - group.generator, - initial, - group.probabilities, - np.column_stack([group.up, group.failures, group.unit_failures]), - ) - return _chain_transient.ChainCurve(chain) + Parameters + ---------- + t : float or array-like + The windows' lengths. + working_nodes : Collection[Hashable], optional + Nodes that always work, by default None. + broken_nodes : Collection[Hashable], optional + Nodes that are always failed, by default None. + method : str, optional + Evaluate the structure function from the minimal path sets + (``"p"`` or ``"paths"``, the default) or the cut sets (``"c"`` + or ``"cuts"``); both give the + same result. + state : dict or str, optional + Start from the components' current states rather than new: + ``{node: NodeState}`` (see + [`NodeState`][repyability.NodeState]), a nested RBD's own such + dict for a nested RBD, or ``"stationary"`` for every component + in its long-run state. A component left out starts new. By + default None: every component new at 0. - def _nested_curve( - self, - horizon: float, - counts: bool = False, - stages: bool = False, - start=None, - ): - """This RBD as a node of another, over time: the structure function - at its own nodes' curves (a ``SystemCurve``), or, with limited - repair crews, its crews' chain (see ``_crew_curve``). ``counts``, - ``stages`` and ``start`` are as for ``_availability_curves``.""" - if self.ccf_groups and not self._crews_couple(): - return self._groups_curve( - horizon, states=start or {}, counts=counts, stages=stages - ) - if not self._crews_couple(): - inner = self._availability_curves( - horizon, set(), counts, stages, start - ) - return SystemCurve(self, inner, *_settling(inner.values())) - curve = self._crew_curve(horizon, states=start or {}, counts=counts) - if stages and self._has_capacity(): - curve.capacity = self._crew_capacity_over( - horizon, set(), set(), start or {} - ) - return curve + Returns + ------- + ExpectedEvents + The system's expected failures, planned outages and down time, + and each component's expected failures, corrective and + preventive actions, tests and down time: floats for one window, + arrays in ``t``'s shape for several. - def _crew_nested(self, forced) -> list: - """The nested RBDs the crews' chain is followed over time with: - those not held working or broken. Raise if there are too many.""" - nested = [ - node - for node, component in self.components.items() - if isinstance(component, RepairableRBD) and node not in forced - ] - if len(nested) > _MAX_CREW_NESTED: - raise NotImplementedError( - f"With limited repair crews, the availability over time is " - "worked out for each pattern of the nested RBDs up and down: " - f"{len(nested)} nested RBDs are more than the " - f"{_MAX_CREW_NESTED} it takes. Simulate it with " - "availability()." - ) - return nested - - def _crew_vectors( - self, chain, values: dict, working_nodes, broken_nodes, method: str - ) -> Tuple[np.ndarray, dict]: - """Over the states of the crews' ``chain``, with the other nodes' - probabilities ``values`` (arrays, one entry per state): whether the - system is up, and each node's Birnbaum importance.""" - size = len(chain.probabilities) - own = { - node: chain.up[:, k].astype(float) - for k, node in enumerate(chain.nodes) - } - own.update(values) - importance, works, fails, _, _ = self._importances( - self._filled(own, size, working_nodes, broken_nodes) - ) - up = works if structure_method(method) == "p" else 1.0 - fails - return np.asarray(up, dtype=float), importance - - def _crew_failing(self, chain, importance: dict) -> np.ndarray: - """The rate of the system's failures in each state of the crews' - ``chain``: a component that is up fails at its rate, and takes the - system down where it is critical (its Birnbaum ``importance`` - there, 1 or 0), as ``system_failure_frequency`` has it.""" - rates = self._crew_chain_rates() - failing = np.zeros(len(chain.probabilities)) - for k, node in enumerate(chain.nodes): - failing += importance[node] * chain.up[:, k] * rates[node][0] - return failing - - def _crew_failing_each(self, chain, importance: dict) -> np.ndarray: - """``_crew_failing`` by component: the rate of the system's failures - by each of the chain's components (columns) in each of its states - (#199).""" - rates = self._crew_chain_rates() - return np.column_stack( - [ - importance[node] * chain.up[:, k] * rates[node][0] - for k, node in enumerate(chain.nodes) - ] - ) + Raises + ------ + ValueError, NotImplementedError + As for ``point_availability``. - def _crew_each( - self, chain, nested: list, working, broken, method: str - ) -> list: - """For each pattern of the ``nested`` RBDs up and down, the rate of - the system's failures by each of the crews' chain's components in - each state (see ``_crew_failing_each``), for ``CrewSystem``'s - ``causes``.""" - size = len(chain.probabilities) - out = [] - for pattern in range(2 ** len(nested)): - _, importance = self._crew_vectors( - chain, - { - node: np.full(size, float((pattern >> j) & 1)) - for j, node in enumerate(nested) - }, - working, - broken, - method, - ) - out.append(self._crew_failing_each(chain, importance)) - return out + Examples + -------- + A pump that wears out, replaced at 600 hours in about 7 (a planned + outage), over a year: - def _crew_start(self, chain, states: dict) -> np.ndarray: - """The probabilities of the crews' chain's states at 0, from the - components' ``states`` (checked by ``_states``): every component up - (a component's age does not matter, its life being exponential), - each in its long-run state (all of them, ``"stationary"``), or as - they say: a component down is in a repair (as its repair is - exponential, however long it has taken), and none waits, so there - can be no more down than crews.""" - named = {node: states[node] for node in chain.nodes if node in states} - stationary = [node for node, s in named.items() if s.stationary] - if stationary: - if len(stationary) < len(chain.nodes): - raise NotImplementedError( - "With limited repair crews, the components' long-run " - "states are tied together by the repair queue: start " - "every component in its long-run state " - "(state='stationary'), or give each its state." - ) - return np.array(chain.probabilities, dtype=float) - down = tuple( - k - for k, node in enumerate(chain.nodes) - if node in named and not named[node].alive + >>> import surpyval as surv + >>> from repyability import RepairableRBD + >>> rbd = RepairableRBD( + ... [("s", "p"), ("p", "t")], + ... { + ... "p": { + ... "reliability": surv.Weibull.from_params([1000, 2.5]), + ... "repairability": surv.LogNormal.from_params([3, 0.5]), + ... "preventive": { + ... "interval": 600.0, + ... "duration": surv.Weibull.from_params([8, 3]), + ... }, + ... } + ... }, + ... ) + >>> year = rbd.expected_events(8760.0) + >>> round(year.system_failures, 3), round(year.node_preventive["p"], 3) + (3.706, 11.275) + >>> round(year.system_planned_outages, 3) # each replacement + 11.275 + """ + return _windows.expected_events( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, ) - assert self.repair_crews is not None # limited crews only - if len(down) > self.repair_crews: - raise ValueError( - f"{len(down)} components are down at the start, in repairs " - "going on, which takes more than the " - f"{self.repair_crews} repair crew(s): a state does not say " - "which jobs wait." - ) - start = np.zeros(len(chain.probabilities)) - start[chain.states.index((down, ()))] = 1.0 - return start - def _crew_curve( + def expected_cost( self, - horizon: float, - working_nodes=frozenset(), - broken_nodes=frozenset(), + t, + working_nodes: Optional[Collection[Hashable]] = None, + broken_nodes: Optional[Collection[Hashable]] = None, method: str = "p", - states: Optional[dict] = None, - counts: bool = False, - ) -> "_chain_transient.CrewCurve": - """The system over time with limited repair crews (see - ``_chain_transient.CrewCurve``): the crews' chain, without the - nodes held working or broken, from the components' ``states`` - (see ``_crew_start``), with the nested RBDs' own curves (each from - its own state in ``states``) to ``horizon``, counting their events - with ``counts``.""" - states = states or {} - working, broken = set(working_nodes), set(broken_nodes) - forced = working | broken - nested = self._require_crew_over_time(frozenset(forced)) - chain = self._crew_chain(frozenset(forced)) - vectors = self._crew_patterns(chain, nested, working, broken, method) - uniformized = self._uniformized( - "The repair crews'", - chain.generator, - self._crew_start(chain, states), - chain.probabilities, - np.column_stack(vectors), - ) - curves = { - node: self.components[node]._nested_curve( - horizon, counts=counts, start=states.get(node) - ) - for node in nested - } - return _chain_transient.CrewCurve(self, uniformized, curves) - - def _crew_patterns( - self, chain, nested: list, working, broken, method: str - ) -> list: - """The crews' chain's vectors for each pattern of the ``nested`` - RBDs up and down (see ``_chain_transient.pattern_weights``): whether - the system is up in each state, for every pattern, then the rate of - its failures by the chain's components, for every pattern (see - ``_chain_transient.CrewSystem``).""" - size = len(chain.probabilities) - ups, failing = [], [] - for pattern in range(2 ** len(nested)): - up, importance = self._crew_vectors( - chain, - { - node: np.full(size, float((pattern >> j) & 1)) - for j, node in enumerate(nested) - }, - working, - broken, - method, - ) - ups.append(up) - failing.append(self._crew_failing(chain, importance)) - return ups + failing - - def _no_crew_window( - self, node=None, what: str = "the expected events" - ) -> NoReturn: - """Raise that with limited repair crews, ``what`` over time (from - their chain) cannot take in a nested RBD (``node``), or (no - ``node``) be given to another RBD that has this one nested, as - yet.""" - where = ( - "which has no place, as yet, for those of a nested RBD (#162): " - f"node {node!r} is one" - if node is not None - else "which gives none to an RBD this one is nested in, as yet " - "(#162)" - ) - raise NotImplementedError( - f"With {self.repair_crews} repair crew(s) for " - f"{len(self._crew_served())} components, {what} over time come " - f"from the crews' Markov chain, {where}. Simulate them with " - "availability() or cost(); the availability over time and the " - "long-run values are exact." - ) + state=None, + *, + discount_rate: float = 0.0, + ) -> ExpectedCost: + """The expected cost of running the system for ``t``, from new: + exact, with no simulation. - def _groups_window( - self, - ends: np.ndarray, - working_nodes, - broken_nodes, - method: str, - states: dict, - nodes: bool, - groups: Optional[dict], - causes: bool = False, - ) -> Tuple[dict, dict]: - """``_window``'s curves and counts with common-cause groups (#158): - the system's up time, failures and planned outages over its groups' - joint states (see ``_ccf_chain.GroupsSystem``), and each node's own - curve, the members' as without their groups, for their own events - and down time.""" - working, broken = set(working_nodes), set(broken_nodes) - horizon = float(np.max(ends)) if len(ends) else 0.0 - system_curve = self._groups_curve( - horizon, working, broken, method, states, counts=True - ) - totals = self._window_counts( - system_curve.curves, - ends, - working, - broken, - method, - nodes=nodes, - groups=groups, - crew=system_curve.system, - causes=causes, - ) - curves = dict(system_curve.curves) - members = [m for group in self.ccf_groups for m in group.members] - own = self._availability_curves( - horizon, - set(self.components) - set(members), - counts=True, - groups=True, - ) - curves.update(own) - return curves, totals + Each category is its events' expected number over ``[0, t)`` (see + ``expected_events``) times its mean cost, as ``cost`` charges them: + ``repair_cost`` and ``replace_cost`` at each corrective action, the + preventive cost at each preventive replacement, the inspection cost + at each test, ``downtime_cost`` over each component's expected down + time, ``downtime_cost_rate`` over the system's, and a maintenance + group's set-up cost once per stop (each failure or replacement of a + member, those at one instant one stop). It is the mean that + ``cost`` estimates by simulation, with its distribution; + ``total_cost`` is the long-run approximation, exact only over a + window long compared with the components' cycles. As ``t`` grows + this approaches ``expected_cost_rate() * t`` plus a constant, of the + early years from new. - def _crew_window( - self, - ends: np.ndarray, - working_nodes, - broken_nodes, - method: str, - states: dict, - nodes: bool, - groups: Optional[dict], - causes: bool = False, - ) -> Tuple[dict, dict]: - """``_window``'s curves and counts with limited repair crews, from - their chain over time: the system's up time and failures (the - integral of the rate of its failures in each state, as - ``system_failure_frequency`` has it), and each component's down - time and failures (at its rate while it is up), each a repair. No - component has scheduled maintenance or tests, and none fails at the - same instant as another.""" - working, broken = set(working_nodes), set(broken_nodes) - forced = working | broken - nested = self._require_crew_over_time(frozenset(forced), "window") - chain = self._crew_chain(frozenset(forced)) - rates = self._crew_chain_rates() - if nested: - # The nested RBDs, independent of the chain, counted as nodes, - # with the system worked out over the chain (#162). - each = ( - self._crew_each(chain, nested, working, broken, method) - if causes - else [] - ) - uniformized = self._uniformized( - "The repair crews'", - chain.generator, - self._crew_start(chain, states), - chain.probabilities, - np.column_stack( - self._crew_patterns(chain, nested, working, broken, method) - + [chain.up.astype(float)] - + each - ), - ) - system = _chain_transient.CrewSystem( - uniformized, nested, chain.nodes, causes - ) - horizon = float(np.max(ends)) if len(ends) else 0.0 - curves = { - node: self.components[node]._nested_curve( - horizon, counts=True, start=states.get(node) - ) - for node in nested - } - totals = self._window_counts( - curves, - ends, - working, - broken, - method, - nodes=nodes, - groups=groups, - crew=system, - causes=causes, - ) - column = 2 * system.patterns - for k, node in enumerate(chain.nodes): - curves[node] = _chain_transient.CrewNodeEvents( - uniformized, column + k, rates[node][0] - ) - return curves, totals - up, importance = self._crew_vectors(chain, {}, working, broken, method) - failing = self._crew_failing(chain, importance) - each = [self._crew_failing_each(chain, importance)] if causes else [] - uniformized = self._uniformized( - "The repair crews'", - chain.generator, - self._crew_start(chain, states), - chain.probabilities, - np.column_stack([up, failing, chain.up.astype(float)] + each), - ) - totals = uniformized.integrals(ends) - zero = np.zeros(len(ends)) - counts: dict = { - "uptime": np.clip(totals[:, 0], 0.0, ends), - "failures": np.maximum(totals[:, 1], 0.0), - "planned": zero, - "downtime": {}, - "overlaps": {name: zero for name in groups or {}}, - } - if causes: - base = 2 + len(chain.nodes) - counts["caused"] = { - node: np.maximum(totals[:, base + k], 0.0) - for k, node in enumerate(chain.nodes) - } - curves = {} - for k, node in enumerate(chain.nodes): - curves[node] = _chain_transient.CrewNodeEvents( - uniformized, 2 + k, rates[node][0] - ) - if nodes: - counts["downtime"][node] = np.clip( - ends - totals[:, 2 + k], 0.0, ends - ) - return curves, counts - - def _crew_capacity( - self, working_nodes, broken_nodes, states: dict - ) -> Tuple[np.ndarray, "_chain_transient.Uniformized"]: - """The capacity over time with limited repair crews and no nested - RBDs: its levels, and the crews' chain followed over time with, as - its vectors, whether the system is at each level in each state (as - ``capacity_distribution`` averages them in the long run).""" - working, broken = set(working_nodes), set(broken_nodes) - forced = working | broken - self._require_crew_over_time(frozenset(forced), "capacity") - chain = self._crew_chain(frozenset(forced)) - size = len(chain.probabilities) - own = { - node: chain.up[:, k].astype(float) - for k, node in enumerate(chain.nodes) - } - arrays, _ = self._node_arrays(self._filled(own, size, working, broken)) - levels, rows = self._capacity_arrays(arrays, size, {}) - uniformized = self._uniformized( - "The repair crews'", - chain.generator, - self._crew_start(chain, states), - chain.probabilities, - rows.T, - ) - return levels, uniformized - - def _crew_capacity_over( - self, horizon: float, working_nodes, broken_nodes, states: dict - ) -> "_CrewCapacity": - """The capacity over time with limited repair crews around nested - RBDs (#162): see ``_CrewCapacity``.""" - working, broken = set(working_nodes), set(broken_nodes) - forced = working | broken - nested = self._require_crew_over_time(frozenset(forced)) - chain = self._crew_chain(frozenset(forced)) - curves = { - node: self.components[node]._nested_curve( - horizon, - stages=node in self._capacity_models(), - start=states.get(node), - ) - for node in nested - } - return _CrewCapacity( - self, - chain, - self._crew_start(chain, states), - curves, - working, - broken, - ) + Every cost is optional, and a cost given as a distribution enters + through its mean. Only this RBD's own costs count, not those inside + a nested ``RepairableRBD``. With nothing priced, not even an + acquisition cost (see ``has_costs``), there is no cost model to + evaluate, and this refuses; with an acquisition cost alone every + category is 0, exactly. - def _capacity_at( - self, curve, x: np.ndarray - ) -> Tuple[np.ndarray, np.ndarray]: - """This RBD's capacity distribution at the times ``x``, as a node of - another, from its curve over time (see ``_nested_curve``, with - ``stages``): from its nodes' curves, or with limited repair crews - from their chain (#162).""" - capacity = getattr(curve, "capacity", None) - if capacity is not None: - return capacity.rows(x) - if isinstance(curve, _ccf_chain.GroupsCurve): - return self._groups_capacity_rows( - curve.curves, curve.system, x, set(), set() - ) - return self._capacity_rows(curve.curves, x, set(), set()) + With a ``discount_rate`` ``r`` each value is its present value, + the costs discounted continuously from when they fall: + ``integral from 0 to t of exp(-r s) dC(s)``, ``C`` the expected + cost from new, worked out by parts, ``exp(-r t) C(t) + r * + integral from 0 to t of exp(-r s) C(s) ds``, the integral by + Gauss-Kronrod quadrature on pieces halved until the total is + within 1e-8 of itself. It is the exact form of ``total_cost``'s + discounted value, which spends the long-run rate from the start: + the early years, which count most, are those that differ from it. + The acquisition cost, at the start, is not discounted. - def _unit_curve( - self, - node, - horizon: float, - counts: bool = False, - stages: bool = False, - start: Optional[NodeState] = None, - unscheduled: bool = False, - ): - """A component's point availability from new over ``[0, horizon]`` - (see ``_point_availability.unit_curve``): on a grid of - ``_POINT_STEPS`` steps over its typical up time, with its - replacement age on the grid, and only as long as it takes to settle - at its long-run availability (to 1e-10), after which the curve holds - that value. With ``counts`` it counts the component's expected - events too, until they grow at their long-run rates (to 1e-7), and - at those rates after. With ``stages``, a degrading component's - curve follows its stages too, until they have settled at their - long-run shares of its up time (to 1e-8). Under block replacement, - see ``_block_curve``. ``start`` starts it from a state rather than - new (see ``point_availability``). ``unscheduled`` leaves out its - preventive maintenance, and follows it to the horizon: the head of a - block-replacement curve, up to its first block time.""" - self._require_time_models(node) - self._require_no_opportunities(node) - component = self.components[node] - if node in self._imperfect: - self._require_minimal_repair(node) - scale = _up_scale(component.reliability, None) - if not (np.isfinite(scale) and scale > 0.0): - scale = horizon if horizon > 0.0 else 1.0 - return MinimalRepairCurve( - component.reliability, scale, _POINT_STEPS, counts - ) - schedule = None if unscheduled else self._preventive.get(node) - if schedule is not None and schedule.policy in ("block", "condition"): - return self._block_curve(node, horizon, start) - age = None if schedule is None else float(schedule.interval) - duration = None if schedule is None else schedule.duration - life = component.reliability - up_sf = life.sf - up_splits = point_knots(life) - repair = component.time_to_replace - - def up_cdf(x): - return 1.0 - _sf_values(up_sf, x) - - def repair_sf(s): - return _sf_values(repair.sf, s) - - def duration_sf(s): - return _sf_values(duration.sf, s) - - maintenance_sf = None if duration is None else duration_sf - try: - if unscheduled: - raise ValueError("followed to the horizon") - long_run: Optional[float] = self._node_availability(node) - rates: Optional[Tuple[float, float, float]] = ( - self._node_frequencies(node) if counts else None - ) - except (ValueError, NotImplementedError): - long_run, rates = None, None - stage_model = None - if stages: - self._require_unscheduled_stages(node) - stage_model = life - if start is not None and start.stationary: - # Long in service: in its long-run state throughout. - if long_run is None: - self._node_availability(node) # raises why it has none - failures, maintained, _ = self._node_frequencies(node) - return SteadyCurve( - float(long_run), # type: ignore[arg-type] - failures, - maintained, - takedown=duration is not None, - fractions=( - None - if stage_model is None - else stage_model.stage_fractions() - ), - ) - first = None - if _draws_start(start): - first = self._first_unit( - node, - start, # type: ignore[arg-type] - up_sf, - up_splits, - age, - stage_model, - ) - scale = _up_scale(life, age) - if schedule is not None: - up, cycle, _, _ = self._maintenance_cycle(node, schedule) - else: - up = model_mean(life) - cycle = up + model_mean(repair) - horizon = max(horizon, 0.0) - step = horizon / 1024.0 if horizon > 0.0 else np.inf - if np.isfinite(scale): - step = min(step, scale / _POINT_STEPS) - if not np.isfinite(step): - step = 1.0 - if age is not None and step < age: - step = age / np.ceil(age / step) - # Units that each reach their age are maintained at nearly fixed - # times, which the curve follows until they have all but ended. - survive = 0.0 - if age is not None and duration is not None: - survive = float(_sf_values(up_sf, np.array([age]))[0]) - end = max(horizon, 16.0 * step) - settles = age is None or survive ** (horizon // age) < 1e-10 - if ( - long_run is not None - and settles - and np.isfinite(cycle) - and cycle > 0.0 - ): - end = min(end, max(8.0 * cycle, 16.0 * step)) - while True: - n = int(np.ceil(end / step)) - if n > _POINT_MAX: - step = end / _POINT_MAX - if age is not None and step < age: - step = age / np.floor(age / step) - if np.isfinite(scale) and step > scale / 250.0: - raise NotImplementedError( - f"Component {node!r} would need more than " - f"{_POINT_MAX} grid points to follow its " - f"availability to {end} (its typical up time is " - f"about {scale:.3g}): estimate it by simulation, " - "with availability()." - ) - n = min(int(np.ceil(end / step)), _POINT_MAX) - try: - with np.errstate(divide="ignore", invalid="ignore"): - curve = unit_curve( - up_cdf, - up_splits, - repair_sf, - point_knots(repair), - step, - n, - age=age, - maintenance_sf=maintenance_sf, - maintenance_splits=( - () if duration is None else point_knots(duration) - ), - counts=counts, - stages=( - None - if stage_model is None - else stage_model.stage_probabilities - ), - first=first, - ) - except NotImplementedError as error: - raise NotImplementedError( - f"Component {node!r}: {error}. Estimate its availability " - "by simulation, with availability()." - ) from None - events = curve.unit_events - if events is not None and rates is not None: - events.failure_rate, events.preventive_rate = rates[:2] - if ( - curve.stages is not None - and stage_model is not None - and long_run is not None - ): - curve.stages.fractions = stage_model.stage_fractions() - if end >= horizon: - break - tail = curve.times[len(curve.times) - len(curve.times) // 4 :] - if ( - long_run is not None - and np.all(np.abs(curve.at(tail) - long_run) < 1e-10) - and (age is None or survive ** (end // age) < 1e-10) - and (events is None or events.settled()) - and (curve.stages is None or curve.stages.settled()) - ): - break - end = min(horizon, _settling_end(curve, long_run, cycle, end)) - curve.long_run = long_run - if curve.unit_events is not None and long_run is None: - # Followed to the horizon: no rates past it. - curve.unit_events.failure_rate = None - curve.unit_events.preventive_rate = None - return curve - - def _block_curve( - self, node, horizon: float, start: Optional[NodeState] = None - ): - """A component's point availability from new under block - replacement, over ``[0, horizon]`` (see - ``_block_replacement.block_availability``), or replaced on condition - (#161, see ``_condition_replacement.condition_availability``); in - its long-run state (a stationary ``start``), its settled cycle from - its phase on; from another state, its own curve up to its first - block time or inspection, and the schedule's from there (see - ``StartedBlockCurve``).""" - component = self.components[node] - schedule = self._preventive[node] - duration = schedule.duration - stationary = start is not None and start.stationary - knots = np.empty(0) if duration is None else point_knots(duration) - life = component.reliability - models = (life, component.time_to_replace, duration, schedule.interval) - - def followed(horizon: float, head=None): - if schedule.policy == "condition": - return condition_availability( - *models, float(schedule.threshold), horizon, node, head - ) - return block_availability(*models, horizon, node, head) + Parameters + ---------- + t : float or array-like + The windows' lengths. + working_nodes : Collection[Hashable], optional + Nodes that always work: they incur no corrective, preventive, + inspection or downtime cost, by default None. + broken_nodes : Collection[Hashable], optional + Nodes that are always failed: they incur no corrective cost, but + their own downtime cost throughout, by default None. + method : str, optional + Evaluate the structure function from the minimal path sets + (``"p"`` or ``"paths"``, the default) or the cut sets (``"c"`` + or ``"cuts"``); both give the + same result. + state : dict or str, optional + Start from the components' current states rather than new: + ``{node: NodeState}`` (see + [`NodeState`][repyability.NodeState]), a nested RBD's own such + dict for a nested RBD, or ``"stationary"`` for every component + in its long-run state. A component left out starts new. By + default None: every component new at 0. + discount_rate : float, optional + The continuous discount rate per unit time of the component + models, as for ``total_cost``, by default 0 (undiscounted). - if start is not None and not start.new and not stationary: - length = schedule.interval - (start.phase or 0.0) - head = self._unit_curve( - node, length, counts=True, start=start, unscheduled=True - ) - down_sf = None - first = None - if not start.alive: - down_sf = self._first_unit( - node, start, None, (), None, None - ).down_sf - else: - # The unit in service now, if unfailed by the first - # inspection, and its age there. - ages = float(start.age) + np.array([0.0, length]) - alive = _sf_values(life.sf, ages) - first = (float(alive[1] / alive[0]), float(ages[1])) - - def failures(u): - return head.events(np.asarray(u, dtype=float))["failures"] - - up = float(head.at(np.array([length]))[0]) - reached = ( - ConditionHead(length, up, failures, down_sf, first) - if schedule.policy == "condition" - else BlockHead(length, up, failures, down_sf) - ) - result = followed(max(horizon - length, 0.0), reached) - return StartedBlockCurve(head, BlockCurve(result, knots), length) - result = followed(np.inf if stationary else max(horizon, 0.0)) - curve = BlockCurve(result, knots) - if not stationary: - return curve - assert start is not None - return ShiftedCurve(curve, curve.settle + (start.phase or 0.0)) - - def _first_unit( - self, node, start: NodeState, up_sf, up_splits, age, stage_model - ) -> First: - """A component's unit at 0 in state ``start`` (see - ``_point_availability.First``): up at its age, with what is left of - its life (the survival function ``R(a + s) / R(a)``) and its - replacement due ``age - a`` from now; or down, with what is left of - its repair or maintenance (``G(r + s) / G(r)``, after ``r`` so - far). ``_check_state`` has checked that it can be in the state.""" - if start.alive: - a = float(start.age) - survive = float(_sf_values(up_sf, np.array([a]))[0]) - - def cdf(s): - return np.clip( - 1.0 - _sf_values(up_sf, a + s) / survive, 0.0, 1.0 - ) + Returns + ------- + ExpectedCost + The expected cost of each window, by category and component, + with the acquisition cost beside it: present values with a + ``discount_rate``. - splits = np.asarray(up_splits, dtype=float) - a + Raises + ------ + ValueError + If nothing is priced, not even an acquisition cost (see + ``has_costs``; as ``routes.Refused``), or as for + ``point_availability``. + NotImplementedError + As for ``point_availability``. - def stages(s): - return stage_model.stage_probabilities(a + s) / survive + Examples + -------- + A pump bought for 20,000, failing on average every 1000 hours, + repaired in 10 at 500 per repair, with lost production at 100 per + hour, owned for a year: - return First( - cdf, - splits[splits > 0.0], - None if age is None else age - a, - stages=None if stage_model is None else stages, - ) - if start.maintenance: - model = self._preventive[node].duration - else: - model = self.components[node].time_to_replace - r = float(start.down_for) - left = float(_sf_values(model.sf, np.array([r]))[0]) + >>> import surpyval as surv + >>> from repyability import RepairableRBD + >>> rbd = RepairableRBD( + ... [("s", "p"), ("p", "t")], + ... { + ... "p": { + ... "reliability": surv.Exponential.from_params([1e-3]), + ... "repairability": surv.Exponential.from_params([0.1]), + ... "repair_cost": 500.0, + ... "acquisition_cost": 20000.0, + ... } + ... }, + ... downtime_cost_rate=100.0, + ... ) + >>> year = rbd.expected_cost(8760.0) + >>> round(year.mean), round(year.total) + (13000, 33000) + >>> round(rbd.total_cost(8760.0)) # at the long-run rate + 33010 + + Owned for ten years at 7% a year, from new and at the long-run rate: - def down_sf(s): - return _sf_values(model.sf, r + s) / left + >>> import math + >>> r = math.log(1.07) / 8760 # per hour + >>> round(rbd.expected_cost(87600.0, discount_rate=r).total) + 114528 + >>> round(rbd.total_cost(87600.0, discount_rate=r)) + 114538 + """ + return _windows.expected_cost( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + state=state, + discount_rate=discount_rate, + ) - splits = point_knots(model) - r - return First(down_sf=down_sf, down_splits=splits[splits > 0.0]) + @contextmanager + def _sharing_curves(self): + """Within it, the curves ``_availability_curves`` builds are kept + and shared, so that an analysis asking for several exact values + builds each node's curve once (#185): keyed by the node, the + horizon, the stages followed and its state, a curve that counts its + events serving one that need not. Out of it, every call builds its + own, so that a change to the diagram or its models is taken.""" + if self._curve_memo is not None: + yield + return + self._curve_memo = {} + try: + yield + finally: + self._curve_memo = None def capacity_distribution( self, @@ -14972,140 +3573,9 @@ def capacity_distribution( >>> round(capacity.mean, 2) # 150 * 10 / 11 136.36 """ - probabilities, weights = self._long_run_probabilities( - working_nodes, broken_nodes, "The capacity distribution" - ) - working_nodes = set(working_nodes or ()) - broken_nodes = set(broken_nodes or ()) - arrays, size = self._node_arrays(probabilities) - own = {} - for node in self._capacity_models(): - levels, shares = self._long_run_capacity(node) - own[node] = self._forced( - (levels, np.repeat(shares[:, None], size, axis=1)), - node, - working_nodes, - broken_nodes, - ) - levels, rows = self._capacity_arrays(arrays, size, own) - return CapacityDistribution(levels, rows @ weights) - - def _require_capacity_models(self) -> None: - """Raise what ``_capacity_refusal`` reports, in the same order: a - degrading component on a schedule, here or in a nested RBD, then no - capacity at all.""" - for node, model in self._capacity_models().items(): - if isinstance(model, RepairableRBD): - model._require_capacity_models() - else: - self._require_unscheduled_stages(node) - self._require_capacity() - - def _capacity_curves( - self, horizon: float, working_nodes, broken_nodes, state=None - ) -> dict: - """The nodes' curves for the capacity over time (see - ``point_capacity``): a degrading component's following its stages, - and a node that takes its capacity from its model kept even when - held working (its levels are then those it is up at).""" - held = set(working_nodes) - set(self._capacity_models()) - return self._availability_curves( - horizon, - held | set(broken_nodes), - stages=True, - state=self._states(state, set(working_nodes) | set(broken_nodes)), - ) - - def _capacity_rows( - self, curves: dict, x: np.ndarray, working_nodes, broken_nodes - ) -> Tuple[np.ndarray, np.ndarray]: - """The distribution of the system's capacity at each of the times - ``x``, from its nodes' curves (see ``_capacity_curves``): its - levels, and one row of probabilities per level, one column per - time. A node with a capacity carries it with its point availability - there; a degrading component is in each stage with the probability - its curve gives, and a nested RBD with capacities brings its own - distribution.""" - size = len(x) - values = {node: curve.at(x) for node, curve in curves.items()} - arrays, _ = self._node_arrays( - self._filled(values, size, working_nodes, broken_nodes) - ) - own = {} - for node, model in self._capacity_models().items(): - if node in broken_nodes: - own[node] = (np.zeros(1), np.ones((1, size))) - continue - curve = curves[node] - if isinstance(model, RepairableRBD): - distribution = model._capacity_at(curve, x) - else: - distribution = _capacity.merged( - np.array((0.0,) + model.capacities), - np.vstack([1.0 - values[node], curve.stages_at(x)]), - ) - own[node] = self._forced( - distribution, node, working_nodes, broken_nodes - ) - return self._capacity_arrays(arrays, size, own) - - def _groups_capacity( - self, horizon: float, working_nodes, broken_nodes, state - ) -> tuple: - """With common-cause groups (#158): the curves of the nodes outside - them for the capacity over time (see ``_capacity_curves``), and the - groups' system (see ``_ccf_chain.GroupsSystem``).""" - working, broken = set(working_nodes), set(broken_nodes) - states = self._states(state, working | broken) - self._require_free_members(working, broken) - self._require_groups_over_time(states) - members = {m for group in self.ccf_groups for m in group.members} - held = working - set(self._capacity_models()) - curves = self._availability_curves( - horizon, - held | broken | members, - stages=True, - state=states, - groups=True, + return _repairable_capacity.capacity_distribution( + self, working_nodes=working_nodes, broken_nodes=broken_nodes ) - return curves, self._groups_system(working, broken, "p") - - def _groups_capacity_rows( - self, curves: dict, system, x: np.ndarray, working_nodes, broken_nodes - ) -> Tuple[np.ndarray, np.ndarray]: - """``_capacity_rows`` with common-cause groups: worked out at the - points the groups' combinations split each time into (see - ``_ccf_chain.GroupsSystem``), and averaged back onto the times with - the combinations' probabilities.""" - size = len(x) - values = {node: curve.at(x) for node, curve in curves.items()} - probabilities, weights, index, _ = system._split(values, x) - points = len(index) - arrays, _ = self._node_arrays(probabilities) - own = {} - for node, model in self._capacity_models().items(): - if node in broken_nodes: - own[node] = (np.zeros(1), np.ones((1, points))) - continue - curve = curves[node] - if isinstance(model, RepairableRBD): - levels, rows = model._capacity_at(curve, x) - else: - levels, rows = _capacity.merged( - np.array((0.0,) + model.capacities), - np.vstack([1.0 - values[node], curve.stages_at(x)]), - ) - own[node] = self._forced( - (levels, np.asarray(rows)[:, index]), - node, - working_nodes, - broken_nodes, - ) - levels, rows = self._capacity_arrays(arrays, points, own) - out = np.array( - [np.bincount(index, weights * row, size) for row in rows] - ).reshape(len(levels), size) - return levels, np.clip(out, 0.0, 1.0) def point_capacity( self, @@ -15137,7 +3607,7 @@ def point_capacity( independent: the probability of each capacity is then that of the states of the crews' Markov chain at it, followed over time by uniformization (see ``point_availability``). The chain takes no - nested RBD's capacities in, as yet (#162). + nested RBD's capacities in, as yet. Parameters ---------- @@ -15199,45 +3669,12 @@ def point_capacity( >>> round(plant.capacity_distribution().meets(100), 4) # long run 0.9767 """ - times = _check_times(x) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - self._require_capacity_models() - ends = times.ravel() - horizon = float(ends.max()) if ends.size else 0.0 - forced = working_nodes | broken_nodes - if self._crews_couple() and self._crew_nested(forced): - capacity = self._crew_capacity_over( - horizon, - working_nodes, - broken_nodes, - self._states(state, forced), - ) - levels, rows = capacity.rows(ends) - elif self._crews_couple(): - levels, chain = self._crew_capacity( - working_nodes, - broken_nodes, - self._states(state, working_nodes | broken_nodes), - ) - rows = np.clip(chain.values(ends).T, 0.0, 1.0) - elif self.ccf_groups: - curves, system = self._groups_capacity( - horizon, working_nodes, broken_nodes, state - ) - levels, rows = self._groups_capacity_rows( - curves, system, ends, working_nodes, broken_nodes - ) - else: - curves = self._capacity_curves( - horizon, working_nodes, broken_nodes, state - ) - levels, rows = self._capacity_rows( - curves, ends, working_nodes, broken_nodes - ) - return CapacityDistribution( - levels, rows[:, 0] if np.ndim(x) == 0 else rows + return _repairable_capacity.point_capacity( + self, + x=x, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + state=state, ) def mission_capacity( @@ -15315,159 +3752,13 @@ def mission_capacity( >>> round(plant.capacity_distribution().delivered_fraction(100), 4) 0.988 """ - windows = _check_times(t) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - self._require_capacity_models() - ends = windows.ravel() - horizon = float(ends.max()) if ends.size else 0.0 - forced = working_nodes | broken_nodes - if self._crews_couple() and self._crew_nested(forced): - capacity = self._crew_capacity_over( - horizon, - working_nodes, - broken_nodes, - self._states(state, forced), - ) - levels, rows = self._mission_rows( - capacity.curves, capacity.rows, ends, horizon - ) - return CapacityDistribution( - levels, rows[:, 0] if np.ndim(t) == 0 else rows - ) - if self._crews_couple(): - levels, chain = self._crew_capacity( - working_nodes, - broken_nodes, - self._states(state, working_nodes | broken_nodes), - ) - positive = ends > 0.0 - rows = chain.values(ends).T - rows[:, positive] = ( - chain.integrals(ends[positive]).T / ends[positive] - ) - rows = np.clip(rows, 0.0, 1.0) - keep = np.any(rows != 0.0, axis=1) - levels, rows = levels[keep], rows[keep] - return CapacityDistribution( - levels, rows[:, 0] if np.ndim(t) == 0 else rows - ) - if self.ccf_groups: - grouped, system = self._groups_capacity( - horizon, working_nodes, broken_nodes, state - ) - levels, rows = self._mission_rows( - [*grouped.values(), system.curve], - lambda x: self._groups_capacity_rows( - grouped, system, x, working_nodes, broken_nodes - ), - ends, - horizon, - ) - return CapacityDistribution( - levels, rows[:, 0] if np.ndim(t) == 0 else rows - ) - curves = self._capacity_curves( - horizon, working_nodes, broken_nodes, state - ) - levels, rows = self._mission_rows( - list(curves.values()), - lambda x: self._capacity_rows( - curves, x, working_nodes, broken_nodes - ), - ends, - horizon, - ) - return CapacityDistribution( - levels, rows[:, 0] if np.ndim(t) == 0 else rows - ) - - def _mission_rows( - self, followed: list, rows_at, ends: np.ndarray, horizon: float - ) -> Tuple[np.ndarray, np.ndarray]: - """The capacity distribution averaged over each window ``[0, end)`` - (see ``mission_capacity``): ``rows_at(x)`` its levels and their - probabilities at the times ``x``, integrated on pieces cut where - the ``followed`` curves bend, and extended exactly past the time - they have settled. Returns the levels and one row per level.""" - settle, period = _settling(followed) - reach = min(horizon, settle if period is None else settle + period) - beyond = ends > reach - fixed = [ends[~beyond]] - cycles = rest = np.empty(0) - if period is not None and beyond.any(): - cycles = np.floor((ends[beyond] - settle) / period) - rest = np.clip( - ends[beyond] - settle - cycles * period, 0.0, period - ) - fixed += [np.array([settle]), settle + rest] - - def estimate(a: np.ndarray, b: np.ndarray) -> dict: - # The probability of each capacity level, integrated. - x, half = _quadrature.points(a, b) - levels, rows = rows_at(x) - return { - float(level): _quadrature.summed(row, half) - for level, row in zip(levels, rows) - } - - try: - edges, finest = _quadrature.pieces( - followed, np.concatenate(fixed), reach, _MISSION_POINTS - ) - edges, integrals = _quadrature.refined( - estimate, edges, finest, _MISSION_POINTS - ) - except _quadrature.TooMany as error: - raise NotImplementedError( - f"Integrating the capacity over [0, {reach}] takes " - f"{error.count} pieces (the components' curves bend that " - "often), more than the limit: estimate it by simulation, " - "with availability(demand=...)." - ) from None - # The levels, as ``estimate`` keyed them. - running: Dict[float, np.ndarray] = { - float(level): _quadrature.running(values, len(edges) - 1) - for level, values in integrals.items() - if isinstance(level, float) - } - settled: Dict[float, float] = {} - if period is None and beyond.any(): - levels, rows = rows_at(np.array([horizon])) - settled = {float(v): float(r) for v, r in zip(levels, rows[:, 0])} - inside = np.searchsorted(edges, ends[~beyond]) - positive = ends > 0.0 - at_zero: Dict[float, float] = {} - if not positive.all(): - levels, rows = rows_at(np.zeros(1)) - at_zero = {float(v): float(r) for v, r in zip(levels, rows[:, 0])} - averages = {} - for level in sorted(set(running) | set(settled) | set(at_zero)): - values = running.get(level, np.zeros(len(edges))) - totals = np.empty(len(ends)) - totals[~beyond] = values[inside] - if beyond.any(): - if period is None: - totals[beyond] = values[-1] + ( - ends[beyond] - reach - ) * settled.get(level, 0.0) - else: - base = values[np.searchsorted(edges, settle)] - partial = values[np.searchsorted(edges, settle + rest)] - totals[beyond] = ( - base + cycles * (values[-1] - base) + (partial - base) - ) - average = np.empty(len(ends)) - average[positive] = totals[positive] / ends[positive] - average[~positive] = at_zero.get(level, 0.0) - averages[level] = average - levels = np.array(sorted(averages), dtype=float) - rows = np.array([averages[level] for level in levels]).reshape( - len(levels), len(ends) + return _repairable_capacity.mission_capacity( + self, + t=t, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + state=state, ) - keep = np.any(rows != 0.0, axis=1) - return levels[keep], rows[keep] def _capacity_models(self) -> dict: """``{node: model}`` for the nodes with no capacity entry whose model @@ -15485,592 +3776,20 @@ def _capacity_models(self) -> dict: out[node] = component.reliability return out - def _long_run_capacity(self, node) -> Tuple[np.ndarray, np.ndarray]: - """The long-run distribution of the capacity of a node whose model - has one (see ``_capacity_models``): a nested RBD's own; for a - degrading component, down a fraction ``1 - A`` of the time and in - each stage the rest of it in proportion to the stage's share of its - working time.""" - component = self.components[node] - if isinstance(component, RepairableRBD): - distribution = component.capacity_distribution() - return distribution.levels, distribution.probabilities - self._require_unscheduled_stages(node) - stages = component.reliability - up = self._node_availability(node) - shares = np.concatenate([[1.0 - up], up * stages.stage_fractions()]) - levels, rows = _capacity.merged( - np.array((0.0,) + stages.capacities), shares[:, None] - ) - return levels, rows[:, 0] - - def _follow_up(self, event: Event, source) -> Event: - """The next event of ``event``'s component, drawn from ``source``. - - A component's ``next_event()`` gives the time *to* its next event, - measured from ``event``. A nested RBD's gives the time *of* its next - state change: its simulation runs on the same clock, from 0. - """ - node = event.component - schedule = self._preventive.get(node) - if schedule is not None: - return self._maintained_follow_up(event, source, schedule) - inspection = self._inspection.get(node) - if inspection is not None: - return self._inspected_follow_up(event, source, inspection) - t, status = source.next_event() - if isinstance(self.components[node], RepairableRBD): - return Event(t, node, status, _planned(source, status)) - return Event(event.time + t, node, status) - - def _strike(self, event: Event, source) -> List[Event]: - """A shared common cause strikes (#158, see ``_Cause``): each member - it names that is up fails now, its own next event superseded (a - member down stays down), and with tests that can miss a failure, - one uniform decides for all its failures. Returns the events to - queue: the members' failures, and the cause's next strike.""" - cause: Any = event.component - t = event.time - status = self.component_status - out: List[Event] = [] - coin = source.coin() if cause.coins else None - for member in cause.struck: - if not status[member]: - continue - pending = self._ccf_pending.pop(member, None) - if pending is not None and pending.time < self.t_simulation: - self._cancelled[id(pending)] = pending - if coin is not None: - self._ccf_coins[member] = coin - out.append(Event(t, member, False)) - later = Event(t + source.gap(), cause, False) - if later.time < self.t_simulation: - out.append(later) - return out - - def _crew_follow_up( - self, event: Event, next_event: Event - ) -> Optional[Event]: - """``next_event`` as the repair crews allow (see ``_Crews``): a job - that falls due when ``event`` takes a component down waits for a - crew (None while it does), and a crew that finishes one at - ``event`` starts the next waiting job, whose end is queued here.""" - crews, node = self._crews, event.component - if crews is None or node not in crews.served: - return next_event - if event.status: - if node in crews.holding: - started = crews.release(node, event.time) - if started is not None: - self._crew_started(started) - return next_event - if next_event.status: - return crews.request(node, event.time, next_event) - return next_event - - def _crew_started(self, started: Tuple[Any, float, Event]) -> None: - """A crew has started a waiting job, ``started`` as - ``_Crews.release`` gives it: queue the end of a component's, or - tell its standby group of a unit's.""" - key, wait, ends = started - if isinstance(key, _Unit): - self._groups[key.node].crew_started(key.unit, ends.time) - return - ends = self._started_late(key, wait, ends) - if ends.time < self.t_simulation: - self._event_queue.put(ends) - - def _started_late(self, node, wait: float, ends: Event) -> Event: - """A job of ``node`` that a crew starts ``wait`` after it fell due, - and so ends with ``ends``: a component off-line for a test does not - age while it waits, so its hidden failure is ``wait`` later.""" - pending = self._pending_failure.get(node) - if pending is not None: - self._pending_failure[node] = pending + wait - return ends - - def _crew_served(self) -> list: - """The jobs the repair crews work on, by key: this RBD's own - components, not a nested RBD's (which has crews of its own), and - each unit of a standby group (``_Unit(node, unit)``).""" - served: list = [] - for node, component in self.components.items(): - if isinstance(component, RepairableRBD): - continue - arrangement = self._standby.get(node) - if arrangement is None: - served.append(node) - else: - served.extend( - _Unit(node, unit) for unit in range(arrangement.units) - ) - return served - - def _crews_limited(self) -> bool: - """Whether there are fewer repair crews than components that may - need one, so that a job may wait.""" - return self.repair_crews is not None and self.repair_crews < len( - self._crew_served() - ) - def _crews_couple(self) -> bool: """Whether a job waiting for a repair crew can tie different nodes together: the crews are limited, and work on more than one node's jobs. With a single standby group's units the only jobs, the waiting stays inside the group, whose own Markov chain counts the crews (see ``_standby_long_run``), and the nodes stay independent.""" - if not self._crews_limited(): + if not _crews._crews_limited(self): return False owners = { key.node if isinstance(key, _Unit) else key - for key in self._crew_served() + for key in _crews._crew_served(self) } return len(owners) > 1 - def _require_unlimited_crews( - self, - assumes: str = "these values assume", - advice: str = "Simulate the system with availability() or cost().", - ) -> None: - """Raise if a job may wait for a repair crew: components then no - longer fail and recover independently, which ``assumes`` (what - assumes it, and ``assume``): do ``advice`` instead.""" - if self._crews_couple(): - raise NotImplementedError( - f"With {self.repair_crews} repair crew(s) for " - f"{len(self._crew_served())} components, a component can " - "wait for a crew, so the components no longer fail and " - f"recover independently, which {assumes}. {advice}" - ) - - def _no_crew_chain(self, why: str) -> NoReturn: - """Raise that the repair crews' Markov chain does not cover this RBD, - because ``why``.""" - raise NotImplementedError( - f"With {self.repair_crews} repair crew(s) for " - f"{len(self._crew_served())} components, a component can wait " - "for a crew, and the exact long-run values come from a Markov " - "chain of the components' states and the repair queue: " - f"{why}. Simulate the system with availability() or cost()." - ) - - def _crew_chain_rates(self) -> Dict[Any, Tuple[float, float]]: - """The failure and repair rates (``inf`` for an instant repair) of - the components the repair crews work on, for their Markov chain - (see ``_crew_chain.py``). Raise if the chain does not cover them: - scheduled maintenance or inspection, a life or repair time that is - not exponential, or more states than it is solved for.""" - assert self.repair_crews is not None # limited crews only - for node in self._standby: - self._no_crew_chain( - "it has no place for a standby group, which component " - f"{node!r} is" - ) - rates: Dict[Any, Tuple[float, float]] = {} - for node in self._crew_served(): - component = self.components[node] - if node in self._preventive: - self._no_crew_chain( - "it has no place for scheduled maintenance, which " - f"component {node!r} has" - ) - if node in self._inspection: - self._no_crew_chain( - "it has no place for inspections, which component " - f"{node!r} has" - ) - if node in self._imperfect: - self._no_crew_chain( - "it has no place for imperfect repair, which component " - f"{node!r} has" - ) - life = _constant_rate(component.reliability) - if life is None: - self._no_crew_chain( - "it needs exponential lives (a constant failure rate), " - f"and the life of component {node!r} is not one" - ) - repair = _repair_rate(component.time_to_replace) - if repair is None: - self._no_crew_chain( - "it needs exponential repair times (or instant repair), " - f"and the repair times of component {node!r} are not" - ) - rates[node] = (life, repair) - count = _crew_chain.state_count( - [self._priority.get(node, 0.0) for node in rates], - [math.isinf(repair) for _, repair in rates.values()], - self.repair_crews, - ) - if count > _crew_chain.MAX_STATES: - self._no_crew_chain( - f"here it has {count:,} states, more than the " - f"{_crew_chain.MAX_STATES:,} it is solved for" - ) - return rates - - def _require_crew_chain(self) -> None: - """Raise if the exact long-run values with limited repair crews - cannot be computed: the Markov chain does not take common-cause - groups in (#251, see ``_ccf_rates``), does not cover the components - (see ``_crew_chain_rates``), or nested RBDs' calendars fall together - (see ``_require_calendars``).""" - self._require_ccf_long_run() - self._crew_chain_rates() - self._require_calendars() - - def _crew_chain( - self, forced: frozenset = frozenset() - ) -> "_crew_chain.CrewChain": - """The Markov chain of the components the repair crews work on, and - its long-run distribution (see ``_crew_chain.py``), leaving out the - nodes in ``forced``: held working a node never fails, and held - broken it is never repaired, so neither needs a crew. Solved once - for each set of rates and kept.""" - self._require_crew_chain() - rates = self._crew_chain_rates() - assert self.repair_crews is not None # limited crews only - nodes = [node for node in rates if node not in forced] - priorities = [self._priority.get(node, 0.0) for node in nodes] - key = ( - self.repair_crews, - tuple(zip(nodes, (rates[node] for node in nodes), priorities)), - ) - cache = self.__dict__.setdefault("_crew_chains", {}) - if key not in cache: - cache[key] = _crew_chain.solve( - nodes, - [rates[node][0] for node in nodes], - [rates[node][1] for node in nodes], - priorities, - self.repair_crews, - ) - return cache[key] - - def _chain_probabilities( - self, working_nodes, broken_nodes - ) -> Tuple[dict, np.ndarray]: - """With limited repair crews, every node's availability in each - state of their Markov chain (1 or 0 for the components the crews - work on and those held working or broken, and a nested RBD's own - long-run availability, as it has crews of its own), and the states' - long-run probabilities: the long-run values are then averages over - the states, as over the times of ``_long_run_grid``.""" - working_nodes = set(working_nodes or ()) - broken_nodes = set(broken_nodes or ()) - self._validate_node_overrides(working_nodes, broken_nodes) - chain = self._crew_chain(frozenset(working_nodes | broken_nodes)) - size = len(chain.probabilities) - out: dict = { - node: chain.up[:, k].astype(float) - for k, node in enumerate(chain.nodes) - } - for node, component in self.components.items(): - if node in working_nodes: - out[node] = np.ones(size) - elif node in broken_nodes: - out[node] = np.zeros(size) - elif node not in out: - out[node] = np.full(size, float(component.mean_availability())) - for node in self.in_or_out: - out[node] = np.ones(size) - return out, chain.probabilities - - def _chain_outage_terms( - self, working_nodes, broken_nodes - ) -> Tuple[List[Tuple[Any, float]], float]: - """``_outage_terms`` with limited repair crews, over the - states of their Markov chain: in each, a component that is up fails - at its constant rate, and takes the system down if it is critical - there. A nested RBD enters through its own frequencies, as it has - crews of its own.""" - availability, unavailability, weights = ( - self._long_run_unavailabilities(working_nodes, broken_nodes) - ) - forced = set(working_nodes or ()) | set(broken_nodes or ()) - rates = self._crew_chain_rates() - birnbaum = super()._birnbaum_importance( - availability, node_failures=unavailability - ) - terms: List[Tuple[Any, float]] = [] - planned = 0.0 - for node in self.components: - if node in forced: - continue - importance = np.asarray(birnbaum[node]) - if node in rates: - life = rates[node][0] - node_failures, node_planned = life * availability[node], 0.0 - else: - node_failures, _, node_planned = self._node_frequencies(node) - terms.append((node, float(weights @ (importance * node_failures)))) - planned += float(weights @ (importance * node_planned)) - return terms, planned - - def _maintained_follow_up( - self, event: Event, source, schedule: _Preventive - ) -> Event: - """The next event of a component under preventive maintenance.""" - node, t = event.component, event.time - if event.inspection: - # Inspected, and kept (replacement on condition): its failure is - # still ahead of it. - return self._condition_next(node, t, schedule) - if event.preventive: - # Replaced: the new unit has its own life ahead of it (the - # failure drawn for the old one never happens). - source.reset() - if not event.status: - # Down for maintenance: back, as new, once it is done. - if isinstance(source, _StreamedComponent): - duration = source.maintenance_time() - else: - duration = schedule.duration.random(1).item() - return Event(t + duration, node, True, True) - elif not event.status: - # Failed: back, as new, once it is repaired. - repair, status = source.next_event() - return Event(t + repair, node, status) - # Working, as new, from t. - return self._renewal(node, t, source, schedule) - - def _renewal( - self, - node, - t: float, - source, - schedule: _Preventive, - life: Optional[float] = None, - start: Optional[float] = None, - ) -> Event: - """The first event of a unit put into service as new at ``t``: its - failure, or its preventive replacement if that is due first (a - failure at the same time comes first); replaced on condition, its - failure or its first inspection (see ``_condition_next``). Started - from a state (see ``_started``), the unit was put into service at - ``start``, before ``t``, and has ``life`` left.""" - if life is None: - life, _ = source.next_event() - if schedule.policy == "condition": - failure = t + life - self._in_service[node] = ( - failure, - self._replaced_at( - node, t if start is None else start, failure, schedule, t - ), - ) - return self._condition_next(node, t, schedule) - if start is None: - # Back from an imperfect repair, the unit is as old as its - # operating time since it was renewed, which age replacement - # counts. - start = t - source.operated if node in self._imperfect else t - due = self._due( - node, schedule, start if schedule.policy == "age" else t - ) - if due < t: - due = t # past its age at the start: replaced at once - if t + life <= due: - event = Event(t + life, node, False) - else: - # Maintenance that takes time is a planned outage; in zero time - # the unit is renewed in place, and stays up. - event = Event(due, node, schedule.duration is None, True) - if node in self._early_members: - # Kept, should a stop of its group renew it early (see _stop). - self._renewed_at[node] = start - self._pending_event[node] = event - return event - - def _own_sources(self, states: Optional[dict] = None) -> dict: - """What the components draw their events from when no streams are - given: themselves, but for an imperfectly repaired one, a stand-in - keeping its virtual age (see ``_ImperfectComponent``), and for one - started from a state in ``states`` that draws what is left of its - life or repair, a stand-in whose next draw can be set (see - ``_started``): drawing from their models and numpy's global - RNG.""" - starting = [ - node - for node, start in (states or {}).items() - if _draws_start(start) - ] - if not self._imperfect and not starting and not self.ccf_groups: - return self.components - sources = dict(self.components) - for group in self.ccf_groups: - # Each member's own cause's life, and the shared causes (#158). - for member in group.members: - own = self._own_rate(member) - if own is not None: - sources[member] = self._own_draws(member, own) - for cause in self._shared_causes(): - sources[cause] = _CauseDraws(cause.rate) - for node, imperfect in self._imperfect.items(): - sources[node] = _ImperfectComponent( - self.components[node], - imperfect, - model=self._duration_model(node), - ) - for node in starting: - component = self.components[node] - sources[node] = _StreamedComponent( - _ModelDraws(component.reliability), - _ModelDraws(component.time_to_replace), - model=self._duration_model(node), - ) - return sources - - def _stop(self, group, t: float, trigger=None) -> List[Event]: - """A stop of maintenance ``group`` at ``t``, opened by ``trigger`` - (a member taken down for a failure or its scheduled replacement) or - by the system going down: each other member that is working, and at - least its opportunity age, is renewed now, as its scheduled - replacement would renew it, and the failure or replacement it had - ahead of it is cancelled. A member whose own failure or replacement - is due at ``t`` keeps it: that joins the stop. Returns the renewals - to queue.""" - out: List[Event] = [] - for member in self._maintenance[group].members: - if ( - member == trigger - or member not in self._early_members - or member not in self._renewed_at - or member in self._renewing - or not self.component_status[member] - ): - continue - schedule = self._preventive[member] - if t - self._renewed_at[member] < schedule.opportunity: - continue - pending = self._pending_event.get(member) - if pending is not None and pending.time <= t: - continue # due now, as part of this stop - self._pending_event.pop(member, None) - if pending is not None and pending.time < self.t_simulation: - self._cancelled[id(pending)] = pending - renewal = Event(t, member, schedule.duration is None, True) - self._early[id(renewal)] = renewal - self._renewing.add(member) - out.append(renewal) - return out - - def _replaced_at( - self, - node, - renewed: float, - failure: float, - schedule: _Preventive, - after: Optional[float] = None, - ) -> float: - """When a unit put into service as new at ``renewed``, and due to - fail at ``failure``, is replaced on condition: at the first - inspection before its failure (and the window's end), and after - ``after`` (by default ``renewed``), at which it is more likely than - the threshold to fail before the next, given its age (the time - since ``renewed``); ``inf`` if at none.""" - end = min(failure, self.t_simulation) - dues = [] - due = self._due(node, schedule, renewed if after is None else after) - while due < end: - dues.append(due) - due = self._due(node, schedule, due) - if not dues: - return math.inf - # Its ages at those inspections, and at the one after the last. - ages = np.array(dues + [due]) - renewed - likely = _failures_between(self.components[node].reliability, ages) - above = np.flatnonzero(likely > schedule.threshold) - return dues[above[0]] if above.size else math.inf - - def _condition_next(self, node, t: float, schedule: _Preventive) -> Event: - """The next event, from ``t``, of a working unit replaced on - condition: its failure, or the next inspection, which replaces the - unit (see ``_replaced_at``) or only checks it. A failure at an - inspection's time comes first.""" - failure, replaced = self._in_service[node] - due = self._due(node, schedule, t) - if failure <= due: - return Event(failure, node, False) - if due >= replaced: - # Replaced, as under block replacement. - return Event(due, node, schedule.duration is None, True) - return Event(due, node, True, inspection=True) - - def _inspected_follow_up( - self, event: Event, source, inspection: _Inspection - ) -> Event: - """The next event of a component whose failures are hidden.""" - node, t = event.component, event.time - if not event.inspection: - if event.status: - # Repaired: as new, from t. - return self._inspected_renewal(node, t, source, inspection) - # Failed, unseen: found by the first inspection at or after t, - # unless that one can miss it and does (a shared common cause - # has decided for all its failures, see _strike). - self._pending_failure[node] = None - found = self._finds(node, inspection, t) - coin = self._ccf_coins.pop(node, None) if self._ccf_coins else None - if not inspection.is_full(found) and not ( - coin < inspection.coverage - if coin is not None - else _test_finds(source, inspection.coverage) - ): - return Event(found, node, False, inspection=True, missed=True) - return Event(found, node, False, inspection=True) - if event.missed: - # A test that missed the failure: the next test misses it too, - # unless it is a full test. - due = self._due(node, inspection, t) - return Event( - due, - node, - False, - inspection=True, - missed=not inspection.is_full(due), - ) - if event.status: - # Tested in zero time, or back from a test: working, with its - # failure still ahead of it. - return self._inspected_next(node, t, inspection) - # A test that takes time, of a failed unit (repaired once the test is - # done) or of a working one (off-line, and not ageing, until then). - duration = 0.0 - if inspection.duration is not None: - if isinstance(source, _StreamedComponent): - duration = source.maintenance_time() - else: - duration = inspection.duration.random(1).item() - failure = self._pending_failure[node] - if failure is None: - repair, status = source.next_event() - return Event(t + duration + repair, node, status) - self._pending_failure[node] = failure + duration - return Event(t + duration, node, True, inspection=True) - - def _inspected_renewal( - self, node, t: float, source, inspection: _Inspection - ) -> Event: - """The first event of a unit with hidden failures put into service - as new at ``t``.""" - life, _ = source.next_event() - self._pending_failure[node] = t + life - return self._inspected_next(node, t, inspection) - - def _inspected_next( - self, node, t: float, inspection: _Inspection - ) -> Event: - """The next event of a working unit with hidden failures, at ``t``: - its failure, or the next inspection if that comes first (a failure - at the same time comes first, and is found by it). A test that takes - time takes the unit off-line; one in zero time leaves it up.""" - failure = self._pending_failure[node] - due = self._due(node, inspection, t) - if failure <= due: # type: ignore[operator] - return Event(failure, node, False) # type: ignore[arg-type] - return Event(due, node, inspection.duration is None, inspection=True) - def next_event(self, method="p", sources: Optional[dict] = None): """Advance the current simulation to the system's next state change. @@ -16143,95 +3862,7 @@ def next_event(self, method="p", sources: Optional[dict] = None): >>> changes[4] (30.0, True) """ - if not hasattr(self, "_event_queue"): - raise ValueError("Need to initialize the event queue") - # The components' draws come from the same sources the queue was - # initialised with (see initialize_event_queue). - if sources is None: - sources = getattr(self, "_step_sources", self.components) - new_system_state = copy(self.system_state) - - # Use a while loop to find the next time/event at which the system - # status changes. - while new_system_state == self.system_state: - if self._event_queue.qsize() == 0: - del self._event_queue - self.last_change_planned = False - return self.t_simulation, self.system_state - - event = self._event_queue.get() - if self._cancelled and self._cancelled.get(id(event)) is event: - del self._cancelled[id(event)] - continue # superseded by an early renewal, or a strike - if event.component.__class__ is _Cause: - # A shared common cause strikes (#158, see _strike). - for follow in self._strike(event, sources[event.component]): - self._event_queue.put(follow) - continue - renewing_early = False - if self._early and self._early.get(id(event)) is event: - del self._early[id(event)] - self._renewing.discard(event.component) - renewing_early = True - group = self._groups.get(event.component) - if group is not None: - # A standby group's own event (see _StandbyGroup). - if not group.holds(event): - continue # superseded by an earlier one - now, _ = group.advance(event.time) - if now == self.component_status[event.component]: - continue - self.component_status[event.component] = now - if now != self.system_state: - new_system_state = self.is_system_working( - self.component_status, method - ) - continue # the group has queued its next event - was = self.component_status[event.component] - self.component_status[event.component] = event.status - node = event.component - if node in self._member_group and not renewing_early: - # A failure, or a scheduled replacement starting (or done in - # place), opens a stop of its maintenance group. - if (not event.status and not event.inspection) or ( - event.preventive and event.status and was - ): - for renewal in self._stop( - self._member_group[node], event.time, node - ): - self._event_queue.put(renewal) - # Only a change against the system's state can change it (the - # structure is coherent; see _replicate). - if event.status != self.system_state: - new_system_state = self.is_system_working( - self.component_status, method - ) - if self.system_state and not new_system_state: - for name, spec in self._maintenance.items(): - if spec.system_down: - for renewal in self._stop(name, event.time): - self._event_queue.put(renewal) - - follow = self._follow_up(event, sources[event.component]) - next_event: Optional[Event] = ( - follow - if self._crews is None - else self._crew_follow_up(event, follow) - ) - # But only queue up the event if it occurs before the end - # of the simulation - if next_event is not None and next_event.time < self.t_simulation: - self._event_queue.put(next_event) - if node in self._ccf_members: - self._ccf_pending[node] = next_event - - self.system_state = new_system_state - # A system taken down by maintenance or a test is a planned outage. - self.last_change_planned = ( - event.preventive or event.inspection - ) and not new_system_state - - return event.time, self.system_state + return _event_loop.next_event(self, method=method, sources=sources) def availability( self, @@ -16383,7 +4014,7 @@ def availability( bit; what the compiled engine does not simulate runs in Python, and another package can add an engine of its own (see ``repyability.rbd.engines``). See - [The compiled engine](guide/simulation.md#the-compiled-engine). + [The compiled engine](../guide/simulation.md#the-compiled-engine). demand : float, optional The demand the delivered fraction is measured against, in the capacities' units, when nodes have capacities. By default the @@ -16420,7 +4051,7 @@ def availability( order, as ``map`` does. The built-in ``map`` runs them here; ``concurrent.futures.ProcessPoolExecutor(...).map`` in other processes; Ray's, Dask's or a batch system's on other machines - (see [Shards](guide/simulation.md#shards)). The result is the + (see [Shards](../guide/simulation.md#shards)). The result is the same to the last bit; a run to a ``tolerance`` maps a round of shards at a time. Each shard is simulated by ``engine`` where it runs, and carries the system as JSON, so the system must save @@ -16438,17 +4069,17 @@ def availability( system's expected values over the window, the result's means (``mean_availability``, the cost's ``mean`` and breakdowns) and their intervals are those values, with no error, the twin - being the system itself (#187, #223); otherwise they may be + being the system itself; otherwise they may be taken given the modules (see ``conditional``). False forces a plain simulation, whose means are the simulations' own. True controls the run by the twin, whose values the result's ``control_variate`` holds (see [`ControlVariate`][repyability.ControlVariate]); every draw must then come from a stream (surpyval parametric models). See - [An exact twin](guide/simulation.md#an-exact-twin). + [An exact twin](../guide/simulation.md#an-exact-twin). conditional : bool, optional Take the expected values given the histories of the dependent - modules (#189): the nodes whose values over time the exact + modules: the nodes whose values over time the exact methods do not work out (``analysis_routes`` names them), every other node independent of them. Each simulation's expected values given its modules' histories are exact, and their mean @@ -16462,7 +4093,7 @@ def availability( fraction of the work, but each simulation's values are then expected values: the cost's ``percentile`` and ``std`` refuse, and ``criticalities`` is None. See - [Conditional runs](guide/simulation.md#conditional-runs). + [Conditional runs](../guide/simulation.md#conditional-runs). Returns ------- AvailabilityResult @@ -16536,26 +4167,24 @@ def availability( >>> {node: round(float(v), 4) for node, v in oci.up.items()} {'a': 1.0, 'b': 1.0} """ - N = 10_000 if mc_samples is None else mc_samples - - return self._simulated( - t_simulation, - working_nodes, - broken_nodes, - method, - N, - verbose, - seed, + return _runs.availability( + self, + t_simulation=t_simulation, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + mc_samples=mc_samples, + verbose=verbose, + seed=seed, tolerance=tolerance, confidence=confidence, - max_N=max_samples, + max_samples=max_samples, antithetic=antithetic, n_jobs=n_jobs, - target="availability", demand=demand, engine=engine, state=state, - curve_points=_curve_points(curve_points), + curve_points=curve_points, shard_map=shard_map, shard_size=shard_size, control_variate=control_variate, @@ -16578,7 +4207,12 @@ def simulate_timelines( """Simulate the system over ``[0, t_simulation]`` and keep each simulation's up/down histories as timelines: every component's and the system's, with the component that caused each of the system's - changes. + changes. A common-cause group's shared cause fails the members it + strikes one after another at one instant, each a failure of the + member's own, so a system failure it makes is the member's whose + fall took the system down (the last the structure could spare), + not the group's: ``failures_by_cause`` counts it under that + member. These are the simulations [`availability`][repyability.RepairableRBD.availability] runs with @@ -16694,18 +4328,18 @@ def simulate_timelines( >>> bool(np.array_equal(runs.system.uptime, result.uptimes)) True """ - return _timeline_runs.simulate( + return _runs.simulate_timelines( self, - t_simulation, - 1000 if mc_samples is None else mc_samples, - seed, - working_nodes, - broken_nodes, - antithetic, - engine, - n_jobs, - start, - state, + t_simulation=t_simulation, + mc_samples=mc_samples, + seed=seed, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + antithetic=antithetic, + engine=engine, + n_jobs=n_jobs, + start=start, + state=state, ) def simulate_chunk( @@ -16735,7 +4369,7 @@ def simulate_chunk( Each simulation draws from streams of its own, seeded from ``seed`` and its position in the run (see - [Random streams](guide/simulation.md#random-streams)), so it + [Random streams](../guide/simulation.md#random-streams)), so it comes out the same wherever and whenever it runs, by either engine, in any company. A run can so be split across processes, machines or preemptible workers: each runs its chunk, saves it @@ -16745,8 +4379,7 @@ def simulate_chunk( simulations ``0`` to ``N - 1`` give the result of ``availability(..., mc_samples=N)``, to the last bit: the same simulations, so the same per-simulation values and timeline, and - the same totals, which are kept exactly however the run is cut - (#151). + the same totals, which are kept exactly however the run is cut. Parameters ---------- @@ -16786,7 +4419,7 @@ def simulate_chunk( of a run share it): the chunk then holds the grid's counts rather than every change. By default None: every change. control_variate : bool, optional - None (the default) or False, as for ``availability`` (#236): + None (the default) or False, as for ``availability``: the means the merged result takes, kept with the chunk (the chunks of a run share it). The simulations are the same either way; False gives the merged result the simulations' own means. @@ -16834,126 +4467,25 @@ def simulate_chunk( >>> bool((merged.uptimes == whole.uptimes).all()) True """ - if seed is None: - raise ValueError( - "A chunk needs the run's seed: the chunks of a run share it." - ) - for name, value in (("start", start), ("stop", stop)): - if isinstance(value, bool) or not isinstance( - value, (int, np.integer) - ): - raise ValueError( - f"{name} must be a whole number, got {value!r}." - ) - if not 0 <= start < stop: - raise ValueError( - f"The chunk must be simulations start to stop - 1, with 0 <= " - f"start < stop; got start={start!r}, stop={stop!r}." - ) - if antithetic and (start % 2 or stop % 2): - raise ValueError( - "With antithetic pairs, start and stop must be even, so that " - "the chunk holds whole pairs." - ) - means = _chunk_means(control_variate, conditional) - working = set() if working_nodes is None else set(working_nodes) - broken = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working, broken) - states = self._simulation_states(state, working | broken) - chunk = self._chunk( - t_simulation, - int(start), - int(stop), - _streams.entropy_of(seed), - working, - broken, - method, - antithetic, - demand, - engine, - None if n_jobs is None else montecarlo.jobs(n_jobs), - verbose, - states, - _curve_points(curve_points), - ) - # Kept only where they differ from availability's defaults, so that - # a chunk is saved as it was before them. - chunk.settings.update( - {name: value for name, value in means.items() if value is False} - ) - return chunk - - def _chunk( - self, - t_simulation: float, - start: int, - stop: int, - entropy: int, - working: set, - broken: set, - method: str, - antithetic: bool, - demand: Optional[float], - engine: str, - jobs: Optional[int], - verbose: bool, - states: dict, - curve_points: Optional[int], - ) -> "SimulationChunk": - """Simulations ``start`` to ``stop - 1`` of the run of ``entropy`` - (see ``simulate_chunk``, whose arguments, checked, these are), as - a chunk with the run's settings.""" - from repyability.rbd.chunks import SimulationChunk - - capacity = self._chunk_capacity(broken, method, demand) - tally = self._run( - t_simulation, - working, - broken, - method, - stop - start, - verbose, - None, - antithetic, - capacity=capacity, - jobs=jobs, + return _runs.simulate_chunk( + self, + t_simulation=t_simulation, + start=start, + stop=stop, + seed=seed, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + antithetic=antithetic, + demand=demand, engine=engine, - entropy=entropy, - first=start, - states=states, + n_jobs=n_jobs, + verbose=verbose, + state=state, curve_points=curve_points, + control_variate=control_variate, + conditional=conditional, ) - settings = { - "t_simulation": float(t_simulation), - "entropy": entropy, - "method": method, - "working_nodes": sorted(working, key=repr), - "broken_nodes": sorted(broken, key=repr), - "antithetic": bool(antithetic), - "demand": None if demand is None else float(demand), - "fingerprint": self._fingerprint(), - "state": _state_key(states), - "curve_points": tally.curve_points, - } - return SimulationChunk([(start, stop)], settings, tally) - - def _chunk_capacity( - self, broken: set, method: str, demand: Optional[float] - ) -> Optional[_CapacityRecorder]: - """Check a chunk's (or shard's) ``method`` and ``demand``, and give - the recorder of the capacities it follows, if the system has - them.""" - self.is_system_working( - {c: c not in broken for c in self.components}, method - ) - if self._has_capacity(): - return _CapacityRecorder(self, demand) - if demand is not None: - raise ValueError( - "A demand is measured against capacities, and no node has " - "one: give them with capacity={node: capacity}." - ) - return None def shards( self, @@ -16998,7 +4530,7 @@ def shards( size : int, optional Simulations to a shard, rounded up to a whole number of the run's widest block of draws (see - [Random streams](guide/simulation.md#random-streams)), so that + [Random streams](../guide/simulation.md#random-streams)), so that no two shards draw the same block; by default as many as make 1024 or more. A shard should run for some seconds, to repay a worker's start. @@ -17051,121 +4583,21 @@ def shards( >>> len(rbd.shards(100.0, 10_000, seed=1, size=4096)) 3 """ - N = 10_000 if mc_samples is None else mc_samples - montecarlo.check_count(N, antithetic, "mc_samples") - working = set() if working_nodes is None else set(working_nodes) - broken = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working, broken) - states = self._simulation_states(state, working | broken) - capacity = self._chunk_capacity(broken, method, demand) - self._require_free_members(working, broken) - self._require_groups_simulated(states) - t_simulation = _check_window(t_simulation) - entropy = _streams.entropy_of(seed) - template = self._shard_template( - t_simulation, - entropy, - working, - broken, - method, - antithetic, - demand, - engine, - states, - _curve_points(curve_points), - ) - # What a run checks before it simulates (see _run). - plan, complete = self._stream_plan( - t_simulation, entropy, antithetic, states - ) - if antithetic and not complete: - raise NotImplementedError(_UNSTREAMED) - self._simulation_engine( - engine, plan, capacity, N, here=False, states=states - ) - step = self._shard_size(plan, size) - return [ - _shard_bytes(template, first, min(first + step, N)) - for first in range(0, N, step) - ] - - def _shard_template( - self, - t_simulation: float, - entropy: int, - working: set, - broken: set, - method: str, - antithetic: bool, - demand: Optional[float], - engine: str, - states: dict, - curve_points: Optional[int], - ) -> dict: - """What every shard of a run holds (see ``shards``): all but its - range.""" - from repyability.rbd.shards import KIND - - return { - "kind": KIND, - "repyability_version": __version__, - "system": self._shard_system(), - "t_simulation": float(t_simulation), - "entropy": int(entropy), - "method": method, - "working_nodes": sorted(working, key=repr), - "broken_nodes": sorted(broken, key=repr), - "antithetic": bool(antithetic), - "demand": None if demand is None else float(demand), - "engine": engine, - "state": _state_key(states), - "curve_points": curve_points, - } - - def _shard_system(self) -> dict: - """This system as a shard carries it: ``to_dict``'s JSON data, with - every model of a class that loads back as itself (see - ``serialisation.exactly``), so that a worker simulates this - system.""" - from repyability.rbd.serialisation import exactly - - try: - with exactly(): - system = self.to_dict() - json.dumps(system) - if type(self) is not RepairableRBD: - raise NotImplementedError( - f"it is a {type(self).__name__}, which loads as a " - "RepairableRBD" - ) - except Exception as error: - raise NotImplementedError( - "A shard carries the system as JSON, and this one cannot be " - f"saved: {str(error).rstrip('.')}. Run it with " - "availability(n_jobs=...) instead." - ) from None - return system - - @staticmethod - def _shard_size(plan: _streams.Plan, size: Optional[int]) -> int: - """Simulations to a shard of the run of ``plan``: ``size`` (by - default 1024), rounded up to a whole number of its widest block of - draws, so that no two shards draw the same block.""" - wanted = 1024 if size is None else size - if ( - isinstance(wanted, bool) - or not isinstance(wanted, (int, np.integer)) - or wanted < 1 - ): - raise ValueError( - "The size of a shard must be a whole number of simulations, " - f"at least 1; got {size!r}." - ) - widest = max( - [plan.columns(spec) for spec in plan.specs.values()] - + [2 if plan.antithetic else 1] + return _runs.shards( + self, + t_simulation=t_simulation, + mc_samples=mc_samples, + seed=seed, + size=size, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + antithetic=antithetic, + demand=demand, + engine=engine, + state=state, + curve_points=curve_points, ) - return int(-(-int(wanted) // widest) * widest) def availability_from_chunks( self, @@ -17186,10 +4618,10 @@ def availability_from_chunks( same per-simulation values (``uptimes``, the costs' ``samples``) and timeline, and the same totals, to the last bit: every total is kept exactly and rounded once, so it does not depend on how the run was - cut (#151). + cut. The chunks must hold simulations ``0`` to ``N - 1`` with none missing, so a lost chunk (a shard that never came back) is not - taken for a smaller run (#176); with ``allow_gaps=True`` they give + taken for a smaller run; with ``allow_gaps=True`` they give the result of whichever simulations they hold. With costs, the result's ``cost`` is the simulated cost distribution, as ``cost`` gives it. @@ -17210,9 +4642,9 @@ def availability_from_chunks( for the result of those they hold. By default False. control_variate : bool, optional None (the default): the means ``availability`` takes by - default, exact where the exact methods work them out (#187). + default, exact where the exact methods work them out. False: the simulations' own, as ``availability(..., - control_variate=False)`` gives them (#236). Chunks made with + control_variate=False)`` gives them. Chunks made with False keep it. conditional : bool, optional None (the default) or False, as for ``availability``: False @@ -17249,150 +4681,14 @@ def availability_from_chunks( >>> rbd.availability_from_chunks(chunk).n_simulations 200 """ - from repyability.rbd.chunks import SimulationChunk - - if isinstance(chunks, (SimulationChunk, dict, bytes, bytearray)): - chunks = [chunks] - chunk = SimulationChunk.merge(chunks) - if mc_samples is not None and chunk.ranges != [(0, int(mc_samples))]: - raise ValueError( - f"The chunks hold simulations {chunk.ranges} (as " - "(start, stop) ranges), not all of simulations 0 to " - f"{int(mc_samples) - 1}: some are missing." - ) - if not allow_gaps and chunk.ranges != [(0, chunk.n_simulations)]: - held = ", ".join(f"{a} to {b - 1}" for a, b in chunk.ranges) - raise ValueError( - f"The chunks hold simulations {held}: those between or " - "before them are missing (a chunk that never came back?). " - "Give them all, or pass allow_gaps=True for the result of " - "the simulations held." - ) - settings = chunk.settings - means = _chunk_means(control_variate, conditional) - for name in ("control_variate", "conditional"): - if settings.get(name) is False: - means[name] = False - fingerprint = self._fingerprint() - if ( - settings["fingerprint"] is not None - and fingerprint is not None - and settings["fingerprint"] != fingerprint - ): - raise ValueError( - "The chunks were simulated with another system (or with " - "this one saved by another RePyability version)." - ) - working = set(settings["working_nodes"]) - broken = set(settings["broken_nodes"]) - capacity = None - if self._has_capacity(): - capacity = _CapacityRecorder(self, settings["demand"]) - initial_up = bool( - self.is_system_working( - {c: c not in broken for c in self.components}, - settings["method"], - ) - ) - # The means the run's result takes by default (#187, #189). - controls, record, breakdown = self._default_means( - chunk._tally, - float(settings["t_simulation"]), - working, - broken, - settings["method"], - int(settings["entropy"]), - bool(settings["antithetic"]), - _states_from_key(self, settings.get("state")), - chunk.ranges, - **means, - ) - return self._availability_result( - chunk._tally, - settings["t_simulation"], - initial_up, - settings["antithetic"], - capacity, - controls, - conditional=record, - breakdown=breakdown, - ) - - def _default_means( - self, - tally: "_Tally", - t_simulation: float, - working: set, - broken: set, - method: str, - entropy: int, - antithetic: bool, - state, - ranges: List[Tuple[int, int]], - control_variate: Optional[bool] = None, - conditional: Optional[bool] = None, - ) -> Tuple[ - Tuple[Optional[ControlVariate], ...], - Optional[ConditionalRun], - Optional[_Breakdown], - ]: - """The means the result of a plain run takes by default, for its - simulations ``ranges`` in ``tally`` (chunks merged, see - ``availability_from_chunks``), as ``availability`` takes them: the - exact methods', where they work them out (#187, see - ``_exact_controls``), or given the histories of its modules, - simulated again from the run's ``entropy`` (#189, see - ``_conditioned_run``), with the expected cost's split found so; - else none. ``control_variate=False`` takes neither, and - ``conditional=False`` not the second (#236), as ``availability`` - does.""" - T = t_simulation - if control_variate is False: - return (None, None), None, None - exact_means = self._exact_means(T, working, broken, method, state) - if exact_means is not None: - return ( - _exact_controls(tally, T, exact_means, antithetic), - None, - exact_means.breakdown, - ) - if conditional is False: - return (None, None), None, None - modules = self._conditional_applies(working, broken, state) - if not modules: - return (None, None), None, None - run = _ModuleRun( + return _runs.availability_from_chunks( self, - T, - working, - broken, - method, - entropy, - antithetic=antithetic, - n_jobs=None, - engine="auto", - curve_points=1, - state=state, - modules=modules, + chunks=chunks, + mc_samples=mc_samples, + allow_gaps=allow_gaps, + control_variate=control_variate, + conditional=conditional, ) - try: - run.prepare() - except NotImplementedError: - return (None, None), None, None - for start, stop in ranges: - run.simulate(start, stop - start) - record = run.means() - return (None, None), record, run.split(record) - - def _fingerprint(self) -> Optional[str]: - """A hash of this system saved as JSON (with the RePyability - version), which chunks of its runs carry; None if it cannot be - saved.""" - try: - text = json.dumps(self.to_dict(), sort_keys=True) - except Exception: - return None - return hashlib.sha256(text.encode()).hexdigest() def compare( self, @@ -17412,8 +4708,8 @@ def compare( where the exact methods work out both systems' expected values, and otherwise by simulation with common random numbers. - By default, as ``availability`` and ``cost`` take their means (#187, - #236), where the exact methods work out both systems' expected + By default, as ``availability`` and ``cost`` take their means, + where the exact methods work out both systems' expected values over the window (``mission_availability``, and ``expected_cost`` their costs), the difference is theirs, exact, with no error (``method="exact"``), and nothing is simulated. @@ -17447,8 +4743,8 @@ def compare( quantity : str, optional ``"availability"`` (the default): the fraction of the window the system is up. ``"cost"``: what owning it for the window from new - costs, its running cost and its components' ``acquisition_cost`` - (#234), as ``total_cost`` counts them (both systems must be + costs, its running cost and its components' ``acquisition_cost``, + as ``total_cost`` counts them (both systems must be priced; a cost given as a distribution draws the same numbers in both too, from a stream of its own). confidence : float, optional @@ -17483,8 +4779,8 @@ def compare( ------ ValueError If ``quantity``, ``mc_samples``, ``confidence``, ``n_jobs`` or - ``engine`` is invalid, or a system to compare by cost has no - costs. + ``engine`` is invalid, or (as ``routes.Refused``) a system to + compare by cost has no costs. NotImplementedError If a component's draws cannot be replayed from a stream of its own (a model sampled its own way), or, with @@ -17529,548 +4825,20 @@ def compare( Two independent runs of 2000 simulations would estimate it with a standard error of about 0.00058. """ - N = 10_000 if mc_samples is None else mc_samples - t_simulation = _check_window(t_simulation) - - if quantity not in ("availability", "cost"): - raise ValueError( - "quantity must be 'availability' or 'cost', got " - f"{quantity!r}." - ) - montecarlo.check_confidence(confidence) - montecarlo.check_count(N, False, "mc_samples") - if control_variate not in (None, False): - raise ValueError( - "compare takes control_variate=None (the exact difference " - "where the exact methods work out both systems' expected " - "values) or False (the simulated difference), got " - f"{control_variate!r}: its common random numbers are its " - "own variance reduction." - ) - if quantity == "cost": - for rbd in (self, other): - if not (rbd.has_costs or rbd.acquisition_cost): - raise ValueError( - "Both systems must be priced to compare their costs." - ) - if control_variate is None: - exact = self._exact_difference( - other, t_simulation, quantity, state - ) - if exact is not None: - return ConfidenceInterval( - estimate=exact, - lower=exact, - upper=exact, - confidence=confidence, - standard_error=0.0, - n_samples=0, - method="exact", - ) - jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) - states = [rbd._simulation_states(state) for rbd in (self, other)] - entropy = _streams.entropy_of(seed) - values = [] - for rbd, start in zip((self, other), states): - tally = rbd._run( - t_simulation, - set(), - set(), - "p", - N, - False, - None, - jobs=jobs, - engine=engine, - entropy=entropy, - common=True, - states=start, - ) - if quantity == "cost": - running = ( - np.asarray(tally.cost_samples, dtype=float) - if rbd.has_costs - else np.zeros(N) - ) - values.append(running + rbd.acquisition_cost) - else: - values.append(np.asarray(tally.uptimes) / t_simulation) - differences = values[0] - values[1] - estimate = float(np.mean(differences)) - standard_error = montecarlo.standard_error(differences, False) - z = montecarlo.z_value(confidence) - return ConfidenceInterval( - estimate=estimate, - lower=estimate - z * standard_error, - upper=estimate + z * standard_error, + return _runs.compare( + self, + other=other, + t_simulation=t_simulation, + mc_samples=mc_samples, + seed=seed, + quantity=quantity, confidence=confidence, - standard_error=standard_error, - n_samples=N, - method="simulated", - ) - - def _exact_difference( - self, - other: "RepairableRBD", - t_simulation: float, - quantity: str, - state, - ) -> Optional[float]: - """``compare``'s difference worked out exactly (#236): this - system's expected value of ``quantity`` over the window less - ``other``'s, where the exact methods work out both (see - ``_exact_means``); else None.""" - values = [] - for rbd in (self, other): - exacts = rbd._exact_means(t_simulation, set(), set(), "p", state) - if exacts is None: - return None - if quantity == "availability": - values.append(exacts.availability) - continue - if exacts.cost is not None: - running = exacts.cost - elif rbd.has_costs: - return None - else: - running = 0.0 - values.append(running + rbd.acquisition_cost) - return float(values[0] - values[1]) - - def _twin(self) -> Tuple["RepairableRBD", List[str]]: - """This system's exact twin (#154), and what it changes, in words. - - The twin has this system's diagram and components, with their - models, failing and repaired independently: without the limit on - repair crews or the maintenance groups, which tie components - together, and, component by component, without what the exact - methods over time do not take (see ``_twin_simpler``). Its expected - values over a window are then exact (``mission_availability``, - ``expected_cost``), and each of its streams is named as this - system's is (a standby group's units as the twin's nested units), - so that, simulated with common random numbers, it follows this - system closely: see ``availability``'s ``control_variate``. - - Raises - ------ - NotImplementedError - If no such twin has exact values over time: a component's model - is a probability, or a life is simulated. - """ - from repyability.rbd import routes as r - - args = self._init_args - changes = [] - if self._crews_limited(): - changes.append("the limit on repair crews") - if self._maintenance: - changes.append("the maintenance groups") - specs: Dict[Any, Any] = {} - for node, value in args["components"].items(): - if isinstance(value, RepairableRBD): - nested, inner = value._twin() - specs[node] = nested - changes.extend(f"{what} in {node!r}" for what in inner) - elif isinstance(value, dict): - spec = { - key: item - for key, item in value.items() - if key not in ("priority", "group") - } - preventive = value.get("preventive") - if preventive and "opportunity" in preventive: - # Opportunities come from a group's stops. - spec["preventive"] = { - key: item - for key, item in preventive.items() - if key != "opportunity" - } - specs[node] = spec - else: - specs[node] = value - twin = self._twin_from(specs) - simpler = {} - for node, spec in specs.items(): - if isinstance(spec, dict): - simpler[node], change = twin._twin_simpler(node, spec) - if change: - changes.append(change) - if any(simpler[node] is not specs[node] for node in simpler): - twin = self._twin_from({**specs, **simpler}) - if twin.ccf_groups and r.refusal( - partial(twin._require_groups_over_time, {}) - ): - # Its groups' chains cannot follow them over time (#158): its - # members fail independently. - twin = self._twin_from({**specs, **simpler}, groups=False) - changes.append("the common-cause groups") - route = twin._over_time() - if route.route == r.REFUSED: - raise NotImplementedError( - "This system has no exact twin to control its simulation " - f"by: {route.reason}" - ) - if route.route == r.SIMULATED: - raise NotImplementedError( - "This system has no exact twin to control its simulation " - "by: its components' values over time are simulated " - f"({route.reason})" - ) - return twin, changes - - def _twin_report(self) -> str: - """What ``analysis_routes`` says of the exact twin a run with - ``control_variate`` is controlled by (see ``AnalysisRoute.twin``): - what it leaves out of this system, or why there is none.""" - try: - _, changes = self._twin() - except NotImplementedError as error: - return f"none. {error}" - _, complete = self._stream_specs(1.0) - if not complete: - return f"none. {_UNSTREAMED}" - if not changes: - return ( - "the system itself, whose values over the window are exact " - "(mission_availability, expected_cost)." - ) - return f"the system without {', '.join(changes)}." - - def _twin_from( - self, components: dict, groups: bool = True - ) -> "RepairableRBD": - """A system of this one's diagram with ``components`` (see - ``_twin``), and its common-cause groups unless not ``groups``.""" - args = self._init_args - return RepairableRBD( - args["edges"], - components, - k=args["k"], - input_node=args["input_node"], - output_node=args["output_node"], - on_infeasible_rbd=args["on_infeasible_rbd"], - downtime_cost_rate=args["downtime_cost_rate"], - ccf_groups=args.get("ccf_groups") if groups else None, - ) - - def _twin_simpler(self, node, spec: dict) -> Tuple[Any, str]: - """A component's spec in an exact twin (see ``_twin``), this system - being the twin so far, and what it changes (or ""): a standby group - whose own chain cannot follow it over time becomes its units - operating together; imperfect repair becomes perfect (but for - minimal repair in no time, whose values over time are exact); - replacement on condition, inspections and block replacement that - the exact methods do not take go.""" - from repyability.rbd import routes as r - - if node in self._standby: - if r.refusal(partial(self._standby_rates, node)): - return ( - _hot_units(spec), - f"the switching of standby group {node!r} (its units " - "operate together)", - ) - return spec, "" - drop: Dict[str, str] = {} - if node in self._imperfect and r.refusal( - partial(self._require_minimal_repair, node) - ): - drop["repair"] = drop["replace_after"] = "imperfect repair" - if node in self._inspection and r.refusal( - partial(self._require_tested_exact, node) - ): - drop["inspection"] = "inspections" - schedule = self._preventive.get(node) - if ( - schedule is not None - and schedule.policy in ("block", "condition") - and r.refusal(partial(self._require_block_models, node)) - ): - drop["preventive"] = ( - "block replacement" - if schedule.policy == "block" - else "replacement on condition" - ) - if not drop: - return spec, "" - what = list(dict.fromkeys(drop[key] for key in drop if key in spec)) - note = " (its failures are revealed)" if "inspection" in drop else "" - return ( - {key: value for key, value in spec.items() if key not in drop}, - f"the {' and '.join(what)} of {node!r}{note}", - ) - - def _simulated( - self, - t_simulation: float, - working_nodes, - broken_nodes, - method: str, - N: int, - verbose: bool, - seed, - *, - tolerance: Optional[float], - confidence: float, - max_N: Optional[int], - antithetic: bool, - n_jobs: Optional[int], - target: str, - demand: Optional[float] = None, - engine: str = "auto", - state=None, - curve_points: Optional[int] = None, - shard_map: Optional[Callable] = None, - shard_size: Optional[int] = None, - control_variate: Optional[bool] = None, - conditional: Optional[bool] = None, - ) -> AvailabilityResult: - """``availability`` (and ``cost``): validate, run the replications - (serially, in parallel, or as shards through ``shard_map``, until - converged if asked; with ``control_variate``, alongside the system's - exact twin; with ``conditional``, of the dependent modules alone, - see ``_conditional_run``) and build the result, its curve on a grid - of ``curve_points`` steps if given. By default its expected values - are exact where the system is its own exact twin (#187), and taken - given the modules' histories where a conditional run applies - (#189, see ``_conditioned_run``).""" - t_simulation = _check_window(t_simulation) - working_nodes = set() if working_nodes is None else set(working_nodes) - broken_nodes = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working_nodes, broken_nodes) - states = self._simulation_states(state, working_nodes | broken_nodes) - # The initial system state with every component up but those forced - # broken, which can make the system start down; a simulation that - # starts down from the components' states records the change at 0 - # (see _replicate). - initial_status = {c: c not in broken_nodes for c in self.components} - initial_up = bool(self.is_system_working(initial_status, method)) - montecarlo.check_count(N, antithetic, "mc_samples") - stop = _stopping_rule( - N, tolerance, confidence, max_N, antithetic, target, t_simulation - ) - _check_shard_map(shard_map, shard_size, n_jobs) - if control_variate is not None and not isinstance( - control_variate, (bool, np.bool_) - ): - raise ValueError( - f"control_variate must be True or False, got " - f"{control_variate!r}." - ) - if conditional is not None and not isinstance( - conditional, (bool, np.bool_) - ): - raise ValueError( - f"conditional must be True, False or None, got " - f"{conditional!r}." - ) - if conditional: - if demand is not None and not self._has_capacity(): - raise ValueError( - "A demand is measured against capacities, and no node " - "has one: give them with capacity={node: capacity}." - ) - return self._conditional_run( - t_simulation, - working_nodes, - broken_nodes, - method, - N, - seed, - stop=stop, - antithetic=antithetic, - n_jobs=n_jobs, - engine=engine, - curve_points=curve_points if target == "availability" else 1, - target=target, - state=state, - shard_map=shard_map, - shard_size=shard_size, - demand=demand, - capacities=target == "availability" and self._has_capacity(), - control_variate=bool(control_variate), - ) - # By default the expected values are the exact methods', where they - # work them out (#187): a run to a tolerance stops at once. - auto = control_variate is None - exact_means = ( - self._exact_means( - t_simulation, working_nodes, broken_nodes, method, state - ) - if auto - else None - ) - twin = None - changes: List[str] = [] - exacts: Optional[_Exacts] = None - if control_variate: - twin, changes = self._twin() - if shard_map is not None and changes: - raise ValueError( - "A run with control_variate simulates the system's twin " - "alongside it, here: leave out shard_map." - ) - exacts = self._twin_exact( - twin, t_simulation, working_nodes, broken_nodes, method, state - ) - # Otherwise, by default, its means are taken given the histories of - # its modules, where a conditional run applies (#189). - modules = ( - self._conditional_applies(working_nodes, broken_nodes, state) - if auto and exact_means is None and conditional is None - else [] - ) - capacity = None - # Shards follow the capacities whatever the target, as chunks do. - if ( - target == "availability" or shard_map is not None - ) and self._has_capacity(): - capacity = _CapacityRecorder(self, demand) - elif demand is not None: - raise ValueError( - "A demand is measured against capacities, and no node has " - "one: give them with capacity={node: capacity}." - ) - entropy = sharded = None - if shard_map is not None: - self._require_groups_simulated() - entropy = _streams.entropy_of(seed) - template = self._shard_template( - t_simulation, - entropy, - working_nodes, - broken_nodes, - method, - antithetic, - demand, - engine, - states, - curve_points, - ) - plan, _ = self._stream_plan( - t_simulation, entropy, antithetic, states - ) - step = self._shard_size(plan, shard_size) - sharded = (shard_map, template, step) - jobs = None if n_jobs is None else montecarlo.jobs(n_jobs) - controls: Tuple[Optional[ControlVariate], ...] = (None, None) - # The expected cost's split, where the means are exact (#223). - breakdown: Optional[_Breakdown] = None - if modules: - done = self._conditioned_run( - modules, - t_simulation, - working_nodes, - broken_nodes, - method, - N, - verbose, - seed, - antithetic, - stop, - capacity=capacity, - jobs=jobs, - n_jobs=n_jobs, - engine=engine, - state=state, - states=states, - curve_points=curve_points, - target=target, - sharded=sharded, - shard_map=shard_map, - shard_size=shard_size, - ) - if done is not None: - tally, record, breakdown = done - return self._availability_result( - tally, - t_simulation, - initial_up, - antithetic, - capacity, - conditional=record, - breakdown=breakdown, - ) - if twin is not None: - assert exacts is not None - tally, controls = self._controlled_run( - twin, - exacts, - t_simulation, - working_nodes, - broken_nodes, - method, - N, - verbose, - seed, - antithetic, - stop, - capacity=capacity, - jobs=jobs, - engine=engine, - state=state, - states=states, - curve_points=curve_points, - target=target, - itself=not changes, - sharded=sharded, - ) - if not changes: - # Its means are the system's exact ones (#223). - breakdown = exacts.breakdown - else: - tally = self._run( - t_simulation, - working_nodes, - broken_nodes, - method, - N, - verbose, - seed, - antithetic, - # Exact means need no more simulations. - None if exact_means is not None else stop, - capacity=capacity, - jobs=jobs, - engine=engine, - entropy=entropy, - states=states, - curve_points=curve_points, - sharded=sharded, - ) - if exact_means is not None: - controls = _exact_controls( - tally, t_simulation, exact_means, antithetic - ) - breakdown = exact_means.breakdown - return self._availability_result( - tally, - t_simulation, - initial_up, - antithetic, - capacity, - controls, - breakdown=breakdown, + n_jobs=n_jobs, + engine=engine, + state=state, + control_variate=control_variate, ) - def _exact_means( - self, - t_simulation: float, - working: set, - broken: set, - method: str, - state, - ) -> Optional[_Exacts]: - """The system's own expected values over the window, where the - exact methods work them out (#187): its mean availability and, if - it is priced, its expected cost (see ``_twin_exact``); else None.""" - try: - return self._twin_exact( - self, t_simulation, working, broken, method, state - ) - except NotImplementedError: - return None - def _conditional_modules(self, working: set, broken: set) -> list: """The nodes a conditional run simulates (#189, see ``_conditional``), in the order of the components: those whose @@ -18085,7 +4853,7 @@ def _conditional_modules(self, working: set, broken: set) -> list: node for node in self.components if node not in held - and self._node_over_time(node, "window")[0] == r.REFUSED + and _curves._node_over_time(self, node, "window")[0] == r.REFUSED } if self._crews_couple(): # A job can wait for a crew behind any other: every node the @@ -18093,7 +4861,7 @@ def _conditional_modules(self, working: set, broken: set) -> list: # (nested RBDs, with crews of their own) taken exactly. chosen |= { key.node if isinstance(key, _Unit) else key - for key in self._crew_served() + for key in _crews._crew_served(self) } - held for name, spec in self._maintenance.items(): members = set(spec.members) @@ -18103,1287 +4871,74 @@ def _conditional_modules(self, working: set, broken: set) -> list: if caught: raise NotImplementedError( f"Maintenance group {name!r} is simulated in a " - f"conditional run, and its member(s) " - f"{sorted(caught, key=str)} held: hold none of its " - "members, or run plainly (conditional=False)." - ) - if spec.system_down: - raise NotImplementedError( - f"Maintenance group {name!r} stops at every outage of " - "the system (system_down), which ties its members to " - "every component: a conditional run would simulate them " - "all. Run it plainly (conditional=False)." - ) - chosen |= members - for group in self.ccf_groups: - caught = set(group.members) & chosen - if caught: - raise NotImplementedError( - f"Common-cause group {list(group.members)}: member(s) " - f"{sorted(caught, key=str)} would be simulated in a " - "conditional run apart from the group's causes, which it " - "takes exactly: run it plainly (conditional=False)." - ) - if len(chosen) > 62: - raise NotImplementedError( - f"A conditional run would simulate {len(chosen)} nodes, more " - "than the 62 it follows the joint states of: run it plainly " - "(conditional=False)." - ) - if chosen and not set(self.components) - held - chosen: - raise NotImplementedError( - "A conditional run would simulate every component (the " - "nodes the exact methods over time do not take, and those " - "they are tied to by repair crews or maintenance groups), " - "leaving none to take exactly given them: run it plainly " - "(conditional=False)." - ) - return [node for node in self.components if node in chosen] - - def _conditional_applies( - self, working: Collection = (), broken: Collection = (), state=None - ) -> list: - """The modules a conditional run would simulate (see - ``_conditional_modules``) with the nodes ``working`` and ``broken`` - held and from the components' ``state``, or none if it would - refuse: the others must be taken exactly given them, which a - structure too meshed for its decision diagram, or common-cause - groups the exact methods over time refuse, prevent.""" - from repyability.rbd import routes as r - - if self._too_meshed() is not None: - return [] - if self.ccf_groups and r.refusal( - partial(self._require_groups_over_time, state or {}) - ): - return [] - try: - return self._conditional_modules(set(working), set(broken)) - except NotImplementedError: - return [] - - def _modules_rbd(self, modules: list) -> "RepairableRBD": - """The ``modules`` of a conditional run (see - ``_conditional_modules``) as a diagram of their own, each between - the input and output nodes with its own spec: their streams are - named as in this system, so its simulations are this system's - modules'. It has this system's repair crews (whose jobs are the - modules' alone: crews that tie the components together make every - node they serve a module), the maintenance groups of its members, - and no system downtime cost.""" - args = self._init_args - options = args["maintenance_groups"] or {} - groups = { - name: spec - for name, spec in options.items() - if set(self._maintenance[name].members) <= set(modules) - } - return RepairableRBD( - [(self.input_node, node) for node in modules] - + [(node, self.output_node) for node in modules], - {node: args["components"][node] for node in modules}, - input_node=self.input_node, - output_node=self.output_node, - repair_crews=args["repair_crews"], - maintenance_groups=groups or None, - ) - - def _capacity_given( - self, - modules: list, - state: int, - working: set, - broken: set, - x: np.ndarray, - others, - demand: Optional[float], - ) -> "_conditional.CapacityGiven": - """The system's capacity given the modules in one joint ``state`` - (see ``_conditional_given``), on the grid ``x``: from its exact - distribution over time with the modules held so (a module up at its - levels by their probabilities, as the simulation weighs them), and - the other nodes from their states ``others``, against ``demand`` - (see ``_conditional.capacity_given``).""" - up = {node for j, node in enumerate(modules) if state >> j & 1} - down = set(modules) - up - distribution = self.point_capacity( - x, working | up, broken | down, state=others - ) - return _conditional.capacity_given( - x, distribution.levels, distribution.probabilities, demand - ) - - def _up_at_start(self, node, start) -> bool: - """Whether ``node`` is up just before 0 from its checked ``start`` - (see ``_states``; None, new): a component unless it starts down, a - nested RBD as its system, from its components'.""" - if start is None: - return True - component = self.components[node] - if isinstance(component, RepairableRBD): - status = { - inner: component._up_at_start(inner, start.get(inner)) - for inner in component.components - } - return bool(component.is_system_working(status, "p")) - return bool(start.alive) - - def _crew_free(self) -> "RepairableRBD": - """This system without its limit on repair crews: where every node - the crews serve is held (a conditional run's modules), they tie - nothing together, and the rest is worked out as if they were - unlimited (a nested RBD keeps its own).""" - if not self._crews_limited(): - return self - return RepairableRBD(**{**self._init_args, "repair_crews": None}) - - def _conditional_given( - self, - modules: list, - state: int, - working: set, - broken: set, - method: str, - x: np.ndarray, - others=None, - ) -> "_conditional.Given": - """The system given the modules in one joint ``state`` (bit ``j`` - set while module ``j`` is up), on the grid ``x``: with the modules - held so, and the other nodes from their states ``others`` (new by - default), its expected up time, failures and planned outages before - each time, and its point availability (see - ``_conditional.Given``).""" - up = {node for j, node in enumerate(modules) if state >> j & 1} - down = set(modules) - up - _, _, _, counts, _, _ = self._window( - x, working | up, broken | down, method, state=others - ) - available = self.point_availability( - x, working | up, broken | down, method, state=others - ) - # Just before 0, every other node is up, but one started down. - started = self._states(others, working | broken | up | down) - status = { - node: node not in broken | down - and self._up_at_start(node, started.get(node)) - for node in self.components - } - return _conditional.Given( - x, - np.asarray(counts["uptime"], dtype=float), - np.asarray(counts["failures"], dtype=float), - np.asarray(counts["planned"], dtype=float), - np.asarray(available, dtype=float), - float(bool(self.is_system_working(status, method))), - ) - - def _conditional_run( - self, - t_simulation: float, - working: set, - broken: set, - method: str, - N: int, - seed, - *, - stop: Optional[Callable[..., int]], - antithetic: bool, - n_jobs: Optional[int], - engine: str, - curve_points: Optional[int], - target: str, - state=None, - shard_map: Optional[Callable] = None, - shard_size: Optional[int] = None, - demand: Optional[float] = None, - capacities: bool = False, - control_variate: bool = False, - ) -> AvailabilityResult: - """``availability`` (and ``cost``) with ``conditional=True`` (#189, - see ``_conditional`` and ``_ModuleRun``): simulate the modules - alone, in rounds while ``stop`` asks for more, and take every other - node exactly given their joint states. From the components' - ``state``, the modules start from theirs and the others from - theirs. With ``shard_map``, the modules' simulations run as shards - (see ``shards``) wherever it sends them. With ``capacities``, the - system's capacity given the modules' states too, against - ``demand``. With ``control_variate``, the exact twin's stand-ins for - the modules (see ``_twin``) are simulated alongside them, with - common random numbers, and each simulation's expected values given - theirs, whose mean is the twin's exact one, control the run's (see - ``ControlVariate``).""" - run = _ModuleRun( - self, - t_simulation, - working, - broken, - method, - _streams.entropy_of(seed), - antithetic=antithetic, - n_jobs=n_jobs, - engine=engine, - curve_points=curve_points, - state=state, - shard_map=shard_map, - shard_size=shard_size, - demand=demand, - capacities=capacities, - control_variate=control_variate, - ) - run.prepare() - first, count = 0, N - while True: - run.simulate(first, count) - first += count - if stop is None: - break - # A run to a tolerance judges the mean asked for (see - # _stopping_rule), controlled if it is: not while the modules - # have not changed state (#215). - count = stop(None, run.judged(target), run.unjudged()) - if not count: - break - return run.result() - - def _conditioned_run( - self, - modules: list, - t_simulation: float, - working: set, - broken: set, - method: str, - N: int, - verbose: bool, - seed, - antithetic: bool, - stop: Optional[Callable[..., int]], - *, - capacity: Optional[_CapacityRecorder], - jobs: Optional[int], - n_jobs: Optional[int], - engine: str, - state, - states: dict, - curve_points: Optional[int], - target: str, - sharded: Optional[tuple], - shard_map: Optional[Callable], - shard_size: Optional[int], - ) -> Optional[Tuple["_Tally", ConditionalRun, Optional[_Breakdown]]]: - """A plain run whose means are taken given its ``modules``' - histories (#189), by default where a conditional run applies: the - simulations, in rounds while ``stop`` asks for more (judged by - those means), and of the same simulations each one's expected up - time and cost given the histories of its modules, worked out as a - conditional run works them out (see ``_ModuleRun``): the modules' - simulations are the plain run's, drawn from the same streams. The - simulations' totals, a record holding those values, and the - expected cost's split given the modules (None unpriced, or while - they have not changed state); or None, before simulating anything, - if the rest cannot be worked out exactly given the modules.""" - entropy = _streams.entropy_of(seed) - run = _ModuleRun( - self, - t_simulation, - working, - broken, - method, - entropy, - antithetic=antithetic, - n_jobs=n_jobs, - # The modules' histories are the event loop's, on every engine. - engine=engine if engine in ("python", "numba") else "auto", - curve_points=1, - state=state, - shard_map=shard_map, - shard_size=shard_size, - modules=modules, - ) - try: - run.prepare() - except NotImplementedError: - return None - tally: Optional[_Tally] = None - first, count = 0, N - while True: - part = self._run( - t_simulation, - working, - broken, - method, - count, - verbose, - None, - antithetic, - capacity=capacity, - jobs=jobs, - engine=engine, - entropy=entropy, - first=first, - states=states, - curve_points=curve_points, - sharded=sharded, - ) - if tally is None: - tally = part - else: - tally.merge(part) - run.simulate(first, count) - first += count - if stop is None: - break - # Its means given the modules, or the simulations' own while - # the modules have not changed state (#215). - more = stop(tally, None if run.unjudged() else run.judged(target)) - if not more: - break - count = more - assert tally is not None - record = run.means() - return tally, record, run.split(record) - - def _twin_exact( - self, - twin: "RepairableRBD", - t_simulation: float, - working: set, - broken: set, - method: str, - state, - ) -> _Exacts: - """The exact ``twin``'s (see ``_twin``) mean availability over the - window and, if both systems are priced, its expected cost, by - category and by component (else None), building each component's - curve once for both (#185).""" - priced = self.has_costs and twin.has_costs - # The cost's curves, which count events, serve the availability too. - with twin._sharing_curves(): - expected = ( - twin.expected_cost( - t_simulation, working, broken, method, state=state - ) - if priced - else None - ) - exact = float( - np.ravel( - twin.mission_availability( - t_simulation, working, broken, method, state=state - ) - )[0] - ) - if expected is None: - return _Exacts(exact) - - def first(value) -> float: - return float(np.ravel(value)[0]) - - return _Exacts( - exact, - first(expected.mean), - ( - {key: first(v) for key, v in expected.by_category.items()}, - {node: first(v) for node, v in expected.by_component.items()}, - ), - ) - - def _controlled_run( - self, - twin: "RepairableRBD", - exacts: _Exacts, - t_simulation: float, - working: set, - broken: set, - method: str, - N: int, - verbose: bool, - seed, - antithetic: bool, - stop: Optional[Callable[..., int]], - *, - capacity: Optional[_CapacityRecorder], - jobs: Optional[int], - engine: str, - state, - states: dict, - curve_points: Optional[int], - target: str, - itself: bool = False, - sharded: Optional[tuple] = None, - ) -> Tuple["_Tally", Tuple[Optional[ControlVariate], ...]]: - """Run this system and its exact ``twin`` (see ``_twin``) with - common random numbers, as ``compare`` does, in rounds while - ``stop`` asks for more, judged by the controlled values; return - this system's totals and the controls of its fractions up and of - its costs (None without costs) by the twin's (see - ``ControlVariate``), ``itself`` if the twin is this system: by its - exact values ``exacts``, its mean availability and its expected - cost (None unpriced; see ``_twin_exact``). A twin that is this - system is not simulated again, so its run can be ``sharded`` (see - ``_run``).""" - assert sharded is None or itself - twin_states = twin._simulation_states(state, working | broken) - exact, exact_cost = exacts.availability, exacts.cost - entropy = _streams.entropy_of(seed) - tally: Optional[_Tally] = None - twin_tally: Optional[_Tally] = None - first, count = 0, N - while True: - part = self._run( - t_simulation, - working, - broken, - method, - count, - verbose, - None, - antithetic, - capacity=capacity, - jobs=jobs, - engine=engine, - entropy=entropy, - common=True, - first=first, - states=states, - curve_points=curve_points, - sharded=sharded, - ) - # A twin that is this system, drawn from the same streams, runs - # as this system does, to the last bit: its run is this one. - twin_part = ( - None - if itself - else twin._run( - t_simulation, - working, - broken, - method, - count, - False, - None, - antithetic, - jobs=jobs, - engine=engine, - entropy=entropy, - common=True, - first=first, - states=twin_states, - curve_points=1, - ) - ) - if tally is None: - tally, twin_tally = part, twin_part - else: - tally.merge(part) - if twin_tally is not None and twin_part is not None: - twin_tally.merge(twin_part) - twins = tally if twin_tally is None else twin_tally - fractions = np.asarray(tally.uptimes, dtype=float) / t_simulation - control = ControlVariate.of( - fractions, - np.asarray(twins.uptimes, dtype=float) / t_simulation, - exact, - antithetic, - itself=itself, - ) - cost_control = None - if exact_cost is not None: - cost_control = ControlVariate.of( - tally.cost_samples, - twins.cost_samples, - exact_cost, - antithetic, - itself=itself, - ) - if stop is None: - break - if target == "cost": - values = ( - tally.cost_samples - if cost_control is None - else cost_control.controlled(tally.cost_samples) - ) - else: - values = control.controlled(fractions) - more = stop(tally, values) - if not more: - break - first, count = first + count, more - return tally, (control, cost_control) - - def _run(self, *args, **kwargs) -> "_Tally": - """A run of simulations (see ``_run_alone``), one at a time in the - process (``SIMULATIONS``, #216): the event loop keeps its state on - the diagram, and draws that cannot be streamed come from numpy's - global RNG. Not a ``sharded`` run, whose shards each run (and take - it) where they are sent: a ``shard_map`` on threads would wait for - it.""" - if kwargs.get("sharded") is not None: - return self._run_alone(*args, **kwargs) - with SIMULATIONS: - return self._run_alone(*args, **kwargs) - - def _run_alone( - self, - t_simulation: float, - working_nodes, - broken_nodes, - method: str, - N: int, - verbose: bool, - seed, - antithetic: bool = False, - stop: Optional[Callable[["_Tally"], int]] = None, - capacity: Optional[_CapacityRecorder] = None, - jobs: Optional[int] = None, - engine: str = "auto", - entropy: Any = None, - common: bool = False, - replacements: bool = False, - first: int = 0, - states: Optional[dict] = None, - curve_points: Optional[int] = None, - sharded: Optional[tuple] = None, - histories: bool = False, - ) -> "_Tally": - """Run ``N`` simulations, from simulation ``first`` (then more, - while ``stop`` asks for them), and return their totals: each from - the components' ``states`` (checked; see ``_simulation_states``), - new by default. - - Every draw comes from a stream of its own (see ``_streams``), seeded - from the run's ``entropy``: the ``seed``, or without one a number - drawn from numpy's global RNG. Draws that cannot be streamed come - from the global RNG, seeded afresh for each simulation. So each - simulation is the same whichever ``engine`` runs it, in however many - processes or threads (``jobs``), and in a run to a tolerance as in a - run of its final size. The global RNG is left as it was, except for - the number drawn for the entropy. ``compare`` passes both systems - the same ``entropy``, and ``common``: that, like - ``antithetic``, needs every draw to come from a stream. With - ``capacity``, each simulation also follows the system's capacity; - with ``replacements``, the tally keeps each simulation's - replacements of each component, which only the Python engine - counts; with ``curve_points``, it counts the changes of state on a - grid (see ``availability``). With ``sharded``, ``(shard_map, - template, step)``, the simulations run as shards (see - ``_ShardRunner``), on the engine each shard's worker chooses. With - ``histories``, the tally keeps each simulation's histories in place - of every total (see ``simulate_timelines``), on the Python engine - or numba's. - """ - self._require_free_members(working_nodes, broken_nodes) - self._require_groups_simulated(states) - t_simulation = _check_window(t_simulation) - from tqdm import tqdm - - state = np.random.get_state() - after = state - tally = _Tally( - list(self.components), self.costs, t_simulation, curve_points - ) - if replacements: - # Only the Python engine counts them. - tally.replacements = [] - engine = "python" - if histories: - tally.histories = _timeline_runs.Records(len(self.components)) - runner: Any = None - try: - if entropy is None: - entropy = _streams.entropy_of(seed) - if seed is None: - after = np.random.get_state() - plan, complete = self._stream_plan( - t_simulation, entropy, antithetic, states - ) - if (antithetic or common) and not complete: - raise NotImplementedError(_UNSTREAMED) - engine = self._simulation_engine( - engine, plan, capacity, N, here=sharded is None, states=states - ) - progress = tqdm( - total=N, disable=not verbose, desc="Running simulations" - ) - if sharded is not None: - runner = _ShardRunner(self, tally, progress, *sharded) - elif engine == "numba": - from repyability.rbd import _compiled - - runner = _compiled.Runner( - self, - plan, - tally, - progress, - working_nodes, - broken_nodes, - method, - jobs, - capacity=capacity, - ) - elif engine != "python": - from repyability.rbd import engines - - runner = engines.registered()[engine].runner( - self, - plan, - tally, - progress, - working_nodes, - broken_nodes, - method, - jobs, - ) - else: - runner = _PythonRunner( - self, - tally, - progress, - ( - t_simulation, - working_nodes, - broken_nodes, - method, - capacity, - entropy, - antithetic, - states, - histories, - ), - jobs, - ) - start, goal = first, first + N - while True: - runner(start, goal) - more = 0 if stop is None else stop(tally) - if not more: - break - start, goal = goal, goal + more - progress.total = goal - progress.refresh() - progress.close() - finally: - if runner is not None: - runner.close() - if sharded is None: - # A sharded run draws nothing here, and another thread's - # run may have the global RNG now. - np.random.set_state(after) - self._forget_run() - return tally - - #: The interim variables of a simulation (see _forget_run). - _RUN_STATE = ( - "_event_queue", - "system_state", - "t_simulation", - "component_status", - "last_change_planned", - "_pending_failure", - "_in_service", - "_renewed_at", - "_pending_event", - "_cancelled", - "_early", - "_renewing", - "_crews", - "_groups", - "_step_sources", - "_phases", - ) - - def _forget_run(self) -> None: - """Clean up the interim variables of a simulation, this RBD's and - its nested RBDs': left behind, a nested standby group's draws (whose - samplers cannot be pickled) would keep the RBD from going to the - processes of a later parallel run.""" - for name in self._RUN_STATE: - self.__dict__.pop(name, None) - for component in self.components.values(): - if isinstance(component, RepairableRBD): - component._forget_run() - - def _simulation_engine( - self, - engine: str, - plan: _streams.Plan, - capacity: Optional[_CapacityRecorder], - N: int, - here: bool = True, - states: Optional[dict] = None, - ) -> str: - """The engine that runs a simulation: ``"python"``, ``"numba"`` or - one another package adds (see ``availability``'s ``engine``), for a - run from the components' ``states``. Not ``here`` (the simulations - run as shards elsewhere), only whether it can simulate the system - is checked, and ``engine`` is kept, for each shard's worker to - choose by.""" - from repyability.rbd import _compiled, engines - - added = engines.registered() - if engine not in ("auto", "python", "numba") and engine not in added: - names = ", ".join( - repr(name) for name in ["auto", "python", "numba", *added] - ) - raise ValueError(f"engine must be one of {names}, got {engine!r}.") - if engine == "python": - return engine - if engine != "auto": - # What the engine does not simulate is refused first: installing - # it would not help. - reason = _compiled.unsupported( - self, plan, capacity, numba=engine == "numba", states=states - ) - if reason is not None: - raise NotImplementedError( - f"The compiled engine does not simulate {reason}: use " - "engine='python', or 'auto', which chooses the engine " - "that can." - ) - if here and engine == "numba": - _compiled.require() - elif here and not added[engine].available(): - raise ImportError( - f"The {engine!r} simulation engine cannot run here." - ) - return _compiled.ready(engine, auto=False) if here else engine - if not here: - return engine - name, _ = _compiled.choice(self, plan, capacity, states) - if name is not None and _compiled.worthwhile(plan, N, name): - return _compiled.ready(name, auto=True) - return "python" - - def _replicate(self, ctx: "_Context", replication: int) -> _Replication: - """Simulation ``replication`` of ``[0, t_simulation]``. - - Besides the counts, it follows each component's up time and its - overlaps with the system's as it goes: when a component changes - state, the time since its last change is added to its up time (if - it was up), and the system's up (or down) time over the same span - to the time both were up (or down), as the difference of the - system's up (or down) time so far at either end. - """ - ctx.run.begin(replication) - t_simulation = ctx.t_simulation - sources = ctx.sources - failure_charges, preventive_charges, inspection_charges = ctx.charges - self._start_queue( - t_simulation, - ctx.working, - ctx.broken, - ctx.method, - sources, - ctx.states, - ) - status = self.component_status - index, plain, works = ctx.position, ctx.plain, ctx.works - inspected = self._inspection - crews = self._crews - groups = self._groups - n = len(index) - # Each component's failures, the system failures they caused, its - # restorations and the system restorations they caused. - failed, caused_down = [0] * n, [0] * n - restored, caused_up = [0] * n, [0] * n - # Each component's replacements: the spares it used. - replaced = [0] * n - # Imperfect repair (#109): a failure repaired, not replaced, uses no - # spare and is charged only its repair. - imperfect = self._imperfect - # Opportunistic maintenance (#108): each member's early renewals. - maintenance, member_group = self._maintenance, self._member_group - cancelled, early = self._cancelled, self._early - opportunistic = [0] * n - failures = restorations = planned = 0 - changes: list = [] - deltas: list = [] - if ctx.initial_up and not self.system_state: - # Down at the start, from the components' states: the curve of - # the availability over time counts it as a change at 0. - changes.append(0.0) - deltas.append(-1) - by_category = [0.0] * len(_CATEGORIES) - by_node = dict.fromkeys(self.costs, 0.0) - rep_cost = 0.0 - - def pay(node, category: int, charges) -> float: - """The next charge of ``charges`` (nothing without any).""" - if charges is None: - return 0.0 - charge = charges.draw() - by_category[category] += charge - by_node[node] += charge - return charge - - trace = None if ctx.capacity is None else ctx.capacity.trace(status) - # With ctx.history (see simulate_timelines): each component's state - # at 0, the times of its changes, and which of them (their places - # among its changes) were planned changes down, maintenance or a - # test off line; and the system's changes: their times, the - # component whose change made each, which of its changes that was, - # and whether it was planned. - history = ctx.history - at_start = [status[node] for node in index] if history else None - times_of: list = [[] for _ in range(n)] if history else [] - planned_of: list = [[] for _ in range(n)] if history else [] - system_record: tuple = ([], [], [], []) if history else () - # The system's state, and its up and down time until ``since``, its - # last change; each component's last change, and the system's up and - # down time until then. - up = self.system_state - started_up = up - system_up = system_down = since = 0.0 - last, up_at, down_at = [0.0] * n, [0.0] * n, [0.0] * n - node_up, both_up, both_down = [0.0] * n, [0.0] * n, [0.0] * n - - # The queue's heap, worked directly (see _EventQueue). - heap = self._event_queue._heap - push, pop = heapq.heappush, heapq.heappop - - # When each group last stopped: the work started at one instant is - # one stop, with one set-up. - stopped: Dict[Hashable, float] = {} - # Common-cause groups' members, whose next events a shared cause can - # supersede (see _strike). - ccf_members, ccf_pending = self._ccf_members, self._ccf_pending - - def corrective(node, c: int) -> float: - """Charge a failure's corrective action, and count the spare a - replacement uses: an imperfect repair (see _ImperfectComponent) - uses none, and is charged only as a repair.""" - charge = 0.0 - renewed = not imperfect or node not in imperfect - renewed = renewed or sources[node].replacing - if renewed: - replaced[c] += 1 - for category, charges in failure_charges.get(node, ()): - if renewed or category == _REPAIR: - charge += pay(node, category, charges) - return charge - - def stop(group, t: float, trigger=None) -> float: - """Open a stop of maintenance ``group`` (see ``_stop``): queue - its early renewals, and return its set-up cost, charged once - per stop, if a member opened it or it renews any.""" - renewals = self._stop(group, t, trigger) - for renewal in renewals: - push(heap, (renewal.time, renewal)) - if (trigger is None and not renewals) or stopped.get(group) == t: - return 0.0 - stopped[group] = t - setup = maintenance[group].setup_cost - if setup: - by_category[6] += setup - return setup - - # No event at or after the end of the window is queued, so the - # simulation runs until the queue is empty. - while heap: - event = pop(heap)[1] - if cancelled and cancelled.get(id(event)) is event: - del cancelled[id(event)] - continue # superseded by an early renewal, or a strike - node = event.component - if node.__class__ is _Cause: - # A shared common cause strikes (#158). - for follow in self._strike(event, sources[node]): - push(heap, (follow.time, follow)) - continue - renewing_early = False - if early and early.get(id(event)) is event: - del early[id(event)] - self._renewing.discard(node) - opportunistic[index[node]] += 1 - renewing_early = True - grouped = False - if groups and node in groups: - # A standby group's own event (see _StandbyGroup): a unit - # failure, charged as a repair, or a repair's end. - group = groups[node] - if not group.holds(event): - continue # superseded by an earlier one - now, broken = group.advance(event.time) - if broken: - replaced[index[node]] += 1 - for category, charges in failure_charges.get(node, ()): - rep_cost += pay(node, category, charges) - if now == status[node]: - continue - event, grouped = Event(event.time, node, now), True - if (event.preventive or event.inspection) and ( - event.status == status[node] - ): - # No change of state: maintenance or a test in zero time of a - # working unit (renewed in place, or found working), or a - # test that finds a hidden failure, whose repair starts now. - if event.preventive: - replaced[index[node]] += 1 - rep_cost += pay(node, 2, preventive_charges.get(node)) - # Replaced on condition: at an inspection, charged too. - rep_cost += pay(node, 3, inspection_charges.get(node)) - if node in member_group and not renewing_early: - # Renewed in place on schedule: a stop of its group. - rep_cost += stop(member_group[node], event.time, node) - else: - rep_cost += pay(node, 3, inspection_charges.get(node)) - if not event.status and not event.missed: - rep_cost += corrective(node, index[node]) - follow = self._follow_up(event, sources[node]) - next_event: Optional[Event] = ( - follow - if crews is None - else self._crew_follow_up(event, follow) + f"conditional run, and its member(s) " + f"{sorted(caught, key=str)} held: hold none of its " + "members, or run plainly (conditional=False)." ) - if next_event is not None and next_event.time < t_simulation: - push(heap, (next_event.time, next_event)) - if ccf_members and node in ccf_members: - ccf_pending[node] = next_event - continue - t = event.time - c = index[node] - if up: - system_up_t, system_down_t = ( - system_up + (t - since), - system_down, + if spec.system_down: + raise NotImplementedError( + f"Maintenance group {name!r} stops at every outage of " + "the system (system_down), which ties its members to " + "every component: a conditional run would simulate them " + "all. Run it plainly (conditional=False)." ) - else: - system_up_t, system_down_t = system_up, system_down + ( - t - since + chosen |= members + for group in self.ccf_groups: + caught = set(group.members) & chosen + if caught: + raise NotImplementedError( + f"Common-cause group {list(group.members)}: member(s) " + f"{sorted(caught, key=str)} would be simulated in a " + "conditional run apart from the group's causes, which it " + "takes exactly: run it plainly (conditional=False)." ) - if status[node]: - node_up[c] += t - last[c] - both_up[c] += system_up_t - up_at[c] - else: - both_down[c] += system_down_t - down_at[c] - last[c], up_at[c], down_at[c] = t, system_up_t, system_down_t - status[node] = event.status - if trace is not None: - trace.change(t, node, event.status) - if history: - times_of[c].append(t) - if not event.status and (event.preventive or event.inspection): - planned_of[c].append(len(times_of[c]) - 1) - if event.status: - restored[c] += 1 - elif event.preventive: - # A planned outage: charged the preventive cost, and the - # inspection's if replaced on condition (a nested RBD's own - # costs are not counted). - replaced[c] += 1 - rep_cost += pay(node, 2, preventive_charges.get(node)) - rep_cost += pay(node, 3, inspection_charges.get(node)) - if node in member_group and not renewing_early: - rep_cost += stop(member_group[node], t, node) - elif event.inspection: - # A test that takes a working unit off-line. - rep_cost += pay(node, 3, inspection_charges.get(node)) - else: - failed[c] += 1 - # Repair and replace are charged per corrective action, at - # the failure that triggers it (for a hidden failure, when an - # inspection finds it; for a standby group, at each unit's). - if node not in inspected and not grouped: - if imperfect and node in imperfect: - rep_cost += corrective(node, c) - else: - replaced[c] += 1 - for category, charges in failure_charges.get(node, ()): - rep_cost += pay(node, category, charges) - if node in member_group: - rep_cost += stop(member_group[node], t, node) - - # The structure is coherent: a restoration can't take the - # system down, nor a failure bring it up, so only a change - # against the system's state needs the structure function. - if event.status != up and works(status) != up: - system_up, system_down, since = system_up_t, system_down_t, t - up = not up - changes.append(t) - if history: - system_record[0].append(t) - system_record[1].append(c) - system_record[2].append(len(times_of[c]) - 1) - system_record[3].append( - not up and bool(event.preventive or event.inspection) - ) - if up: - deltas.append(1) - restorations += 1 - caused_up[c] += 1 - else: - deltas.append(-1) - if event.preventive or event.inspection: - planned += 1 - else: - failures += 1 - caused_down[c] += 1 - for name, spec in maintenance.items(): - if spec.system_down: - # Any system outage is a stop of this group. - rep_cost += stop(name, t) - - if grouped: - continue # the group has queued its next event - # The component's next event: its next failure if it has just - # been restored, or its restoration if it has just gone down; - # queued only if it falls inside the window. - if node in plain: - delay, next_status = sources[node].next_event() - next_event = Event(t + delay, node, next_status) - else: - next_event = self._follow_up(event, sources[node]) - if crews is not None: - # A job waits for a repair crew (see _Crews). - next_event = self._crew_follow_up(event, next_event) - if next_event is not None and next_event.time < t_simulation: - push(heap, (next_event.time, next_event)) - if ccf_members and node in ccf_members: - ccf_pending[node] = next_event - self.system_state = up - - # Close the system's time, and each component's, at the end of the - # window. - if up: - system_up += t_simulation - since - else: - system_down += t_simulation - since - for c, node in enumerate(index): - if status[node]: - node_up[c] += t_simulation - last[c] - both_up[c] += system_up - up_at[c] - else: - both_down[c] += system_down - down_at[c] - - rates = ctx.downtime_cost_rates - if rates: - for c, node in enumerate(index): - rate = rates.get(node) - if rate is not None: - charge = rate * (t_simulation - node_up[c]) - rep_cost += charge - by_category[4] += charge - by_node[node] += charge - if ctx.has_costs: - charge = self.downtime_cost_rate * (t_simulation - system_up) - rep_cost += charge - by_category[5] += charge - - rec = _Replication() - rec.uptime = system_up - rec.node_up, rec.both_up, rec.both_down = node_up, both_up, both_down - rec.counts = (failed, caused_down, restored, caused_up) - rec.failures, rec.restorations = failures, restorations - rec.planned = planned - rec.changes, rec.deltas = changes, deltas - rec.cost = rep_cost if ctx.has_costs else None - rec.by_category, rec.by_node = by_category, by_node - rec.capacity_changes = rec.capacity_time = rec.delivered = None - rec.replacements = replaced - rec.opportunistic = opportunistic - rec.history = ( - (at_start, times_of, planned_of, started_up, system_record) - if history - else None - ) - if trace is not None: - trace.finish(t_simulation, rec) - return rec - - def _availability_result( - self, - tally: "_Tally", - t_simulation: float, - initial_up: bool, - antithetic: bool, - capacity: Optional[_CapacityRecorder] = None, - controls: Tuple[Optional[ControlVariate], ...] = (None, None), - conditional: Optional[ConditionalRun] = None, - breakdown: Optional[_Breakdown] = None, - ) -> AvailabilityResult: - """The ``AvailabilityResult`` of the replications in ``tally``: its - exact totals rounded, once; with ``controls``, the controls of its - fractions up and its costs by an exact twin (see - ``_controlled_run``); with ``conditional``, the record of its means - taken given its modules' histories (see ``_conditioned_run``); with - ``breakdown``, the split of the expected cost its mean is found - with, exact or given the modules (#223), in place of the - simulations' own.""" - N = tally.n - nodes = tally.nodes - tally._fold() - - def rounded(totals) -> list: - return [float(total) for total in totals] - - system_uptime = float(tally.system_uptime) - system_downtime = float(tally.system_downtime) - both_up = rounded(tally.intersection_uptime) - both_down = rounded(tally.intersection_downtime) - # Collect Importance/Criticality measures from the simulation - # reference: https://www.weibull.com/pubs/2004rm_05B_02.pdf - # Operational Criticality Index - oci_down = { - k: _safe_ratio(v, system_downtime) - for k, v in zip(nodes, both_down) - } - oci_up = { - k: _safe_ratio(v, system_uptime) for k, v in zip(nodes, both_up) - } - # Intersection Over Union Importance - iou_up = { - k: _safe_ratio(both, either) - for k, both, either in zip( - nodes, both_up, rounded(tally.union_uptime) + if len(chosen) > 62: + raise NotImplementedError( + f"A conditional run would simulate {len(chosen)} nodes, more " + "than the 62 it follows the joint states of: run it plainly " + "(conditional=False)." ) - } - iou_down = { - k: _safe_ratio(both, either) - for k, both, either in zip( - nodes, both_down, rounded(tally.union_downtime) + if chosen and not set(self.components) - held - chosen: + raise NotImplementedError( + "A conditional run would simulate every component (the " + "nodes the exact methods over time do not take, and those " + "they are tied to by repair crews or maintenance groups), " + "leaving none to take exactly given them: run it plainly " + "(conditional=False)." ) - } - FCI, RCI = tally.criticality_counts() - # Failure Criticality Index Importance - fci_sys = failure_criticality_index_per_system_failures( - FCI, tally.system_failures - ) - fci_comp = failure_criticality_index_per_component_failures(FCI) - # Restoration Criticality Index Importance - rci_sys = restoration_criticality_index_by_system( - RCI, tally.system_restorations - ) - rci_comp = restoration_criticality_index_by_component(RCI) - criticalities = Criticalities( - operational_criticality_index=UpDownImportance( - up=oci_up, down=oci_down - ), - iou=UpDownImportance(up=iou_up, down=iou_down), - failure_criticality_index=FailureCriticalityIndex( - per_system_failure=fci_sys, per_component_failure=fci_comp - ), - restoration_criticality_index=RestorationCriticalityIndex( - by_system=rci_sys, by_component=rci_comp - ), - ) + return [node for node in self.components if node in chosen] - # The availability from t=0..t_simulation: how many of the N - # simulated systems work after each time at which one changed state - # (and at 0 and t_simulation whether or not any did), over N. - if tally.binned is not None: - # On the grid (#153): how many work at each of its times. - assert tally.edges is not None - time = tally.edges.copy() - working = (N if initial_up else 0) + np.cumsum(tally.binned) - else: - changed_at, deltas = tally.state_changes() - time, working = _working_over_time( - changed_at, - deltas, - t_simulation, - N if initial_up else 0, - ) - system_availability = working / N - - cost_result = _acquisition_only(self, N, t_simulation, antithetic) - if self.has_costs: - # Per-component means cover each node's repair, replace, - # preventive and own downtime cost. (System downtime is a - # system-level quantity and is not attributed to components.) - by_category, by_component = ( - breakdown - if breakdown is not None - else ( - { - k: float(v) / N - for k, v in tally.cost_by_category.items() - }, - { - k: float(v) / N - for k, v in tally.cost_by_component.items() - }, - ) - ) - cost_result = CostResult( - samples=np.asarray(tally.cost_samples, dtype=float), - t_simulation=t_simulation, - n_simulations=N, - acquisition_cost=self.acquisition_cost, - by_category=by_category, - by_component=by_component, - antithetic=antithetic, - control_variate=controls[1], - conditional=conditional, - ) + # A simulation in progress (``initialize_event_queue``, ``next_event``) + # keeps its state in a ``_Run`` (see ``_events._Run``); these read it. - capacity_fields: dict = {} - if capacity is not None: - # The mean capacity from t=0..t_simulation: the simulated - # systems' total expected capacity after each time at which one - # changed (unlimited while one can carry an unlimited amount). - if tally.capacity_binned is not None: - # On the grid (#190): the total after each of its times. - assert tally.edges is not None - assert tally.unlimited_binned is not None - times = tally.edges.copy() - totals = np.cumsum( - bin_totals(tally.capacity_binned, len(times)) - ) - unlimited = np.cumsum(tally.unlimited_binned) > 0 - else: - at, steps, starts, free_at, counts = tally.capacity_changes() - change = _group_totals(steps, starts) - del steps, starts # (their memory serves what follows) - times, totals, unlimited = _capacity_totals( - at, change, free_at, counts, t_simulation - ) - del at, change - # The mean over the systems, never below 0, and unlimited while - # one can carry an unlimited amount: np.where(unlimited, inf, - # np.maximum(totals / N, 0.0)), worked out in place. - curve = np.divide(totals, N, out=totals) - np.maximum(curve, 0.0, out=curve) - curve[unlimited] = np.inf - capacity_fields = dict( - capacity_timeline=times, - capacity=curve, - capacity_time={ - level: float(tally.capacity_time[level]) - for level in sorted(tally.capacity_time) - }, - demand=capacity.demand, - delivered=( - None - if capacity.demand is None - else np.asarray(tally.delivered, dtype=float) - ), + def _running(self, name: str): + run = self.__dict__.get("_run_state") + if run is None: + raise AttributeError( + f"{name} is a simulation's: call initialize_event_queue first" ) + return getattr(run, name) - return AvailabilityResult( - timeline=time, - availability=system_availability, - system_uptime=system_uptime, - time_simulated_to=t_simulation, - criticalities=criticalities, - node_uptime=dict(zip(nodes, rounded(tally.node_uptime))), - node_downtime=dict(zip(nodes, rounded(tally.node_downtime))), - system_downtime=system_downtime, - system_failures=tally.system_failures, - system_restorations=tally.system_restorations, - n_simulations=N, - cost=cost_result, - system_planned_outages=tally.system_planned_outages, - uptimes=np.asarray(tally.uptimes, dtype=float), - antithetic=antithetic, - opportunistic_renewals=( - dict(zip(nodes, tally.opportunistic)) - if self._maintenance - else None - ), - control_variate=controls[0], - conditional=conditional, - **capacity_fields, - ) + @property + def system_state(self) -> bool: + """Whether the system works, in the simulation in progress.""" + return self._running("system_state") + + @property + def component_status(self) -> Dict[Hashable, bool]: + """Whether each component works, in the simulation in progress.""" + return self._running("component_status") + + @property + def t_simulation(self) -> float: + """The end of the window of the simulation in progress.""" + return self._running("t_simulation") + + @property + def last_change_planned(self) -> bool: + """Whether the last change of the system's state, in the + simulation in progress, was planned (see ``next_event``).""" + return self._running("last_change_planned") def cost( self, @@ -19406,7 +4961,7 @@ def cost( shard_size: Optional[int] = None, control_variate: Optional[bool] = None, conditional: Optional[bool] = None, - ) -> Optional[CostResult]: + ) -> CostResult: """Simulate the cost of running the system for ``t_simulation``. Runs the ``availability`` simulation with cost accumulation and @@ -19427,11 +4982,11 @@ def cost( When nothing is priced but the components' ``acquisition_cost`` (see ``has_costs``), the result has no running cost, exactly, and - the acquisition beside it, without a simulation (#234); when - nothing at all is priced, it is None: there is no cost model to - evaluate. (The same result is available as ``availability(...).cost`` + the acquisition beside it, without a simulation; when nothing at + all is priced there is no cost model to evaluate, and this + refuses. (The same result is available as ``availability(...).cost`` if you also want the availability outputs from the same - simulations.) + simulations; there it is None when nothing at all is priced.) Parameters ---------- @@ -19499,12 +5054,12 @@ def cost( twin's exact ``expected_cost``, and a ``tolerance`` is judged on it. By default None: where the exact methods work out the system's expected cost over the window, ``mean`` is that cost, - with no error, and the breakdowns its split (#187, #223), as + with no error, and the breakdowns its split, as for ``availability``. False forces a plain simulation, whose ``mean`` is ``sample_mean``. conditional : bool, optional Take the expected cost given the histories of the dependent - modules, as for ``availability`` (#189): each simulation's + modules, as for ``availability``: each simulation's expected cost given them is the modules' own costs as simulated, the other nodes' exact expected costs and the system downtime's expected cost. By default None: where a @@ -19518,16 +5073,15 @@ def cost( Returns ------- - CostResult or None - The simulated costs, or None if nothing is priced (not even an - ``acquisition_cost``). + CostResult + The simulated costs. Raises ------ ValueError - As for ``availability``, when something is priced. (With nothing - priced this returns None without any checks; with only an - ``acquisition_cost``, only ``mc_samples`` is checked.) + If nothing at all is priced (see ``has_costs``; as + ``routes.Refused``), or as for ``availability`` (with only an + ``acquisition_cost``, only ``mc_samples`` is checked). NotImplementedError As for ``availability``. ImportError @@ -19561,44 +5115,37 @@ def cost( >>> round(result.cost_rate, 2), round(rbd.expected_cost_rate(), 2) (13.6, 13.64) """ - N = 10_000 if mc_samples is None else mc_samples - t_simulation = _check_window(t_simulation) - - if not self.has_costs: - if self.acquisition_cost: - montecarlo.check_count(N, antithetic, "mc_samples") - return _acquisition_only(self, N, t_simulation, antithetic) - return self._simulated( - t_simulation, - working_nodes, - broken_nodes, - method, - N, - verbose, - seed, + return _runs.cost( + self, + t_simulation=t_simulation, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + mc_samples=mc_samples, + verbose=verbose, + seed=seed, tolerance=tolerance, confidence=confidence, - max_N=max_samples, + max_samples=max_samples, antithetic=antithetic, n_jobs=n_jobs, - target="cost", engine=engine, state=state, - # The cost result has no curve: count the changes, not keep them. - curve_points=1, shard_map=shard_map, shard_size=shard_size, control_variate=control_variate, conditional=conditional, - ).cost + ) def node_availability(self) -> dict[Hashable, float]: """Returns each node's long-run (steady-state) availability. Exact: ``MTTF / (MTTF + MTTR)`` for each component (1.0 if it is - repaired instantly), a nested ``RepairableRBD``'s own - ``mean_availability``, and 1.0 for the input and output nodes. These - are the availabilities the importance measures are evaluated at. + repaired instantly), and a nested ``RepairableRBD``'s own + ``mean_availability``; the components alone, as ``node_sf`` and + the importance measures list them (the input and output nodes + are no components). These are the availabilities the importance + measures are evaluated at. A component under age replacement at ``T`` renews at a failure or at a preventive replacement, whichever comes first. By the @@ -19634,1234 +5181,38 @@ def node_availability(self) -> dict[Hashable, float]: ``mean_availability``): waiting for a crew, it is down. Returns - ------- - dict[Hashable, float] - Node name -> long-run availability, in ``[0, 1]``. - - Raises - ------ - NotImplementedError - If a component is under block replacement with models its exact - values do not cover (a lifetime that is not a surpyval - parametric model with a density, dead-on-arrival units, repairs - that may never end, or repairs or maintenance far longer than - the interval), or has hidden failures its numerical values do - not cover (see ``mean_availability``); or, while a component can - wait for a repair crew, as for ``mean_availability``. - - Examples - -------- - >>> import surpyval as surv - >>> from repyability import RepairableRBD - >>> E = surv.Exponential.from_params - >>> rbd = RepairableRBD( - ... [("s", "a"), ("a", "b"), ("b", "t")], - ... { - ... "a": {"reliability": E([0.2]), "repairability": E([1.0])}, - ... "b": {"reliability": E([0.5]), "repairability": "instant"}, - ... }, - ... ) - >>> availability = rbd.node_availability() - >>> {node: round(a, 4) for node, a in availability.items()} - {'a': 0.8333, 'b': 1.0, 's': 1.0, 't': 1.0} - """ - node_av: dict[Hashable, float] = {} - for node_name in self.components: - node_av[node_name] = self._node_availability(node_name) - - for node_name in self.in_or_out: - node_av[node_name] = 1.0 - - return node_av - - def _node_availability(self, node) -> float: - """A component's long-run availability (see ``node_availability``): - with limited repair crews, its long-run probability of being up in - their Markov chain, or a nested RBD's own, as it has crews of its - own.""" - self._require_perfect_repair(node) - if self._crews_couple(): - chain = self._crew_chain() - if node in chain.nodes: - return chain.availability(node) - return float(self.components[node].mean_availability()) - if node in self._standby: - return self._standby_long_run(node).availability - if node in self._inspection: - unit = self._tested_unit(node) - if unit is not None: - return unit.long_run.availability - life = self._tested_life(node) - if life is not None: - return life.availability - rate, interval = self._inspected_rate(node) - inspection = self._inspection[node] - if inspection.partial: - # Up with probability rho ** k * exp(-rate * u) (see - # _tested_profile): averaged over a full test's cycle, - # (1 - rho ** m) / ((1 - coverage) * rate * full_test). - kept = _log_kept(rate, inspection) - return float( - -np.expm1(inspection.per_full_test * kept) - / ((1.0 - inspection.coverage) * rate * inspection.period) - ) - return float(-np.expm1(-rate * interval) / (rate * interval)) - schedule = self._preventive.get(node) - if schedule is None: - component = self.components[node] - return float(np.atleast_1d(component.mean_availability())[0]) - up, cycle, _, _ = self._maintenance_cycle(node, schedule) - return min(1.0, up / cycle) - - def _node_unavailability(self, node) -> float: - """A component's long-run unavailability, as ``_node_availability`` - gives its availability, but worked out in its own right (from a - standby group's down states, or the mean down time over a cycle), - so that a small one keeps its precision. For a component whose - value is constant over the long-run grid, and while the crews do - not couple the components (see ``_long_run_unavailabilities``).""" - self._require_perfect_repair(node) - if node in self._standby: - return self._standby_long_run(node).unavailability - component = self.components[node] - schedule = self._preventive.get(node) - if schedule is None: - return float(np.atleast_1d(component.mean_unavailability())[0]) - up, cycle, fails, survives = self._maintenance_cycle(node, schedule) - if schedule.policy == "block": - return max(0.0, (cycle - up) / cycle) - # Down for a repair after a failure before the age (its probability - # from the model's own ff), else for the maintenance. - maintenance = ( - 0.0 if schedule.duration is None else model_mean(schedule.duration) - ) - down = ( - fails * model_mean(component.time_to_replace) - + survives * maintenance - ) - return min(1.0, down / cycle) - - def _require_one_inspected(self) -> None: - """Raise if more than one component has hidden failures, when no - intervals are given to choose from (``optimal_inspection_intervals`` - with ``allowed=None``).""" - if len(self._inspection) > 1: - raise ValueError( - "More than one component has hidden failures: give the " - "intervals to choose from in allowed (tests are made on " - "a calendar, and the long-run values depend on how the " - "schedules line up)." - ) - - def _require_capacities_given(self) -> None: - """Raise if a node takes its capacity from its model (a - ``DegradingNode``'s stages, or a nested RBD's capacities), which - the availability simulation does not follow.""" - own = self._capacity_models() - if own: - raise NotImplementedError( - f"Node(s) {sorted(own, key=str)} take their capacity from " - "their models (a DegradingNode's stages, or a nested RBD's " - "capacities), which the simulation does not follow. Give " - "them a capacity, or use capacity_distribution() for the " - "long run." - ) - - def _require_reliabilities(self, node) -> None: - """Raise, as the model itself does, if a component's life or repair - model has no exact or numerical reliability: a ``StandbyModel`` or - ``LoadSharingModel`` that only simulations take (#149).""" - component = self.components[node] - for model in ( - getattr(component, "reliability", None), - getattr(component, "time_to_replace", None), - ): - if ( - isinstance(model, (StandbyModel, LoadSharingModel)) - and model.is_simulated - ): - raise model._no_reliability() - - def _require_time_models(self, node) -> None: - """Raise if a component's life or repair model is a probability, - not a distribution of times: its availability over time then has - no exact value. (First, if one has no reliability: see - ``_require_reliabilities``.)""" - self._require_reliabilities(node) - component = self.components[node] - for what, model in [ - ("reliability", component.reliability), - ("repairability", component.time_to_replace), - ]: - if is_fixed_probability(model): - raise NotImplementedError( - f"Component {node!r}: its {what} model is a probability, " - "not a distribution of times, so its availability over " - "time has no exact value. Estimate it by simulation, " - "with availability()." - ) - - def _require_unscheduled_stages(self, node) -> None: - """Raise if a degrading component is maintained or inspected on a - schedule: its long-run time in each stage has no exact value.""" - if node in self._preventive or node in self._inspection: - raise NotImplementedError( - f"Component {node!r} degrades through stages and is " - "maintained or inspected on a schedule: its long-run time in " - "each stage has no exact value here. Give it a capacity " - "instead." - ) - - def _imperfect_phrase(self, node) -> str: - """How a component is repaired imperfectly, in words.""" - imperfect = self._imperfect[node] - kind = "I" if imperfect.kijima == "kijima1" else "II" - phrase = f"Kijima {kind}, q = {imperfect.q:g}" - if imperfect.replace_after is not None: - phrase += ( - f", replaced at failure {imperfect.replace_after} since it " - "was renewed" - ) - return phrase - - def _require_perfect_repair(self, node) -> None: - """Raise if a component is repaired imperfectly (a spec's - ``"repair"``): a repair does not renew it, so its long-run values - are known only by simulation (its values over time, only by - simulation too, but for minimal repair in no time: see - ``_require_minimal_repair``).""" - if node in self._imperfect: - over_time = ( - "its values over a window from new are exact, as it fails " - "as often as its life's cumulative hazard" - if self._minimal_repair_blocker(node) is None - else "nor its values over time" - ) - raise NotImplementedError( - f"Component {node!r} is repaired imperfectly " - f"({self._imperfect_phrase(node)}), so a repair does not " - f"renew it: its long-run values have no exact value here " - f"({over_time}). Estimate them by simulation, with " - "availability() or cost()." - ) - - def _minimal_repair_blocker(self, node) -> Optional[str]: - """Why an imperfectly repaired component's values over time have no - exact value, or None if they have: when it is minimally repaired - (Kijima, ``q = 1``) in no time, with no ``replace_after``, - preventive maintenance or tests, and a life that works at 0. Its - failures are then a non-homogeneous Poisson process whose intensity - is its life's hazard at its age (its age is its operating time, - which is all the time), so that it is up throughout and fails - ``H(t)`` times by ``t`` on average, ``H`` the life's cumulative - hazard (see ``MinimalRepairCurve``).""" - imperfect = self._imperfect[node] - component = self.components[node] - if imperfect.q < 1.0: - return "its repairs take away some of its age (q below 1)" - if imperfect.replace_after is not None: - return ( - f"it is replaced at its failure {imperfect.replace_after} " - "since it was renewed" - ) - repair = component.time_to_replace - if float(np.ravel(_sf_values(repair.sf, np.zeros(1)))[0]) > 0.0: - return "its repairs take time" - if node in self._preventive: - return "it has preventive maintenance" - if node in self._inspection: - return "its failures are found only by its tests" - life = component.reliability - if float(np.ravel(_sf_values(life.sf, np.zeros(1)))[0]) < 1.0: - return "its life may end at 0 (dead on arrival)" - return None - - def _require_minimal_repair(self, node) -> None: - """Raise unless an imperfectly repaired component's values over time - are exact: minimal repair in no time (see - ``_minimal_repair_blocker``).""" - blocker = self._minimal_repair_blocker(node) - if blocker is not None: - raise NotImplementedError( - f"Component {node!r} is repaired imperfectly " - f"({self._imperfect_phrase(node)}), and {blocker}: its " - "availability over time has no exact value here (it has " - "for minimal repair, q = 1, in no time, with no " - "replace_after, maintenance or tests). Estimate it by " - "simulation, with availability() or cost()." - ) - - def _require_no_opportunities(self, node) -> None: - """Raise if a component can be renewed early at the stops of its - maintenance group (an ``"opportunity"`` below its interval): when - depends on the other members, so its long-run values and its - availability over time are known only by simulation.""" - if node in self._early_members: - raise NotImplementedError( - f"Component {node!r} is renewed early at the stops of its " - f"maintenance group {self._member_group[node]!r}, which " - "depend on the other members, so its long-run values and " - "its availability over time have no exact value here. " - "Estimate them by simulation, with availability() or cost()." - ) - - def _require_separate_setups(self) -> None: - """Raise if two members of a maintenance group with a set-up cost - can be replaced at the same instants, again and again: on block - schedules, or never failing before an age replacement in zero time. - Such replacements share one stop, and its set-up, which the exact - cost rate, charging a set-up for each member's failures and - replacements, does not count.""" - for group, spec in self._maintenance.items(): - if not spec.setup_cost: - continue - clocked = [node for node in spec.members if self._on_a_clock(node)] - if len(clocked) > 1: - raise NotImplementedError( - f"Components {clocked} of maintenance group {group!r} " - "are replaced on a clock (on block schedules, or never " - "failing before an age replacement in zero time), so " - "their replacements can fall at the same instants and " - "share a set-up, which the exact cost rate does not " - "count: estimate it by simulation, with cost()." - ) - - def _on_a_clock(self, node) -> bool: - """Whether a component's replacements fall on a fixed lattice of - times: under block replacement, or under age replacement in zero - time without a failure before it.""" - schedule = self._preventive.get(node) - if schedule is None or not math.isfinite(schedule.interval): - return False - if schedule.policy == "block": - return True - if schedule.policy != "age" or schedule.duration is not None: - return False - survives = self.components[node].reliability_function( - schedule.interval - ) - return bool(np.ravel(survives)[0] >= 1.0) - - def _require_block_models(self, node) -> None: - """Raise unless the exact block-replacement values cover the - component's models (the checks ``block_cycle`` makes first).""" - from repyability.rbd._block_replacement import _check_life, _Duration - - component = self.components[node] - _check_life(component.reliability, node) - _Duration(component.time_to_replace, "repair", node) - _Duration(self._preventive[node].duration, "replacement", node) - - def _require_calendars(self) -> None: - """Raise if the exact long-run values cannot average over the - components' calendars (block replacement and inspection): a nested - RBD's calendar with another here, intervals with no common period, - or one repeating too often in it (before any computation).""" - nested = [ - node - for node, c in self.components.items() - if isinstance(c, RepairableRBD) and c._has_calendar() - ] - blocks = self._block_nodes() - if nested and len(nested) + len(self._inspection) + len(blocks) > 1: - raise NotImplementedError( - f"Node(s) {sorted(nested, key=str)} are RBDs with hidden " - "failures or block replacement, and other nodes' inspections " - "or block replacements here fall at the same times: " - "estimate the long-run values by simulation, with " - "availability() or cost()." - ) - intervals = {self._preventive[node].interval for node in blocks} - intervals |= { - self._inspection[node].interval for node in self._inspection - } - if not intervals: - return - period = _common_period(intervals) - if any(round(period / interval) > 100_000 for interval in intervals): - if blocks: - raise NotImplementedError( - f"The block-replacement and inspection intervals " - f"{sorted(intervals)} repeat together only after too many " - "intervals to average over: estimate the long-run values " - "by simulation, with availability() or cost()." - ) - raise NotImplementedError( - f"The inspection intervals {sorted(intervals)} repeat " - "together only after too many inspections to average " - "over: estimate the long-run values by simulation, with " - "availability() or cost()." - ) - - def _instant_tests(self, node) -> bool: - """Whether a component with hidden failures is tested, and - repaired, in no time.""" - return ( - self._inspection[node].duration is None - and model_mean(self.components[node].time_to_replace) == 0.0 - ) - - def _tested_kind(self, node) -> str: - """How a component with hidden failures' values are worked out: - ``"closed"``, closed forms, for a constant failure rate and tests - and repair in no time; ``"life"``, summed over the test intervals - (``TestedLife``, #144), for any other life with those and tests - that find every failure; and ``"unit"``, its renewal cycle followed - test by test on a grid (``TestedUnit``, #159), for tests or repairs - that take time, or tests that can miss a failure of a life that is - not exponential.""" - instant = self._instant_tests(node) - if instant and _constant_rate(self.components[node].reliability): - return "closed" - if instant and not self._inspection[node].partial: - return "life" - return "unit" - - def _require_tested_exact(self, node) -> None: - """Raise unless a component with hidden failures has exact or - numerical values (long-run, or over time): with tests and repair in - no time, any life (#144); with tests or repairs that take time, or - tests that can miss a failure, a surpyval parametric life with a - density and repair and test times that end, the tests within the - interval (#159).""" - if self._tested_kind(node) != "unit": - return - component = self.components[node] - inspection = self._inspection[node] - check_tested( - component.reliability, - component.time_to_replace, - inspection.duration, - inspection.interval, - node, - ) - - def _inspected_rate(self, node) -> Tuple[float, float]: - """The constant failure rate and the inspection interval of a - component with hidden failures, for the closed forms of an - exponential life tested and repaired in no time (see - ``_tested_life`` and ``_tested_unit`` for the others).""" - self._require_tested_exact(node) - if self._tested_kind(node) != "closed": - raise NotImplementedError( - f"Component {node!r} has hidden failures, and a life that is " - "not exponential or tests or repairs that take time, which " - "this does not take: estimate it by simulation, with " - "availability() or cost()." - ) - rate = _constant_rate(self.components[node].reliability) - assert rate is not None - return rate, self._inspection[node].interval - - def _tested_life(self, node) -> Optional[TestedLife]: - """For a component with hidden failures, a life other than - exponential, and tests and repair in no time that find every - failure, its long run under its tests (see ``_hidden_life``), - worked out once; None for any other (see ``_tested_kind``).""" - self._require_tested_exact(node) - if self._tested_kind(node) != "life": - return None - component = self.components[node] - interval = float(self._inspection[node].interval) - cache = self.__dict__.setdefault("_tested_lives", {}) - key = (node, id(component.reliability), interval) - if key not in cache: - cache[key] = TestedLife(component.reliability, interval) - return cache[key] - - def _tested_unit( - self, node, any_kind: bool = False - ) -> Optional[TestedUnit]: - """For a component with hidden failures whose tests or repairs - take time, or whose tests can miss a failure of a life that is not - exponential, its values under its tests (see ``_hidden_tests``), - worked out once; None for any other (see ``_tested_kind``), but - with ``any_kind``, for its spares (whose tests the caller has - checked).""" - if not any_kind: - self._require_tested_exact(node) - if self._tested_kind(node) != "unit": - return None - component = self.components[node] - inspection = self._inspection[node] - cache = self.__dict__.setdefault("_tested_units", {}) - key = ( - node, - id(component.reliability), - id(component.time_to_replace), - id(inspection.duration), - float(inspection.interval), - float(inspection.coverage), - inspection.per_full_test, - ) - if key not in cache: - cache[key] = TestedUnit( - component.reliability, - component.time_to_replace, - inspection.duration, - inspection.interval, - inspection.coverage, - inspection.per_full_test, - node, - _constant_rate(component.reliability), - ) - return cache[key] - - def _unit_phase(self, node, times: np.ndarray) -> np.ndarray: - """The time since a tested component's last full test at each of - ``times`` on its calendar (its full tests at its offset and every - full test's interval from it): where its long run's profile is - read (see ``TestedLongRun``).""" - inspection = self._inspection[node] - position = times - inspection.offset if inspection.offset else times - period = inspection.period - return position - period * np.floor(position / period) - - def _tested_rate(self, node) -> float: - """How fast a tested component's long-run availability falls - between tests: its failure rate, or for any other life the rate of - its profile's mean decay (see ``TestedLife.rate``), or its failures - per unit of up time (``TestedLongRun.rate``), for the long-run - grid's spacing.""" - unit = self._tested_unit(node) - if unit is not None: - return unit.long_run.rate - life = self._tested_life(node) - return self._inspected_rate(node)[0] if life is None else life.rate - - def _tested_scale(self, node) -> float: - """The rate that sets the scale of a tested component's interval: - its failure rate, or one over the mean of any other life.""" - self._require_tested_exact(node) - life = self.components[node].reliability - rate = _constant_rate(life) - return float(rate) if rate else 1.0 / float(model_mean(life)) - - def _tested_phase(self, node, times: np.ndarray) -> np.ndarray: - """The time since a tested component's last test at each of - ``times`` on its calendar (its tests at its offset and every - interval from it).""" - inspection = self._inspection[node] - position = times - inspection.offset if inspection.offset else times - interval = inspection.interval - return position - interval * np.floor(position / interval) - - def _tested_intensity( - self, node, times: np.ndarray, availability - ) -> np.ndarray: - """A tested component's long-run rate of failing at each of - ``times`` (``availability`` its availability there): its constant - rate while it is up, or for any other life the rate of its profile - (see ``TestedLife.intensity``).""" - unit = self._tested_unit(node) - if unit is not None: - return unit.long_run.intensity(self._unit_phase(node, times)) - life = self._tested_life(node) - if life is None: - rate, _ = self._inspected_rate(node) - return rate * availability - return life.intensity(self._tested_phase(node, times)) - - def _block_nodes(self) -> list: - """The components renewed on a calendar, whose long-run values vary - with it: under block replacement, or replaced on condition at - inspections (#145).""" - return [ - node - for node, schedule in self._preventive.items() - if schedule.policy in ("block", "condition") - ] - - def _block_cycle(self, node): - """The renewal cycle of a component under block replacement (see - ``_block_replacement``) or replaced on condition (see - ``_condition_replacement``), computed once and kept.""" - component = self.components[node] - schedule = self._preventive[node] - cache = self.__dict__.setdefault("_block_cycles", {}) - key = ( - node, - id(component.reliability), - id(component.time_to_replace), - float(schedule.interval), - id(schedule.duration), - schedule.policy, - schedule.threshold, - ) - if key not in cache: - if schedule.policy == "condition": - cache[key] = condition_cycle( - component.reliability, - component.time_to_replace, - schedule.duration, - schedule.interval, - float(schedule.threshold), - node, - ) - else: - cache[key] = block_cycle( - component.reliability, - component.time_to_replace, - schedule.duration, - schedule.interval, - node, - ) - return cache[key] - - def _has_calendar(self) -> bool: - """Whether a component here, or in a nested RBD, is inspected or - replaced on a calendar: its long-run availability then varies with - the time of the schedule.""" - return ( - bool(self._inspection) - or bool(self._block_nodes()) - or any( - isinstance(c, RepairableRBD) and c._has_calendar() - for c in self.components.values() - ) - ) - - def _unit_nodes(self) -> list: - """The components with hidden failures whose values are worked out - on a grid (see ``_tested_kind``).""" - return [ - node - for node in self._inspection - if self._tested_kind(node) == "unit" - ] - - def _calendar_grid( - self, blocks: list, units: Optional[list] = None - ) -> Tuple[np.ndarray, np.ndarray]: - """``_long_run_grid`` with components under block replacement, or - tested on a grid (``units``): the middles of cells over one common - period of the block and inspection intervals. The cells' edges are - those of every block-replaced component's profile (its long-run - availability over its interval, cell by cell), of every such tested - component's (its grid's points across each test interval, and where - its test's and its repair's CDFs bend), the block and inspection - times, and enough points in between for an inspected component's - availability to vary little across a cell; so each cell lies in one - cell of every profile, and the mean over the cells is as exact as - the profiles.""" - intervals = {self._preventive[node].interval for node in blocks} - intervals |= { - self._inspection[node].interval for node in self._inspection - } - schedules = list(self._inspection.values()) - period = _common_period(intervals | {s.period for s in schedules}) - - def too_long() -> NoReturn: - raise NotImplementedError( - f"The block-replacement and inspection intervals " - f"{sorted(intervals)} repeat together only after too long a " - "time to average over finely enough: estimate the long-run " - "values by simulation, with availability() or cost()." - ) - - # The grid's size, before it is built: a common period of many - # repeats of the profiles (near-equal intervals, say) would not fit - # in memory. - size = 0 - for node in blocks: - phase = self._block_cycle(node).phase - size += int(round(period / phase[-1])) * len(phase) - for node in units or (): - unit = self._tested_unit(node).long_run # type: ignore - size += int(round(period / unit.period)) * len(unit.edges()) - for node in self._inspection: - interval = self._inspection[node].interval - per_interval = np.ceil(256.0 * self._tested_rate(node) * interval) - size += int(per_interval) * int(round(period / interval)) - if size > 4_000_000: - too_long() - pieces = [np.array([0.0, period])] - for interval in intervals: - pieces.append(interval * np.arange(int(round(period / interval)))) - for schedule in schedules: - pieces.append(np.array(sorted(_tests_in(schedule, period)))) - for node in blocks: - phase = self._block_cycle(node).phase - repeats = int(round(period / phase[-1])) - pieces.append( - ( - phase[-1] * np.arange(repeats)[:, None] + phase[None, :-1] - ).ravel() - ) - for node in units or (): - long_run = self._tested_unit(node).long_run # type: ignore - repeats = int(round(period / long_run.period)) - edges = ( - self._inspection[node].offset - + long_run.period * np.arange(repeats)[:, None] - + long_run.edges()[None, :] - ).ravel() - pieces.append(edges - period * np.floor(edges / period)) - for node in self._inspection: - rate = self._tested_rate(node) - interval = self._inspection[node].interval - per_interval = int(np.ceil(256.0 * rate * interval)) - pieces.append( - np.linspace( - 0.0, - period, - 1 + per_interval * int(round(period / interval)), - ) - ) - edges = np.concatenate(pieces) - if len(edges) > 4_000_000: - too_long() - # Edges closer than rounding are one. - edges = np.unique(np.round(edges / period, 12)) * period - middle, width = 0.5 * (edges[1:] + edges[:-1]), np.diff(edges) - if not units: - return middle, width / period - # A tested component's test and repair times have exact CDFs in its - # profile, which may bend sharply within a cell (a test of hours, - # say): two Gauss-Legendre points a cell, exact for its linear grid - # values, and to the cell's fourth power for the rest. - side = 0.5 * width / np.sqrt(3.0) - times = np.stack([middle - side, middle + side], axis=1).ravel() - return times, np.repeat(0.5 * width / period, 2) - - def _block_profile(self, node, times: np.ndarray, rates: bool = False): - """A block-replaced component's long-run availability (or failure - intensity) at each of ``times``: the value of its profile's cell - (over a block interval) that the time falls in.""" - cycle = self._block_cycle(node) - interval = float(cycle.phase[-1]) - values = cycle.failure_rate if rates else cycle.availability - phase = times - interval * np.floor(times / interval) - cell = np.searchsorted(cycle.phase, phase, side="right") - 1 - return values[np.clip(cell, 0, len(values) - 1)] - - def _calendar_outages(self, working_nodes, broken_nodes) -> float: - """The system's planned outages per unit time, in the long run, - at exact times on the calendar: replacements at block times that - take time, and tests that take time, each taking a unit working - then off line. At each such time, the probability that the system - is up just before and down just after, which (as they only take - units down) is the fall in the system availability. Units due at - the same time go down together; an instant inspection due then - comes first.""" - blocks = [ - node - for node in self._block_nodes() - if self._preventive[node].duration is not None - ] - units = [ - node - for node in self._unit_nodes() - if self._tested_unit(node).setup.timed # type: ignore - ] - if not blocks and not units: - return 0.0 - intervals = {self._preventive[node].interval for node in blocks} - intervals |= { - self._inspection[node].period for node in self._inspection - } - period = _common_period(intervals) - # At each instant (as a share of the period), the nodes due then, - # with their values up and down just before and just after. - due: dict = {} - for node in blocks: - cycle = self._block_cycle(node) - values = ( - (cycle.before, 1.0 - cycle.before), - (cycle.after, 1.0 - cycle.after), - ) - interval = self._preventive[node].interval - for k in range(int(round(period / interval))): - instant = round(k * interval / period, 12) - due.setdefault(instant, []).append((node, values)) - for node in units: - inspection = self._inspection[node] - long_run = self._tested_unit(node).long_run # type: ignore - for k in range(int(round(period / inspection.interval))): - time = inspection.offset + k * inspection.interval - time -= period * math.floor(time / period) - due.setdefault(round(time / period, 12), []).append( - (node, long_run.around(k % long_run.per)) - ) - instants = np.array(sorted(due)) * period - # Just after each instant, before its outages start (an - # inspection due then is done). - just_after = instants + 1e-9 * period - up, down = self._availabilities_at(just_after), ( - self._unavailabilities_at(just_after) - ) - before, after = dict(up), dict(up) - before_down, after_down = dict(down), dict(down) - for column, key in enumerate(sorted(due)): - for node, values in due[key]: - for works, fails, (value, failed) in zip( - (before, after), (before_down, after_down), values - ): - works[node] = np.array(works[node], dtype=float) - fails[node] = np.array(fails[node], dtype=float) - works[node][column] = value - fails[node][column] = failed - # The fall in the system availability, as the rise in its - # unavailability: a difference of small values in a reliable - # system, not of values near 1. - rise = self._system_unreliability( - self._probabilities_with_overrides( - after, working_nodes, broken_nodes - ), - self._failures_with_overrides( - after_down, working_nodes, broken_nodes - ), - ) - self._system_unreliability( - self._probabilities_with_overrides( - before, working_nodes, broken_nodes - ), - self._failures_with_overrides( - before_down, working_nodes, broken_nodes - ), - ) - return float(np.sum(rise)) / period - - def _long_run_grid(self) -> Tuple[np.ndarray, np.ndarray]: - """The times, and their weights (which sum to 1), that the exact - long-run values average over. - - Components with hidden failures are up with a probability that falls - between inspections and is restored at each one, and components - inspected at the same times are down together, so the system's - long-run values are averages over one period of the inspection - schedules: by Gauss-Legendre quadrature between consecutive - inspections, on pieces short enough (a failure rate times their - length at most 1) for it to be exact to rounding. With no such - components they are constant: one time, of weight 1. A nested RBD - with hidden failures enters through its own long-run values, which - is exact only if nothing else varies with the inspections. With a - component whose profile is on a grid (block-replaced, or tested - with tests or repairs that take time: see ``_calendar_grid``), the - middles of the grids' cells. - """ - self._require_unlimited_crews() - self._require_calendars() - blocks = self._block_nodes() - units = self._unit_nodes() - if blocks or units: - return self._calendar_grid(blocks, units) - if not self._inspection: - return np.zeros(1), np.ones(1) - rates = { - node: (self._tested_rate(node), self._inspection[node].interval) - for node in self._inspection - } - intervals = {interval for _, interval in rates.values()} - schedules = list(self._inspection.values()) - period = _common_period(intervals | {s.period for s in schedules}) - breaks = {0.0, period} - for interval in intervals: - count = int(round(period / interval)) - breaks.update(k * interval for k in range(1, count)) - for schedule in schedules: - breaks.update(_tests_in(schedule, period)) - fastest = max(rate for rate, _ in rates.values()) - # The tests of components whose lives are not exponential: a unit - # renewed there may not age smoothly from it (a Weibull life's - # survival, say), so the pieces after it close in on it. - graded: set = set() - for node, schedule in self._inspection.items(): - life = self._tested_life(node) - if life is None: - continue - count = int(round(period / schedule.interval)) - starts = schedule.offset + schedule.interval * np.arange(count) - starts = np.where(starts >= period, starts - period, starts) - graded.update(round(float(s) / period, 12) for s in starts) - # Where its life bends within an interval (a threshold, say). - bends = (starts[:, None] + life.bends[None, :]).ravel() - breaks.update(float(b) for b in np.mod(bends, period)) - edges = np.array(sorted(breaks)) - points, weights = np.polynomial.legendre.leggauss(16) - times, masses = [], [] - for a, b in zip(edges[:-1], edges[1:]): - pieces = np.linspace( - a, b, max(1, math.ceil((b - a) * fastest)) + 1 - ) - if round(float(a) / period, 12) in graded: - width = pieces[1] - pieces[0] - pieces = np.concatenate( - [ - pieces[:1], - a + width * 2.0 ** -np.arange(48.0, 0.0, -1.0), - pieces[1:], - ] - ) - for lo, hi in zip(pieces[:-1], pieces[1:]): - half = 0.5 * (hi - lo) - times.append(lo + half * (points + 1.0)) - masses.append(half * weights) - return np.concatenate(times), np.concatenate(masses) / period - - def _tested_profile( - self, node, times: np.ndarray - ) -> Tuple[np.ndarray, np.ndarray]: - """A component with hidden failures: its probabilities of being up - and down at each of ``times``, in the long run. - - Tested at ``offset`` and every ``interval`` after it, it is up with - probability ``exp(-rate * u)``, ``u`` the time since its last test. - A test that can miss a failure (a ``coverage`` ``c`` below 1) finds - what it can: up just after the ``k``-th test since a full one with - probability ``rho ** k``, where ``rho = 1 - (1 - c) * (1 - - exp(-rate * interval))`` keeps the units that were up or whose - failure it found; so up with probability ``rho ** k * exp(-rate * - u)``. Down is worked out in its own right, by ``expm1``. Any other - life's profile is its ``TestedLife``'s (#144), and with tests or - repairs that take time, or tests that can miss a failure of a life - that is not exponential, its ``TestedLongRun``'s (#159).""" - unit = self._tested_unit(node) - if unit is not None: - return unit.long_run.profile(self._unit_phase(node, times)) - life = self._tested_life(node) - if life is not None: - return life.profile(self._tested_phase(node, times)) - rate, interval = self._inspected_rate(node) - inspection = self._inspection[node] - position = times - inspection.offset if inspection.offset else times - if not inspection.partial: - since = position - interval * np.floor(position / interval) - return np.exp(-rate * since), -np.expm1(-rate * since) - cycle = inspection.period - within = position - cycle * np.floor(position / cycle) - tests = np.clip( - np.floor(within / interval), 0, inspection.per_full_test - 1 - ) - exponent = tests * _log_kept(rate, inspection) - rate * ( - within - tests * interval - ) - return np.exp(exponent), -np.expm1(exponent) - - def _availabilities_at(self, times: np.ndarray) -> dict: - """Every node's availability at each of ``times`` (see - ``_long_run_grid``): ``exp(-lambda * u)``, ``u`` the time since the - last inspection, for a component with hidden failures; its constant - long-run availability for any other.""" - out: dict = {} - blocks = set(self._block_nodes()) - for node in self.components: - if node in self._inspection: - out[node] = self._tested_profile(node, times)[0] - elif node in blocks: - out[node] = self._block_profile(node, times) - else: - out[node] = np.full(len(times), self._node_availability(node)) - for node in self.in_or_out: - out[node] = np.ones(len(times)) - return out - - def _unavailabilities_at(self, times: np.ndarray) -> dict: - """Every node's unavailability at each of ``times``, as - ``_availabilities_at`` gives their availabilities, each worked out - in its own right so that a small one keeps its precision: - ``1 - exp(-lambda * u)`` by ``expm1`` at a time ``u`` since a test, - for a component with hidden failures, and ``_node_unavailability`` - for one whose value is constant. - (A block-replaced component's profile is numerical, to about 1e-7, - so one less it loses nothing.)""" - out: dict = {} - blocks = set(self._block_nodes()) - for node in self.components: - if node in self._inspection: - out[node] = self._tested_profile(node, times)[1] - elif node in blocks: - out[node] = 1.0 - self._block_profile(node, times) - else: - out[node] = np.full( - len(times), self._node_unavailability(node) - ) - for node in self.in_or_out: - out[node] = np.zeros(len(times)) - return out - - def _long_run_unavailabilities( - self, working_nodes, broken_nodes - ) -> Tuple[dict, dict, np.ndarray]: - """``_long_run_probabilities``, with every node's unavailability - too (at each time of ``_long_run_grid``, or in each state of the - repair crews' Markov chain), each worked out in its own right, so - that the system's unavailability keeps a small one's precision.""" - probabilities, failures, weights, _ = self._long_run_points( - working_nodes, broken_nodes - ) - return probabilities, failures, weights - - def _long_run_points( - self, working_nodes, broken_nodes - ) -> Tuple[dict, dict, np.ndarray, Optional[np.ndarray]]: - """``_long_run_unavailabilities``, and each point's time's position - in ``_long_run_grid``, or None in the states of the repair crews' - chain.""" - if self._crews_couple(): - probabilities, weights = self._chain_probabilities( - working_nodes, broken_nodes - ) - forced = set(working_nodes or ()) | set(broken_nodes or ()) - chain = self._crew_chain(frozenset(forced)) - # In each state a node in the chain, or held working or broken, - # is up or down for certain: one less it is exact. - failures = { - node: 1.0 - value for node, value in probabilities.items() - } - for node, component in self.components.items(): - if node not in forced and node not in chain.nodes: - failures[node] = np.full( - len(weights), float(component.mean_unavailability()) - ) - return probabilities, failures, weights, None - times, weights = self._long_run_grid() - probabilities = self._probabilities_with_overrides( - self._availabilities_at(times), working_nodes, broken_nodes - ) - failures = self._failures_with_overrides( - self._unavailabilities_at(times), working_nodes, broken_nodes - ) - # (Common-cause groups are conditioned on module by module, see - # _ccf_tabled, by every method that would come here with them.) - assert not self.ccf_groups - return probabilities, failures, weights, np.arange(len(times)) - - def _long_run_probabilities( - self, working_nodes, broken_nodes, what: str = "This analysis" - ) -> Tuple[dict, np.ndarray]: - """The node availabilities the long-run values are evaluated at - (with the forced nodes held at 1 or 0), over the times of - ``_long_run_grid``, and those times' weights; or, with limited - repair crews, over the states of their Markov chain (see - ``_chain_probabilities``). Each value is then a ratio of averages - of system quantities. With common-cause groups, the times are split - by every combination of their states (see ``_with_ccf_groups``, - which ``what`` refuses where they are too many): for the capacity - distribution, which takes the nodes' joint states.""" - if self.ccf_groups: - # The groups' chains refuse the crews (see _ccf_rates). - self._require_ccf_long_run() - if self._crews_couple(): - return self._chain_probabilities(working_nodes, broken_nodes) - times, weights = self._long_run_grid() - probabilities = self._probabilities_with_overrides( - self._availabilities_at(times), working_nodes, broken_nodes - ) - if self.ccf_groups: - self._require_free_members(working_nodes, broken_nodes) - probabilities, _, weights, _ = self._with_ccf_groups( - times, probabilities, None, weights, what=what - ) - return probabilities, weights - - def _importance_probabilities( - self, working_nodes, broken_nodes - ) -> Tuple[dict, dict, np.ndarray, Optional[np.ndarray]]: - """``_long_run_points`` for the importance measures: with limited - repair crews the states of the crews' Markov chain, in each of - which every node the crews work on is up or down for certain - (#146). (With common-cause groups, see ``_ccf_long_run_measure``.)""" - return self._long_run_points(working_nodes, broken_nodes) - - def _importance_over_time( - self, - spec: "_importance_time.Spec", - working_nodes, - broken_nodes, - x, - window, - state, - ) -> dict: - """An importance measure at the times ``x`` or over the window - ``[0, window)`` (#191, see ``_importance_time``), from new or from - the components' ``state``.""" - if x is None and window is None: - raise ValueError( - "state is where the components start from, for a measure at " - "times (x) or over a window (window): give one of them. The " - "long-run measure does not depend on it." - ) - if spec.name == _importance_time.FUSSELL_VESELY: - if spec.fv_type not in ("c", "p"): - raise ValueError( - "fv_type must be either 'c' (cut-set) or 'p' (path-set), " - f"fv_type={spec.fv_type!r} was given." - ) - if spec.method not in ("exact", "rare_event"): - raise ValueError( - "method must be 'exact' or 'rare_event', " - f"got {spec.method!r}." - ) - if spec.name == _importance_time.CRITICALITY and spec.kind not in ( - "failure", - "success", - ): - raise ValueError( - f"kind must be 'failure' or 'success', got {spec.kind!r}." - ) - return _importance_time.measure( - self, - spec, - working_nodes, - broken_nodes, - x=x, - window=window, - state=state, - ) + ------- + dict[Hashable, float] + Node name -> long-run availability, in ``[0, 1]``. - def _crew_held_importance( - self, measure: str, working_nodes, broken_nodes - ) -> dict: - """With limited repair crews (#146), an importance measure of each - node from the system's long-run unavailability with the node held - working, ``U(1_i)``, and held failed, ``U(0_i)``: the crews' Markov - chain solved without it, as held it needs no crew. ``"birnbaum"`` - is ``U(0_i) - U(1_i)``, ``"improvement"`` ``U - U(1_i)``, ``"raw"`` - ``U(0_i) / U`` and ``"rrw"`` ``U / U(1_i)``. A node already held - is held the other way for its own measure.""" - working = set(working_nodes or ()) - broken = set(broken_nodes or ()) - self._validate_node_overrides(working, broken) - base = np.float64(self.mean_unavailability(working, broken)) - out: dict = {} - for node in self.nodes: - up = np.float64( - self.mean_unavailability(working | {node}, broken - {node}) - ) - down = np.float64( - self.mean_unavailability(working - {node}, broken | {node}) - ) - with np.errstate(divide="ignore", invalid="ignore"): - out[node] = { - "birnbaum": down - up, - "improvement": base - up, - "raw": down / base, - "rrw": base / up, - }[measure] - return _squeeze_values(out) - - def _standby_rates(self, node) -> Tuple[float, float]: - """A standby group's units' failure and repair rates, for its exact - long-run values; raise if their lives and repair times are not - exponential.""" - component = self.components[node] - life = _constant_rate(component.reliability) - repair = _constant_rate(component.time_to_replace) - if life is None or repair is None: - raise NotImplementedError( - f"Component {node!r} is a standby group, whose exact " - "long-run values come from a Markov chain of its units, " - "which needs their lives and repair times exponential: " - "simulate it with availability() or cost()." - ) - return life, repair - - def _standby_long_run(self, node) -> "_standby_chain.StandbyLongRun": - """A standby group's long-run values, from its Markov chain (see - ``_standby_chain``). The crews do not tie it to other nodes here (see - ``_crews_couple``): its units' repairs wait only for each other, - with ``repair_crews`` crews when they are the crews' only jobs.""" - life, repair = self._standby_rates(node) - arrangement = self._standby[node] - return _standby_chain.long_run( - arrangement.units, - arrangement.k, - life, - repair, - arrangement.dormancy_factor, - arrangement.switching_probability, - self.repair_crews if self._crews_limited() else None, - ) + Raises + ------ + NotImplementedError + If a component is under block replacement with models its exact + values do not cover (a lifetime that is not a surpyval + parametric model with a density, dead-on-arrival units, repairs + that may never end, or repairs or maintenance far longer than + the interval), or has hidden failures its numerical values do + not cover (see ``mean_availability``); or, while a component can + wait for a repair crew, as for ``mean_availability``. - def _node_frequencies(self, node) -> Tuple[float, float, float]: - """A component's long-run failures, preventive replacements and - planned outages, per unit time: recursive for a nested - RepairableRBD (whose own preventive replacements are not this RBD's - to count), ``1 / (MTTF + MTTR)`` failures for a NonRepairable, and - from its renewal cycle for one under age replacement.""" - self._require_perfect_repair(node) - component = self.components[node] - if isinstance(component, RepairableRBD): - failures, planned = component._outage_frequencies() - return failures, 0.0, planned - if node in self._standby: - return self._standby_long_run(node).failure_frequency, 0.0, 0.0 - if node in self._inspection: - # At most one failure per inspection interval: the unit, down - # from its failure, is renewed at the inspection that finds it. - unit = self._tested_unit(node) - if unit is not None: - # Its tests that take it off line, working, are planned - # outages. - long_run = unit.long_run - return long_run.failures, 0.0, long_run.planned - life = self._tested_life(node) - if life is not None: - return life.failures, 0.0, 0.0 - rate, interval = self._inspected_rate(node) - if self._inspection[node].partial: - # It fails at its constant rate while it is up. - return rate * self._node_availability(node), 0.0, 0.0 - return float(-np.expm1(-rate * interval) / interval), 0.0, 0.0 - schedule = self._preventive.get(node) - if schedule is None: - return component.failure_frequency(), 0.0, 0.0 - _, cycle, failures, maintenances = self._maintenance_cycle( - node, schedule - ) - maintained = maintenances / cycle - planned = 0.0 if schedule.duration is None else maintained - return failures / cycle, maintained, planned - - def _maintenance_cycle( - self, node, schedule: _Preventive - ) -> Tuple[float, float, float, float]: - """A component's renewal cycle under preventive maintenance: its mean - up time, mean length, mean number of failures and mean number of - preventive replacements. - - Under age replacement a cycle ends at a failure or at a preventive - replacement, whichever comes first. Under block replacement it runs - from one block time at which the unit is up (and replaced) to the - next, with any failures and repairs in between (see - ``_block_replacement``); replaced on condition, from one inspection - that replaces the unit to the next (see ``_condition_replacement``). - It is computed once and kept.""" - self._require_no_opportunities(node) - self._require_perfect_repair(node) - component = self.components[node] - if schedule.policy == "block": - block = self._block_cycle(node) - return block.up, block.length, block.failures, 1.0 - if schedule.policy == "condition": - kept = self._block_cycle(node) - return kept.up, kept.length, kept.failures, kept.replaced - # Kept by interval, as a search over intervals revisits them. - cache = self.__dict__.setdefault("_age_cycles", {}) - key = ( - node, - id(component.reliability), - id(component.time_to_replace), - float(schedule.interval), - id(schedule.duration), - ) - if key in cache: - return cache[key] - up = float(component.avg_replacement_time(schedule.interval)) - survives = float( - np.ravel(component.reliability_function(schedule.interval))[0] - ) - fails = _failed_by(component, schedule.interval) - maintenance = ( - 0.0 if schedule.duration is None else model_mean(schedule.duration) - ) - cycle = ( - up - + fails * model_mean(component.time_to_replace) - + survives * maintenance - ) - cache[key] = (up, cycle, fails, survives) - return cache[key] + Examples + -------- + >>> import surpyval as surv + >>> from repyability import RepairableRBD + >>> E = surv.Exponential.from_params + >>> rbd = RepairableRBD( + ... [("s", "a"), ("a", "b"), ("b", "t")], + ... { + ... "a": {"reliability": E([0.2]), "repairability": E([1.0])}, + ... "b": {"reliability": E([0.5]), "repairability": "instant"}, + ... }, + ... ) + >>> availability = rbd.node_availability() + >>> {node: round(a, 4) for node, a in availability.items()} + {'a': 0.8333, 'b': 1.0} + """ + return _long_run.node_availability(self) def system_failure_frequency( self, @@ -20943,83 +5294,9 @@ def system_failure_frequency( >>> round(rbd.system_failure_frequency(working_nodes=["a"]), 4) 0.3333 """ - return self._outage_frequencies(working_nodes, broken_nodes)[0] - - def _outage_frequencies( - self, - working_nodes: Optional[Collection[Hashable]] = None, - broken_nodes: Optional[Collection[Hashable]] = None, - ) -> Tuple[float, float]: - """The system's long-run failures and planned outages per unit time: - ``sum_i I_B(i) * omega_i`` over the nodes' failures, and the same - over their planned outages (a node's outage takes the system down - when the node is critical, which it is with probability - ``I_B(i)``): the sum of ``_outage_terms``.""" - terms, planned = self._outage_terms(working_nodes, broken_nodes) - failures = 0.0 - for _, term in terms: - failures += term - return failures, planned - - def _outage_terms( - self, - working_nodes: Optional[Collection[Hashable]] = None, - broken_nodes: Optional[Collection[Hashable]] = None, - ) -> Tuple[List[Tuple[Any, float]], float]: - """The system's long-run failures per unit time, as the terms of - their sum, each with what causes it (a node, or a common-cause - group's members together, for its causes that strike more than one - of them), and its planned outages per unit time (see - ``_outage_frequencies``). With limited repair crews, see - ``_chain_outage_terms``; with common-cause groups, - ``_ccf_outage_terms``.""" - if self.ccf_groups: - return self._ccf_outage_terms(working_nodes, broken_nodes) - if self._crews_couple(): - return self._chain_outage_terms(working_nodes, broken_nodes) - times, weights = self._long_run_grid() - availability = self._probabilities_with_overrides( - self._availabilities_at(times), working_nodes, broken_nodes + return _long_run.system_failure_frequency( + self, working_nodes=working_nodes, broken_nodes=broken_nodes ) - unavailability = self._failures_with_overrides( - self._unavailabilities_at(times), working_nodes, broken_nodes - ) - forced = (set() if working_nodes is None else set(working_nodes)) | ( - set() if broken_nodes is None else set(broken_nodes) - ) - # Each node's Birnbaum importance, from the nodes' unavailabilities - # as well as their availabilities, so that a small one (a node - # backed up by reliable redundancy) keeps its precision. - birnbaum = super()._birnbaum_importance( - availability, node_failures=unavailability - ) - terms: List[Tuple[Any, float]] = [] - planned = 0.0 - blocks = set(self._block_nodes()) - for node in self.components: - if node in forced: - # A forced node never changes state, so it contributes no - # system failures. - continue - importance = np.asarray(birnbaum[node]) - if node in self._inspection: - # It fails at its constant rate whenever it is up (any other - # life: at its profile's rate). - node_failures: Any = self._tested_intensity( - node, times, availability[node] - ) - node_planned: Any = 0.0 - elif node in blocks: - # Its failure intensity varies over its block interval; its - # replacements fall at block times (counted below). - node_failures = self._block_profile(node, times, rates=True) - node_planned = 0.0 - else: - node_failures, _, node_planned = self._node_frequencies(node) - terms.append((node, float(weights @ (importance * node_failures)))) - planned += float(weights @ (importance * node_planned)) - planned += self._calendar_outages(working_nodes, broken_nodes) - return terms, planned def mean_time_between_failures( self, @@ -21077,8 +5354,9 @@ def mean_time_between_failures( >>> round(rbd.mean_time_between_failures(), 4) # MUT + MDT 2.5714 """ - omega = self.system_failure_frequency(working_nodes, broken_nodes) - return 1.0 / omega if omega > 0.0 else float("inf") + return _long_run.mean_time_between_failures( + self, working_nodes=working_nodes, broken_nodes=broken_nodes + ) def mean_up_time( self, @@ -21136,11 +5414,9 @@ def mean_up_time( >>> round(rbd.mean_up_time(working_nodes=["a"]), 4) # b alone 2.0 """ - availability = self.mean_availability(working_nodes, broken_nodes) - omega = sum(self._outage_frequencies(working_nodes, broken_nodes)) - if omega > 0.0: - return float(availability) / omega - return float("inf") if availability > 0.0 else 0.0 + return _long_run.mean_up_time( + self, working_nodes=working_nodes, broken_nodes=broken_nodes + ) def mean_down_time( self, @@ -21198,11 +5474,9 @@ def mean_down_time( >>> rbd.mean_down_time(working_nodes=["a"]) # never down 0.0 """ - unavailability = self.mean_unavailability(working_nodes, broken_nodes) - omega = sum(self._outage_frequencies(working_nodes, broken_nodes)) - if omega > 0.0: - return unavailability / omega - return 0.0 if unavailability <= 0.0 else float("inf") + return _long_run.mean_down_time( + self, working_nodes=working_nodes, broken_nodes=broken_nodes + ) @_times_first() def birnbaum_importance( @@ -21218,7 +5492,7 @@ def birnbaum_importance( evaluated at the nodes' long-run availabilities. In the guide's Greeks it is *delta*: how far the system moves with each - component (see [Sensitivities: the Greeks](guide/greeks.md)). + component (see [Sensitivities: the Greeks](../guide/greeks.md)). Exact, with no simulation: ``I_B(i) = A_sys(A_i = 1) - A_sys(A_i = 0)``, the system's long-run availability with node i @@ -21251,7 +5525,7 @@ def birnbaum_importance( improvement potential, RAW and RRW are built on the same values; the criticality and Fussell-Vesely measures are probabilities over the chain's states. With a crew for each component they are the - independent ones (#146). + independent ones. Parameters ---------- @@ -21266,7 +5540,12 @@ def birnbaum_importance( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21314,30 +5593,13 @@ def birnbaum_importance( >>> round(rbd.birnbaum_importance(working_nodes=["a"])["b"], 4) 1.0 """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec(_importance_time.BIRNBAUM, "failure"), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "birnbaum", working_nodes, broken_nodes - ) - if self._crews_couple(): - return self._crew_held_importance( - "birnbaum", working_nodes, broken_nodes - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._birnbaum_importance( - node_probabilities, weights, node_failures - ) + return _repairable_importance.birnbaum_importance( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, ) @_times_first() @@ -21373,7 +5635,12 @@ def improvement_potential( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21417,30 +5684,13 @@ def improvement_potential( >>> {node: round(p, 4) for node, p in potential.items()} {'a': 0.1111, 'b': 0.2778} """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec(_importance_time.IMPROVEMENT, "failure"), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "improvement", working_nodes, broken_nodes - ) - if self._crews_couple(): - return self._crew_held_importance( - "improvement", working_nodes, broken_nodes - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._improvement_potential( - node_probabilities, weights, node_failures - ) + return _repairable_importance.improvement_potential( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, ) @_times_first() @@ -21479,7 +5729,12 @@ def risk_achievement_worth( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21526,30 +5781,13 @@ def risk_achievement_worth( >>> {node: round(r, 4) for node, r in raw.items()} {'a': 2.25, 'b': 2.25} """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec(_importance_time.RAW, "failure"), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "raw", working_nodes, broken_nodes - ) - if self._crews_couple(): - return self._crew_held_importance( - "raw", working_nodes, broken_nodes - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._risk_achievement_worth( - node_probabilities, weights, node_failures - ) + return _repairable_importance.risk_achievement_worth( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, ) @_times_first() @@ -21589,7 +5827,12 @@ def risk_reduction_worth( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21633,30 +5876,13 @@ def risk_reduction_worth( >>> {node: round(r, 4) for node, r in rrw.items()} {'a': 1.3333, 'b': 2.6667} """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec(_importance_time.RRW, "failure"), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "rrw", working_nodes, broken_nodes - ) - if self._crews_couple(): - return self._crew_held_importance( - "rrw", working_nodes, broken_nodes - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._risk_reduction_worth( - node_probabilities, weights, node_failures - ) + return _repairable_importance.risk_reduction_worth( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, ) @_times_first() @@ -21709,7 +5935,12 @@ def criticality_importance( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21769,29 +6000,14 @@ def criticality_importance( >>> {node: round(c, 4) for node, c in criticality.items()} {'a': 1.0, 'b': 1.0} """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec(_importance_time.CRITICALITY, kind), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "criticality", working_nodes, broken_nodes, kind=kind - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._criticality_importance( - node_probabilities, - kind, - weights=weights, - node_failures=node_failures, - ) + return _repairable_importance.criticality_importance( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + kind=kind, + x=x, + window=window, + state=state, ) @_times_first() @@ -21853,7 +6069,12 @@ def fussell_vesely( the long run, by default None: at each, the nodes are up with their point availabilities then (see ``point_availability``), and with limited repair crews the crews' chain is followed to - it (#191). + it. At the instant of a test or a scheduled replacement the + measure is of the state just after it (as the point + availability is): where that state cannot be down (a pair + tested together, both just found working), a ratio measure + is 0/0, NaN; evaluate just before the instant for the state + before it. window : float or array-like, optional The length of a window ``[0, window)`` to evaluate it over instead, by default None: a ratio measure is then the ratio of @@ -21901,39 +6122,15 @@ def fussell_vesely( >>> {node: round(v, 4) for node, v in fv.items()} {'a': 0.375, 'b': 0.75} """ - if x is not None or window is not None or state is not None: - return self._importance_over_time( - _importance_time.Spec( - _importance_time.FUSSELL_VESELY, - "failure", - fv_type=fv_type, - method=method, - ), - working_nodes, - broken_nodes, - x, - window, - state, - ) - if self.ccf_groups: - return self._ccf_long_run_measure( - "fussell_vesely", - working_nodes, - broken_nodes, - fv_type=fv_type, - method=method, - ) - node_probabilities, node_failures, weights, _ = ( - self._importance_probabilities(working_nodes, broken_nodes) - ) - return _squeeze_values( - super()._fussell_vesely( - node_probabilities, - fv_type, - method, - weights=weights, - node_failures=node_failures, - ) + return _repairable_importance.fussell_vesely( + self, + fv_type=fv_type, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + method=method, + x=x, + window=window, + state=state, ) @_times_first() @@ -21951,10 +6148,10 @@ def parameter_sensitivity( ) -> dict: """How the system's availability (or its cost rate) moves with each lever: the derivative in each of its components' parameters, and - the change one more standby unit or repair crew makes (#192). + the change one more standby unit or repair crew makes. In the guide's Greeks it is the levers' *deltas* (see [Sensitivities: - the Greeks](guide/greeks.md)). + the Greeks](../guide/greeks.md)). The levers are each component's life and repair models' parameters (``"reliability.alpha"``, ``"repairability.beta"``, @@ -22045,7 +6242,12 @@ def parameter_sensitivity( ``{key: {lever: sensitivity}}``, the key a node, a common-cause group's tuple of members, or None for the repair crews; floats, but for an array ``x`` arrays in its shape. - With a tuple ``of``, a dict of these by quantity. + With a tuple ``of``, a dict of these by quantity. Over time + (``x`` or ``window``) the ``repair.q`` lever of a minimally + repaired component (``q = 1``, the one imperfect repair the + values over time take) is left out: the derivative in ``q`` + does not exist there (``levers()`` still lists it, for + ``with_levers``). Raises ------ @@ -22079,13 +6281,13 @@ def parameter_sensitivity( >>> round(sensitivity["repairability.failure_rate"], 4) 0.8264 """ - return _sensitivity.sensitivity( + return _sensitivity.parameter_sensitivity( self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, x=x, window=window, state=state, - working_nodes=working_nodes, - broken_nodes=broken_nodes, rel_step=rel_step, of=of, unit_costs=unit_costs, @@ -22093,7 +6295,7 @@ def parameter_sensitivity( def levers(self) -> List[Lever]: """The levers that ``parameter_sensitivity`` moves, in the order it - reports them (#244): each component's life and repair models' + reports them: each component's life and repair models' parameters, its maintenance's and tests' options, its standby group's and its imperfect repair's, each common-cause group's, and the repair crews. @@ -22105,6 +6307,10 @@ def levers(self) -> List[Lever]: others. Nothing is worked out, so a report can name the levers and show their values without the sensitivities; [`with_levers`][repyability.RepairableRBD.with_levers] sets them. + The levers are the components' own models' parameters and + options: those of the units inside a composite life model (a + ``StandbyModel``, a ``LoadSharingModel``, a ``DegradingNode``) or + of a nested diagram are not reached (see the guide's Greeks). Returns ------- @@ -22128,7 +6334,7 @@ def levers(self) -> List[Lever]: return _sensitivity.public_levers(self) def with_levers(self, values) -> "RepairableRBD": - """A copy of this RBD with levers at new values (#244), each moved + """A copy of this RBD with levers at new values, each moved as ``parameter_sensitivity`` moves it, so that a what-if agrees with the sensitivities: a test interval takes the full tests with it (every so many tests, as before), and a common-cause group's @@ -22180,33 +6386,6 @@ def with_levers(self, values) -> "RepairableRBD": """ return _sensitivity.with_levers(self, values) - def _uncertainty( - self, of: str, x, uncertainty, n_draws, seed, sampling, state - ) -> UncertaintyResult: - """The quantity ``of`` over draws of the uncertain models (see - ``_repairable_uncertainty``), as an ``UncertaintyResult``.""" - from repyability.rbd import _repairable_uncertainty as drawn - - drawn.check(self, of, x, state) - # The diagram's own value first: what it refuses, every draw would. - nominal = drawn.value(self, of, x, state) - inputs, groups = drawn.sources(self, uncertainty) - models, group_models = drawn.draws( - self, inputs, groups, n_draws, seed, sampling - ) - samples = drawn.samples( - self, of, x, state, inputs, models, group_models, n_draws - ) - if x is None or np.ndim(x) == 0: - return UncertaintyResult( - samples=samples[:, 0], - nominal=float(nominal[0]), - n_draws=n_draws, - ) - return UncertaintyResult( - samples=samples, nominal=nominal, n_draws=n_draws - ) - def mean_availability_uncertainty( self, uncertainty: Optional[Dict[Hashable, Any]] = None, @@ -22216,7 +6395,7 @@ def mean_availability_uncertainty( sampling: str = "random", ) -> UncertaintyResult: """The long-run availability over plausible models of the components - (parameter uncertainty, #200). + (parameter uncertainty). A component's models are estimated from data, so their parameters are uncertain: *epistemic* uncertainty, about what the models are, @@ -22240,13 +6419,13 @@ def mean_availability_uncertainty( inputs, a role of it under one: a fleet's life fit, shared by two pumps whose repairs were recorded apart, is ``{("a", "b"): {"reliability": "fit"}, "a": {"repairability": "fit"}, "b": - {"repairability": "fit"}}``, the life drawn once a draw (#214). An + {"repairability": "fit"}}``, the life drawn once a draw. An input drawn without other nodes that hold the same model object, which then keep it as fitted, is warned about. A node's uncertainty is - ``"fit"``: every one of its models that is a surpyval fit with a - parameter covariance (``hess_inv``), drawn from its normal + parameter covariance (``covariance()``), drawn from its normal approximation (on the log scale for a positive parameter, the logit scale for one in (0, 1)); - ``{role: uncertainty}``: each model named drawn as its @@ -22261,7 +6440,9 @@ def mean_availability_uncertainty( By default, every model that is a fit with a parameter covariance, drawn once for all the nodes holding that fitted object in that role, and for one common-cause group's members - together. + together. The models inside a nested ``RepairableRBD`` are + not drawn, by default or given: the nested diagram's own + uncertainty methods take them. n_draws : int, optional The number of draws, by default 1000: one evaluation each. seed : int, optional @@ -22285,9 +6466,11 @@ def mean_availability_uncertainty( If ``uncertainty`` names an unknown node, a nested RBD or a junction, a node twice, nodes of a population with different models, a model a node does not have, or a common-cause group's - members apart; if a model's uncertainty cannot be drawn; if - ``n_draws`` or ``sampling`` is invalid; or as - ``mean_availability`` does for a draw. + members apart; if no ``uncertainty`` is given and no + component's model is a fit (as ``routes.Refused``); if a + model's uncertainty cannot be drawn; if ``n_draws`` or + ``sampling`` is invalid; or as ``mean_availability`` does for + a draw. Examples -------- @@ -22320,14 +6503,12 @@ def mean_availability_uncertainty( >>> round(lower, 4), round(upper, 4) (0.9392, 0.9577) """ - return self._uncertainty( - "mean_availability", - None, - uncertainty, - n_draws, - seed, - sampling, - None, + return _repairable_uncertainty.mean_availability_uncertainty( + self, + uncertainty=uncertainty, + n_draws=n_draws, + seed=seed, + sampling=sampling, ) def point_availability_uncertainty( @@ -22341,7 +6522,7 @@ def point_availability_uncertainty( state=None, ) -> UncertaintyResult: """The point availability at times ``x`` (from new, or from - ``state``) over plausible models of the components (#200), each draw + ``state``) over plausible models of the components, each draw worked out as ``point_availability`` works it out: the inputs and their uncertainty as for ``mean_availability_uncertainty``. @@ -22375,14 +6556,14 @@ def point_availability_uncertainty( As for ``mean_availability_uncertainty`` and ``point_availability``. """ - return self._uncertainty( - "point_availability", - x, - uncertainty, - n_draws, - seed, - sampling, - state, + return _repairable_uncertainty.point_availability_uncertainty( + self, + x=x, + uncertainty=uncertainty, + n_draws=n_draws, + seed=seed, + sampling=sampling, + state=state, ) def mission_availability_uncertainty( @@ -22396,7 +6577,7 @@ def mission_availability_uncertainty( state=None, ) -> UncertaintyResult: """The mission availability over ``[0, t]`` (from new, or from - ``state``) over plausible models of the components (#200), each draw + ``state``) over plausible models of the components, each draw worked out as ``mission_availability`` works it out: the inputs and their uncertainty as for ``mean_availability_uncertainty``. @@ -22428,14 +6609,14 @@ def mission_availability_uncertainty( As for ``mean_availability_uncertainty`` and ``mission_availability``. """ - return self._uncertainty( - "mission_availability", - t, - uncertainty, - n_draws, - seed, - sampling, - state, + return _repairable_uncertainty.mission_availability_uncertainty( + self, + t=t, + uncertainty=uncertainty, + n_draws=n_draws, + seed=seed, + sampling=sampling, + state=state, ) def expected_cost_rate_uncertainty( @@ -22446,8 +6627,8 @@ def expected_cost_rate_uncertainty( seed=None, sampling: str = "random", ) -> UncertaintyResult: - """The long-run cost rate over plausible models of the components - (#200), each draw worked out as ``expected_cost_rate`` works it + """The long-run cost rate over plausible models of the components, + each draw worked out as ``expected_cost_rate`` works it out: the inputs and their uncertainty as for ``mean_availability_uncertainty``. @@ -22474,14 +6655,12 @@ def expected_cost_rate_uncertainty( As for ``mean_availability_uncertainty`` and ``expected_cost_rate``. """ - return self._uncertainty( - "expected_cost_rate", - None, - uncertainty, - n_draws, - seed, - sampling, - None, + return _repairable_uncertainty.expected_cost_rate_uncertainty( + self, + uncertainty=uncertainty, + n_draws=n_draws, + seed=seed, + sampling=sampling, ) def uncertainty_importance( @@ -22499,11 +6678,11 @@ def uncertainty_importance( ) -> UncertaintyImportance: """Which input's parameter uncertainty makes the availability (or the cost rate) uncertain: each uncertain input's share of its - variance (vega, #200), the inputs as for + variance (vega), the inputs as for ``mean_availability_uncertainty``. In the guide's Greeks it is *vega*: whose uncertainty widens the answer - (see [Sensitivities: the Greeks](guide/greeks.md)). + (see [Sensitivities: the Greeks](../guide/greeks.md)). ``of`` is the quantity: ``"mean_availability"`` (the default, the long run), ``"point_availability"`` at the times ``x``, @@ -22516,7 +6695,7 @@ def uncertainty_importance( its derivatives in input ``k``'s parameters (``parameter_ sensitivity``'s, of every one of its uncertain models; a population's summed over its nodes, which move together) and - ``Sigma_k`` their covariance (a fit's ``hess_inv``, or the + ``Sigma_k`` their covariance (a fit's ``covariance()``, or the variances of the distributions given). The inputs are independent, so each one's part is its own term, and the shares add up to 1. A list of models has no parameters to move: it needs @@ -22598,46 +6777,17 @@ def uncertainty_importance( >>> {n: round(s, 2) for n, s in parts.first_order.items()} {'pump': 0.74, 'valve': 0.26} """ - from repyability.rbd import _repairable_uncertainty as drawn - - # parameter_sensitivity's name for the cost rate is taken too (#232). - of = drawn.QUANTITY_NAMES.get(of, of) if isinstance(of, str) else of - drawn.check(self, of, x, state) - if method not in ("delta", "sobol"): - raise ValueError( - f"method must be 'delta' or 'sobol', got {method!r}." - ) - inputs, groups = drawn.sources(self, uncertainty) - keys = [item.key for item in inputs] + [ - groups[i][0] for i in sorted(groups) - ] - if method == "delta": - parts = drawn.delta(self, of, x, state, inputs, groups, rel_step) - variance = np.sum(parts, axis=0) - with np.errstate(divide="ignore", invalid="ignore"): - shares = [ - np.where(variance > 0.0, part / variance, np.nan) - for part in parts - ] - first = total = dict(zip(keys, shares)) - else: - firsts, totals, variance = drawn.sobol( - self, of, x, state, inputs, groups, n_draws, seed, sampling - ) - first, total = dict(zip(keys, firsts)), dict(zip(keys, totals)) - scalar = x is None or np.ndim(x) == 0 - - def shaped(values): - values = np.asarray(values, dtype=float).reshape(-1) - if scalar: - return float(values[0]) - return values.reshape(np.shape(x)) - - return UncertaintyImportance( + return _repairable_uncertainty.uncertainty_importance( + self, + x=x, + uncertainty=uncertainty, + of=of, method=method, - variance=shaped(variance), - first_order={key: shaped(v) for key, v in first.items()}, - total={key: shaped(v) for key, v in total.items()}, + n_draws=n_draws, + seed=seed, + sampling=sampling, + rel_step=rel_step, + state=state, ) @_times_first() @@ -22658,11 +6808,11 @@ def differential_importance( ) -> dict: """Each node's (or lever's) share of the change in the system's availability when they all change together: the differential - importance measure (DIM, Borgonovo & Apostolakis, 2001; #193), in + importance measure (DIM, Borgonovo & Apostolakis, 2001), in the long run, at times ``x`` or over a window. In the guide's Greeks it is *DIM*, the shares of a change (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``DIM_i = dA/dtheta_i dtheta_i / sum_j dA/dtheta_j dtheta_j``, so the shares add up to 1, and a group's share is the sum of its @@ -22708,9 +6858,11 @@ def differential_importance( Nodes held failed, likewise. x : float or array-like, optional Times from new (or from ``state``), as for - ``birnbaum_importance``; by default the long run. - window : float, optional - Over ``[0, window)`` instead, as for ``birnbaum_importance``. + ``birnbaum_importance`` (at a test's instant, of the state + just after it); by default the long run. + window : float or array-like, optional + Over ``[0, window)`` instead, as for ``birnbaum_importance`` + (over the nodes, one share per window of several). state : dict or str, optional With ``x`` or ``window``, the components' current states. over : str, optional @@ -22736,7 +6888,8 @@ def differential_importance( ------- dict The shares, by node, ``(key, lever)`` or group name: floats - but for an array ``x`` (NaN where the total is 0). + but for an array ``x`` or ``window`` (NaN where the total is + 0). Raises ------ @@ -22774,354 +6927,22 @@ def differential_importance( >>> round(shares["b"], 4) 0.7143 """ - from ._differential import check, flattened, shares - - check(over, change, kind) - held = set(working_nodes or ()) | set(broken_nodes or ()) - timed = { - key: value - for key, value in (("x", x), ("window", window), ("state", state)) - if value is not None - } - if over == "components": - if change == "uniform": - values = self.birnbaum_importance( - working_nodes, broken_nodes, **timed - ) - else: - values = self.criticality_importance( - working_nodes, broken_nodes, kind, **timed - ) - contributions = { - node: (0.0 * np.asarray(v) if node in held else v) - for node, v in values.items() - } - else: - derivatives = flattened( - self.parameter_sensitivity( - working_nodes, - broken_nodes, - x=x, - window=window, - state=state, - rel_step=rel_step, - ) - ) - continuous = { - (lever.key, lever.name): lever.value - for lever in _sensitivity._levers(self) - if not lever.discrete - } - contributions = { - key: ( - np.asarray(d, dtype=float) - * (continuous[key] if change == "proportional" else 1.0) - ) - for key, d in derivatives.items() - if key in continuous - } - scalar = x is None or np.ndim(x) == 0 - return shares(contributions, groups, scalar, improving) - - def _independent_rates( - self, flat, horizon: float, scale: float, working, broken, states - ) -> Tuple[dict, np.ndarray, dict]: - """``availability_rate``'s parts with independent components: each - component's Birnbaum importance times its own rate, the times the - components jump, and each one's part in each jump (see - ``_rates``).""" - # A little past the last time, for the differences there. - curves = self._availability_curves( - 1.01 * scale, working | broken, state=states - ) - - def importances(values: dict) -> dict: - size = len(next(iter(values.values()))) if values else len(flat) - return self._importances( - self._filled(values, size, working, broken) - )[0] - - importance = importances( - {node: curve.at(flat) for node, curve in curves.items()} - ) - parts = { - node: importance[node] - * _rates.derivative(curves[node], flat, scale) - for node in curves - } - jump_times, before, after = _rates.jumps(curves, horizon) - return ( - parts, - jump_times, - _rates.split_jumps(importances, before, after), - ) - - def _crew_rates( - self, flat, horizon: float, scale: float, working, broken, states - ) -> Tuple[dict, np.ndarray, dict]: - """``availability_rate``'s parts with limited repair crews (#199). - The system is up with probability ``sum_m p(t) u_m w_m(t)`` (see - ``_chain_transient.CrewCurve``), so its rate is ``sum_m p(t) Q u_m - w_m(t)`` plus ``sum_m p(t) u_m w_m'(t)``. Split by the component - whose failures and repairs make each transition, ``Q = sum_i Q_i`` - (``_crew_chain.CrewChain.split``), the first is the chain's - components' parts, each ``p(t) Q_i u_m`` one more vector of the - crews' uniformized chain, exact; the second, the nested RBDs', each - one's Birnbaum importance over the chain times its own rate, as for - independent components, and their jumps are split likewise.""" - nested = self._require_crew_over_time(frozenset(working | broken)) - chain = self._crew_chain(frozenset(working | broken)) - size, count = len(chain.probabilities), len(chain.nodes) - patterns = 2 ** len(nested) - ups = [ - self._crew_vectors( - chain, - { - node: np.full(size, float((m >> j) & 1)) - for j, node in enumerate(nested) - }, - working, - broken, - "p", - )[0] - for m in range(patterns) - ] - uniformized = self._uniformized( - "The repair crews'", - chain.generator, - self._crew_start(chain, states), - chain.probabilities, - np.column_stack(ups + [chain.split(up) for up in ups]), + return _repairable_importance.differential_importance( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, + over=over, + change=change, + kind=kind, + improving=improving, + groups=groups, + rel_step=rel_step, ) - curves = { - node: self.components[node]._nested_curve( - 1.01 * scale, start=states.get(node) - ) - for node in nested - } - def importances(values: dict, times: np.ndarray) -> dict: - columns = uniformized.values(times)[:, :patterns] - return dict( - zip( - nested, - _chain_transient.pattern_importances( - columns, [values[node] for node in nested] - ), - ) - ) - - columns = uniformized.values(flat) - available = { - node: np.asarray(curve.at(flat), dtype=float) - for node, curve in curves.items() - } - weights = _chain_transient.pattern_weights( - [available[node] for node in nested] - ) - parts: Dict[Hashable, np.ndarray] = {} - for k, node in enumerate(chain.nodes): - own = columns[:, patterns + k + count * np.arange(patterns)] - parts[node] = np.sum(own * weights, axis=1) - importance = importances(available, flat) - for node in nested: - parts[node] = importance[node] * _rates.derivative( - curves[node], flat, scale - ) - jump_times, before, after = _rates.jumps(curves, horizon) - - def along(values: dict) -> dict: - # Each jump's time, for each point of the path between the - # values either side of it. - points = len(next(iter(values.values()))) // len(jump_times) - return importances(values, np.repeat(jump_times, points)) - - return parts, jump_times, _rates.split_jumps(along, before, after) - - def _groups_rates( - self, flat, horizon: float, scale: float, working, broken, states - ) -> Tuple[dict, np.ndarray, dict]: - """``availability_rate``'s parts with common-cause groups (#199). - The nodes outside the groups are independent of the groups and of - each other, so each one's part is its Birnbaum importance (over the - groups' joint states, see ``_ccf_chain.GroupsSystem``) times its own - rate. A group's members' states move by its chain, ``p_g'(t) = - p_g(t) Q_g``, so its part is ``p_g(t) Q_g a_g(t)``, ``a_g(t)`` the - system's availability with the group in each of its states then. - Split by whose each move is (``_ccf_chain.Move``), ``Q_g`` gives - each member the part of its own cause and its repairs, and the - group, under the tuple of its members, that of its causes that - strike more than one. The jumps are split by ``_groups_jumps``.""" - self._require_free_members(working, broken) - self._require_groups_over_time(states) - curve = self._groups_curve(1.01 * scale, working, broken, "p", states) - curves, system = curve.curves, curve.system - values = { - node: np.asarray(c.at(flat), dtype=float) - for node, c in curves.items() - } - importance = system.importance(values, flat) - parts: Dict[Any, np.ndarray] = { - node: importance[node] * _rates.derivative(c, flat, scale) - for node, c in curves.items() - } - for number, group in enumerate(system.groups): - # The system's unavailability with the group in each of its - # combinations, and the group's chain's states. - down = system.given(values, flat, number) - chances = group.raw(flat) - for label, begin, end, rate in group.moves: - key = ( - group.members[label] - if label != _ccf_chain.SHARED - else tuple(group.members) - ) - fall = ( - down[:, group.combination[end]] - - down[:, group.combination[begin]] - ) - part = -np.sum(chances[:, begin] * rate * fall, axis=1) - parts[key] = parts.get(key, 0.0) + part - jump_times, split = self._groups_jumps(curves, system, horizon) - return parts, jump_times, split - - def _groups_jumps( - self, curves: dict, system, horizon: float - ) -> Tuple[np.ndarray, dict]: - """The times in ``(0, horizon]`` at which the system with - common-cause groups jumps (a scheduled event of a node outside the - groups, a hidden group's tests), and each one's part in each jump. - Where only nodes outside the groups jump, independent and the - system multilinear in them, ``_rates.split_jumps`` splits it. Where - a test is among what jumps, the members' states are no longer - independent, and the jump is split by the Shapley value of what - jumps then (the nodes, and each test, under its member's name), - the system worked out exactly with each subset of them done: for - independent nodes alone, the same split.""" - out_times, before, after = _rates.jumps(curves, horizon) - tests = [group.tests(horizon) for group in system.groups] - tested = sorted({t for by in tests for t in by}) - times = np.unique(np.concatenate([out_times, np.array(tested)])) - split: Dict[Any, np.ndarray] = {} - - def part(key) -> np.ndarray: - if key not in split: - split[key] = np.zeros(len(times)) - return split[key] - - alone = np.array([t not in set(tested) for t in out_times], bool) - if alone.any(): - at = out_times[alone] - - def importances(values: dict) -> dict: - points = len(next(iter(values.values()))) // len(at) - return system.importance(values, np.repeat(at, points)) - - shares = _rates.split_jumps( - importances, - {node: v[alone] for node, v in before.items()}, - {node: v[alone] for node, v in after.items()}, - ) - rows = np.searchsorted(times, at) - for node, values in shares.items(): - part(node)[rows] += values - for time in tested: - row = int(np.searchsorted(times, time)) - just = np.nextafter(time, -np.inf) - low = { - node: float(np.ravel(c.at(np.array([just])))[0]) - for node, c in curves.items() - } - high = { - node: float(np.ravel(c.at(np.array([time])))[0]) - for node, c in curves.items() - } - players: List[Tuple[Any, Any]] = [ - (node, None) - for node in curves - if abs(high[node] - low[node]) > _rates.JUMP - ] - for number, by in enumerate(tests): - for test in by.get(time, []): - players.append((number, test)) - for (who, test), value in zip( - players, self._shapley(system, players, low, high, time) - ): - key = ( - who - if test is None - else system.groups[who].members[test.member] - ) - part(key)[row] += value - return times, split - - #: The most that may jump at one instant with a hidden common-cause - #: group's tests among them (the Shapley value takes every subset). - _MAX_PLAYERS = 12 - - def _shapley( - self, system, players: list, low: dict, high: dict, time: float - ) -> np.ndarray: - """Each player's Shapley value in the system's jump at ``time`` - (see ``_groups_jumps``): the system's availability with each - subset of the players done, a node at its value after the jump - (``high``) rather than before (``low``), and a test applied to its - group's states just before the time.""" - count = len(players) - if count > self._MAX_PLAYERS: - raise NotImplementedError( - f"{count} nodes and tests change at time {time:g}, with a " - "common-cause group's tests among them: their jump is split " - f"for at most {self._MAX_PLAYERS}, as yet. Simulate it with " - "availability()." - ) - subsets = np.arange(2**count) - values = {} - for node in low: - if (node, None) in players: - j = players.index((node, None)) - values[node] = np.where( - (subsets >> j) & 1, high[node], low[node] - ).astype(float) - else: - values[node] = np.full(len(subsets), high[node]) - chances = [] - just = np.array([np.nextafter(time, -np.inf)]) - for number, group in enumerate(system.groups): - mine = [ - (j, test) - for j, (who, test) in enumerate(players) - if test is not None and who == number - ] - if not mine: - chances.append( - np.repeat( - group.probabilities(np.array([time])), len(subsets), 0 - ) - ) - continue - start = group.raw(just) - table = np.empty((len(subsets), len(group.down))) - for subset in subsets: - done = [test for j, test in mine if (subset >> j) & 1] - table[subset] = group.grouped(group.tested(start, done))[0] - chances.append(table) - worth = system.availability(values, chances) - out = np.zeros(count) - weight = [ - math.factorial(k) - * math.factorial(count - k - 1) - / math.factorial(count) - for k in range(count) - ] - sizes = np.array([bin(int(s)).count("1") for s in subsets]) - for j in range(count): - without = subsets[(subsets >> j) & 1 == 0] - out[j] = sum( - weight[sizes[s]] * (worth[s | (1 << j)] - worth[s]) - for s in without - ) - return out + _MAX_PLAYERS = _repairable_importance._MAX_PLAYERS def availability_rate( self, @@ -23133,10 +6954,10 @@ def availability_rate( ) -> RateBreakdown: """How fast the system's availability is changing at each time ``x`` from new (or from the components' ``state``), and which - components are moving it (#195). + components are moving it. In the guide's Greeks it is *theta*: what is moving the system now (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). The components failing and recovering independently, the system's point availability is multilinear in theirs, so @@ -23153,7 +6974,7 @@ def availability_rate( numerical: to about ``1e-5`` of their size, less at 0 where a life's density is not smooth there. The down times kept off the grid (a repair or maintenance that starts at a known time) are - differentiated on their own scale, however short (#240), so the + differentiated on their own scale, however short, so the rates just after a scheduled event are as close. At a scheduled event a component's availability can jump: a block @@ -23166,7 +6987,7 @@ def availability_rate( With limited repair crews or common-cause groups the components no longer fail and recover independently, and the parts come from - the crews' or the groups' Markov chains (#199). The system's rate + the crews' or the groups' Markov chains. The system's rate is ``p(t) Q u``, ``p(t)`` the chain's states' probabilities, ``Q`` its generator and ``u`` the system up in each state; each transition is one component's failure or repair (a crew finishing @@ -23241,56 +7062,12 @@ def availability_rate( >>> rate.node_rate["p1"].round(5).tolist() [-3e-05, -0.00156] """ - times = _check_times(x) - working = set() if working_nodes is None else set(working_nodes) - broken = set() if broken_nodes is None else set(broken_nodes) - self._validate_node_overrides(working, broken) - forced = working | broken - states = self._states(state, forced) - flat = times.ravel() - horizon = float(flat.max()) if flat.size else 0.0 - scale = horizon if horizon > 0.0 else 1.0 - if self._crews_couple(): - parts, jump_times, split = self._crew_rates( - flat, horizon, scale, working, broken, states - ) - elif self.ccf_groups: - parts, jump_times, split = self._groups_rates( - flat, horizon, scale, working, broken, states - ) - else: - parts, jump_times, split = self._independent_rates( - flat, horizon, scale, working, broken, states - ) - keys = list(self.components) + [ - key for key in {**parts, **split} if key not in self.components - ] - node_rate: Dict[Hashable, np.ndarray] = {} - rate = np.zeros(len(flat)) - node_jumps: Dict[Hashable, np.ndarray] = {} - jumps = np.zeros(len(jump_times)) - for key in keys: - part = np.asarray(parts.get(key, np.zeros(len(flat))), dtype=float) - node_rate[key] = part - rate = rate + part - part = np.asarray( - split.get(key, np.zeros(len(jump_times))), dtype=float - ) - node_jumps[key] = part - jumps = jumps + part - - def shaped(values: np.ndarray): - if np.ndim(x) == 0: - return float(values[0]) - return values.reshape(times.shape) - - return RateBreakdown( - x=shaped(flat), - rate=shaped(rate), - node_rate={node: shaped(v) for node, v in node_rate.items()}, - jump_times=jump_times, - jumps=jumps, - node_jumps=node_jumps, + return _repairable_importance.availability_rate( + self, + x=x, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + state=state, ) @_times_first("window") @@ -23304,11 +7081,11 @@ def barlow_proschan_importance( ) -> dict: """Each component's share of the system's failures: the probability that a system failure is caused by the component's (Barlow & - Proschan, 1975; #195), in the long run or over a window from new + Proschan, 1975), in the long run or over a window from new (or from the components' ``state``). In the guide's Greeks it is *theta*, integrated: who caused the - failures (see [Sensitivities: the Greeks](guide/greeks.md)). + failures (see [Sensitivities: the Greeks](../guide/greeks.md)). A component's failure fails the system when the component is critical then, which it is with probability ``I_B^i(t)``, so the @@ -23328,7 +7105,7 @@ def barlow_proschan_importance( one that strikes a member alone for that member; with limited repair crews, the shares come from the crews' chain, its components' failures at their rates in each state, over a window - from its transient probabilities (#199). + from its transient probabilities. Parameters ---------- @@ -23380,59 +7157,13 @@ def barlow_proschan_importance( >>> {node: round(share, 4) for node, share in shares.items()} {'a': 0.2857, 'b': 0.7143} """ - if window is None: - if state is not None: - raise ValueError( - "state is where the components start from, for the " - "shares over a window: give its length (window). The " - "long-run shares do not depend on it." - ) - terms, _ = self._outage_terms(working_nodes, broken_nodes) - parts: Dict[Any, Any] = {node: 0.0 for node in self.components} - for cause, term in terms: - parts[cause] = parts.get(cause, 0.0) + term - scalar = True - else: - ends = _check_times(window) - if np.any(ends <= 0.0): - raise ValueError( - "A window must be a positive length of time, got " - f"{window!r}." - ) - windows, flat, _, counts, _, _ = self._window( - window, - working_nodes, - broken_nodes, - "p", - state=state, - causes=True, - ) - caused = counts["caused"] - parts = { - key: np.asarray( - caused.get(key, np.zeros(len(flat))), dtype=float - ) - for key in [*self.components, *caused] - } - scalar = np.ndim(window) == 0 - total: Any = 0.0 - for value in parts.values(): - total = total + np.asarray(value, dtype=float) - with np.errstate(divide="ignore", invalid="ignore"): - shares = { - key: np.where( - total > 0.0, np.asarray(value, dtype=float) / total, np.nan - ) - for key, value in parts.items() - } - if scalar: - return { - key: float(np.ravel(value)[0]) for key, value in shares.items() - } - return { - key: np.asarray(value).reshape(windows.shape) - for key, value in shares.items() - } + return _repairable_importance.barlow_proschan_importance( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + window=window, + state=state, + ) @_times_first() def joint_importance( @@ -23446,10 +7177,10 @@ def joint_importance( ) -> dict: """The joint (second-order) importance of each pair of components: whether improving the two together is worth more than improving - each (#194), in the long run, at times ``x`` or over a window. + each, in the long run, at times ``x`` or over a window. In the guide's Greeks it is *gamma*: complements or substitutes (see - [Sensitivities: the Greeks](guide/greeks.md)). + [Sensitivities: the Greeks](../guide/greeks.md)). ``JRI(i, j) = d2A / dA_i dA_j = A(1_i, 1_j) - A(1_i, 0_j) - A(0_i, 1_j) + A(0_i, 0_j)``, the system's availability with @@ -23461,7 +7192,12 @@ def joint_importance( other held up, less that with it held down, as ``birnbaum_importance`` works them out, with ``x``, ``window`` and ``state``; with limited repair crews, each pair held in the crews' - chain. + chain. With common-cause groups, the system's availability given + each of the pair up and down is conditioned on their states, as + ``birnbaum_importance`` conditions on one: two members of one + group on their joint state, from the group's chain, and a + component outside the groups held (see [Common-cause + failures](../guide/common-cause.md)). Parameters ---------- @@ -23471,7 +7207,8 @@ def joint_importance( Components held down, likewise. x : float or array-like, optional Times from new (or from ``state``), as for - ``birnbaum_importance``; by default the long run. + ``birnbaum_importance`` (at a test's instant, of the state + just after it); by default the long run. window : float, optional Over ``[0, window)`` instead, as for ``birnbaum_importance``. state : dict or str, optional @@ -23489,8 +7226,7 @@ def joint_importance( ValueError As ``birnbaum_importance`` does. NotImplementedError - With common-cause groups, whose members cannot be held, or as - ``birnbaum_importance`` does. + As ``birnbaum_importance`` does. Examples -------- @@ -23508,16 +7244,11 @@ def joint_importance( >>> {pair: round(v, 4) for pair, v in rbd.joint_importance().items()} {('a', 'b'): -0.9091, ('a', 'c'): 0.0909, ('b', 'c'): 0.0909} """ - timed = { - key: value - for key, value in (("x", x), ("window", window), ("state", state)) - if value is not None - } - with self._sharing_curves(): - return self._joint_pairs( - lambda working, broken: self.birnbaum_importance( - sorted(working, key=str), sorted(broken, key=str), **timed - ), - working_nodes, - broken_nodes, - ) + return _repairable_importance.joint_importance( + self, + working_nodes=working_nodes, + broken_nodes=broken_nodes, + x=x, + window=window, + state=state, + ) diff --git a/repyability/rbd/repeated_node.py b/repyability/rbd/repeated_node.py index e81d0480..496795b4 100644 --- a/repyability/rbd/repeated_node.py +++ b/repyability/rbd/repeated_node.py @@ -2,7 +2,7 @@ from repyability.utils.checks import simulation_options, whole_number from repyability.utils.deprecation import refuse_removed_names -from repyability.utils.wrappers import numpy_seed +from repyability.utils.wrappers import conditional_survival, numpy_seed from ._mean_lifetime import mean_lifetime, model_kinks, model_knots from ._sampling import RowSampler, inverse_sampler @@ -249,6 +249,5 @@ def cs(self, x, X): ``sf(X)`` is 0: a float if ``x`` and ``X`` are both scalars, otherwise an array. """ - from repyability.utils.wrappers import conditional_survival return conditional_survival(self, x, X) diff --git a/repyability/rbd/repeated_standby_node.py b/repyability/rbd/repeated_standby_node.py index e574e725..71fa0a10 100644 --- a/repyability/rbd/repeated_standby_node.py +++ b/repyability/rbd/repeated_standby_node.py @@ -1,7 +1,7 @@ import numpy as np from repyability.utils.checks import whole_number -from repyability.utils.wrappers import numpy_seed +from repyability.utils.wrappers import conditional_survival, numpy_seed from ._sampling import RowSampler, column, inverse_sampler from .numerical_convolution import ( @@ -153,7 +153,7 @@ def draw(u): probs = switch_success_probs(self.switching_probability, self.repeats) - def draw(u): + def draw_switched(u): x = column(u, 0, unit) running = np.ones(len(u), dtype=bool) for i, p in enumerate(probs): @@ -161,7 +161,7 @@ def draw(u): x = x + np.where(running, column(u, 2 + 2 * i, unit), 0.0) return x - return RowSampler(1 + 2 * len(probs), draw) + return RowSampler(1 + 2 * len(probs), draw_switched) def mean(self, *args, **kwargs): """Mean lifetime (MTTF) of the node, from the convolution. @@ -247,6 +247,5 @@ def cs(self, x, X): ``sf(X)`` is 0: a float if ``x`` and ``X`` are both scalars, otherwise an array. """ - from repyability.utils.wrappers import conditional_survival return conditional_survival(self, x, X) diff --git a/repyability/rbd/results.py b/repyability/rbd/results.py index de7747e8..ced3b480 100644 --- a/repyability/rbd/results.py +++ b/repyability/rbd/results.py @@ -18,8 +18,8 @@ ``stock(probability)``), as on the RBDs, where what is given is an attribute (``acquisition_cost``) and what is worked out from arguments a method (``total_cost(t)``). ``SparesDemand``'s ``mean`` and ``std`` were -methods until 0.12 (#184): calling them still works, with a -``FutureWarning``, until 0.13. +methods until 0.12 (#184), and ``CapacityDistribution``'s ``mean`` until +0.13 (#235). """ import dataclasses @@ -27,13 +27,12 @@ import warnings from collections.abc import Mapping from dataclasses import dataclass, field -from typing import Any, Dict, Hashable, Optional, Tuple +from typing import Any, Dict, Hashable, Optional, Tuple, cast import numpy as np from scipy.special import ndtri from repyability.rbd import _montecarlo as montecarlo -from repyability.utils.deprecation import called from repyability.utils.wrappers import outside_level @@ -64,7 +63,7 @@ def _json_key(key) -> Any: def plain(value) -> Any: - """``value`` as plain data, for JSON (#235): a result (or anything + """``value`` as plain data, for JSON: a result (or anything with a ``to_dict``) as its ``to_dict()``, a dataclass as a dict of its fields, an array or a tuple as a list, a numpy number as a Python one, a mapping with its keys as JSON holds them (see ``_json_key``), and @@ -126,7 +125,8 @@ def __len__(self): return len(self._field_names()) def _field_names(self): - return tuple(f.name for f in dataclasses.fields(self)) + # Every result class is a dataclass (the base is not). + return tuple(f.name for f in dataclasses.fields(cast(Any, self))) @dataclass @@ -165,12 +165,15 @@ class ConfidenceInterval(_ResultMapping): ``"control_variate"``, of their values controlled by an exact twin (see [`ControlVariate`][repyability.ControlVariate]); ``"exact"``, a run controlled by an exact twin that is the system itself, whose - estimate is then its exact value (#179, #187); ``"conditional"``, + estimate is then its exact value; ``"conditional"``, the mean of each simulation's expected values given its modules' - histories (#189, see + histories (see [`ConditionalRun`][repyability.ConditionalRun]). Where the method chooses how to simulate (``NonRepairableRBD.unreliability_interval``), - the way it chose; None otherwise. + the way it chose; ``"simulated"`` for the other simulated intervals + (``mean_time_to_failure_interval``, + ``PhasedMission.reliability_interval``), and ``"exact"`` where + ``compare`` works the difference out exactly. Examples -------- @@ -203,9 +206,9 @@ class ConfidenceInterval(_ResultMapping): @dataclass class ControlVariate(_ResultMapping): - """The exact twin a simulation run was controlled by (#154), with + """The exact twin a simulation run was controlled by, with ``RepairableRBD.availability``'s or ``cost``'s ``control_variate``; by - default, the system itself where the exact methods take it (#187, see + default, the system itself where the exact methods take it (see ``itself``). The twin is the system with its components failing and repaired @@ -245,7 +248,7 @@ class ControlVariate(_ResultMapping): only as the exact methods take them: crews, standby groups or common-cause groups their chains follow): its exact value is then the system's, which the intervals give, with no error. A run takes - it by default (#187). + it by default. Examples -------- @@ -480,8 +483,7 @@ def percentile(self, q: float) -> Any: ---------- q : float The percentile, in [0, 100], as numpy takes it: ``q`` strictly - between 0 and 1 warns, as it is more likely a fraction meant - (#233). + between 0 and 1 warns, as it is more likely a fraction meant. Returns ------- @@ -768,8 +770,8 @@ class Criticalities(_ResultMapping): @dataclass class ConditionalRun(_ResultMapping): - """How a run's expected values were taken given its modules' histories - (#189): ``RepairableRBD.availability`` or ``cost`` on a system with + """How a run's expected values were taken given its modules' histories: + ``RepairableRBD.availability`` or ``cost`` on a system with dependent modules, by default or with ``conditional=True``. The modules are the nodes whose values over time the exact methods do @@ -804,7 +806,7 @@ class ConditionalRun(_ResultMapping): met: the system was worked out exactly given each. Just one, and the modules never changed state in the run: their outages were not sampled, and every simulation's expected values given them are the - same, which says nothing of the error (#215). The means and their + same, which says nothing of the error. The means and their intervals are then the simulations' own (``method="simulated"``) for a whole run, and have no error to give (``nan``) for a run of the modules alone, which a ``tolerance`` does not stop: run more @@ -825,6 +827,11 @@ class ConditionalRun(_ResultMapping): costs : numpy.ndarray, optional With ``whole``, each simulation's expected cost given them, in order, if anything is priced; None otherwise. + delivered : numpy.ndarray, optional + With ``whole``, each simulation's expected fraction of the demand + delivered given them, in order, if a demand was given; None + otherwise (with ``conditional=True`` they are the result's own + ``delivered``). """ modules: Tuple[Hashable, ...] @@ -833,6 +840,7 @@ class ConditionalRun(_ResultMapping): whole: bool = False uptimes: Optional[np.ndarray] = None costs: Optional[np.ndarray] = None + delivered: Optional[np.ndarray] = None def __repr__(self) -> str: # A summary (#235): each simulation's values are in the arrays. @@ -850,13 +858,13 @@ def __repr__(self) -> str: def informative(self) -> bool: """Whether the modules changed state in the run, so that the spread of the simulations' expected values given them estimates - the error (#215), or there are none, and the values are exact.""" + the error, or there are none, and the values are exact.""" return self.states > 1 or not self.modules @property def unknown(self) -> bool: """Whether the error of the run's means is unknown: a run of the - modules alone in which they never changed state (#215).""" + modules alone in which they never changed state.""" return not self.whole and not self.informative @@ -888,7 +896,7 @@ class CostResult(_ResultMapping): ``percentile(90)`` answers "what could a bad window cost", which a mean cannot. - One expected value (#223): ``mean`` is the run's estimate of the + One expected value: ``mean`` is the run's estimate of the expected cost of a window, the one ``mean_interval`` gives an interval for, and ``cost_rate`` and the breakdowns follow it. By default it is exact where the exact methods work it out (``method="exact"``, see @@ -954,8 +962,8 @@ class CostResult(_ResultMapping): control_variate : ControlVariate, optional The exact twin the run was controlled by (see [`ControlVariate`][repyability.ControlVariate]): by default the - system itself, where the exact methods work out its expected cost - (#187), whose ``mean`` is then that cost; with + system itself, where the exact methods work out its expected cost, + whose ``mean`` is then that cost; with ``control_variate=True``, the system without what ties its components together, whose ``mean`` is the controlled estimate. ``samples`` and ``sample_mean`` stay the simulations' own. None @@ -1062,7 +1070,7 @@ def _estimate(self) -> Tuple[float, float, str]: @property def mean(self) -> float: - """The run's estimate of the expected cost of a window (#223): the + """The run's estimate of the expected cost of a window: the estimate ``mean_interval`` gives, exact where the exact methods work it out, given the modules' histories where those apply, and otherwise the simulations' own, ``sample_mean`` (see the class's @@ -1078,7 +1086,7 @@ def mean(self) -> float: @property def sample_mean(self) -> float: """The simulations' own mean cost over the window: the average of - ``samples``, whatever ``mean`` is (#223). + ``samples``, whatever ``mean`` is. Returns ------- @@ -1116,7 +1124,7 @@ def mean_se(self) -> float: simulations' own (``sample_se``), controlled or conditional ones; for an antithetic run, of the pairs' means. nan where it cannot be known (fewer than two simulations, or a conditional run whose - modules never changed state, #215). + modules never changed state). Returns ------- @@ -1219,7 +1227,7 @@ def percentile(self, q) -> float: ---------- q : float The percentile, between 0 and 100: ``q`` strictly between 0 - and 1 warns, as it is more likely a fraction meant (#233). + and 1 warns, as it is more likely a fraction meant. Returns ------- @@ -1289,10 +1297,14 @@ class ExpectedEvents(_ResultMapping): node_corrective : dict Each component's expected number of corrective actions, at each of which ``repair_cost`` and ``replace_cost`` are charged: its failures, - or for hidden failures, those found by a test in the window. + or for hidden failures, those found by a test in the window (a + nested RBD's: its system failures). node_preventive : dict Each component's expected number of preventive replacements (age or - block), at each of which its preventive cost is charged. + block), at each of which its preventive cost is charged (a nested + RBD's: its planned outages, as the simulation counts them for the + node; its own components' maintenance is counted by its own + ``expected_events``). node_inspections : dict Each component's expected number of tests (of hidden failures). node_downtime : dict @@ -1338,7 +1350,7 @@ class RateBreakdown(_ResultMapping): """How fast a system's availability (or reliability) is changing at each time, and each node's part in it: returned by ``RepairableRBD.availability_rate`` and - ``NonRepairableRBD.reliability_rate`` (#195). + ``NonRepairableRBD.reliability_rate``. With independent nodes the system's value is multilinear in theirs, so its rate is the sum, over the nodes, of each one's Birnbaum importance @@ -1397,8 +1409,8 @@ class RateBreakdown(_ResultMapping): @dataclass class UncertaintyImportance(_ResultMapping): - """Each uncertain input's part in the uncertainty of a system quantity - (#196): returned by ``NonRepairableRBD.uncertainty_importance``. + """Each uncertain input's part in the uncertainty of a system quantity: + returned by ``NonRepairableRBD.uncertainty_importance``. The inputs are the uncertainties given (a node, a tuple of nodes of one population, a common-cause group's model), by the keys they were given @@ -1485,7 +1497,7 @@ class ExpectedCost(_ResultMapping): acquisition_cost : float The one-off cost of buying the components, not in ``mean``. discount_rate : float - The continuous rate the costs were discounted at (#231): with one + The continuous rate the costs were discounted at: with one above 0, every value but ``acquisition_cost`` (paid at the start) is a present value. By default 0. @@ -1785,6 +1797,35 @@ class DemonstrationPlan(_ResultMapping): producer_risk: float +@dataclass +class FailureForecast(_ResultMapping): + """The failures of a part replaced when it fails, over a window from + its current unit's age: returned by + [`NonRepairable.failures`][repyability.NonRepairable.failures]. + + Attributes + ---------- + window : float or numpy.ndarray + The window's length, in the models' time unit. + age : float + The current unit's age at the window's start. + probability : float or numpy.ndarray + The chance the current unit fails in the window. + expected_failures : float or numpy.ndarray + The expected number of failures in the window, each a replacement: + the current unit's, and its replacements'. + expected_cost : float, numpy.ndarray or None + Their expected cost: the unplanned cost times + ``expected_failures``; None if the part has no unplanned cost. + """ + + window: Any + age: float + probability: Any + expected_failures: Any + expected_cost: Any + + @dataclass class MaintenancePlan(_ResultMapping): """The result of ``RepairableRBD.optimal_replacement_intervals()`` and @@ -1810,7 +1851,7 @@ class MaintenancePlan(_ResultMapping): For ``optimal_inspection_intervals``, node name -> the time of its first test (from 0 to less than its interval): those chosen with ``offset_shares``, or else each keeping its share of the interval, - so that plans compare as they are (#222). ``with_intervals`` takes + so that plans compare as they are. ``with_intervals`` takes them as they are. None for ``optimal_replacement_intervals``. """ @@ -1823,7 +1864,7 @@ class MaintenancePlan(_ResultMapping): @dataclass(frozen=True) class Lever(_ResultMapping): """One lever of a diagram: a value that its ``parameter_sensitivity`` - moves (#244). + moves. [`RepairableRBD.levers`][repyability.RepairableRBD.levers] and [`NonRepairableRBD.levers`][repyability.NonRepairableRBD.levers] list @@ -1843,7 +1884,9 @@ class Lever(_ResultMapping): ``RepairableRBD`` ``"reliability.alpha"``, ``"inspection.interval"``, ``"standby.units"``, ``"ccf_beta"``, ``"repair_crews"``, ...; in a ``NonRepairableRBD`` the parameter's - own name (``"alpha"``) or the group's (``"ccf_beta"``). + own name (``"alpha"``) or the group's (``"ccf_beta"``), or a + regression node's covariate (``"covariate.load"``, its feature + name, or ``"covariate.0"``, its place). value : float Its value in the diagram. discrete : bool @@ -1976,9 +2019,7 @@ def mean(self) -> Any: In the long run, the average capacity over time. Infinite if the capacity can be infinite (see ``levels``). A property, as the other - results' values are (#235): it was a method until 0.13, and - calling it, ``mean()``, still gives it, with a ``FutureWarning``, - until 0.14. + results' values are (a method until 0.13, #235). Returns ------- @@ -1992,10 +2033,7 @@ def mean(self) -> Any: parts = np.where( self.probabilities > 0, levels * self.probabilities, 0.0 ) - return called( - self._per_time(np.sum(parts, axis=0)), - "CapacityDistribution.mean", - ) + return self._per_time(np.sum(parts, axis=0)) def delivered_fraction(self, demand: float) -> Any: """The expected fraction of a demand the system delivers: @@ -2083,7 +2121,7 @@ class AvailabilityResult(_ResultMapping): and the criticality measures, all summed over the ``n_simulations`` simulations of the window ``[0, time_simulated_to]``. - One expected value (#223): ``mean_availability`` is the run's estimate + One expected value: ``mean_availability`` is the run's estimate of the expected availability over the window, the one ``mean_availability_interval`` gives an interval for. By default it is exact where the exact methods work it out (``method="exact"``, see @@ -2094,7 +2132,12 @@ class AvailabilityResult(_ResultMapping): simulations' own (``"simulated"``). The simulations' own average, ``system_uptime / (n_simulations * time_simulated_to)``, is ``sample_mean_availability`` whatever the method, as the curve, the - totals and the counts are theirs. Its repr summarises the run. + totals and the counts are theirs. Against a demand, the + ``delivered_fraction`` is taken the same way (exact where + ``mission_capacity`` works it out, given the modules' histories, or + the simulations' own, ``sample_delivered_fraction``), and + ``delivered_fraction_interval`` says which. Its repr summarises the + run. The properties ``mean_up_time``, ``mean_down_time`` and ``failure_frequency`` are simulation *estimates* derived from the fields @@ -2187,6 +2230,12 @@ class AvailabilityResult(_ResultMapping): Each simulation's delivered fraction, in order: the integral of ``min(capacity, demand)`` over the window, over ``demand`` times its length. None without a demand. + delivered_control : ControlVariate, optional + With a demand, the control of the delivered fractions by the + system itself where ``mission_capacity`` works out the expected + fraction delivered (as ``control_variate`` controls the + availability's): ``delivered_fraction`` is then that exact value. + None otherwise. opportunistic_renewals : dict, optional With maintenance groups (see ``RepairableRBD``): each component's early renewals at another's stop, summed over the simulations. @@ -2194,7 +2243,7 @@ class AvailabilityResult(_ResultMapping): control_variate : ControlVariate, optional The exact twin the run was controlled by (see [`ControlVariate`][repyability.ControlVariate]): by default the - system itself, where it is its own exact twin (#187), or with + system itself, where it is its own exact twin, or with ``control_variate=True`` the system without what ties its components together. ``mean_availability_interval`` is then the controlled estimate's (for the system itself, its exact value), @@ -2266,6 +2315,7 @@ class AvailabilityResult(_ResultMapping): opportunistic_renewals: Optional[Dict[Hashable, int]] = None control_variate: Optional[ControlVariate] = None conditional: Optional[ConditionalRun] = None + delivered_control: Optional[ControlVariate] = None @property def mean_capacity(self) -> Optional[float]: @@ -2289,10 +2339,14 @@ def mean_capacity(self) -> Optional[float]: @property def delivered_fraction(self) -> Optional[float]: - """Simulation estimate of the fraction of the demand delivered over - the window: the production availability. The mean of - ``delivered``; over a long window it approaches the exact long-run - ``capacity_distribution().delivered_fraction(demand)``. + """The run's estimate of the expected fraction of the demand + delivered over the window: the production availability, the + estimate ``delivered_fraction_interval`` gives, exact where + ``mission_capacity`` works it out, given the modules' histories + where those apply, and otherwise the simulations' own, + ``sample_delivered_fraction`` (as ``mean_availability`` is taken, + see the class's notes). Over a long window it approaches the + exact long-run ``capacity_distribution().delivered_fraction(demand)``. Returns ------- @@ -2300,21 +2354,69 @@ def delivered_fraction(self) -> Optional[float]: The delivered fraction, in ``[0, 1]``, or None without a demand. """ + if self.delivered is None: + return None + return self._delivered_estimate()[0] + + @property + def sample_delivered_fraction(self) -> Optional[float]: + """The simulations' own delivered fraction over the window, the + mean of ``delivered``, whatever ``delivered_fraction`` is. + + Returns + ------- + float or None + The fraction of the demand the simulated systems delivered, + or None without a demand. + """ if self.delivered is None: return None return float(np.mean(self.delivered)) + def _delivered_estimate(self) -> Tuple[float, float, str]: + """The run's estimate of the expected delivered fraction over the + window, its standard error and how it was found (see + ``delivered_fraction_interval``), as ``_estimate`` finds the + availability's.""" + control, conditional = self.delivered_control, self.conditional + if control is not None and control.itself: + return control.exact, 0.0, "exact" + fractions = np.asarray(self.delivered, dtype=float) + method = "simulated" + if conditional is not None and ( + conditional.informative or not conditional.whole + ): + if not conditional.whole: + # A run of the modules alone: its own values are theirs. + method = "conditional" + elif conditional.delivered is not None: + method = "conditional" + fractions = np.asarray(conditional.delivered, dtype=float) + error = ( + math.nan + if conditional is not None and conditional.unknown + else montecarlo.standard_error(fractions, self.antithetic) + ) + return float(np.mean(fractions)), error, method + def delivered_fraction_interval( self, confidence: float = 0.95 ) -> ConfidenceInterval: """Confidence interval for the expected delivered fraction over the window. - As ``mean_availability_interval`` is for the availability: the - estimate is the mean of the simulations' delivered fractions, with - standard error ``std / sqrt(n)`` (of antithetic pairs' means, for an - antithetic run), and the interval, clipped to [0, 1], describes the - simulation error. + Around ``delivered_fraction``, as ``mean_availability_interval`` + is around the availability: an exact one has no error, and the + interval is that value; otherwise it is the mean of the + simulations' delivered fractions (the conditional ones, given the + modules' histories, where those apply), with standard error ``std + / sqrt(n)`` (of antithetic pairs' means, for an antithetic run), + and the interval, clipped to [0, 1], describes the simulation + error. Its ``method`` says how the estimate was found: + ``"exact"`` (``mission_capacity`` works it out, see + ``delivered_control``), ``"conditional"`` (see + [`ConditionalRun`][repyability.ConditionalRun]) or + ``"simulated"``, the simulations' own. Parameters ---------- @@ -2324,7 +2426,8 @@ def delivered_fraction_interval( Returns ------- ConfidenceInterval - The estimate, bounds, standard error and number of simulations. + The estimate, bounds, standard error, number of simulations and + method. Raises ------ @@ -2339,9 +2442,7 @@ def delivered_fraction_interval( "This result has no delivered fractions: the RBD had no " "capacities, or no demand to measure them against." ) - fractions = np.asarray(self.delivered, dtype=float) - estimate = float(np.mean(fractions)) - se = montecarlo.standard_error(fractions, self.antithetic) + estimate, se, method = self._delivered_estimate() z = montecarlo.z_value(confidence) return ConfidenceInterval( estimate=estimate, @@ -2349,7 +2450,8 @@ def delivered_fraction_interval( upper=min(1.0, estimate + z * se), confidence=confidence, standard_error=se, - n_samples=len(fractions), + n_samples=len(self.delivered), + method=method, ) def _estimate(self) -> Tuple[float, float, str]: @@ -2387,8 +2489,8 @@ def _estimate(self) -> Tuple[float, float, str]: @property def mean_availability(self) -> float: - """The run's estimate of the expected availability over the window - (#223): the estimate ``mean_availability_interval`` gives, exact + """The run's estimate of the expected availability over the window: + the estimate ``mean_availability_interval`` gives, exact where the exact methods work it out, given the modules' histories where those apply, and otherwise the simulations' own, ``sample_mean_availability`` (see the class's notes). The @@ -2411,7 +2513,7 @@ def mean_availability(self) -> float: def sample_mean_availability(self) -> float: """The simulations' own mean availability over the window, ``system_uptime / (n_simulations * time_simulated_to)``, whatever - ``mean_availability`` is (#223). + ``mean_availability`` is. Returns ------- @@ -2724,7 +2826,7 @@ def __repr__(self) -> str: @property def mean(self) -> float: """The expected number of spares used (a property, as the other - results' values are, #184). + results' values are). Returns ------- diff --git a/repyability/rbd/routes.py b/repyability/rbd/routes.py index 8334b982..8aa28f2f 100644 --- a/repyability/rbd/routes.py +++ b/repyability/rbd/routes.py @@ -63,7 +63,7 @@ class AnalysisRoute: twin: str = "" def to_dict(self) -> dict: - """The route as plain data, ready for ``json.dumps`` (#235): its + """The route as plain data, ready for ``json.dumps``: its fields by name, the nodes as a list. Returns @@ -88,12 +88,71 @@ def __str__(self) -> str: return text +class AnalysisRoutes(dict): + """The routes of a diagram's analyses, as ``analysis_routes()`` gives + them: a dict of [`AnalysisRoute`][repyability.AnalysisRoute] by method + name, which prints one method a line, ``name: route -- reason`` (the + reason's first sentence). + + Examples + -------- + >>> import surpyval as surv + >>> from repyability import NonRepairableRBD + >>> rbd = NonRepairableRBD( + ... [("s", "c"), ("c", "t")], + ... {"c": surv.Exponential.from_params([0.01])}, + ... ) + >>> routes = rbd.analysis_routes() + >>> routes["mean"].route + 'numerical' + >>> print(routes) # doctest: +ELLIPSIS + Hf: exact -- The structure function over the node reliabilities, exactly. + allocate_redundancy: exact -- ... + ... + unreliability_interval: simulated -- ... + """ + + def to_dict(self) -> dict: + """The routes as plain data, ready for ``json.dumps``: each + route's ``to_dict()`` by method name. + + Returns + ------- + dict + ``{method: route.to_dict()}``. + """ + return {name: route.to_dict() for name, route in self.items()} + + def __str__(self) -> str: + return "\n".join( + f"{name}: {route.route} -- {_first_sentence(route.reason)}" + for name, route in self.items() + ) + + __repr__ = __str__ + + +def _first_sentence(reason: str) -> str: + """``reason`` on one line: its first sentence, when it has more.""" + text = " ".join(reason.split()) + cut = text.find(". ") + return text if cut < 0 else text[: cut + 1] + + +class Refused(ValueError): + """An analysis refused because the diagram has nothing it works on (no + capacities, no component under age replacement, nothing priced, ...). + A ``ValueError``, as these always were; refusals of what an analysis + cannot work out are ``NotImplementedError``.""" + + def refusal(check: Callable[[], object]) -> Optional[str]: - """The message of the ``NotImplementedError`` or ``ValueError`` that - ``check`` raises, or None if it passes.""" + """The message of the refusal (``NotImplementedError`` or ``Refused``) + that ``check`` raises, or None if it passes. Any other error is a + fault, and is raised.""" try: check() - except (NotImplementedError, ValueError) as error: + except (NotImplementedError, Refused) as error: return str(error) return None diff --git a/repyability/rbd/serialisation.py b/repyability/rbd/serialisation.py index d6d8809b..a2781bfd 100644 --- a/repyability/rbd/serialisation.py +++ b/repyability/rbd/serialisation.py @@ -31,6 +31,7 @@ from repyability._version import __version__ from repyability.non_repairable import NonRepairable +from repyability.rbd._degradation import is_degradation from repyability.rbd._model_utils import distribution_name, is_mixture from repyability.rbd.degrading_node import DegradingNode from repyability.rbd.helper_classes import ( @@ -164,7 +165,11 @@ def _serialise_model(model: Any) -> dict: "load": model.load, "k": model.k, } - if distribution_name(model) is not None or is_mixture(model): + if ( + distribution_name(model) is not None + or is_mixture(model) + or is_degradation(model) + ): # surpyval's own format: everything surpyval keeps (an offset, p, # f0, a fit's covariance, a mixture's components) round-trips, # whatever it adds later. @@ -307,6 +312,8 @@ def _serialise_component(value) -> dict: } if value.get("replace_after") is not None: out["replace_after"] = int(value["replace_after"]) + if value.get("duty") is not None: + out["duty"] = float(value["duty"]) for key in ("preventive", "inspection"): if value.get(key) is not None: out[key] = _serialise_schedule(value[key]) @@ -322,14 +329,22 @@ def _serialise_component(value) -> dict: def _serialise_schedule(spec: dict) -> dict: # A component's preventive-maintenance or inspection schedule: the - # interval (and policy, and a condition policy's threshold) as they are, - # the duration ("instant" or a model) and the costs (each a number or a - # distribution). + # interval (and policy, and a condition policy's threshold or level) as + # they are, the duration ("instant" or a model) and the costs (each a + # number or a distribution). + if spec.get("policy") == "calendar": + raise NotImplementedError( + "Cannot save a 'calendar' preventive policy: a CalendarPolicy " + "is worked out from its part and calendar, not saved. Save the " + "diagram without it, and work the policy out again." + ) out: dict[str, Any] = {"interval": float(spec["interval"])} if "policy" in spec: out["policy"] = spec["policy"] if spec.get("threshold") is not None: out["threshold"] = float(spec["threshold"]) + if spec.get("level") is not None: + out["level"] = float(spec["level"]) if spec.get("opportunity") is not None: out["opportunity"] = float(spec["opportunity"]) # An inspection's offset, coverage and full tests, when given (so a @@ -387,6 +402,8 @@ def _deserialise_component(d: dict) -> Any: out["repair"] = dict(d["repair"]) if "replace_after" in d: out["replace_after"] = d["replace_after"] + if "duty" in d: + out["duty"] = d["duty"] for key in ("preventive", "inspection"): if key in d: out[key] = _deserialise_schedule(d[key]) @@ -448,7 +465,7 @@ def _capacity_from_list(capacity_list): def _ccf_to_list(ccf_groups): - from repyability.rbd.ccf import MGL, BetaFactor + from repyability.rbd.ccf import MGL, AlphaFactor, BetaFactor if not ccf_groups: return None @@ -458,6 +475,12 @@ def _ccf_to_list(ccf_groups): model = {"kind": "beta_factor", "beta": group.model.beta} elif isinstance(group.model, MGL): model = {"kind": "mgl", "letters": list(group.model.letters)} + elif isinstance(group.model, AlphaFactor): + model = { + "kind": "alpha_factor", + "alphas": list(group.model.alphas), + "testing": group.model.testing, + } else: raise NotImplementedError( f"Cannot serialise CCF model {type(group.model).__name__}." @@ -473,7 +496,7 @@ def _ccf_to_list(ccf_groups): def _ccf_from_list(ccf_list): - from repyability.rbd.ccf import MGL, BetaFactor, CCFGroup + from repyability.rbd.ccf import MGL, AlphaFactor, BetaFactor, CCFGroup if not ccf_list: return None @@ -490,6 +513,13 @@ def _ccf_from_list(ccf_list): basis=basis, shocks=model_dict.get("shocks"), ) + elif kind == "alpha_factor": + model = AlphaFactor( + model_dict["alphas"], + testing=model_dict["testing"], + basis=basis, + shocks=model_dict.get("shocks"), + ) else: raise ValueError(f"Unknown CCF model kind {kind!r}.") members = [_node_name(m) for m in entry["members"]] diff --git a/repyability/rbd/shards.py b/repyability/rbd/shards.py index feaaabfc..dba6434b 100644 --- a/repyability/rbd/shards.py +++ b/repyability/rbd/shards.py @@ -69,9 +69,12 @@ def run_shard(shard) -> bytes: >>> merged.system_uptime == whole.system_uptime True """ - from repyability.rbd.repairable_rbd import RepairableRBD, _states_from_key + from repyability.rbd._runs import _states_from_key + from repyability.rbd.repairable_rbd import RepairableRBD from repyability.rbd.serialisation import _node_name + from . import _runs + data = _shard(shard) rbd = RepairableRBD.from_dict(data["system"]) if not isinstance(rbd, RepairableRBD): @@ -80,7 +83,8 @@ def run_shard(shard) -> bytes: return _modules_partial(rbd, data) working = {_node_name(node) for node in data["working_nodes"]} broken = {_node_name(node) for node in data["broken_nodes"]} - chunk = rbd._chunk( + chunk = _runs._chunk( + rbd, float(data["t_simulation"]), int(data["start"]), int(data["stop"]), @@ -105,7 +109,7 @@ def _modules_partial(rbd, data: Dict[str, Any]) -> bytes: ``_conditional.partial_bytes``), ``rbd`` being the modules' own diagram (``RepairableRBD._modules_rbd``).""" from repyability.rbd import _conditional, _timeline_runs - from repyability.rbd.repairable_rbd import _states_from_key + from repyability.rbd._runs import _states_from_key start, stop = int(data["start"]), int(data["stop"]) histories, tally = _timeline_runs.with_costs( diff --git a/repyability/rbd/standby_node.py b/repyability/rbd/standby_node.py index 0ea35755..0317d85e 100644 --- a/repyability/rbd/standby_node.py +++ b/repyability/rbd/standby_node.py @@ -1,11 +1,12 @@ from queue import PriorityQueue +from typing import Any import numpy as np from surpyval import Hypoexponential from repyability.utils.checks import simulation_options, whole_number -from repyability.utils.deprecation import ignored, refuse_removed_names -from repyability.utils.wrappers import numpy_seed +from repyability.utils.deprecation import refuse_removed_names +from repyability.utils.wrappers import conditional_survival, numpy_seed from ._dependent_lifetimes import ( ColdPairSurvival, @@ -23,13 +24,13 @@ def drawn_mean(model, name: str, exact, mc_samples, seed, method) -> float: - """A node model's ``mean(mc_samples, seed, method=...)`` (#233): + """A node model's ``mean(mc_samples, seed, method=...)``: ``exact()``, its exact or numerical mean (a NotImplementedError where it has none), by default and with ``method="exact"``; with ``method="simulate"``, or by default where it has no exact mean and the draws' options are given, the mean of ``mc_samples`` draws of - ``random`` (10_000 by default). Options given to an exact mean by - default are ignored, with a ``FutureWarning``: 0.14 refuses them.""" + ``random`` (10_000 by default). An exact mean refuses the draws' + options, as it draws nothing (they warned in 0.13, #233).""" if method not in (None, "exact", "simulate"): raise ValueError( f"method must be 'exact' or 'simulate' (or None), got {method!r}." @@ -44,15 +45,8 @@ def drawn_mean(model, name: str, exact, mc_samples, seed, method) -> float: if method == "exact" or (mc_samples is None and seed is None): raise return float(model.random(count, seed=seed).mean()) - if method == "exact": - simulation_options(f"{name}.mean(method='exact')", options) - else: - ignored( - f"{name}.mean()", - "the mean is worked out exactly (or numerically), so nothing is " - "drawn; method='simulate' estimates it from draws instead.", - options, - ) + shown = "mean(method='exact')" if method == "exact" else "mean()" + simulation_options(f"{name}.{shown}", options) return value @@ -307,7 +301,7 @@ def __init__( "switching_probability is only supported for cold " "standby (dormancy_factor == 0)." ) - closed_form = None + closed_form: Any = None if rate is not None: # Identical Exponential units: with j units alive, k operate # at rate `rate` and j - k sit dormant at `dormancy_factor * @@ -656,8 +650,9 @@ def _row_sampler(self): be replayed. Columns follow the order ``random(1)`` draws in: one per unit, and under imperfect k=1 switching a switch draw before each spare's.""" - units = [inverse_sampler(m) for m in self.reliabilities] - if any(unit is None for unit in units): + found = [inverse_sampler(m) for m in self.reliabilities] + units = [unit for unit in found if unit is not None] + if len(units) < len(found): return None if self.dormancy_factor > 0.0: @@ -735,18 +730,17 @@ def mean(self, mc_samples=None, seed=None, *, method=None): ``is_simulated``) has no exact mean, and refuses, unless asked for an estimate: the mean of ``mc_samples`` new draws of ``random``. ``method="simulate"`` estimates it so whatever the arrangement, to - check the exact mean against draws, say (#233). + check the exact mean against draws, say. Parameters ---------- mc_samples : int, optional The number of draws to estimate the mean from, 10_000 by default: with ``method="simulate"``, or for an arrangement with - no exact mean. An exact mean draws nothing, and ignores it with - a ``FutureWarning`` (0.14 will refuse it). + no exact mean. An exact mean draws nothing, and refuses it. seed : int or None, optional Seed for those draws (see ``random``), by default None. - Ignored as ``mc_samples`` is. + Refused as ``mc_samples`` is. method : {None, "exact", "simulate"}, optional None (the default): the exact or numerical mean, or for an arrangement with neither, the draws' estimate when @@ -765,9 +759,8 @@ def mean(self, mc_samples=None, seed=None, *, method=None): If the arrangement has no exact mean and neither ``mc_samples`` nor ``seed`` is given (or ``method="exact"``). ValueError - If ``method`` is not one of these. - TypeError - If ``method="exact"`` is given with ``mc_samples`` or ``seed``. + If ``method`` is not one of these, or ``mc_samples`` or ``seed`` + is given to an exact mean. Examples -------- @@ -906,6 +899,5 @@ def cs(self, x, X): >>> round(StandbyModel([unit, unit]).cs(50.0, 100.0), 4) 0.7582 """ - from repyability.utils.wrappers import conditional_survival return conditional_survival(self, x, X) diff --git a/repyability/rbd/uncertainty.py b/repyability/rbd/uncertainty.py index 188e2179..236d2fb3 100644 --- a/repyability/rbd/uncertainty.py +++ b/repyability/rbd/uncertainty.py @@ -11,10 +11,14 @@ in surpyval, already provides. A node's uncertainty is one of - ``"fit"``: the parameters are drawn from the normal approximation of the - fitted model's maximum-likelihood estimate (surpyval's ``hess_inv``, the - inverse Hessian of the negative log-likelihood), on the log scale for a + fitted model's maximum-likelihood estimate (surpyval's ``covariance()``, + the inverse of the observed information), on the log scale for a parameter that must be positive and the logit scale for one in (0, 1), - so that every draw is valid; + so that every draw is valid. They are every parameter the fit + estimated: the distribution's, and a limited failure population's share + that ever fails (``lfp_p``) and a zero-inflated one's share dead on + arrival (``f0``) (#267). An offset is held where it was fitted, since + surpyval's covariance leaves it out (SurPyval#830), with a warning; - a mapping ``{parameter name: distribution}``: each named parameter is drawn independently from its distribution (anything with ``qf`` or ``ppf``, such as a surpyval or scipy.stats distribution), the others @@ -36,11 +40,22 @@ import difflib import warnings from collections.abc import Mapping, Sequence -from typing import Any, List, Optional +from typing import ( + Any, + Callable, + Dict, + Hashable, + Iterable, + List, + Optional, + Tuple, +) import numpy as np -from repyability.rbd._model_utils import is_mixture +from repyability.rbd._model_utils import is_mixture, parametric_spec +from repyability.utils.checks import seed as check_seed +from repyability.utils.wrappers import outside_level #: The draws of a node's model that ``"fit"`` asks for. FIT = "fit" @@ -111,6 +126,45 @@ def integers(self, high, size) -> np.ndarray: return np.minimum((u * high).astype(int), int(high) - 1) +def check_draws(n_draws) -> int: + """``n_draws``, checked: a positive integer.""" + if isinstance(n_draws, bool) or not isinstance(n_draws, (int, np.integer)): + raise ValueError(f"n_draws must be an integer, got {n_draws!r}.") + if n_draws < 1: + raise ValueError(f"n_draws must be at least 1, got {n_draws}.") + return int(n_draws) + + +def draw_with( + draw: Callable[[int, Any], Any], + n_draws: int, + seed, + sampling: str = "random", + paired: bool = False, + join: Optional[Callable[[Any, Any], Any]] = None, +) -> Any: + """``draw(n, source)``'s draws, ``n_draws`` of them, with their + randomness from ``seed``: ``sampling="random"``, numpy's generator; + ``"sobol"``, the points of a scrambled Sobol sequence, the dimensions + the draws take counted first (see the module docstring). ``paired``: + two independent sets of ``n_draws``, one after the other, for the + Sobol indices: one draw of twice as many, or, from the Sobol points, + two draws from dimensions of their own, put together by ``join``.""" + check_sampling(sampling) + rng = np.random.default_rng(check_seed(seed)) + if sampling == "random": + return draw(2 * n_draws if paired else n_draws, rng) + counter = Counter() + draw(1, counter) + width = counter.dimensions + table = sobol_table(n_draws, 2 * width if paired else width, rng) + first = draw(n_draws, SobolPoints(table[:, :width])) + if not paired: + return first + assert join is not None + return join(first, draw(n_draws, SobolPoints(table[:, width:]))) + + def sobol_table(n: int, dimensions: int, rng) -> np.ndarray: """``n`` points of a scrambled Sobol sequence in ``dimensions`` dimensions, scrambled by ``rng`` (numpy's generator). Its balance is @@ -126,9 +180,9 @@ def sobol_table(n: int, dimensions: int, rng) -> np.ndarray: return engine.random(n) -def _parametric(model, label: str): - """The model's distribution, or a ValueError for a model whose - parameters cannot be redrawn.""" +def _spec(model, label: str): + """The model's parameters (``parametric_spec``), or a ValueError for a + model whose parameters cannot be redrawn.""" if is_mixture(model): # Its dist is its components' alone (#227). raise ValueError( @@ -138,14 +192,14 @@ def _parametric(model, label: str): "so they cannot be drawn. Give a list of alternative models " "instead (fitted to resampled data, say)." ) - dist = getattr(model, "dist", None) - if dist is None or not hasattr(dist, "from_params"): + spec = parametric_spec(model) + if spec is None: raise ValueError( f"{label}: its model has no parameters to draw (it is not a " "surpyval parametric distribution). Give a list of alternative " "models instead." ) - return dist + return spec class _Scale: @@ -187,20 +241,24 @@ def inverse(self, z): return a + (b - a) / (1.0 + np.exp(-z)) -def _fit_covariance(model, label: str): - """The fitted model's distribution, parameters, parameter covariance - (surpyval's ``hess_inv``) and the parameters' bounds, checked.""" - dist = _parametric(model, label) - covariance = getattr(model, "hess_inv", None) - params = np.atleast_1d(np.asarray(model.params, dtype=float)) +def _fit_covariance(model, label: str, warn: bool = False): + """The fitted model's parameters (``parametric_spec``), their + covariance (surpyval's ``covariance()``) and their bounds, checked; + with ``warn``, a warning that an offset is held where it was fitted.""" + spec = _spec(model, label) + try: + covariance = model.covariance() + except (AttributeError, TypeError, ValueError): + covariance = None if covariance is None: raise ValueError( f"{label}: 'fit' draws from the fitted model's parameter " - "covariance (surpyval's hess_inv, from a maximum-likelihood " + "covariance (surpyval's covariance(), from a maximum-likelihood " "fit), which this model does not have. Fit it with surpyval, or " "give distributions over its parameters or a list of models." ) covariance = np.atleast_2d(np.asarray(covariance, dtype=float)) + params = np.asarray(spec.params, dtype=float) k = len(params) if covariance.shape != (k, k) or not np.all(np.isfinite(covariance)): raise ValueError( @@ -208,8 +266,7 @@ def _fit_covariance(model, label: str): f"finite {k} x {k} matrix; give distributions over its " "parameters or a list of models instead." ) - bounds = list(getattr(dist, "bounds", [(None, None)] * k))[:k] - for (lower, upper), value in zip(bounds, params): + for (lower, upper), value in zip(spec.bounds, params): if (lower is not None and value <= lower) or ( upper is not None and value >= upper ): @@ -219,11 +276,21 @@ def _fit_covariance(model, label: str): "approximation there; give distributions over its " "parameters or a list of models instead." ) - return dist, params, covariance, bounds + if warn and getattr(model, "offset", False): + warnings.warn( + f"{label}: its model was fitted with an offset, gamma = " + f"{spec.extras.get('gamma')!r}, which surpyval's parameter " + "covariance leaves out (SurPyval#830), so 'fit' holds it where " + "it was fitted and the spread drawn is too narrow. For the " + "offset's uncertainty give a list of models (refits to " + "resampled data, say).", + stacklevel=outside_level(), + ) + return spec, params, covariance, list(spec.bounds) def _fit_draws(model, n: int, rng: np.random.Generator, label: str) -> list: - _, params, covariance, bounds = _fit_covariance(model, label) + spec, params, covariance, bounds = _fit_covariance(model, label, n > 0) k = len(params) scales = [_Scale(lower, upper) for lower, upper in bounds] # The delta method: the covariance on the unbounded scale. @@ -237,7 +304,7 @@ def _fit_draws(model, n: int, rng: np.random.Generator, label: str) -> list: drawn = np.column_stack( [sc.inverse(z[:, j]) for j, sc in enumerate(scales)] ) - return [model.with_params(list(row)) for row in drawn] + return [spec.build(row) for row in drawn] def is_fit(model) -> bool: @@ -250,6 +317,39 @@ def is_fit(model) -> bool: return True +def fitted_together( + models: Mapping, joined: Iterable[Iterable[Hashable]] = () +) -> Dict[Hashable, str]: + """The uncertainty drawn when none is given (#184): ``"fit"`` for each + of ``models`` (by name) that is a surpyval fit with a parameter + covariance, the names that share one model object, or are ``joined`` + (a common-cause group's members, say), given together in a tuple, so + that they share their draws.""" + fitted = [name for name, model in models.items() if is_fit(model)] + parent = {name: name for name in fitted} + + def root(name): + while parent[name] != name: + parent[name] = parent[parent[name]] + name = parent[name] + return name + + holder: Dict[int, Hashable] = {} + for name in fitted: + parent[root(name)] = root(holder.setdefault(id(models[name]), name)) + for names in joined: + inside = [name for name in names if name in parent] + for name in inside[1:]: + parent[root(name)] = root(inside[0]) + together: Dict[Hashable, list] = {} + for name in fitted: + together.setdefault(root(name), []).append(name) + return { + names[0] if len(names) == 1 else tuple(names): FIT + for names in together.values() + } + + def _quantiles( prior, n: int, rng: np.random.Generator, label: str, name ) -> np.ndarray: @@ -280,22 +380,20 @@ def _closest(unknown: list, names: list) -> str: def _parameter_draws( model, priors: Mapping, n: int, rng: np.random.Generator, label: str ) -> list: - dist = _parametric(model, label) - names = list(getattr(dist, "parameter_names", [])) + spec = _spec(model, label) + names = spec.names unknown = [p for p in priors if p not in names] if unknown: raise ValueError( f"{label}: {sorted(map(str, unknown))} are not parameters of its " - f"{getattr(dist, 'name', 'model')} model, whose parameters are " - f"{names}.{_closest(unknown, names)}" + f"{getattr(model.dist, 'name', 'model')} model, whose parameters " + f"are {names}.{_closest(unknown, names)}" ) - params = np.atleast_1d(np.asarray(model.params, dtype=float)) - bounds = list(getattr(dist, "bounds", [(None, None)] * len(names))) - columns = np.tile(params, (n, 1)) + columns = np.tile(np.asarray(spec.params, dtype=float), (n, 1)) for name, prior in priors.items(): values = _quantiles(prior, n, rng, label, name) j = names.index(name) - lower, upper = bounds[j] if j < len(bounds) else (None, None) + lower, upper = spec.bounds[j] if not np.all(np.isfinite(values)) or ( (lower is not None and np.any(values <= lower)) or (upper is not None and np.any(values >= upper)) @@ -305,7 +403,7 @@ def _parameter_draws( f"outside its range ({lower}, {upper})." ) columns[:, j] = values - return [model.with_params(list(row)) for row in columns] + return [spec.build(row) for row in columns] def _ensemble_draws( @@ -350,11 +448,14 @@ def draw_ccf_models( ) -> List[Any]: """``n`` plausible common-cause models of a ``CCFGroup``, as ``spec`` says: ``{parameter name: distribution}`` over its model's parameters - (``beta``, and an MGL model's ``gamma``, ``delta``, ...; see - ``ccf.parameters``), each drawn independently and in ``[0, 1]``, the - others kept; or a list of models (``BetaFactor`` or ``MGL``, for the - group's size and with its model's basis), drawn with replacement.""" - from .ccf import MGL, BetaFactor, parameters, with_parameters + (``beta``, an MGL model's ``gamma``, ``delta``, ..., an alpha-factor + model's ``alpha_2``, ...; see ``ccf.parameters``), each drawn + independently and in ``[0, 1]``, the others kept (alpha factors + drawn to sum past 1 are refused, ``alpha_1`` being the remainder); + or a list of models (``BetaFactor``, ``MGL`` or ``AlphaFactor``, for + the group's size and with its model's basis), drawn with + replacement.""" + from .ccf import _Model, parameters, with_parameters model = group.model label = f"Common-cause group {list(group.members)!r}" @@ -385,20 +486,17 @@ def draw_ccf_models( if not spec: raise ValueError(f"{label}: the list of models is empty.") for m in spec: - size = ( - m.required_group_size() - if isinstance(m, (BetaFactor, MGL)) - else None - ) + size = m.required_group_size() if isinstance(m, _Model) else None if ( - not isinstance(m, (BetaFactor, MGL)) + not isinstance(m, _Model) or size not in (None, len(group.members)) or m.basis != model.basis or m.shocks != model.shocks ): raise ValueError( f"{label}: every alternative model must be a " - f"BetaFactor or an MGL model for {len(group.members)} " + "BetaFactor or an MGL or AlphaFactor model for " + f"{len(group.members)} " f"members, splitting the {model.basis} as its model " f"does, with {model.shocks} shocks; got {m!r}." ) @@ -439,9 +537,9 @@ def _quantiles_at(prior, u: np.ndarray, label: str, name) -> np.ndarray: def varied_parameters(model, spec: Any, label: str): - """For the delta method (#196): the parameters a node's uncertainty + """For the delta method: the parameters a node's uncertainty varies, as their positions among the model's parameters, their values, - their covariance (a fit's ``hess_inv``, or the variances of the + their covariance (a fit's ``covariance()``, or the variances of the distributions given) and their bounds. A list of models has no parameters to vary.""" if isinstance(spec, str): @@ -450,22 +548,22 @@ def varied_parameters(model, spec: Any, label: str): f"{label}: unknown uncertainty {spec!r}; give 'fit', a dict " "of parameter distributions or a list of models." ) - _, params, covariance, bounds = _fit_covariance(model, label) + _, params, covariance, bounds = _fit_covariance(model, label, True) return list(range(len(params))), params, covariance, bounds if isinstance(spec, Mapping): if not spec: raise ValueError(f"{label}: no parameter distributions given.") - dist = _parametric(model, label) - names = list(getattr(dist, "parameter_names", [])) + parameters = _spec(model, label) + names = parameters.names unknown = [p for p in spec if p not in names] if unknown: raise ValueError( f"{label}: {sorted(map(str, unknown))} are not parameters of " - f"its {getattr(dist, 'name', 'model')} model, whose " + f"its {getattr(model.dist, 'name', 'model')} model, whose " f"parameters are {names}." ) - params = np.atleast_1d(np.asarray(model.params, dtype=float)) - bounds = list(getattr(dist, "bounds", [(None, None)] * len(names))) + params = np.asarray(parameters.params, dtype=float) + bounds = parameters.bounds positions = [names.index(name) for name in spec] covariance = np.diag( [_variance(prior, label, name) for name, prior in spec.items()] @@ -474,10 +572,7 @@ def varied_parameters(model, spec: Any, label: str): positions, params[positions], covariance, - [ - bounds[j] if j < len(bounds) else (None, None) - for j in positions - ], + [bounds[j] for j in positions], ) if isinstance(spec, Sequence): raise ValueError( @@ -490,6 +585,44 @@ def varied_parameters(model, spec: Any, label: str): ) +def split_group_levers(key, spec, groups) -> Tuple[Optional[int], dict, Any]: + """An uncertainty given under the tuple of a common-cause group's + members (``groups``), with the group's own parameters named as its + levers are (``ccf_beta``, ``ccf_alpha_2``, ..., as ``levers()`` and + ``parameter_sensitivity`` key them): the group's index, its own + parameters' distributions by their names (``beta``, ...), and what is + left of ``spec`` for the members' one model. ``(None, {}, spec)`` for + any other key or spec.""" + if not (isinstance(key, tuple) and isinstance(spec, Mapping)): + return None, {}, spec + index = next( + (i for i, g in enumerate(groups) if set(g.members) == set(key)), None + ) + if index is None: + return None, {}, spec + from .ccf import parameters + + own = { + name[len("ccf_") :]: prior + for name, prior in spec.items() + if isinstance(name, str) and name.startswith("ccf_") + } + levers = [f"ccf_{name}" for name in parameters(groups[index].model)] + unknown = [f"ccf_{name}" for name in own if f"ccf_{name}" not in levers] + if unknown: + raise ValueError( + f"Common-cause group {list(key)!r}: {unknown} are not levers of " + f"its model, {groups[index].model!r}, whose levers are " + f"{levers}.{_closest(unknown, levers)}" + ) + rest = { + name: prior + for name, prior in spec.items() + if not (isinstance(name, str) and name.startswith("ccf_")) + } + return index, own, rest + + def varied_ccf_parameters(group, spec: Any): """``varied_parameters`` for a common-cause group's model: the names of the parameters its uncertainty varies, their values, their diff --git a/repyability/repairable.py b/repyability/repairable.py index c03009ef..04bc750d 100644 --- a/repyability/repairable.py +++ b/repyability/repairable.py @@ -53,7 +53,7 @@ from scipy.special import gammaln from repyability.maintenance import FailureLimitPolicy, MaintenancePolicy -from repyability.rbd._model_utils import failure_time_scale +from repyability.rbd._model_utils import MODEL_ERRORS, failure_time_scale from repyability.utils.deprecation import refuse_removed_names from repyability.utils.wrappers import numpy_seed, outside_level @@ -523,7 +523,7 @@ def _timescale(self) -> float: m = failure_time_scale(baseline) if np.isfinite(m) and m > 0.0: return m - except Exception: + except MODEL_ERRORS: pass return 1.0 @@ -542,7 +542,7 @@ def _resolvable_age(self) -> Optional[float]: return None try: age = float(np.ravel(baseline.qf(1.0 - 1e-10))[0]) - except Exception: + except MODEL_ERRORS: return None return age if np.isfinite(age) and age > 0.0 else None diff --git a/repyability/tests/capabilities.py b/repyability/tests/capabilities.py new file mode 100644 index 00000000..b1c33f03 --- /dev/null +++ b/repyability/tests/capabilities.py @@ -0,0 +1,545 @@ +"""The README's capability matrices, made from the code (see +``test_readme_capabilities.py``, which checks the README holds what this +makes). + +The first says which analyses each model type has: a public method of +the class, or a gap. The others say, for each kind of node or component +a diagram can hold, how each family of analyses is worked out on a +diagram with it (``analysis_routes()`` on the catalogue's diagrams): a +refusal that the diagram has nothing the analysis works on +(``routes.Refused``: nothing priced, no capacities, ...) is no gap, and +is left out; one the analysis cannot work out (``NotImplementedError``) +is. + +Run ``python -m repyability.tests.capabilities`` to write them into the +README, between its ``capabilities`` markers. +""" + +import pathlib +import warnings +from typing import Dict, List, Tuple + +from repyability import ( + FaultTree, + Network, + NonRepairableRBD, + PhasedMission, + RepairableRBD, +) +from repyability.rbd import routes + +START = "" +END = "" + +MODELS = { + "NonRepairableRBD": NonRepairableRBD, + "RepairableRBD": RepairableRBD, + "FaultTree": FaultTree, + "Network": Network, + "PhasedMission": PhasedMission, +} + +#: Each capability, by section: for each model type, the methods that give +#: it (the first a model has is shown), or none. +CAPABILITIES: List[Tuple[str, List[Tuple[str, Tuple[str, ...]]]]] = [ + ( + "Reliability and lifetime", + [ + ("Reliability R(t)", ("sf", "reliability")), + ("Unreliability F(t)", ("ff", "unreliability")), + ("Density, hazard, cumulative hazard", ("df", "hf", "Hf")), + ("Mean time to (first) failure", ("mean", "mean_time_to_failure")), + ( + "Each node's own R(t), F(t) and MTTF", + ("node_sf", "node_ff", "node_mttf"), + ), + ( + "B-life, time to a reliability", + ("bx_life", "time_to_reliability"), + ), + ("Given the system's age", ("cs",)), + ("Given the components' ages or states", ("sf_given_state",)), + ( + "Mean residual and remaining life", + ("mean_residual_life", "remaining_life"), + ), + ("Random lifetimes", ("random", "random_block")), + ("Top event probability", ("top_event_probability",)), + ( + "Each phase's failure probability", + ("phase_failure_probabilities",), + ), + ], + ), + ( + "Availability and outages", + [ + ("Long-run availability", ("mean_availability",)), + ("Each node's availability", ("node_availability",)), + ( + "Availability over time", + ("point_availability", "point_unavailability"), + ), + ( + "Over a mission", + ("mission_availability", "mission_unavailability"), + ), + ("Unavailability (precise when small)", ("mean_unavailability",)), + ( + "Failure frequency, MTBF", + ("system_failure_frequency", "mean_time_between_failures"), + ), + ("Mean up and down times", ("mean_up_time", "mean_down_time")), + ( + "Expected failures and events in a window", + ("expected_events", "expected_failures"), + ), + ( + "Simulated availability (distribution)", + ( + "availability", + "availability_from_chunks", + "simulate_chunk", + "shards", + ), + ), + ("Simulated histories", ("simulate_timelines",)), + ("Compare two designs", ("compare",)), + ], + ), + ( + "Costs and spares", + [ + ("Long-run cost rate", ("expected_cost_rate",)), + ("Expected cost in a window", ("expected_cost",)), + ("Simulated cost (distribution)", ("cost",)), + ("Total cost of ownership", ("total_cost",)), + ("Spares demand and stock", ("spares_demand", "spares_stock")), + ], + ), + ( + "Structure", + [ + ( + "Minimal path and cut sets", + ( + "minimal_cut_sets", + "cut_sets", + "minimal_path_sets", + "path_sets", + "get_min_cut_sets", + "get_min_path_sets", + "get_all_path_sets", + ), + ), + ("Each path set's probability", ("path_set_probabilities",)), + ("Cut sets ranked by probability", ("ranked_cut_sets",)), + ("Irrelevant components", ("find_irrelevant_components",)), + ("Convert to and from an RBD", ("to_rbd", "from_rbd")), + ( + "Capacity distribution", + ("capacity_distribution", "system_capacity"), + ), + ("Capacity over time", ("point_capacity", "mission_capacity")), + ( + "How each analysis is computed", + ( + "analysis_routes", + "is_analytically_solvable", + "get_non_analytic_nodes", + ), + ), + ( + "Save and load (dict, JSON)", + ("to_dict", "to_json", "from_dict", "from_json"), + ), + ], + ), + ( + "Importance measures", + [ + ("Birnbaum", ("birnbaum_importance",)), + ("Structural", ("structural_importance",)), + ("Improvement potential", ("improvement_potential",)), + ("Risk achievement worth", ("risk_achievement_worth",)), + ("Risk reduction worth", ("risk_reduction_worth",)), + ("Criticality", ("criticality_importance",)), + ("Fussell-Vesely", ("fussell_vesely",)), + ("Differential (DIM)", ("differential_importance",)), + ("Joint (second order)", ("joint_importance",)), + ("Barlow-Proschan", ("barlow_proschan_importance",)), + ( + "Rate of change over time", + ("reliability_rate", "availability_rate"), + ), + ("Uncertainty importance", ("uncertainty_importance",)), + ("All measures given a state", ("importances_given_state",)), + ( + "Parameter sensitivity, levers", + ("parameter_sensitivity", "levers", "with_levers"), + ), + ], + ), + ( + "Uncertainty", + [ + ( + "Of the reliability or availability", + ( + "sf_uncertainty", + "mean_availability_uncertainty", + "reliability_interval", + "unreliability_interval", + "point_availability_uncertainty", + "mission_availability_uncertainty", + ), + ), + ( + "Of the MTTF", + ("mean_uncertainty", "mean_time_to_failure_interval"), + ), + ( + "Of the B-life", + ("bx_life_uncertainty", "time_to_reliability_uncertainty"), + ), + ("Of the cost rate", ("expected_cost_rate_uncertainty",)), + ], + ), + ( + "Design and maintenance", + [ + ( + "Reliability allocation", + ( + "simple_allocation", + "equal_allocation", + "cost_based_allocation", + "improvement_allocation", + "minimum_effort_allocation", + ), + ), + ( + "Availability, MTTF and MTTR allocation", + ("availability_allocation", "mttf_mttr_allocation"), + ), + ("Redundancy allocation", ("allocate_redundancy",)), + ( + "Reliability-redundancy allocation", + ("allocate_reliability_redundancy",), + ), + ("Cost-reliability front", ("redundancy_front",)), + ( + "Optimal replacement intervals", + ("optimal_replacement_intervals", "with_intervals"), + ), + ( + "Optimal proof-test intervals", + ("optimal_inspection_intervals",), + ), + ], + ), +] + +#: Public methods that are not analyses: the structure evaluated at one +#: state, the event loop's steps and the node names. +NOT_ANALYSES = frozenset( + { + "initialize_event_queue", + "is_system_working", + "next_event", + "node_names", + "occurs", + "system_probability", + "system_timeline", + } +) + +#: The families of analyses the node tables give, by model type: the +#: methods of each. +FAMILIES: Dict[str, List[Tuple[str, Tuple[str, ...]]]] = { + "NonRepairableRBD": [ + ("Reliability", ("sf", "ff")), + ("MTTF", ("mean",)), + ("B-life", ("bx_life", "time_to_reliability")), + ("Given a state", ("sf_given_state", "mean_residual_life")), + ( + "Importance", + ( + "birnbaum_importance", + "criticality_importance", + "fussell_vesely", + "risk_achievement_worth", + "risk_reduction_worth", + "improvement_potential", + "differential_importance", + "joint_importance", + "barlow_proschan_importance", + ), + ), + ("Sensitivity", ("parameter_sensitivity",)), + ("Uncertainty", ("sf_uncertainty", "mean_uncertainty")), + ("Allocation", ("allocate_redundancy", "simple_allocation")), + ("Capacity", ("capacity_distribution",)), + ("Simulation", ("random",)), + ], + "RepairableRBD": [ + ("Long run", ("mean_availability", "mean_unavailability")), + ( + "Frequency", + ( + "system_failure_frequency", + "mean_up_time", + "mean_down_time", + ), + ), + ( + "Over time", + ("point_availability", "mission_availability", "expected_events"), + ), + ("Costs", ("expected_cost_rate", "expected_cost")), + ( + "Importance", + ( + "birnbaum_importance", + "criticality_importance", + "fussell_vesely", + "risk_achievement_worth", + "risk_reduction_worth", + "improvement_potential", + "differential_importance", + "joint_importance", + "barlow_proschan_importance", + "availability_rate", + ), + ), + ("Sensitivity", ("parameter_sensitivity",)), + ( + "Uncertainty", + ( + "mean_availability_uncertainty", + "expected_cost_rate_uncertainty", + ), + ), + ( + "Allocation", + ("allocate_redundancy", "availability_allocation"), + ), + ( + "Intervals", + ("optimal_replacement_intervals", "optimal_inspection_intervals"), + ), + ("Spares", ("spares_demand", "spares_stock")), + ("Capacity", ("capacity_distribution", "point_capacity")), + ("Simulation", ("availability", "cost", "simulate_timelines")), + ], +} + +#: The node tables' rows: a kind of node or component, and the catalogue's +#: diagram with one. +NODES: Dict[str, List[Tuple[str, str]]] = { + "NonRepairableRBD": [ + ("A surpyval model", "plain"), + ("A fitted model (with its covariance)", "fitted"), + ("A fit with a cure share and zero-inflation", "fitted shares"), + ("A probability per demand", "fixed"), + ("A mixture (`MixtureModel`)", "mixture"), + ("A regression node (covariates)", "regression"), + ("A degrading node", "degrading"), + ("A fitted degradation process", "degradation process"), + ("Cold standby (convolved)", "convolved standby"), + ("Standby with no reliability of its own", "simulated standby"), + ("Load sharing", "load sharing"), + ("A repeated node", "repeated"), + ("A repeated standby node", "repeated standby"), + ("A nested RBD", "nested, worked out"), + ("A junction", "junction in series"), + ("A beta-factor common-cause group", "common cause"), + ("An MGL common-cause group", "common cause, MGL"), + ("An alpha-factor common-cause group", "common cause, alpha factor"), + ("Capacities", "capacities"), + ("A structure too meshed to work out", "too meshed"), + ], + "RepairableRBD": [ + ("Any life and repair", "costed_pairs"), + ("A fitted life", "fitted life"), + ("A fitted Crow-AMSAA process", "fitted Crow-AMSAA process"), + ("A probability per demand", "fixed probability"), + ("A mixture life", "mixture life"), + ("A fitted degradation process life", "degradation process life"), + ("Operated part of the time (duty)", "operated part of the time"), + ( + "A standby or load-sharing life, simulated", + "simulated standby life", + ), + ("k-out-of-n", "koon"), + ("A nested RBD", "nested_one_level"), + ("A junction", "junction, bought"), + ("Capacities", "capacities"), + ("Age replacement", "age replacement, priced"), + ("Block replacement", "block replacement"), + ("Opportunistic maintenance", "opportunistic maintenance"), + ("Replacement on condition", "replaced on condition"), + ( + "Replacement on condition by level", + "replaced on condition by level", + ), + ("Hidden failures, tested", "tested, Weibull"), + ("Tests that take time", "tested, taking time"), + ("Tests that can miss a failure", "tested, missing"), + ("Imperfect repair (Kijima)", "imperfect repair"), + ("Minimal repair", "minimal repair in no time"), + ( + "Minimal repair, repairs that take time", + "minimal repair, repairs that take time", + ), + ("Limited repair crews", "one repair crew"), + ("A standby group", "standby group"), + ("A beta-factor group, revealed failures", "common cause, revealed"), + ("A beta-factor group, tested", "common cause, tested"), + ( + "An alpha-factor group, revealed failures", + "common cause, alpha factor", + ), + ( + "A group whose tests take time", + "common cause, timed tests and repairs", + ), + ("A structure too meshed to work out", "too meshed"), + ], +} + +SYMBOLS = { + routes.EXACT: "E", + routes.NUMERICAL: "N", + routes.SIMULATED: "S", +} + + +def _model_table() -> str: + names = list(MODELS) + lines = [ + "| | " + " | ".join(f"`{n}`" for n in names) + " |", + "|---|" + "---|" * len(names), + ] + for section, rows in CAPABILITIES: + lines.append(f"| **{section}** |" + " |" * len(names)) + for label, methods in rows: + cells = [] + for cls in MODELS.values(): + have = [m for m in methods if callable(getattr(cls, m, None))] + cells.append(f"`{have[0]}`" if have else "—") + lines.append(f"| {label} | " + " | ".join(cells) + " |") + return "\n".join(lines) + + +def _gap_kind(call, rbd) -> bool: + """Whether a refused analysis is a gap: the method raises + ``NotImplementedError``, not ``routes.Refused`` (nothing to work on).""" + try: + call(rbd) + except routes.Refused: + return False + except NotImplementedError: + return True + return True + + +def _node_table(model: str) -> str: + from repyability.tests.catalogue import ( + nonrepairable_kinds, + repairable_kinds, + ) + from repyability.tests.test_analysis_routes import ( + NONREPAIRABLE_CALLS, + REPAIRABLE_CALLS, + ) + + kinds, calls = ( + (nonrepairable_kinds(), NONREPAIRABLE_CALLS) + if model == "NonRepairableRBD" + else (repairable_kinds(), REPAIRABLE_CALLS) + ) + families = FAMILIES[model] + lines = [ + "| | " + " | ".join(name for name, _ in families) + " |", + "|---|" + "---|" * len(families), + ] + for label, kind in NODES[model]: + rbd = kinds[kind] + report = rbd.analysis_routes() + cells = [] + for _, methods in families: + seen, gaps = [], [] + for method in methods: + route = report[method] + if route.route != routes.REFUSED: + if SYMBOLS[route.route] not in seen: + seen.append(SYMBOLS[route.route]) + elif _gap_kind(calls[method], rbd): + gaps.append(method) + seen.sort(key="ENS".index) + cell = " ".join(seen) + if gaps: + if len(gaps) == len(methods): + refused = "✗" + elif len(gaps) <= 2: + refused = "✗ " + ", ".join(f"`{m}`" for m in gaps) + else: + refused = f"✗ {len(gaps)} of {len(methods)}" + cell = f"{cell}; {refused}" if cell else refused + cells.append(cell or "–") + lines.append(f"| {label} | " + " | ".join(cells) + " |") + return "\n".join(lines) + + +def matrix() -> str: + """The README's capability section, between its markers.""" + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + nonrepairable = _node_table("NonRepairableRBD") + repairable = _node_table("RepairableRBD") + return "\n".join( + [ + START, + "", + "Which analyses each model type has (the method; — where it " + "has none):", + "", + _model_table(), + "", + "How each family of analyses is worked out on a diagram with " + "each kind of node or component: **E** exact, **N** numerical " + "(to a stated accuracy), **S** simulated, **✗** refused as not " + "worked out (naming the methods, or counting them, where only " + "some are), – where the diagram has nothing the analysis works " + "on. Made from `analysis_routes()` on the test " + "catalogue's diagrams.", + "", + "
NonRepairableRBD, by node" + "", + "", + nonrepairable, + "", + "
", + "", + "
RepairableRBD, by component" + "", + "", + repairable, + "", + "
", + "", + END, + ] + ) + + +def write(path: pathlib.Path) -> None: + """Put ``matrix()`` into the README at ``path``, between its markers.""" + text = path.read_text() + start, end = text.index(START), text.index(END) + len(END) + path.write_text(text[:start] + matrix() + text[end:]) + + +if __name__ == "__main__": + write(pathlib.Path(__file__).resolve().parents[2] / "README.md") diff --git a/repyability/tests/catalogue.py b/repyability/tests/catalogue.py index 9069d0b5..c2f8f7ff 100644 --- a/repyability/tests/catalogue.py +++ b/repyability/tests/catalogue.py @@ -8,16 +8,24 @@ fails until it is. """ +import copy +import functools + import numpy as np import surpyval as surv from repyability import ( MGL, + AlphaFactor, BetaFactor, + Calendar, CCFGroup, DegradingNode, + FaultTree, LoadSharingModel, + NonRepairable, NonRepairableRBD, + Opportunity, PerfectReliability, PerfectUnreliability, RegressionNode, @@ -73,6 +81,20 @@ ] +def _fitted_once(fit): + """``fit``, run once: each call gets its own copy of the fitted model, + as the diagrams are built many times a run and a test may change the + model it is given.""" + once = functools.cache(fit) + + @functools.wraps(fit) + def fitted(): + return copy.deepcopy(once()) + + return fitted + + +@_fitted_once def aft(): """A pump's life at its load (surpyval's ExponentialAFT): 100 at load 1, 25 at load 2.""" @@ -81,11 +103,46 @@ def aft(): return surv.ExponentialAFT.fit(x, Z=load) +@_fitted_once def fitted_life(): """A Weibull fitted to 30 failures: a fit with a parameter covariance.""" return surv.Weibull.fit(np.random.default_rng(7).weibull(2, 30) * 100) +@_fitted_once +def shares_life(): + """A Weibull fitted to 30 failures, 20 units that never failed and 5 + dead on arrival: a fit of the share that ever fails (``lfp_p``) and + the share dead on arrival (``f0``) too (#267).""" + x = np.r_[ + np.random.default_rng(7).weibull(2, 30) * 100, + np.full(20, 300.0), + np.zeros(5), + ] + c = np.r_[np.zeros(30), np.ones(20), np.zeros(5)].astype(int) + return surv.Weibull.fit(x, c=c, lfp=True, zi=True) + + +def wear(): + """A gamma degradation process fitted to the wear of 12 units, each + measured 10 times; it fails at a wear of 8 (#271).""" + g = np.random.default_rng(7) + hours = np.tile(np.arange(1.0, 11.0) * 30.0, 12) + units = np.repeat(np.arange(12), 10) + worn = np.concatenate( + [np.cumsum(g.gamma(2.0, 0.25, 10)) for _ in range(12)] + ) + return surv.GammaProcess.fit(hours, worn, units, threshold=8.0) + + +def crow_amsaa(): + """A Crow-AMSAA process fitted to 60 failures of a repairable unit + (#269): minimal repair of a Weibull life.""" + gaps = np.random.default_rng(7).weibull(1.5, 60) * 300 + return surv.CrowAMSAA.fit(np.cumsum(gaps)) + + +@_fitted_once def mixture_life(): """A two-mode population, infant mortality and wear-out (#227).""" g = np.random.default_rng(7) @@ -159,6 +216,13 @@ def nonrepairable_kinds(): **rest, }, ), + "nested, worked out": NonRepairableRBD( + EDGES, + { + "a": NonRepairableRBD([("s", "x"), ("x", "t")], {"x": unit}), + **rest, + }, + ), "junction in series": NonRepairableRBD( JUNCTION, {"a": unit, **rest, "j": PerfectReliability} ), @@ -167,6 +231,23 @@ def nonrepairable_kinds(): {"a": unit, "b": unit, "c": E([0.002])}, ccf_groups=[CCFGroup(["a", "b"], MGL(0.1))], ), + "common cause, alpha factor": NonRepairableRBD( + VOTE, + { + "a": unit, + "b": unit, + "d": unit, + "c": E([0.002]), + "j": PerfectReliability, + }, + k={"j": 2}, + ccf_groups=[ + CCFGroup( + ["a", "b", "d"], + AlphaFactor([0.95, 0.04, 0.01], testing="non-staggered"), + ) + ], + ), "a branch that never works": NonRepairableRBD( EDGES, {"a": unit, "b": PerfectUnreliability, "c": E([0.002])} ), @@ -188,6 +269,8 @@ def nonrepairable_kinds(): EDGES, {"a": RegressionNode(aft(), covariates=[1.0]), **rest} ), "fitted": NonRepairableRBD(EDGES, {"a": fitted_life(), **rest}), + "fitted shares": NonRepairableRBD(EDGES, {"a": shares_life(), **rest}), + "degradation process": NonRepairableRBD(EDGES, {"a": wear(), **rest}), "mixture": NonRepairableRBD(EDGES, {"a": mixture_life(), **rest}), "degrading": NonRepairableRBD( EDGES, @@ -213,6 +296,54 @@ def nonrepairable_kinds(): } +def fault_tree_kinds(): + """The fault trees of every kind the tree's analyses are checked on + (#314): events given as probabilities, as lifetime models (surpyval + distributions, a fit with its covariance) and as both, a repeated + event, and common-cause groups of each model and basis.""" + unit = W([100, 2]) + gates = {"top": ("or", ["pair", "c"]), "pair": ("and", ["a", "b"])} + rest = {"b": W([80, 1.5]), "c": E([0.002])} + alike = {"a": unit, "b": unit, "c": E([0.002])} + return { + "fixed": FaultTree(gates, {"a": 0.1, "b": 0.2, "c": 0.05}), + "lifetimes": FaultTree(gates, {"a": unit, **rest}), + "mixed": FaultTree(gates, {"a": unit, "b": 0.2, "c": L([4.0, 0.5])}), + "fitted": FaultTree(gates, {"a": fitted_life(), **rest}), + "repeated": FaultTree( + { + "top": ("and", ["A", "B"]), + "A": ("or", ["s", "a"]), + "B": ("or", ["s", "b"]), + }, + {"s": E([0.001]), "a": unit, "b": W([80, 1.5])}, + ), + "common cause": FaultTree( + gates, alike, ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))] + ), + "common cause by rate": FaultTree( + gates, + alike, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1, basis="rate"))], + ), + "common cause, MGL": FaultTree( + { + "top": ("or", ["vote", "d"]), + "vote": ("vote", 2, ["a", "b", "c"]), + }, + {"a": unit, "b": unit, "c": unit, "d": E([0.002])}, + ccf_groups=[ + CCFGroup(["a", "b", "c"], MGL(0.1, 0.3, basis="rate")) + ], + ), + "common cause, fixed": FaultTree( + gates, + {"a": 0.05, "b": 0.05, "c": 0.01}, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], + ), + } + + def systems_of_every_kind(): """The repairable systems every simulation is checked on: those of ``test_performance_equivalence``, with draws that cannot be streamed @@ -251,6 +382,15 @@ def systems_of_every_kind(): return systems +def calendar_policy(life): + """A calendar policy for ``life`` (#319): shuts every 60 at 2, and a + failure at 10.""" + calendar = Calendar( + [Opportunity("shut", 2.0, period=60.0)], failure=10.0, anytime=None + ) + return NonRepairable(life).optimal_calendar_policy(calendar) + + def repairable_kinds(): life, repair = W([500, 1.5]), E([0.5]) @@ -288,6 +428,27 @@ def system(a, **options): replace_cost=10.0, ) ), + "replaced on condition by level": system( + unit( + reliability=wear(), + preventive={ + "interval": 60.0, + "policy": "condition", + "level": 6.0, + "inspection_cost": 1.0, + }, + replace_cost=10.0, + ) + ), + "replaced on a calendar policy": system( + unit( + preventive={ + "policy": "calendar", + "calendar": calendar_policy(life), + }, + replace_cost=10.0, + ) + ), "opportunistic maintenance": RepairableRBD( EDGES, { @@ -302,6 +463,30 @@ def system(a, **options): "train": {"setup_cost": 50.0, "system_down": True} }, ), + # A calendar component in a maintenance group (#349): renewed + # early at the age-replaced member's stops, its own + # replacements stops that renew that member early. + "calendar policy in a maintenance group": RepairableRBD( + EDGES, + { + "a": unit( + preventive={ + "policy": "calendar", + "calendar": calendar_policy(life), + "opportunity": 100.0, + }, + group="train", + replace_cost=10.0, + ), + "b": unit( + preventive={"interval": 300.0, "opportunity": 200.0}, + group="train", + replace_cost=10.0, + ), + "c": unit(), + }, + maintenance_groups={"train": {"setup_cost": 50.0}}, + ), "grouped, no opportunities": RepairableRBD( EDGES, { @@ -339,6 +524,9 @@ def system(a, **options): "replace_cost": 10.0, } ), + "minimal repair, repairs that take time": system( + unit(repair={"model": "kijima1", "q": 1.0}, repair_cost=1.0) + ), "imperfect repair, replaced and maintained": system( unit( repair={"model": "kijima2", "q": 0.8}, @@ -392,6 +580,13 @@ def system(a, **options): "mixture life": system( unit(reliability=mixture_life(), replace_cost=10.0) ), + "fitted Crow-AMSAA process": system( + unit(reliability=crow_amsaa(), repair_cost=1.0) + ), + "degradation process life": system(unit(reliability=wear())), + "operated part of the time": system( + unit(duty=0.25, preventive={"interval": 300.0}) + ), "mixture life, maintained": system( unit( reliability=mixture_life(), @@ -486,15 +681,25 @@ def system(a, **options): "reliability": E([0.002]), "repairability": E([0.5]), } - for node in "ab" - } - | { - "c": unit( - preventive={"interval": 300.0, "policy": "block"} - ) + for node in "abc" }, ccf_groups=[CCFGroup(["a", "b"], MGL(0.2))], ), + "common cause, alpha factor": RepairableRBD( + VOTE, + { + node: { + "reliability": E([0.002]), + "repairability": E([0.5]), + } + for node in "abdc" + } + | {"j": PerfectReliability}, + k={"j": 2}, + ccf_groups=[ + CCFGroup(["a", "b", "d"], AlphaFactor([0.95, 0.04, 0.01])) + ], + ), "common cause, Weibull": system( unit(), ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))] ), diff --git a/repyability/tests/keyed_draws.py b/repyability/tests/keyed_draws.py index aa74195e..64635a6c 100644 --- a/repyability/tests/keyed_draws.py +++ b/repyability/tests/keyed_draws.py @@ -9,7 +9,7 @@ import numpy as np -from repyability.rbd import _streams +from repyability.rbd import _event_loop, _streams def reference_uniform(entropy, spec, r: int, k: int) -> float: @@ -40,7 +40,9 @@ class KeyedDraws: def __init__(self, rbd, t_simulation, seed, antithetic=False): self.entropy = _streams.entropy_of(seed) - self.plan, _ = rbd._stream_plan(t_simulation, self.entropy, antithetic) + self.plan, _ = _event_loop._stream_plan( + rbd, t_simulation, self.entropy, antithetic + ) self.antithetic = antithetic self.taken: dict = defaultdict(int) diff --git a/repyability/tests/test_analysis_routes.py b/repyability/tests/test_analysis_routes.py index a86e6ceb..d5188bf4 100644 --- a/repyability/tests/test_analysis_routes.py +++ b/repyability/tests/test_analysis_routes.py @@ -18,8 +18,11 @@ from repyability import ( AnalysisRoute, DegradingNode, + FaultTree, LoadSharingModel, + Network, NonRepairableRBD, + PhasedMission, RepairableRBD, RepeatedNode, RepeatedStandbyNode, @@ -32,6 +35,7 @@ EDGES, E, W, + fault_tree_kinds, nonrepairable_kinds, repairable_kinds, ) @@ -366,6 +370,110 @@ def test_every_analysis_is_called(kinds, calls): assert set(calls) == set(report) +# -- networks (#315) -------------------------------------------------------- + + +def uncertain_element(net): + """A link's first parameter known to within 5%.""" + for name, model in net.models.items(): + spec = parametric_spec(model) + if spec is not None: + parameter, value = spec.names[0], spec.params[0] + return {name: {parameter: st.uniform(0.95 * value, 0.1 * value)}} + raise AssertionError("No element's model has parameters.") + + +NETWORK_CALLS = { + **{ + name: (lambda name: lambda net: getattr(net, name)(X))(name) + for name in ( + "sf", + "ff", + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "fussell_vesely", + "differential_importance", + "joint_importance", + "barlow_proschan_importance", + ) + }, + "structural_importance": lambda net: net.structural_importance(), + "cut_sets": lambda net: net.cut_sets(), + "path_sets": lambda net: net.path_sets(), + "minimal_cut_sets": lambda net: net.minimal_cut_sets(), + "minimal_path_sets": lambda net: net.minimal_path_sets(), + "mean": lambda net: net.mean(), + "time_to_reliability": lambda net: net.time_to_reliability(0.5), + "bx_life": lambda net: net.bx_life(10), + "random": lambda net: net.random(200, seed=1), + "sf_uncertainty": lambda net: net.sf_uncertainty( + X, uncertain_element(net), n_draws=20, seed=1 + ), + "mean_uncertainty": lambda net: net.mean_uncertainty( + uncertain_element(net), n_draws=5, seed=1 + ), +} + +BRIDGE_LINKS = { + "a": ("s", "x"), + "b": ("s", "y"), + "c": ("x", "y"), + "d": ("x", "t"), + "e": ("y", "t"), +} + + +def networks(): + """Networks of each kind: the bridge with lifetime models and a + failing node, and one of fixed probabilities.""" + return { + "bridge": Network( + { + name: (u, v, W([100 + 10 * i, 1.5])) + for i, (name, (u, v)) in enumerate(BRIDGE_LINKS.items()) + }, + "s", + "t", + nodes={"x": E([0.01])}, + ), + "fixed bridge": Network( + { + name: (u, v, 0.05 + 0.01 * i) + for i, (name, (u, v)) in enumerate(BRIDGE_LINKS.items()) + }, + "s", + "t", + ), + } + + +@pytest.mark.parametrize("name", sorted(networks())) +def test_every_network_method_does_what_the_report_says(name): + net = networks()[name] + report = net.analysis_routes() + assert set(NETWORK_CALLS) == set(report) + for method, call in NETWORK_CALLS.items(): + behaves_as_reported(report[method], lambda: call(net)) + + +def test_the_network_report_covers_every_public_analysis(): + public = { + name + for name, _ in inspect.getmembers(Network, callable) + if not name.startswith("_") + } + assert set(networks()["bridge"].analysis_routes()) == public - { + "analysis_routes", + "from_dict", + "from_json", + "to_dict", + "to_json", + } + + @pytest.mark.parametrize("cls", [NonRepairableRBD, RepairableRBD]) def test_the_report_covers_every_public_analysis(cls): # Everything public but building, saving and the structure itself. @@ -407,6 +515,69 @@ def test_the_report_covers_every_public_analysis(cls): assert set(rbd.analysis_routes()) == public - not_analyses +PHASED_CALLS = { + **{ + name: (lambda name: lambda mission: getattr(mission, name)())(name) + for name in ( + "reliability", + "unreliability", + "sf", + "ff", + "phase_failure_probabilities", + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "fussell_vesely", + ) + }, + "reliability_interval": ( + lambda mission: mission.reliability_interval(200, seed=1) + ), +} + + +def mission(): + """A mission of a bridge then a vote (see ``test_phased_mission``).""" + from repyability.tests.test_phased_mission import BRIDGE, VOTE + + return PhasedMission([("up", 30.0, BRIDGE), ("on", 30.0, VOTE)]) + + +@pytest.mark.parametrize( + "limit, expected", + [(None, {"exact", "simulated"}), (5, {"refused", "simulated"})], + ids=["worked out", "too large"], +) +def test_every_mission_method_does_what_the_report_says( + limit, expected, monkeypatch +): + # A mission's analyses are exact, or refused once its decision diagram + # passes the limit, and its interval simulated (#316). + if limit is not None: + monkeypatch.setattr(phased_mission, "MAX_NODES", limit) + phased = mission() + report = phased.analysis_routes() + assert {route.route for route in report.values()} == expected + for method, call in PHASED_CALLS.items(): + behaves_as_reported(report[method], lambda: call(phased)) + assert set(PHASED_CALLS) == set(report) + + +def test_the_mission_report_covers_every_public_analysis(): + # Everything public but the report itself and the structure. + public = { + name + for name, _ in inspect.getmembers(PhasedMission, callable) + if not name.startswith("_") + } + assert set(mission().analysis_routes()) == public - { + "analysis_routes", + "minimal_cut_sets", + } + + def test_nodes_are_routed_by_how_their_reliability_is_found(): unit = W([100, 2]) cases = [ @@ -505,10 +676,14 @@ def test_the_engine_is_the_one_auto_would_run(monkeypatch): assert "not installed" in route.engine_reason monkeypatch.setattr(_compiled, "available", lambda: True) assert plain.analysis_routes()["availability"].engine == "numba" - capacities = repairable_kinds()["capacities"].analysis_routes() + with_capacities = repairable_kinds()["capacities"] + capacities = with_capacities.analysis_routes() # numba's own loop follows capacities (#155). assert capacities["availability"].engine == "numba" - assert capacities["cost"].engine == "numba" + priced = RepairableRBD( + **{**with_capacities._init_args, "downtime_cost_rate": 1.0} + ) + assert priced.analysis_routes()["cost"].engine == "numba" def test_a_load_sharing_group_is_simulated_only_with_different_units(): @@ -550,6 +725,7 @@ def test_the_guide_s_table_agrees_with_the_report(): EDGES, {n: {"reliability": life, "repairability": repair} for n in "abc"}, capacity=capacity, + downtime_cost_rate=1.0, # priced: the cost analyses refuse else ), } reports = {cls: rbd.analysis_routes() for cls, rbd in plain.items()} @@ -590,9 +766,21 @@ def alone(node): crew = repairable["one repair crew, exponential"] group = repairable["standby group"] minimal = repairable["minimal repair in no time"] + timed_minimal = repairable["minimal repair, repairs that take time"] tested_group = repairable["common cause, tested"] revealed_group = repairable["common cause, revealed"] timed_group = repairable["common cause, timed tests and repairs"] + # A unit dead on arrival, repaired and block replaced in no time (#322). + dead_on_arrival = RepairableRBD( + [("s", "a"), ("a", "t")], + { + "a": { + "reliability": W([100, 2], f0=0.1), + "repairability": "instant", + "preventive": {"interval": 60.0, "policy": "block"}, + } + }, + ) claims = { "Sampled lifetimes or histories, and distributions or percentiles " "of an outcome over a window": [ @@ -719,7 +907,7 @@ def alone(node): (timed_group, "mean_availability", "numerical"), (timed_group, "birnbaum_importance", "numerical"), (timed_group, "point_availability", "numerical"), - (timed_group, "system_failure_frequency", "refused"), + (timed_group, "system_failure_frequency", "numerical"), (timed_group, "availability", "simulated"), (tested_group, "mean_availability", "exact"), (tested_group, "system_failure_frequency", "exact"), @@ -780,6 +968,19 @@ def alone(node): (minimal, "point_availability", "numerical"), (minimal, "expected_cost", "numerical"), (minimal, "mean_availability", "refused"), + (timed_minimal, "expected_failures", "numerical"), + (timed_minimal, "point_availability", "numerical"), + (timed_minimal, "expected_cost", "numerical"), + (timed_minimal, "mean_availability", "refused"), + ], + "Replacement on condition by a measured degradation level": [ + (repairable["replaced on condition by level"], name, route) + for name, route in ( + ("mean_availability", "refused"), + ("point_availability", "refused"), + ("expected_cost", "refused"), + ("availability", "simulated"), + ) ], "Spares with repair crews, standby groups, opportunistic " "maintenance or imperfect repair": [ @@ -802,6 +1003,8 @@ def alone(node): "spares_stock", "numerical", ), + (dead_on_arrival, "spares_stock", "numerical"), + (dead_on_arrival, "spares_demand", "numerical"), ( repairable["tested, constant rate"], "spares_demand", @@ -825,3 +1028,146 @@ def alone(node): assert phased_mission.MAX_STATES == 200_000 assert network.MAX_PATHS == 100_000 assert network.MAX_STATES == 5_000_000 + + +# -- fault trees (#314) ------------------------------------------------------- + + +def uncertain_event(tree): + """An event's model parameter known to within 5%: the first event, + outside the common-cause groups, whose model has parameters (a + probability's is ``p``).""" + members = {m for g in tree.ccf_groups for m in g.members} + for event in tree.events: + spec = parametric_spec(tree._event_model(event)) + if spec is not None and event not in members: + name, value = spec[2][0], spec[1][0] + return {event: {name: st.uniform(0.95 * value, 0.1 * value)}} + raise AssertionError("No event's model has parameters.") + + +FAULT_TREE_CALLS = { + **{ + name: (lambda name: lambda tree: getattr(tree, name)(X))(name) + for name in ( + "top_event_probability", + "sf", + "ff", + "Hf", + "df", + "hf", + "ranked_cut_sets", + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "fussell_vesely", + "differential_importance", + "joint_importance", + "barlow_proschan_importance", + "parameter_sensitivity", + ) + }, + "time_to_reliability": lambda tree: tree.time_to_reliability(0.5), + "bx_life": lambda tree: tree.bx_life(10), + "mean": lambda tree: tree.mean(), + "random": lambda tree: tree.random(200, seed=1), + "structural_importance": lambda tree: tree.structural_importance(), + "sf_uncertainty": lambda tree: tree.sf_uncertainty( + X, uncertain_event(tree), n_draws=2, seed=1 + ), + "mean_uncertainty": lambda tree: tree.mean_uncertainty( + uncertain_event(tree), n_draws=2, seed=1 + ), +} + + +@pytest.mark.parametrize("name", sorted(fault_tree_kinds())) +def test_every_fault_tree_method_does_what_the_report_says(name): + tree = fault_tree_kinds()[name] + report = tree.analysis_routes() + for method, call in FAULT_TREE_CALLS.items(): + behaves_as_reported(report[method], lambda: call(tree)) + + +def test_every_fault_tree_analysis_is_called(): + report = next(iter(fault_tree_kinds().values())).analysis_routes() + assert set(FAULT_TREE_CALLS) == set(report) + + +def test_the_fault_tree_report_covers_every_public_analysis(): + # Everything public but building, saving, converting, the logic at one + # state, the cut and path sets, and the levers. + not_analyses = { + "analysis_routes", + "from_dict", + "from_json", + "from_rbd", + "to_dict", + "to_json", + "to_rbd", + "occurs", + "get_min_cut_sets", + "get_min_path_sets", + "minimal_cut_sets", + "minimal_path_sets", + "levers", + "with_levers", + } + public = { + name + for name, _ in inspect.getmembers(FaultTree, callable) + if not name.startswith("_") + } + tree = next(iter(fault_tree_kinds().values())) + assert set(tree.analysis_routes()) == public - not_analyses + + +def test_a_fault_tree_s_routes_follow_its_events_and_groups(): + kinds = fault_tree_kinds() + fixed = kinds["fixed"].analysis_routes() + assert fixed["top_event_probability"].route == routes.EXACT + assert fixed["mean"].route == routes.REFUSED + assert "does not vary with time" in fixed["mean"].reason + assert fixed["bx_life"].route == routes.REFUSED + assert fixed["sf_uncertainty"].route == routes.SIMULATED + timed = kinds["lifetimes"].analysis_routes() + assert timed["mean"].route == routes.NUMERICAL + assert timed["bx_life"].route == routes.NUMERICAL + assert timed["df"].route == routes.NUMERICAL + assert timed["structural_importance"].route == routes.EXACT + assert timed["barlow_proschan_importance"].route == routes.NUMERICAL + assert timed["mean_uncertainty"].route == routes.SIMULATED + grouped = kinds["common cause"].analysis_routes() + assert grouped["sf"].route == routes.EXACT + assert "common-cause" in grouped["sf"].reason + assert grouped["mean"].route == routes.REFUSED + assert "split a failure probability" in grouped["mean"].reason + assert grouped["mean_uncertainty"].route == routes.REFUSED + assert grouped["joint_importance"].route == routes.EXACT + by_rate = kinds["common cause by rate"].analysis_routes() + assert by_rate["mean"].route == routes.NUMERICAL + assert by_rate["parameter_sensitivity"].route == routes.NUMERICAL + for route in by_rate.values(): + assert route.to_dict()["route"] == route.route + + +def test_a_fault_in_a_check_is_raised_not_reported_as_a_refusal(): + # A refusal (NotImplementedError, or routes.Refused for a diagram with + # nothing the analysis works on) is reported with its message; any + # other error in a check is a fault, and is raised. + def refused(): + raise routes.Refused("No node has a capacity.") + + def cannot(): + raise NotImplementedError("Not exactly.") + + def faulty(): + raise ValueError("operands could not be broadcast together") + + assert routes.refusal(refused) == "No node has a capacity." + assert routes.refusal(cannot) == "Not exactly." + assert routes.refusal(lambda: None) is None + with pytest.raises(ValueError, match="broadcast"): + routes.refusal(faulty) diff --git a/repyability/tests/test_api_consistency.py b/repyability/tests/test_api_consistency.py index 14807d41..4df96b54 100644 --- a/repyability/tests/test_api_consistency.py +++ b/repyability/tests/test_api_consistency.py @@ -33,6 +33,7 @@ from repyability._version import __version__ from repyability.rbd.ccf import with_parameters from repyability.rbd.uncertainty import draw_ccf_models +from repyability.utils import deprecation from repyability.utils.deprecation import NEXT_REMOVAL W, E = surv.Weibull.from_params, surv.Exponential.from_params @@ -260,17 +261,52 @@ def test_the_spares_demand_values_are_properties(): assert demand.std == pytest.approx(np.sqrt(10.0), rel=1e-4) -def test_what_0_13_deprecates_goes_in_0_14(): - # optimal_inspection_intervals(offsets=), renamed offset_shares (#222); - # CapacityDistribution.mean() called, now a property (#235); and the - # simulation options of StandbyModel, LoadSharingModel and - # DegradingNode's exact mean, which it ignores (#233). And the private - # names callers used before the levers were public, kept as they were - # for 0.13: rbd._sensitivity's levers and Lever (#244). +def test_what_0_14_deprecates_goes_in_0_15(): + # NodeState's phase, renamed since_scheduled (#350): giving phase= and + # reading .phase warn, and NEXT_REMOVAL (0.15) refuses them. version = tuple(map(int, __version__.split("."))) assert version < tuple(map(int, NEXT_REMOVAL.split("."))) +def test_a_deprecation_warns_until_the_next_removal(): + # The helpers a deprecation warns through. + match = f"{NEXT_REMOVAL} will refuse it" + with pytest.warns(FutureWarning, match=f"renamed new=.*{match}"): + deprecation.renamed("f", "old", "new", "it is clearer") + with pytest.warns(FutureWarning, match=f"f ignores seed: why.*{match}"): + deprecation.ignored("f", "why.", {"seed": 1, "lower": None}) + number = deprecation.called(2.0, "Result.value") + array = deprecation.called(np.array([1.0, 2.0]), "Result.values") + assert number == 2.0 and np.array_equal(array, [1.0, 2.0]) + with pytest.warns(FutureWarning, match=r"write value, not value\(\)"): + assert number() == 2.0 + with pytest.warns(FutureWarning, match=f"refused in {NEXT_REMOVAL}"): + assert np.array_equal(array(), [1.0, 2.0]) + + +def test_a_node_state_takes_phase_as_since_scheduled_for_now(): + with pytest.warns( + FutureWarning, match="phase= is renamed since_scheduled" + ): + state = NodeState(age=10.0, phase=4.0) + assert state.since_scheduled == 4.0 + assert state == NodeState(age=10.0, since_scheduled=4.0) + assert ( + repr(state) == "NodeState(age=10.0, alive=True, since_scheduled=4.0)" + ) + with pytest.warns(FutureWarning, match="phase is renamed since_scheduled"): + assert state.phase == 4.0 + with pytest.raises(TypeError, match="since_scheduled alone"): + NodeState(phase=4.0, since_scheduled=4.0) + with warnings.catch_warnings(): + warnings.simplefilter("error") + assert NodeState(since_scheduled=4.0).since_scheduled == 4.0 + assert ( + NodeState(stationary=True, since_scheduled=4.0).since_scheduled + == 4.0 + ) + + def test_an_uncertainty_result_shows_a_summary(): pump = surv.Weibull.fit(np.linspace(200, 1800, 50)) rbd = NonRepairableRBD([("s", "pump"), ("pump", "t")], {"pump": pump}) @@ -452,3 +488,71 @@ def test_the_shocks_are_kept(): np.random.default_rng(0), ) assert all(m.shocks == "independent" for m in drawn) + + +# -- the routes print one method a line (0.14 persona review) ---------------- + + +def test_the_routes_print_one_method_a_line(): + import json + + from repyability import AnalysisRoutes, FaultTree + from repyability.rbd.routes import AnalysisRoute + + unit = {"reliability": W([100, 2]), "repairability": E([0.5])} + diagrams = [ + NonRepairableRBD(BRIDGE, {n: W([100, 2]) for n in "abcde"}), + RepairableRBD(BRIDGE, {n: dict(unit) for n in "abcde"}), + FaultTree({"top": ("and", ["a", "b"])}, {"a": W([100, 2]), "b": 0.1}), + Network( + {"a": ("s", "t", W([100, 2])), "b": ("s", "t", 0.1)}, "s", "t" + ), + PhasedMission( + [ + ( + "p", + 10.0, + NonRepairableRBD( + BRIDGE, {n: W([100, 2]) for n in "abcde"} + ), + ) + ] + ), + ] + for diagram in diagrams: + report = diagram.analysis_routes() + assert isinstance(report, AnalysisRoutes) + assert isinstance(report, dict) + assert all(isinstance(r, AnalysisRoute) for r in report.values()) + lines = str(report).splitlines() + assert repr(report) == str(report) + assert len(lines) == len(report) + for line, (name, route) in zip(lines, report.items()): + head, rest = line.split(": ", 1) + assert head == name + assert rest.startswith(route.route + " -- ") + assert "\n" not in rest + assert route.reason.startswith(rest[len(route.route) + 4 :]) + plain = report.to_dict() + assert plain == json.loads(json.dumps(plain)) + first = next(iter(report)) + assert plain[first] == report[first].to_dict() + assert dict(report) == {k: v for k, v in report.items()} + # A long refusal is cut to its first sentence. + long = AnalysisRoutes( + { + "mean": AnalysisRoute( + "refused", "First sentence. Then more.\nAnd more." + ) + } + ) + assert str(long) == "mean: refused -- First sentence." + + +def test_simulated_intervals_say_so(): + rbd = NonRepairableRBD(BRIDGE, {n: W([100, 2]) for n in "abcde"}) + assert rbd.mean_time_to_failure_interval(mc_samples=50, seed=1).method == ( + "simulated" + ) + mission = PhasedMission([("p", 10.0, rbd)]) + assert mission.reliability_interval(50, seed=1).method == "simulated" diff --git a/repyability/tests/test_api_docstrings.py b/repyability/tests/test_api_docstrings.py index 0b4a0c12..1fdb4739 100644 --- a/repyability/tests/test_api_docstrings.py +++ b/repyability/tests/test_api_docstrings.py @@ -15,6 +15,7 @@ import pytest import repyability +from repyability.tests.repository import source def _heading(title: str) -> str: @@ -105,3 +106,19 @@ def test_documented(qualname, obj, kind): def test_the_walk_finds_the_api(): # Guard against the walk silently finding nothing. assert len(MEMBERS) > 150 + + +def test_the_api_pages_document_every_public_name(): + # docs/api/ is split into pages; each public name is on one of them, + # and nothing is documented that is not public. + pages = source("docs/api").glob("*.md") + documented = { + name + for page in pages + for name in re.findall( + r"^::: repyability\.(\w+)", page.read_text(), re.M + ) + } + public = set(repyability.__all__) - {"__version__"} + assert public - documented == set() + assert documented - public <= {"timelines"} # the merge functions diff --git a/repyability/tests/test_availability_allocation.py b/repyability/tests/test_availability_allocation.py index fb7e654f..202eb5e3 100644 --- a/repyability/tests/test_availability_allocation.py +++ b/repyability/tests/test_availability_allocation.py @@ -471,7 +471,7 @@ def test_mttf_mttr_allocation_errors(kwargs, message): def test_a_design_cut_short_still_meets_the_target(monkeypatch): - import repyability.rbd.repairable_rbd as module + import repyability.rbd._repairable_allocation as module real = module.minimize diff --git a/repyability/tests/test_block_replacement.py b/repyability/tests/test_block_replacement.py index 0fa99b7d..ac6ae570 100644 --- a/repyability/tests/test_block_replacement.py +++ b/repyability/tests/test_block_replacement.py @@ -16,6 +16,7 @@ import repyability.rbd._block_replacement as block_replacement from repyability import RepairableRBD +from repyability.rbd import _long_run from repyability.rbd._block_replacement import block_cycle E = surv.Exponential.from_params @@ -107,7 +108,7 @@ def test_a_replacement_that_takes_time(): assert rbd.system_failure_frequency() == pytest.approx( lam * (T - d) / T, rel=1e-6 ) - failures, planned = rbd._outage_frequencies() + failures, planned = _long_run._outage_frequencies(rbd) assert planned == pytest.approx(1.0 / T, rel=1e-9) @@ -192,7 +193,7 @@ def test_replacements_at_the_same_time_take_the_units_down_together(parallel): spec = component(E([0.01]), "instant", T, FIXED([d])) rbd = pair(spec, dict(spec), parallel) assert rbd.mean_availability() == pytest.approx((T - d) / T, abs=1e-6) - failures, planned = rbd._outage_frequencies() + failures, planned = _long_run._outage_frequencies(rbd) assert planned == pytest.approx(1.0 / T, rel=1e-6) if parallel: # One unit failing while the other is up takes nothing down. @@ -210,7 +211,7 @@ def test_replacements_every_other_interval(): assert rbd.mean_availability() == pytest.approx( 1.0 - d / (2 * T), abs=1e-6 ) - _, planned = rbd._outage_frequencies() + _, planned = _long_run._outage_frequencies(rbd) assert planned == pytest.approx(1.0 / (2 * T), rel=1e-6) @@ -227,7 +228,7 @@ def test_a_synchronised_pair_matches_the_simulation(): window = result.mean_availability_interval() assert abs(exact - window.estimate) < 4 * window.standard_error t = result.n_simulations * result.time_simulated_to - _, planned = rbd._outage_frequencies() + _, planned = _long_run._outage_frequencies(rbd) assert result.system_planned_outages / t == pytest.approx( planned, rel=0.02 ) diff --git a/repyability/tests/test_capacity_simulation.py b/repyability/tests/test_capacity_simulation.py index d4b4525c..86c77e60 100644 --- a/repyability/tests/test_capacity_simulation.py +++ b/repyability/tests/test_capacity_simulation.py @@ -14,7 +14,7 @@ import surpyval as surv from repyability import DegradingNode, RepairableRBD -from repyability.rbd.repairable_rbd import _CapacityRecorder +from repyability.rbd._tally import _CapacityRecorder E = surv.Exponential.from_params X = surv.ExactEventTime.from_params @@ -60,9 +60,22 @@ def test_the_long_run_average_matches_the_exact_capacity(): exact = plant.capacity_distribution() result = plant.availability(2000, mc_samples=200, seed=2, demand=100) assert result.mean_capacity == pytest.approx(exact.mean, rel=2e-3) + # The delivered fraction is exact over the window by default (as the + # mean availability is), close to the long run; the simulations' own + # covers the long run. interval = result.delivered_fraction_interval(0.999) - assert interval.lower <= exact.delivered_fraction(100) <= interval.upper + assert interval.method == "exact" and interval.standard_error == 0.0 assert result.delivered_fraction == interval.estimate + assert result.delivered_fraction == pytest.approx( + exact.delivered_fraction(100), rel=1e-4 + ) + own = plant.availability( + 2000, mc_samples=200, seed=2, demand=100, control_variate=False + ) + interval = own.delivered_fraction_interval(0.999) + assert interval.method == "simulated" + assert interval.lower <= exact.delivered_fraction(100) <= interval.upper + assert own.delivered_fraction == result.sample_delivered_fraction total = sum(result.capacity_time.values()) assert total == pytest.approx(200 * 2000) assert list(result.capacity_time) == exact.levels.tolist() @@ -213,8 +226,10 @@ def test_capacities_from_models_are_not_simulated(): ) result = given.availability(10, mc_samples=2, seed=0) assert result.demand == 80.0 - # The cost simulation does not follow the capacity. - assert staged.cost(10, mc_samples=2, seed=0) is None + # The cost simulation does not follow the capacity (and refuses a + # diagram with nothing priced). + with pytest.raises(ValueError, match="Nothing is priced"): + staged.cost(10, mc_samples=2, seed=0) BRIDGE = [ @@ -305,3 +320,43 @@ def test_the_capacity_on_a_grid_is_the_curve_at_its_times(engine): ) assert grid.capacity_time == whole.capacity_time np.testing.assert_array_equal(grid.delivered, whole.delivered) + + +def test_the_delivered_fraction_is_exact_where_mission_capacity_is(): + # By default a run's delivered fraction is the exact expected fraction + # of the demand delivered over the window (mission_capacity), as its + # mean availability is exact (#187): the sample's with + # control_variate=False, and the same from merged chunks. + plant = pumps() + exact = plant.mission_capacity(50.0).delivered_fraction(100) + result = plant.availability(50.0, mc_samples=200, seed=2, demand=100) + assert result.delivered_fraction == exact + assert result.delivered_control is not None + assert result.delivered_control.itself + interval = result.delivered_fraction_interval() + assert (interval.estimate, interval.lower, interval.upper) == ( + exact, + exact, + exact, + ) + assert interval.method == "exact" and interval.standard_error == 0.0 + assert result.sample_delivered_fraction == pytest.approx( + float(np.mean(result.delivered)) + ) + assert result.mean_availability_interval().method == "exact" + own = plant.availability( + 50.0, mc_samples=200, seed=2, demand=100, control_variate=False + ) + assert own.delivered_control is None + assert own.delivered_fraction == result.sample_delivered_fraction + assert own.delivered_fraction_interval().method == "simulated" + assert own.delivered_fraction_interval().standard_error > 0.0 + chunks = [ + plant.simulate_chunk(50.0, 0, 100, seed=2, demand=100), + plant.simulate_chunk(50.0, 100, 200, seed=2, demand=100), + ] + merged = plant.availability_from_chunks(chunks) + assert merged.delivered_fraction == exact + assert merged.delivered_fraction_interval().method == "exact" + # Through JSON, as every result goes. + assert merged.to_dict()["delivered_control"]["exact"] == exact diff --git a/repyability/tests/test_catalogue.py b/repyability/tests/test_catalogue.py index 2aace550..d60871b4 100644 --- a/repyability/tests/test_catalogue.py +++ b/repyability/tests/test_catalogue.py @@ -36,6 +36,7 @@ StandbyModel, ) from repyability.rbd import routes +from repyability.rbd._degradation import is_degradation from repyability.rbd._model_utils import is_fixed_probability, is_mixture from repyability.rbd.results import _ResultMapping from repyability.rbd.uncertainty import is_fit @@ -46,8 +47,12 @@ #: the catalogue: a new export is either here or a node of the diagrams #: that take it (below). NOT_NODES = { + "AlphaFactor", "AnalysisRoute", + "AnalysisRoutes", "BetaFactor", + # A calendar of opportunities, priced (#319): a calendar policy's input. + "Calendar", "CCFGroup", "ComponentOption", "FailureLimitPolicy", @@ -56,10 +61,13 @@ "MaintenancePolicy", "Network", "NodeState", + "Opportunity", "PhasedMission", "RBD", # A component analysed alone: a diagram takes a NonRepairable. "Repairable", + # A part's maintenance plan, valued alone (#319). + "ReplacementPlan", "SimulationChunk", "Timeline", "Timelines", @@ -153,6 +161,8 @@ def model_kinds(model) -> set: return {"NonRepairable"} | model_kinds(model.reliability) if is_mixture(model): return {"surpyval MixtureModel"} + if is_degradation(model): + return {"surpyval degradation process"} if "Regression" in cls.__name__: return {"surpyval regression model"} if is_fixed_probability(model): @@ -240,11 +250,13 @@ def required_repairable() -> set: | { "BetaFactor common cause, probability basis", "MGL common cause, probability basis", + "AlphaFactor common cause, probability basis", "common cause among tested components", } | {"a structure too meshed to work out", "StandbyModel, simulated"} | {node.__name__ for node in REPAIRABLE_NODES} | SURPYVAL_KINDS + | {"surpyval degradation process"} ) @@ -253,11 +265,13 @@ def required_nonrepairable() -> set: {node.__name__ for node in NONREPAIRABLE_NODES} | SURPYVAL_KINDS | {"surpyval MixtureModel", "surpyval regression model"} + | {"surpyval degradation process"} | {"a k-out-of-n vote", "capacities"} | { "BetaFactor common cause, probability basis", "BetaFactor common cause, rate basis", "MGL common cause, probability basis", + "AlphaFactor common cause, probability basis", } | {"a structure too meshed to work out", "StandbyModel, simulated"} ) diff --git a/repyability/tests/test_ccf.py b/repyability/tests/test_ccf.py index 96f8ede0..665690b4 100644 --- a/repyability/tests/test_ccf.py +++ b/repyability/tests/test_ccf.py @@ -14,6 +14,7 @@ from repyability import ( MGL, + AlphaFactor, BetaFactor, CCFGroup, NodeState, @@ -223,11 +224,22 @@ def test_betafactor_validation(): BetaFactor(1.5) assert BetaFactor(0.2) == BetaFactor(0.2) assert BetaFactor(0.2) != BetaFactor(0.3) + # A number: not a bool (which would read as 1), nor text (#233). + for given in (True, "0.1", None, [0.1]): + with pytest.raises(ValueError, match="beta must be a number"): + BetaFactor(given) + for given in (True, "0.3"): + with pytest.raises(ValueError, match="MGL parameters must be num"): + MGL(0.1, given) + assert MGL(0.1, np.float64(0.3)).letters == (0.1, 0.3) def test_ccfgroup_validation(): with pytest.raises(ValueError, match="at least 2"): CCFGroup(["a"], BetaFactor(0.1)) + # One name is not a collection of them (it would read as its letters). + with pytest.raises(ValueError, match="not one name"): + CCFGroup("v1", BetaFactor(0.1)) with pytest.raises(ValueError, match="distinct"): CCFGroup(["a", "a"], BetaFactor(0.1)) with pytest.raises(ValueError, match="BetaFactor"): @@ -325,3 +337,271 @@ def test_mgl_validation(): # An MGL model's parameter count must match the group size. with pytest.raises(ValueError, match="members"): CCFGroup(["a", "b"], MGL(0.1, 0.3)) # 2 letters -> needs 3 members + + +# -- Alpha-factor (#311) ------------------------------------------------------ + +# The alpha-factor model's basic-event probabilities (NUREG/CR-5485, +# Mosleh et al.), for a group of m members with alpha_1..alpha_m summing to +# 1 and each member's total Q: under staggered testing +# Q_k = alpha_k / C(m-1, k-1) * Q, +# and under non-staggered testing +# Q_k = k / C(m-1, k-1) * alpha_k / alpha_t * Q, alpha_t = sum k alpha_k. +# A three-member group with alpha = (0.95, 0.04, 0.01) and Q = 1e-3 then has, +# by hand: staggered Q_1 = 9.5e-4, Q_2 = 0.04 / 2 * 1e-3 = 2e-5, Q_3 = 1e-5; +# non-staggered alpha_t = 0.95 + 0.08 + 0.03 = 1.06, Q_1 = 0.95 / 1.06 * 1e-3, +# Q_2 = 0.04 / 1.06 * 1e-3, Q_3 = 0.03 / 1.06 * 1e-3. +ALPHAS = (0.95, 0.04, 0.01) +BY_HAND = { + "staggered": (9.5e-4, 2e-5, 1e-5), + "non-staggered": (0.95e-3 / 1.06, 0.04e-3 / 1.06, 0.03e-3 / 1.06), +} + + +def _specific_sets(model, members, Q): + """``Q_1, ..., Q_m`` from ``decompose``: the independent part, then + one shock's probability for each size (every set of a size alike).""" + q_independent, shocks = model.decompose(members, Q) + by_size = {} + for subset, p in shocks: + by_size.setdefault(len(subset), set()).add(float(p[0])) + assert all(len(values) == 1 for values in by_size.values()) + return (float(q_independent[0]),) + tuple( + by_size[k].pop() for k in range(2, len(members) + 1) + ) + + +@pytest.mark.parametrize("testing", ["staggered", "non-staggered"]) +def test_alpha_factor_specific_sets_by_hand(testing): + model = AlphaFactor(ALPHAS, testing=testing) + assert model.group_size == 3 + found = _specific_sets(model, list("abc"), 1e-3) + assert found == pytest.approx(BY_HAND[testing], rel=1e-12) + # The Q_k of the sets holding a member sum to its Q. + Q1, Q2, Q3 = found + assert Q1 + 2 * Q2 + Q3 == pytest.approx(1e-3, rel=1e-12) + + +def _mgl_letters(fractions): + """The MGL letters of the specific-set fractions ``Q_k / Q``, + ``k = 1..m``: ``rho_k`` is the share of a member's failures shared + by at least ``k`` members among those shared by at least ``k - 1``.""" + from math import comb + + m = len(fractions) + + def at_least(k): + return sum( + comb(m - 1, j - 1) * fractions[j - 1] for j in range(k, m + 1) + ) + + return [at_least(k) / at_least(k - 1) for k in range(2, m + 1)] + + +@pytest.mark.parametrize("testing", ["staggered", "non-staggered"]) +@pytest.mark.parametrize( + "alphas", [(0.95, 0.04, 0.01), (0.9, 0.06, 0.03, 0.01), (0.8, 0.2)] +) +@pytest.mark.parametrize( + "options", [{}, {"shocks": "independent"}, {"basis": "rate"}] +) +def test_alpha_factor_is_the_equivalent_mgl(testing, alphas, options): + # Its outcomes, and so every analysis, are those of the MGL model with + # the letters its specific-set probabilities give. + model = AlphaFactor(alphas, testing=testing, **options) + letters = _mgl_letters( + _specific_sets( + AlphaFactor(alphas, testing=testing), + list("abcd")[: len(alphas)], + 1.0, + ) + ) + mgl = MGL(*letters, **options) + members = list("abcd")[: len(alphas)] + Q = np.array([1e-4, 0.01, 0.1, 0.4]) + ours = model._split(members, Q, 1.0 - Q) + theirs = mgl._split(members, Q, 1.0 - Q) + np.testing.assert_allclose(ours[0], theirs[0], rtol=1e-12) + np.testing.assert_allclose(ours[1], theirs[1], rtol=1e-12) + assert [s for s, _ in ours[2]] == [s for s, _ in theirs[2]] + for (_, p), (_, q) in zip(ours[2], theirs[2]): + np.testing.assert_allclose(p, q, rtol=1e-12) + own, causes = model._causes(members) + own_mgl, causes_mgl = mgl._causes(members) + assert own == pytest.approx(own_mgl, rel=1e-12) + assert [s for s, _ in causes] == [s for s, _ in causes_mgl] + assert [c for _, c in causes] == pytest.approx( + [c for _, c in causes_mgl], rel=1e-12 + ) + # And a 2-out-of-m system agrees. + edges = [("s", n) for n in members] + [(n, "t") for n in members] + nodes = {n: Weibull.from_params([1000.0, 1.5]) for n in members} + x = np.array([30.0, 300.0]) + assert NonRepairableRBD( + edges, nodes, k={"t": 2}, ccf_groups=[CCFGroup(members, model)] + ).ff(x) == pytest.approx( + NonRepairableRBD( + edges, nodes, k={"t": 2}, ccf_groups=[CCFGroup(members, mgl)] + ).ff(x), + rel=1e-12, + ) + + +def test_alpha_factor_on_a_pair_is_the_beta_factor(): + # Staggered, alpha_2 is the share of a member's failures that are + # shared: the beta factor. Non-staggered, each shared event counts + # twice, so beta = 2 alpha_2 / (alpha_1 + 2 alpha_2). + Q, beta = 0.1, 0.15 + bf = float(_parallel(Q, [CCFGroup(["a", "b"], BetaFactor(beta))]).sf()) + staggered = AlphaFactor([1.0 - beta, beta]) + assert float(_parallel(Q, [CCFGroup(["a", "b"], staggered)]).sf()) == ( + pytest.approx(bf, rel=1e-12) + ) + alpha_2 = beta / (2.0 - beta) + non_staggered = AlphaFactor( + [1.0 - alpha_2, alpha_2], testing="non-staggered" + ) + assert float( + _parallel(Q, [CCFGroup(["a", "b"], non_staggered)]).sf() + ) == pytest.approx(bf, rel=1e-12) + + +def test_alpha_factor_serialisation_roundtrip(): + edges = [("s", n) for n in "abc"] + [(n, "t") for n in "abc"] + nodes = {n: FixedEventProbability.from_params(0.01) for n in "abc"} + for model in ( + AlphaFactor(ALPHAS), + AlphaFactor(ALPHAS, testing="non-staggered"), + AlphaFactor(ALPHAS, shocks="independent"), + AlphaFactor(ALPHAS, testing="non-staggered", basis="rate"), + ): + rbd = NonRepairableRBD( + edges, nodes, ccf_groups=[CCFGroup(list("abc"), model)] + ) + saved = rbd.to_dict() + assert saved["ccf_groups"][0]["model"]["kind"] == "alpha_factor" + assert saved["ccf_groups"][0]["model"]["testing"] == model.testing + for restored in ( + NonRepairableRBD.from_dict(saved), + NonRepairableRBD.from_json(rbd.to_json()), + ): + assert restored.ccf_groups[0].model == model + assert repr(restored.ccf_groups[0].model) == repr(model) + assert float(restored.sf()) == pytest.approx( + float(rbd.sf()), rel=1e-12 + ) + + +def test_alpha_factor_parameters_round_trip(): + from repyability.rbd.ccf import parameters, with_parameters + + model = AlphaFactor(ALPHAS, testing="non-staggered", shocks="independent") + assert parameters(model) == {"alpha_2": 0.04, "alpha_3": 0.01} + assert with_parameters(model, {}) == model + moved = with_parameters(model, {"alpha_2": 0.1}) + assert moved == AlphaFactor( + [0.89, 0.1, 0.01], testing="non-staggered", shocks="independent" + ) + # alpha_1 is the remainder, and cannot go negative. + with pytest.raises(ValueError, match="sum to at most 1"): + with_parameters(model, {"alpha_2": 0.995}) + assert with_parameters(model, {"alpha_2": 0.99}).alphas == pytest.approx( + (0.0, 0.99, 0.01) + ) + with pytest.raises(ValueError, match="not parameters"): + with_parameters(model, {"alpha_1": 0.5}) + + +def test_alpha_factor_levers(): + edges = [("s", n) for n in "abc"] + [(n, "t") for n in "abc"] + nodes = {n: Weibull.from_params([100.0, 2.0]) for n in "abc"} + rbd = NonRepairableRBD( + edges, + nodes, + k={"t": 2}, + ccf_groups=[CCFGroup(list("abc"), AlphaFactor(ALPHAS))], + ) + levers = {(lever.key, lever.name): lever for lever in rbd.levers()} + assert set(levers) == { + (("a", "b", "c"), "alpha"), + (("a", "b", "c"), "beta"), + (("a", "b", "c"), "ccf_alpha_2"), + (("a", "b", "c"), "ccf_alpha_3"), + } + assert levers[(("a", "b", "c"), "ccf_alpha_2")].value == 0.04 + assert levers[(("a", "b", "c"), "ccf_alpha_3")].bounds == (0.0, 1.0) + moved = rbd.with_levers({(("a", "b", "c"), "ccf_alpha_2"): 0.1}) + assert moved.ccf_groups[0].model == AlphaFactor([0.89, 0.1, 0.01]) + with pytest.raises(ValueError, match="between 0 and 1"): + rbd.with_levers({(("a", "b", "c"), "ccf_alpha_2"): 1.5}) + with pytest.raises(ValueError, match="sum to at most 1"): + rbd.with_levers({(("a", "b", "c"), "ccf_alpha_2"): 0.995}) + # The sensitivity moves each alpha_k with alpha_1 the remainder. + x = 50.0 + report = rbd.parameter_sensitivity(x) + step = 1e-5 + up = rbd.with_levers({(("a", "b", "c"), "ccf_alpha_3"): 0.01 + step}) + down = rbd.with_levers({(("a", "b", "c"), "ccf_alpha_3"): 0.01 - step}) + assert report[("a", "b", "c")]["ccf_alpha_3"] == pytest.approx( + (float(up.sf(x)) - float(down.sf(x))) / (2 * step), rel=1e-5 + ) + + +def test_alpha_factor_validation(): + with pytest.raises(ValueError, match="at least two"): + AlphaFactor([1.0]) + with pytest.raises(ValueError, match="in a list"): + AlphaFactor(0.9) + with pytest.raises(ValueError, match=r"\[0, 1\]"): + AlphaFactor([1.5, -0.5]) + with pytest.raises(ValueError, match=r"\[0, 1\]"): + AlphaFactor([0.9, "0.1"]) + with pytest.raises(ValueError, match="sum to 1"): + AlphaFactor([0.9, 0.2]) + with pytest.raises(ValueError, match="testing must be"): + AlphaFactor([0.9, 0.1], testing="weekly") + with pytest.raises(ValueError, match="shocks must be"): + AlphaFactor([0.9, 0.1], shocks="both") + with pytest.raises(ValueError, match="strikes independently"): + AlphaFactor([0.9, 0.1], basis="rate", shocks="exclusive") + # A rounded table's alphas are scaled to sum to 1. + rounded = AlphaFactor([0.958, 0.0263, 0.0086, 0.007]) + assert sum(rounded.alphas) == pytest.approx(1.0, abs=1e-15) + assert rounded.alphas[1] / rounded.alphas[0] == pytest.approx( + 0.0263 / 0.958 + ) + assert AlphaFactor(ALPHAS) == AlphaFactor(list(ALPHAS)) + assert AlphaFactor(ALPHAS) != AlphaFactor(ALPHAS, testing="non-staggered") + assert AlphaFactor(ALPHAS) != MGL(0.05, 0.2) + assert len({AlphaFactor(ALPHAS), AlphaFactor(ALPHAS)}) == 1 + assert repr(AlphaFactor(ALPHAS)) == "AlphaFactor([0.95, 0.04, 0.01])" + assert repr( + AlphaFactor(ALPHAS, testing="non-staggered", basis="rate") + ) == ( + "AlphaFactor([0.95, 0.04, 0.01], testing='non-staggered', " + "basis='rate')" + ) + assert repr(AlphaFactor(ALPHAS, shocks="independent")) == ( + "AlphaFactor([0.95, 0.04, 0.01], shocks='independent')" + ) + assert AlphaFactor(ALPHAS).required_group_size() == 3 + with pytest.raises(ValueError, match="members"): + CCFGroup(["a", "b"], AlphaFactor(ALPHAS)) + with pytest.raises(ValueError, match="BetaFactor, MGL or AlphaFactor"): + CCFGroup(["a", "b"], "alpha") + + +def test_alpha_factor_warns_naming_its_rate_model(): + edges = [("s", n) for n in "abc"] + [(n, "t") for n in "abc"] + nodes = {n: FixedEventProbability.from_params(0.3) for n in "abc"} + rbd = NonRepairableRBD( + edges, + nodes, + ccf_groups=[ + CCFGroup(list("abc"), AlphaFactor(ALPHAS, testing="non-staggered")) + ], + ) + with pytest.warns( + UserWarning, match="testing='non-staggered', basis='rate'" + ): + rbd.sf() diff --git a/repyability/tests/test_ccf_modules.py b/repyability/tests/test_ccf_modules.py index f5705774..49e8a77f 100644 --- a/repyability/tests/test_ccf_modules.py +++ b/repyability/tests/test_ccf_modules.py @@ -19,6 +19,7 @@ from repyability import ( MGL, + AlphaFactor, BetaFactor, CCFGroup, FaultTree, @@ -37,53 +38,129 @@ KEYS = ("works", "fails", "up_ok", "down_ok", "up_bad", "down_bad") -def every_combination(rbd, x): - """What ``_ccf_importances`` and the Fussell-Vesely numerators give, - conditioned on every combination of every group's outcomes.""" +def chosen_pairs(rbd) -> list: + """Pairs of nodes of each kind there is: two members of one group, + members of two groups, a member and a node outside the groups, and two + outside; the first of each.""" + group_of = {m: k for k, g in enumerate(rbd.ccf_groups) for m in g.members} + kinds: dict = {} + for i, j in itertools.combinations(rbd.nodes, 2): + a, b = group_of.get(i), group_of.get(j) + kind = ( + "outside" + if a is None and b is None + else ( + "mixed" + if a is None or b is None + else "within" if a == b else "across" + ) + ) + kinds.setdefault(kind, (i, j)) + return list(kinds.values()) + + +def every_combination(rbd, x, pairs=()): + """What ``_ccf_modules.Evaluation.states`` gives of each node and of + each pair of ``pairs``, and the Fussell-Vesely numerators, conditioned + on every combination of every group's outcomes.""" p = {n: np.atleast_1d(rbd.reliabilities[n].sf(x)) for n in rbd.nodes} q = {n: np.atleast_1d(rbd.reliabilities[n].ff(x)) for n in rbd.nodes} grouped = {m for g in rbd.ccf_groups for m in g.members} sums: dict = {key: {} for key in KEYS} R = Q = 0.0 fv: dict = {} + states = {(i, j): {} for i, j in pairs} for w, given, failing in shock_outcomes(rbd.ccf_groups, p, q): up, down = rbd._system_probabilities(given, failing) R, Q = R + w * up, Q + w * down + + def held(hold: dict): + """The system with each node of ``hold`` failed (1.0) or + working (0.0).""" + down = {n: s * np.ones_like(up) for n, s in hold.items()} + return rbd._system_probabilities( + {**given, **{n: 1.0 - d for n, d in down.items()}}, + {**failing, **down}, + ) + for node in rbd.nodes: works, fails = given[node], failing[node] - one, zero = np.ones_like(works), np.zeros_like(works) - u1, d1 = rbd._system_probabilities( - {**given, node: one}, {**failing, node: zero} - ) - u0, d0 = rbd._system_probabilities( - {**given, node: zero}, {**failing, node: one} - ) + u1, d1 = held({node: 0.0}) + u0, d0 = held({node: 1.0}) on, off = (works, fails) if node in grouped else (1.0, 1.0) for key, value in zip( KEYS, (works, fails, on * u1, on * d1, off * u0, off * d0), ): sums[key][node] = sums[key].get(node, 0.0) + w * value - pairs, failures, size = rbd._node_pairs(given, failing) + for i, j in pairs: + for a, b in itertools.product((True, False), repeat=2): + chance = w + for node, up_ in ((i, a), (j, b)): + if node in grouped: + chance = chance * (given if up_ else failing)[node] + u, d = held({i: 0.0 if a else 1.0, j: 0.0 if b else 1.0}) + was = states[i, j].get((a, b), (0.0, 0.0, 0.0)) + states[i, j][a, b] = ( + was[0] + chance, + was[1] + chance * u, + was[2] + chance * d, + ) + pairs_p, failures, size = rbd._node_pairs(given, failing) structure = rbd._set_structure() for node, value in structure.failed_cut_sets( - pairs, failures, shape=size + pairs_p, failures, shape=size ).items(): fv[node] = fv.get(node, 0.0) + w * value - return R, Q, sums, fv + return R, Q, sums, fv, states + + +def joint_of(states: dict) -> np.ndarray: + """The joint importance from a pair's states (see + ``every_combination``): the system given each combination.""" + given = {combo: up / chance for combo, (chance, up, _) in states.items()} + return ( + given[True, True] + - given[True, False] + - given[False, True] + + given[False, False] + ) def agree(rbd, x, rel=1e-10): - R, Q, sums, fv = every_combination(rbd, x) + pairs = chosen_pairs(rbd) + R, Q, sums, fv, states = every_combination(rbd, x, pairs) up, down = rbd._sf_and_ff(x, None, None) np.testing.assert_allclose(up, R, rtol=rel) np.testing.assert_allclose(down, Q, rtol=rel) - ours = rbd._ccf_importances(x, None, None) - for key in KEYS: - for node in rbd.nodes: + evaluation = rbd._ccf_evaluation( + *rbd._importance_inputs(x, set(), set()), set(), set() + ) + grouped = {m for g in rbd.ccf_groups for m in g.members} + for node in rbd.nodes: + ours = evaluation.states((node,)) + (works, up_ok, down_ok), (fails, up_bad, down_bad) = ( + ours[True,], + ours[False,], + ) + if node not in grouped: + assert works == 1.0 and fails == 1.0 + works, fails = sums["works"][node], sums["fails"][node] + for key, value in zip( + KEYS, (works, fails, up_ok, down_ok, up_bad, down_bad) + ): np.testing.assert_allclose( - ours[key][node], sums[key][node], rtol=rel, err_msg=key + value, sums[key][node], rtol=rel, err_msg=key ) + joint = rbd.joint_importance(x) + for pair in pairs: + np.testing.assert_allclose( + joint[pair], + joint_of(states[pair]), + rtol=1e-8, + atol=1e-13, + err_msg=str(pair), + ) p, q = rbd._importance_inputs(x, set(), set()) shares = rbd._ccf_fv_numerators(p, q, set(), set(), "c", "exact") for node in rbd.nodes: @@ -137,6 +214,14 @@ def paired_tree(n: int, model=lambda: BetaFactor(0.1)) -> FaultTree: (3, lambda i: MGL(0.1, 0.3, basis="rate")), (3, lambda i: MGL(0.1, 0.3) if i % 2 else BetaFactor(0.15)), (3, lambda i: MGL(0.1, 0.0)), # no independent equivalent + (3, lambda i: AlphaFactor([0.9, 0.07, 0.03])), + (3, lambda i: AlphaFactor([0.9, 0.07, 0.03], testing="non-staggered")), + (3, lambda i: AlphaFactor([0.9, 0.07, 0.03], shocks="independent")), + (3, lambda i: AlphaFactor([0.9, 0.07, 0.03], basis="rate")), + ( + 3, + lambda i: AlphaFactor([0.9, 0.1, 0.0]), + ), # no independent equivalent ], ) def test_trains_agree_with_every_combination(kinds, model): @@ -197,6 +282,7 @@ def random_diagram(rng: random.Random): models += [ MGL(rng.uniform(0.05, 0.3), rng.uniform(0.1, 0.5)), MGL(0.2, 0.3, shocks="independent"), + AlphaFactor([0.9, 0.07, 0.03], testing="non-staggered"), ] groups.append(CCFGroup(members, rng.choice(models))) rate = {v: rng.uniform(0.002, 0.03) for v in nodes} @@ -268,6 +354,12 @@ def test_many_groups_across_trains_are_quick(model): assert ours[event] == pytest.approx( float(np.ravel(theirs[event])[0]), rel=1e-9 ) + joint, pairs = tree.joint_importance(200.0), small.joint_importance(200.0) + assert set(joint) == set(pairs) + for pair in chosen_pairs(small): + assert joint[pair] == pytest.approx( + float(np.ravel(pairs[pair])[0]), rel=1e-9 + ) @pytest.mark.parametrize("Q", [1e-6, 1e-3, 0.05, 0.3, 0.9]) @@ -276,6 +368,11 @@ def test_many_groups_across_trains_are_quick(model): [ (MGL(0.1, 0.3), ("a", "b", "c")), (MGL(0.2, 0.5, 0.4), ("a", "b", "c", "d")), + (AlphaFactor([0.9, 0.07, 0.03]), ("a", "b", "c")), + ( + AlphaFactor([0.8, 0.1, 0.06, 0.04], testing="non-staggered"), + ("a", "b", "c", "d"), + ), ], ) def test_exclusive_causes_as_independent_ones(model, members, Q): diff --git a/repyability/tests/test_ccf_over_time.py b/repyability/tests/test_ccf_over_time.py index 7c873ce0..fc289c73 100644 --- a/repyability/tests/test_ccf_over_time.py +++ b/repyability/tests/test_ccf_over_time.py @@ -12,7 +12,14 @@ from scipy import integrate, linalg from surpyval import Exponential, Weibull -from repyability import MGL, BetaFactor, CCFGroup, NodeState, RepairableRBD +from repyability import ( + MGL, + AlphaFactor, + BetaFactor, + CCFGroup, + NodeState, + RepairableRBD, +) from repyability.rbd import routes as r E, W = Exponential.from_params, Weibull.from_params @@ -183,6 +190,16 @@ def failing(u): {n: revealed(0.01, 0.2) for n in "abd"} | {"c": OTHER}, ccf_groups=[CCFGroup(["a", "b", "d"], MGL(0.2, 0.4))], ), + RepairableRBD( + THREE_THEN_C, + {n: revealed(0.01, 0.2) for n in "abd"} | {"c": OTHER}, + ccf_groups=[ + CCFGroup( + ["a", "b", "d"], + AlphaFactor([0.8, 0.12, 0.08], testing="non-staggered"), + ) + ], + ), RepairableRBD( PAIR, { @@ -192,7 +209,7 @@ def failing(u): ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], ), ], - ids=["revealed", "MGL", "staggered tests"], + ids=["revealed", "MGL", "alpha factor", "staggered tests"], ) def test_it_settles_into_the_long_run(rbd): late = 1e6 + np.array([0.0, 125.0, 250.0, 625.0, 900.0]) diff --git a/repyability/tests/test_ccf_repairable_modules.py b/repyability/tests/test_ccf_repairable_modules.py index 5cfb99e9..f8b4c4ff 100644 --- a/repyability/tests/test_ccf_repairable_modules.py +++ b/repyability/tests/test_ccf_repairable_modules.py @@ -12,6 +12,7 @@ its, to rounding. """ +import itertools import random import time @@ -21,12 +22,13 @@ from repyability import ( MGL, + AlphaFactor, BetaFactor, CCFGroup, PerfectReliability, RepairableRBD, ) -from repyability.rbd import _ccf_chain, _ccf_modules +from repyability.rbd import _ccf_chain, _ccf_groups, _ccf_modules, _long_run from repyability.rbd.rbd import RBD E = surv.Exponential.from_params @@ -47,14 +49,16 @@ def revealed(rate, repair): def split(rbd, working=(), broken=()): """The long-run points split by every combination of every group's states, as before #218.""" - times, weights = rbd._long_run_grid() + times, weights = _long_run._long_run_grid(rbd) p = rbd._probabilities_with_overrides( - rbd._availabilities_at(times), working, broken + _long_run._availabilities_at(rbd, times), working, broken ) q = rbd._failures_with_overrides( - rbd._unavailabilities_at(times), working, broken + _long_run._unavailabilities_at(rbd, times), working, broken + ) + p, q, weights, index = _ccf_groups._with_ccf_groups( + rbd, times, p, q, weights ) - p, q, weights, index = rbd._with_ccf_groups(times, p, q, weights) return p, q, weights, index @@ -93,6 +97,76 @@ def per_time(values): return out +def pair_measure(rbd, pair, p, q, weights, index, held=()): + """A pair's joint importance over the split points: conditioned on + the pair's states at each time (a member's by its state in the + points, a node outside the groups held), the system with the pair + held where their states have no chance at a time; 0 for a pair with + a node held.""" + if set(held) & set(pair): + return 0.0 + pairs_p, pairs_q, _ = rbd._node_pairs(p, q) + grouped = {m for g in rbd.ccf_groups for m in g.members} + times = int(index.max()) + 1 + + def per_time(values): + return np.bincount(index, weights=weights * values, minlength=times) + + up, down = rbd._system_probabilities(pairs_p, pairs_q) + R_t, Q_t = per_time(up), per_time(down) + weight = per_time(np.ones_like(weights)) + given = {} + for combo in itertools.product((True, False), repeat=2): + hold_p, hold_q = dict(pairs_p), dict(pairs_q) + chance = np.ones_like(weights) + for node, state in zip(pair, combo): + if node in grouped: + chance = chance * (pairs_p if state else pairs_q)[node] + hold_p[node] = np.where(state, 1.0, 0.0) * np.ones_like(weights) + hold_q[node] = 1.0 - hold_p[node] + u, d = rbd._system_probabilities(hold_p, hold_q) + some = per_time(chance) > 0.0 + with np.errstate(divide="ignore", invalid="ignore"): + given[combo] = ( + np.where(some, per_time(chance * u) / per_time(chance), 0.0) + + np.where(some, 0.0, per_time(u) / weight), + np.where(some, per_time(chance * d) / per_time(chance), 0.0) + + np.where(some, 0.0, per_time(d) / weight), + ) + + def second(k): + return ( + given[True, True][k] + - given[True, False][k] + - given[False, True][k] + + given[False, False][k] + ) + + change = np.where(Q_t <= R_t, -second(1), second(0)) + return float(weight @ change) + + +def chosen_pairs(rbd) -> list: + """Pairs of nodes of each kind there is: two members of one group, + members of two groups, a member and a node outside the groups, and two + outside; the first of each.""" + group_of = {m: k for k, g in enumerate(rbd.ccf_groups) for m in g.members} + kinds: dict = {} + for i, j in itertools.combinations(rbd.nodes, 2): + a, b = group_of.get(i), group_of.get(j) + kind = ( + "outside" + if a is None and b is None + else ( + "mixed" + if a is None or b is None + else "within" if a == b else "across" + ) + ) + kinds.setdefault(kind, (i, j)) + return list(kinds.values()) + + BASE = { "birnbaum": RBD._birnbaum_importance, "improvement": RBD._improvement_potential, @@ -165,6 +239,16 @@ def _agree(rbd, working, broken, rel): assert got[node] == pytest.approx( float(want[node][0]), rel=1e-9, abs=1e-13 ), ("fv", fv_type, method, node) + joint = rbd.joint_importance(working, broken) + assert {frozenset(pair) for pair in joint} == { + frozenset(pair) for pair in itertools.combinations(rbd.nodes, 2) + } + for pair in chosen_pairs(rbd): + assert joint[pair] == pytest.approx( + pair_measure(rbd, pair, p, q, weights, index, working | broken), + rel=1e-9, + abs=1e-13, + ), ("joint", pair) def pairs(n, **options): @@ -339,6 +423,13 @@ def random_diagram(rng: random.Random): models = [BetaFactor(rng.uniform(0.05, 0.4))] if size == 3: models.append(MGL(rng.uniform(0.05, 0.3), rng.uniform(0.1, 0.5))) + shared = rng.uniform(0.05, 0.3) + models.append( + AlphaFactor( + [1.0 - shared, 0.7 * shared, 0.3 * shared], + testing=rng.choice(["staggered", "non-staggered"]), + ) + ) groups.append(CCFGroup(members, rng.choice(models))) rate = {v: rng.uniform(1e-4, 3e-3) for v in nodes} for group in groups: @@ -368,12 +459,34 @@ def test_over_time_agrees_with_every_combination(): # The groups' system over time, against the times split by every # combination (as GroupsSystem evaluated it before #218). for rbd in (nested(), trains(2, 3), bridge()): - curve = rbd._groups_curve(2000.0) + curve = _ccf_groups._groups_curve(rbd, 2000.0) x = np.array([1.0, 30.0, 400.0, 1500.0]) values = {n: c.at(x) for n, c in curve.curves.items()} system = curve.system importance, up, down, rate, own = system.evaluate(values, x) probabilities, weights, index, _ = system._split(values, x) + # The joint importance at the times, each pair conditioned on its + # states in the split (#317), the curves followed to the last time + # as the measure follows them. + joint = rbd.joint_importance(x=x) + to_last = _ccf_groups._groups_curve(rbd, float(x.max())) + at_x = {n: c.at(x) for n, c in to_last.curves.items()} + p, w, i, _ = to_last.system._split(at_x, x) + q = {n: 1.0 - v for n, v in p.items()} + for pair in chosen_pairs(rbd): + for k in range(len(x)): + at = i == k + want = pair_measure( + rbd, + pair, + {n: v[at] for n, v in p.items()}, + {n: v[at] for n, v in q.items()}, + w[at], + np.zeros(int(at.sum()), dtype=int), + ) + assert joint[pair][k] == pytest.approx( + want, rel=1e-9, abs=1e-13 + ), ("joint", pair, k) def total(v): v = np.broadcast_to(np.asarray(v, dtype=float), weights.shape) diff --git a/repyability/tests/test_ccf_timed.py b/repyability/tests/test_ccf_timed.py index 01896be5..83e54b68 100644 --- a/repyability/tests/test_ccf_timed.py +++ b/repyability/tests/test_ccf_timed.py @@ -16,6 +16,7 @@ from surpyval import ExactEventTime, Exponential, Weibull from repyability import BetaFactor, CCFGroup, RepairableRBD +from repyability.rbd import _ccf_groups, _long_run from repyability.rbd import routes as r E, X, W = ( @@ -147,10 +148,79 @@ def test_the_failure_frequency_with_repairs_that_take_time(): failures, abs=4.0 * np.sqrt(failures / n) ) assert rbd.mean_time_between_failures() > 0.0 - # Tests that take time are planned outages, not worked out with groups. - timed = pair(0.2, TIMINGS["fixed test and repair"]) - with pytest.raises(NotImplementedError, match="tests that take time"): - timed.mean_time_between_failures() + + +def test_planned_outages_at_tests_that_take_time(): + # #293: a test that takes time takes a working member off line, and + # the system down if the other is down then: a planned outage, at the + # tests' times on the calendar, from the groups' states just before + # and just after each (the test starting is one of the chain's jumps). + rbd = pair( + 0.2, TIMINGS["fixed test and repair"], rate=2e-3, interval=100.0 + ) + terms, planned = _long_run._outage_terms(rbd) + failures = sum(term for _, term in terms) + assert failures == pytest.approx(rbd.system_failure_frequency()) + T, n = 20000.0, 400 + result = rbd.availability( + T, mc_samples=n, seed=5, control_variate=False, conditional=False + ) + for exact, counts in ( + (failures, result.system_failures), + (planned, result.system_planned_outages), + ): + total = float(np.sum(counts)) + assert total / (n * T) == pytest.approx( + exact, abs=4.0 * np.sqrt(total) / (n * T) + ) + # Mean up and down times take the planned outages with the failures. + up = rbd.mean_availability() + assert rbd.mean_up_time() == pytest.approx(up / (failures + planned)) + assert rbd.mean_down_time() == pytest.approx( + (1.0 - up) / (failures + planned) + ) + routes = rbd.analysis_routes() + for name in ("system_failure_frequency", "mean_up_time", "mean_down_time"): + assert routes[name].route != r.REFUSED, name + + +@pytest.mark.parametrize("block", [False, True]) +def test_planned_outages_with_no_shared_cause_are_the_independent(block): + # With no shared cause, the planned outages at the tests (and at a + # block replacement that takes time, of a node outside the group) are + # those of the independent members, which their own models give. + def build(beta): + timing = TIMINGS["fixed test, exponential repair"] + a = inspected(3e-4, 500.0, **timing) + b = inspected(3e-4, 500.0, offset=170.0, **timing) + edges, components = PAIR, {"a": a, "b": b} + if block: + edges = [ + ("s", "a"), + ("s", "b"), + ("a", "c"), + ("b", "c"), + ("c", "t"), + ] + components["c"] = { + "reliability": W([5000.0, 2.0]), + "repairability": E([0.125]), + "preventive": { + "interval": 250.0, + "policy": "block", + "duration": E([0.2]), + }, + } + groups = ( + [] if beta is None else [CCFGroup(["a", "b"], BetaFactor(beta))] + ) + return RepairableRBD(edges, components, ccf_groups=groups) + + free, grouped = build(None), build(0.0) + for name in ("system_failure_frequency", "mean_up_time", "mean_down_time"): + assert getattr(grouped, name)() == pytest.approx( + getattr(free, name)(), rel=3e-5 + ), name def refusal(**timing): @@ -176,9 +246,9 @@ def test_what_the_chain_does_not_take_says_what_to_do(): def test_copies_of_members_whose_repairs_take_time_are_refused(): rbd = pair(0.1, TIMINGS["fixed repair"]) group = rbd.ccf_groups[0] - times, _ = rbd._long_run_grid() + times, _ = _long_run._long_run_grid(rbd) with pytest.raises(NotImplementedError, match="copies of them"): - rbd._group_states(group, times, counts=[2, 1]) + _ccf_groups._group_states(rbd, group, times, counts=[2, 1]) def test_the_ends_of_tests_and_repairs_are_jumps_over_time(): @@ -223,3 +293,180 @@ def test_the_ends_of_tests_and_repairs_are_jumps_over_time(): jumps = rate.jumps[rate.jump_times < end].sum() change = rbd.point_availability(end) - rbd.point_availability(0.0) assert integral + jumps == pytest.approx(change, abs=1e-7) + + +MEASURES = ( + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "differential_importance", +) + + +@pytest.mark.parametrize( + "name", + [ + "fixed test and repair", + "fixed test, instant repair", + "fixed test, exponential repair", + ], +) +def test_measures_with_tests_that_take_time_tend_to_the_independent(name): + # #294: while a member's test keeps it off line, it cannot be up, and + # the system given it up was 0/0. It is the system with it held up + # then, as for a node outside the groups: so with no shared cause the + # measures are the independent members' own (to the members' own + # model's grid, as above), where leaving those times out would move + # them by the tests' share of the time (0.4% here). + timing = TIMINGS[name] + free, tiny = pair(None, timing), pair(0.0, timing) + grouped = pair(0.1, timing) + for measure in MEASURES: + values = getattr(grouped, measure)() + assert all(np.isfinite(v) or v == np.inf for v in values.values()) + own, near = getattr(free, measure)(), getattr(tiny, measure)() + for node in "ab": + assert near[node] == pytest.approx(own[node], rel=3e-5), measure + + +def test_a_member_in_its_test_is_held_up_and_down(): + # Member b is off line for its test over [170, 172): the pair is down + # just when a is, so b's Birnbaum measure then (b held up, then held + # down) is the system's unavailability. + rbd = pair(0.1, TIMINGS["fixed test, exponential repair"]) + x = np.array([171.0]) + birnbaum = rbd.birnbaum_importance(x=x)["b"] + assert birnbaum == pytest.approx(rbd.point_unavailability(x), rel=1e-9) + + +def trapezoid_between_tests(rbd, stop: float, points: int) -> float: + """The integral from 0 to ``stop`` of the point unavailability by the + trapezoid rule on ``points`` a piece between the tests on the + calendar (finely after each, where an outage starts), each piece + ending just before the next test: the value at a test's time is the + value just after it.""" + tests = np.unique( + np.concatenate( + [ + schedule.offset + schedule.interval * np.arange(0.0, 200.0) + for schedule in rbd._inspection.values() + ] + ) + ) + edges = np.unique(np.concatenate([[0.0, stop], tests])) + edges = edges[(edges >= 0.0) & (edges <= stop)] + total = 0.0 + for lo, hi in zip(edges[:-1], edges[1:]): + x = np.unique( + np.concatenate( + [ + np.linspace(lo, min(lo + 60.0, hi), points), + np.linspace(lo, hi, points), + ] + ) + ) + x[-1] = np.nextafter(hi, -np.inf) + total += np.trapezoid(rbd.point_unavailability(x), x) + return total + + +def test_expected_events_count_the_tests_that_take_time(): + # Each test that takes time takes a working member off line, and the + # pair down where the other is not up then: a planned outage of the + # system, counted over a window as the long run counts it (#293), + # from the group's chain just before and at each test. With no + # shared cause the counts are the independent members' own; the + # down time is the window less the mission availability's share; + # and the simulation counts the same. + def build(beta): + timing = TIMINGS["fixed test and repair"] + a = inspected(2e-3, 100.0, **timing) + groups = ( + [] if beta is None else [CCFGroup(["a", "b"], BetaFactor(beta))] + ) + return RepairableRBD(PAIR, {"a": a, "b": dict(a)}, ccf_groups=groups) + + T = 1050.0 + free, tiny = build(None).expected_events(T), build(0.0).expected_events(T) + assert free.system_planned_outages > 9.0 + assert tiny.system_planned_outages == pytest.approx( + free.system_planned_outages, rel=3e-5 + ) + # The down times to the members' own model's grid, integrated. + assert tiny.system_downtime == pytest.approx( + free.system_downtime, rel=2e-4 + ) + for node in "ab": + for counted in ("node_inspections", "node_failures"): + assert getattr(tiny, counted)[node] == pytest.approx( + getattr(free, counted)[node], rel=3e-5 + ), counted + assert tiny.node_downtime[node] == pytest.approx( + free.node_downtime[node], rel=2e-4 + ) + grouped = build(0.2) + events = grouped.expected_events(T) + assert events.system_downtime == pytest.approx( + T * grouped.mission_unavailability(T), rel=1e-9 + ) + # A shared cause fails both members together more often, so a test + # finds both down more often, with no outage to start. + assert events.system_planned_outages < free.system_planned_outages + assert events.system_planned_outages > 0.9 * free.system_planned_outages + n = 600 + result = grouped.availability( + T, mc_samples=n, seed=3, control_variate=False, conditional=False + ) + for exact, counts in ( + (events.system_failures, result.system_failures), + (events.system_planned_outages, result.system_planned_outages), + ): + total = float(np.sum(counts)) + assert total / n == pytest.approx( + exact, abs=4.0 * max(np.sqrt(total), 1.0) / n + ) + interval = result.mean_availability_interval() + assert abs(interval.estimate - (1.0 - events.system_downtime / T)) < ( + 4.0 * interval.standard_error + ) + + +@pytest.mark.parametrize( + "name", ["exponential test and repair", "fixed test, exponential repair"] +) +def test_the_mission_unavailability_sees_tests_of_an_exponential_length( + name, +): + # A test or repair of an exponential length starts a transient at its + # rate's scale, far shorter than the interval, over which the pair is + # down until the tests are over: the chain's knots follow it, and the + # integral over a window refines to that scale (as it does to a + # unit's closed form, #237). Against the trapezoid rule on a fine + # grid, and the long run after the first period. + timing = {"repair": E([1 / 24.0]), "test": E([1 / 8.0])} + if name == "fixed test, exponential repair": + timing["test"] = X([8.0]) + rate, interval, T = 2e-6, 8760.0, 3 * 8760.0 + a = inspected(rate, interval, **timing) + rbd = RepairableRBD( + PAIR, + {"a": a, "b": dict(a)}, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], + ) + mission = rbd.mission_unavailability(T) + assert mission * T == pytest.approx( + trapezoid_between_tests(rbd, T, 4001), rel=1e-3 + ) + assert mission == pytest.approx( + 1.0 - rbd.mission_availability(T), rel=1e-6 + ) + # Two of the three periods start with a test and its outage: the + # long-run's share, the first period's hidden failures a little + # fewer. + long_run = rbd.mean_unavailability() + assert 0.6 * long_run < mission < long_run + events = rbd.expected_events(T) + assert events.system_downtime == pytest.approx(mission * T, rel=1e-9) + assert events.system_planned_outages == pytest.approx(2.0, abs=0.01) diff --git a/repyability/tests/test_chains_over_time.py b/repyability/tests/test_chains_over_time.py index 4bd8813a..4ee68c33 100644 --- a/repyability/tests/test_chains_over_time.py +++ b/repyability/tests/test_chains_over_time.py @@ -558,11 +558,13 @@ def plant(crews): independent, chained = plant(2), plant(2) chained._crews_couple = lambda: True # type: ignore[method-assign] windows = np.array([5.0, 60.0, 400.0]) + # Each path integrates by its own quadrature: over the shortest window + # both are within about 2e-6 of the closed form (the chain's closer). for name in ("system_failures", "system_downtime"): np.testing.assert_allclose( chained.expected_events(windows)[name], independent.expected_events(windows)[name], - rtol=1e-6, + rtol=5e-6, atol=1e-10, ) np.testing.assert_allclose( diff --git a/repyability/tests/test_condition_long_run.py b/repyability/tests/test_condition_long_run.py index a6eb5ad7..8665331d 100644 --- a/repyability/tests/test_condition_long_run.py +++ b/repyability/tests/test_condition_long_run.py @@ -13,6 +13,7 @@ from repyability import RepairableRBD from repyability.rbd import _condition_replacement as condition +from repyability.rbd import _curves, _long_run from repyability.rbd import routes as r W, E, LN = ( @@ -83,16 +84,26 @@ def test_a_threshold_of_0_replaces_at_every_inspection(duration): def test_a_threshold_no_inspection_reaches_runs_to_failure(): - never = RepairableRBD(PAIR, {"a": unit(threshold=1.0), "b": unit()}) + # Priced, as the cost analyses refuse a diagram with nothing priced. + never = RepairableRBD( + PAIR, {"a": unit(threshold=1.0, repair_cost=100.0), "b": unit()} + ) plain = RepairableRBD( PAIR, - {"a": {"reliability": LIFE, "repairability": REPAIR}, "b": unit()}, + { + "a": { + "reliability": LIFE, + "repairability": REPAIR, + "repair_cost": 100.0, + }, + "b": unit(), + }, ) for name in LONG_RUN: assert getattr(never, name)() == pytest.approx( getattr(plain, name)(), rel=1e-6 ) - cycle = never._block_cycle("a") + cycle = _long_run._block_cycle(never, "a") assert cycle.replaced == 0.0 @@ -144,7 +155,7 @@ def test_the_simulation_agrees(): assert cost.mean() == pytest.approx(rbd.expected_cost_rate(), rel=0.015) # Each inspection the unit is up at is charged. inspections = result.cost.by_category["inspection"] - cycle = rbd._block_cycle("a") + cycle = _long_run._block_cycle(rbd, "a") assert inspections / result.time_simulated_to == pytest.approx( 2.0 * cycle.before / 30.0, rel=0.015 ) @@ -160,6 +171,6 @@ def test_the_routes(): # Over time too (#161): followed from one inspection to the next. over = report["point_availability"] assert over.route == r.NUMERICAL - route, reason = rbd._node_over_time("a", "availability") + route, reason = _curves._node_over_time(rbd, "a", "availability") assert route == r.NUMERICAL and "replacement on condition" in reason assert 0.0 < float(np.ravel(rbd.point_availability(10.0))[0]) < 1.0 diff --git a/repyability/tests/test_condition_over_time.py b/repyability/tests/test_condition_over_time.py index 80587379..8a511ed8 100644 --- a/repyability/tests/test_condition_over_time.py +++ b/repyability/tests/test_condition_over_time.py @@ -12,6 +12,7 @@ from scipy.optimize import brentq from repyability import NodeState, RepairableRBD +from repyability.rbd import _runs from repyability.rbd import routes as r W, E, LN = ( @@ -53,7 +54,7 @@ def categories(cost) -> dict: @pytest.mark.parametrize( "state", - [None, {"c": NodeState(age=30.0, phase=50.0)}], + [None, {"c": NodeState(age=30.0, since_scheduled=50.0)}], ids=["new", "from a state"], ) def test_a_threshold_of_0_is_block_replacement(state): @@ -82,7 +83,7 @@ def test_it_settles_into_its_long_run_cycle(threshold, phase): # intervals are the long-run ones. # Over [T, 4T): at phase 0 the inspection at the start is past. rbd = single(unit(threshold=threshold)) - state = {"c": NodeState(stationary=True, phase=phase)} + state = {"c": NodeState(stationary=True, since_scheduled=phase)} ends = np.array([INTERVAL, 4 * INTERVAL]) mission = np.ravel(rbd.mission_availability(ends, state=state)) up = (ends[1] * mission[1] - ends[0] * mission[0]) / (3 * INTERVAL) @@ -115,7 +116,7 @@ def test_the_first_inspection_decides_at_the_unit_s_own_age(side): first = INTERVAL - phase age = threshold_age(0.2) - first + side * 1e-3 rbd = single(unit(threshold=0.2)) - state = {"c": NodeState(age=age, phase=phase)} + state = {"c": NodeState(age=age, since_scheduled=phase)} replaced = categories(rbd.expected_cost(first + 1.0, state=state)) unfailed = LIFE.sf(age + first) / LIFE.sf(age) expected = unfailed if side > 0 else 0.0 @@ -126,8 +127,8 @@ def test_the_first_inspection_decides_at_the_unit_s_own_age(side): "state", [ None, - {"c": NodeState(age=700.0, phase=50.0)}, - {"c": NodeState(alive=False, down_for=1.0, phase=150.0)}, + {"c": NodeState(age=700.0, since_scheduled=50.0)}, + {"c": NodeState(alive=False, down_for=1.0, since_scheduled=150.0)}, ], ids=["new", "up and kept so far", "in a repair"], ) @@ -156,7 +157,7 @@ def test_against_the_simulation(state): def test_the_twin_keeps_it_and_a_run_to_a_tolerance_takes_exact_values(): rbd = single(unit()) - twin, changes = rbd._twin() + twin, changes = _runs._twin(rbd) assert twin._preventive["c"].policy == "condition" and not changes report = rbd.analysis_routes() for name in ( diff --git a/repyability/tests/test_condition_replacement.py b/repyability/tests/test_condition_replacement.py index 96dfb01b..f396e202 100644 --- a/repyability/tests/test_condition_replacement.py +++ b/repyability/tests/test_condition_replacement.py @@ -8,7 +8,7 @@ from repyability import RepairableRBD from repyability.rbd import routes as r -from repyability.rbd.repairable_rbd import _failures_between +from repyability.rbd._event_loop import _failures_between E, W, L = ( surv.Exponential.from_params, @@ -335,7 +335,10 @@ def test_the_values_over_time_are_numerical_and_the_spares_simulated(): {"interval": 10.0, "policy": "block", "inspection_cost": 1.0}, "only to the 'condition'", ), - ({"interval": 10.0, "policy": "cbm"}, "'age', 'block' or 'condition'"), + ( + {"interval": 10.0, "policy": "cbm"}, + "'age', 'block', 'condition' or 'calendar'", + ), ], ) def test_the_spec_is_checked(preventive, message): diff --git a/repyability/tests/test_conditional.py b/repyability/tests/test_conditional.py index 987bfa40..97452e96 100644 --- a/repyability/tests/test_conditional.py +++ b/repyability/tests/test_conditional.py @@ -15,7 +15,7 @@ from repyability import NodeState, PerfectReliability, RepairableRBD from repyability.rbd import routes as r -from repyability.rbd.repairable_rbd import _ModuleRun +from repyability.rbd._tally import _ModuleRun E, W = Exponential.from_params, Weibull.from_params LIFE, REPAIR = 0.004, 0.25 # y's failure and repair rates @@ -922,3 +922,52 @@ def test_modules_that_changed_state_give_the_conditional_error(): assert cost.mean_se == interval.standard_error assert cost.sample_mean == pytest.approx(np.mean(cost.samples)) assert math.fsum(cost.by_category.values()) == pytest.approx(cost.mean) + + +def test_the_delivered_fraction_is_taken_given_the_modules(): + # A line with a standby group (a module) and capacities: by default a + # plain run's delivered fraction is each simulation's expected + # fraction given the modules' histories, the same values a run of the + # modules alone gives from the same seed, and the simulations' own + # stays as sample_delivered_fraction. + line = { + f"u{i}": { + "reliability": W([40.0 + 5 * i, 1.8]), + "repairability": E([0.5]), + } + for i in range(3) + } + pumps = { + "reliability": W([8.0, 2.0]), + "repairability": E([0.5]), + "standby": {"units": 2}, + } + names = [*line, "pumps"] + rbd = RepairableRBD( + [("s", names[0])] + list(zip(names, names[1:])) + [(names[-1], "t")], + {**line, "pumps": pumps}, + capacity={n: 100 for n in names}, + ) + options = dict(mc_samples=300, seed=1, demand=80.0) + alone = rbd.availability(100.0, conditional=True, **options) + plain = rbd.availability(100.0, **options) + own = rbd.availability(100.0, conditional=False, **options) + assert alone.delivered_fraction_interval().method == "conditional" + assert plain.delivered_fraction_interval().method == "conditional" + assert own.delivered_fraction_interval().method == "simulated" + assert plain.conditional is not None and plain.conditional.whole + np.testing.assert_allclose( + plain.conditional.delivered, alone.delivered, rtol=1e-12 + ) + assert plain.delivered_fraction == pytest.approx( + alone.delivered_fraction, rel=1e-12 + ) + assert plain.sample_delivered_fraction == own.delivered_fraction + assert plain.delivered_fraction != plain.sample_delivered_fraction + # The expected values vary less than the simulations' own. + assert plain.delivered_fraction_interval().standard_error < ( + own.delivered_fraction_interval().standard_error + ) + # Within a few standard errors of each other, both being unbiased. + spread = own.delivered_fraction_interval().standard_error + assert abs(plain.delivered_fraction - own.delivered_fraction) < 4 * spread diff --git a/repyability/tests/test_conditional_survival.py b/repyability/tests/test_conditional_survival.py new file mode 100644 index 00000000..a7d51e7b --- /dev/null +++ b/repyability/tests/test_conditional_survival.py @@ -0,0 +1,82 @@ +"""Conditional survival at old ages (#268): the chance of surviving a +further ``x`` given survival to ``X`` is worked out from the cumulative +hazard, ``exp(-(H(X + x) - H(X)))``, where the model has one, so it keeps +its precision where ``R(X + x)`` and ``R(X)`` are both too small for a +float, rather than coming out 0 as their ratio did.""" + +import numpy as np +import pytest +import surpyval as surv + +from repyability import NodeState, NonRepairableRBD +from repyability.utils.wrappers import conditional_survival + +W = surv.Weibull.from_params + + +def weibull_cs(x, X, alpha, beta): + return np.exp(-(((X + x) / alpha) ** beta - (X / alpha) ** beta)) + + +@pytest.mark.parametrize("X", [50.0, 300.0, 600.0, 1000.0, 2000.0]) +def test_a_weibull_far_past_its_life(X): + # At X = 1000 the Weibull's R(X) = exp(-1000) is below the smallest + # float; the further 10 hours' survival is exp(-30.3) = 6.9e-14. + model = W([100, 3]) + expected = weibull_cs(10.0, X, 100, 3) + assert conditional_survival(model, 10.0, X) == pytest.approx( + expected, rel=1e-9 + ) + + +def test_a_limited_failure_population_far_past_its_life(): + # Long after the units that fail have failed, the survivors never + # fail: the conditional survival is 1. + model = W([100, 3], lfp_p=0.4) + assert conditional_survival(model, 50.0, 1e4) == pytest.approx(1.0) + at = 100.0 + f = lambda t: 1 - np.exp(-((t / 100) ** 3)) # noqa: E731 + expected = (1 - 0.4 * f(at + 20)) / (1 - 0.4 * f(at)) + assert conditional_survival(model, 20.0, at) == pytest.approx( + expected, rel=1e-12 + ) + + +def test_an_age_that_cannot_be_reached_survives_nothing(): + # A model whose life ends: past the end, nothing has survived to X. + model = surv.Uniform.from_params([0.0, 100.0]) + np.testing.assert_array_equal( + conditional_survival(model, [10.0, 50.0], 150.0), [0.0, 0.0] + ) + + +def test_it_agrees_with_the_ratio_where_the_ratio_is_precise(): + model = W([100, 2]) + x = np.linspace(0.0, 200.0, 21) + for X in (0.0, 10.0, 80.0, 150.0): + np.testing.assert_allclose( + conditional_survival(model, x, X), + model.sf(x + X) / model.sf(X), + rtol=1e-12, + ) + + +def test_a_node_state_far_past_its_life(): + # sf_given_state conditions each node on its age through it. + rbd = NonRepairableRBD( + [("s", "a"), ("a", "b"), ("b", "t")], + {"a": W([100, 3]), "b": surv.Exponential.from_params([0.001])}, + ) + state = {"a": NodeState(age=1000.0)} + expected = weibull_cs(10.0, 1000.0, 100, 3) * np.exp(-0.01) + assert rbd.sf_given_state(10.0, state) == pytest.approx(expected, rel=1e-9) + + +def test_a_diagrams_own_conditional_survival(): + # The diagram's cs is its own Hf's: two units in series, aged 600. + rbd = NonRepairableRBD( + [("s", "a"), ("a", "b"), ("b", "t")], + {"a": W([100, 3]), "b": W([100, 3])}, + ) + expected = weibull_cs(10.0, 600.0, 100, 3) ** 2 + assert rbd.cs(10.0, 600.0) == pytest.approx(expected, rel=1e-9) diff --git a/repyability/tests/test_control_variate.py b/repyability/tests/test_control_variate.py index d0eac79f..5cd03095 100644 --- a/repyability/tests/test_control_variate.py +++ b/repyability/tests/test_control_variate.py @@ -10,7 +10,7 @@ import surpyval as surv from repyability import ControlVariate, NodeState, RepairableRBD -from repyability.rbd import _streams +from repyability.rbd import _curves, _event_loop, _runs, _streams from repyability.tests.test_performance_equivalence import binomial_first from repyability.tests.test_simulation_chunks import identical @@ -93,7 +93,7 @@ def test_the_controlled_estimate_is_unbiased_and_more_precise(): ) assert control.twin.shape == (2000,) assert control.exact == pytest.approx( - rbd._twin()[0].mission_availability(1000.0) + _runs._twin(rbd)[0].mission_availability(1000.0) ) fractions = controlled.uptimes / 1000.0 assert c.estimate == pytest.approx(control.controlled(fractions).mean()) @@ -186,7 +186,7 @@ def test_from_a_state_and_with_held_nodes(): broken_nodes=["B"], control_variate=True, ) - exact = rbd._twin()[0].mission_availability(500.0, broken_nodes=["B"]) + exact = _runs._twin(rbd)[0].mission_availability(500.0, broken_nodes=["B"]) assert held.control_variate.exact == pytest.approx(exact) @@ -216,7 +216,7 @@ def test_the_twin_leaves_out_what_ties_components_together(): repair_crews=2, maintenance_groups={"g": {}}, ) - twin, changes = rbd._twin() + twin, changes = _runs._twin(rbd) assert changes == [ "the limit on repair crews", "the maintenance groups", @@ -230,13 +230,13 @@ def test_the_twin_leaves_out_what_ties_components_together(): # The standby group's twin is its units, as a nested RBD whose streams # are the units'. assert isinstance(twin.components["G"], RepairableRBD) - mine, _ = rbd._stream_specs(500.0) - theirs, _ = twin._stream_specs(500.0) + mine, _ = _event_loop._stream_specs(rbd, 500.0) + theirs, _ = _event_loop._stream_specs(twin, 500.0) for unit_index in range(3): for kind in (_streams.FAILURE, _streams.REPAIR): assert (("G", unit_index), kind) in mine assert (("G", unit_index), kind) in theirs - assert twin._over_time().route == "numerical" + assert _curves._over_time(twin).route == "numerical" route = rbd.analysis_routes()["availability"] assert route.twin.startswith("the system without the limit on repair") assert "Exact twin: the system without" in str(route) @@ -328,13 +328,13 @@ def test_a_system_that_is_its_own_twin_is_simulated_once(monkeypatch): # Its twin's run, drawn from the same streams, is its own to the last # bit (#186): one run a round, where a twin that differs runs beside it. runs = [] - plain = RepairableRBD._run + plain = _event_loop._run - def counted(self, *args, **kwargs): - runs.append(self) - return plain(self, *args, **kwargs) + def counted(rbd, *args, **kwargs): + runs.append(rbd) + return plain(rbd, *args, **kwargs) - monkeypatch.setattr(RepairableRBD, "_run", counted) + monkeypatch.setattr(_event_loop, "_run", counted) own = RepairableRBD(EDGES, {n: unit() for n in "ABC"}) result = own.availability( 300.0, mc_samples=100, seed=3, control_variate=True @@ -383,16 +383,16 @@ def test_the_twin_builds_each_curve_once(monkeypatch): rbd = RepairableRBD( EDGES, {n: unit(scale) for n, scale in zip("ABC", (90, 100, 110))} ) - twin, _ = rbd._twin() + twin, _ = _runs._twin(rbd) apart = (twin.expected_cost(300.0).mean, twin.mission_availability(300.0)) built = [] - plain = RepairableRBD._unit_curve + plain = _curves._unit_curve - def counted(self, node, *args, **kwargs): + def counted(rbd, node, *args, **kwargs): built.append(node) - return plain(self, node, *args, **kwargs) + return plain(rbd, node, *args, **kwargs) - monkeypatch.setattr(RepairableRBD, "_unit_curve", counted) + monkeypatch.setattr(_curves, "_unit_curve", counted) with twin._sharing_curves(): together = ( twin.expected_cost(300.0).mean, @@ -463,7 +463,7 @@ def test_a_run_takes_exact_values_by_default(monkeypatch): def refuse(*args, **kwargs): raise NotImplementedError("too many grid points") - monkeypatch.setattr(RepairableRBD, "_twin_exact", refuse) + monkeypatch.setattr(_runs, "_twin_exact", refuse) with pytest.warns(RuntimeWarning, match="did not converge"): fallen = rbd.availability( 300.0, mc_samples=50, seed=3, tolerance=1e-6, max_samples=100 diff --git a/repyability/tests/test_crew_chain.py b/repyability/tests/test_crew_chain.py index 49542fc6..bc15f2d9 100644 --- a/repyability/tests/test_crew_chain.py +++ b/repyability/tests/test_crew_chain.py @@ -73,7 +73,7 @@ def test_k_of_n_units_match_the_machine_repair_model(n, lam, mu, crews, k): (1 - up) / frequency, rel=1e-8 ) unit = sum(p[j] * (n - j) / n for j in range(n + 1)) - expected = {**{i: unit for i in range(n)}, "s": 1.0, "t": 1.0} + expected = {i: unit for i in range(n)} assert rbd.node_availability() == pytest.approx(expected, rel=1e-12) diff --git a/repyability/tests/test_degradation.py b/repyability/tests/test_degradation.py new file mode 100644 index 00000000..4f0abe6a --- /dev/null +++ b/repyability/tests/test_degradation.py @@ -0,0 +1,244 @@ +"""surpyval's fitted degradation processes as lives, and replacement on +condition by the measured level (#271).""" + +import numpy as np +import pytest +import surpyval as surv +from scipy import stats + +from repyability import NodeState, NonRepairableRBD, RepairableRBD +from repyability.rbd._degradation import level_after +from repyability.rbd.serialisation import rbd_from_json, rbd_to_json +from repyability.tests.catalogue import wear + +EDGES = [("s", "a"), ("a", "t")] +REPAIR = surv.Exponential.from_params([1 / 20.0]) + + +def drift(): + """A Wiener process fitted to 10 units' readings; it fails at 8.""" + g = np.random.default_rng(3) + hours = np.tile(np.arange(1.0, 11.0) * 30.0, 10) + units = np.repeat(np.arange(10), 10) + worn = np.concatenate( + [ + np.cumsum(0.6 + 0.15 * np.sqrt(30.0) * g.standard_normal(10)) + for _ in range(10) + ] + ) + return surv.WienerProcess.fit(hours, worn, units, threshold=8.0) + + +def levelled(model, level, interval=60.0, repair_cost=0.0, cost=0.0): + """One component replaced on condition at ``level``, repaired in no + time.""" + spec = { + "reliability": model, + "repairability": "instant", + "repair_cost": repair_cost, + "preventive": { + "policy": "condition", + "interval": interval, + "level": level, + "cost": cost, + }, + } + return RepairableRBD(EDGES, {"a": spec}) + + +def reference(model, level, interval, horizon, n, dt=0.25, seed=0): + """Each simulation's failures and replacements, from the process's + definition: its path stepped on a grid of ``dt`` (a Wiener step crossing + the threshold within it with its Brownian bridge's probability), failed + at the threshold and repaired as new at once, and inspected at every + multiple of ``interval``. Exact but for failures falling on the grid.""" + g = np.random.default_rng(seed) + threshold, new = float(model.threshold), float(model.y0) + a, b = (float(p) for p in model.params) + wiener = type(model).__name__.startswith("Wiener") + y = np.full(n, new) + failures, replacements = np.zeros(n), np.zeros(n) + every = round(interval / dt) + for k in range(1, round(horizon / dt) + 1): + if wiener: + step = y + a * dt + b * np.sqrt(dt) * g.standard_normal(n) + gap, after = threshold - y, np.maximum(threshold - step, 0.0) + bridge = np.exp(-2.0 * gap * after / (b * b * dt)) + failed = (step >= threshold) | (g.random(n) < bridge) + else: + step = y + g.gamma(a * dt, 1.0 / b, n) + failed = step >= threshold + failures += failed + y = np.where(failed, new, step) + if k % every == 0: + replaced = y >= level + replacements += replaced + y = np.where(replaced, new, y) + return failures, replacements + + +# A degradation process as a life ------------------------------------------ + + +@pytest.mark.parametrize("make", [wear, drift]) +def test_a_degradation_life_is_its_first_passage(make): + model = make() + rbd = NonRepairableRBD(EDGES, {"a": model}) + x = np.array([100.0, 300.0, 600.0]) + np.testing.assert_allclose(rbd.sf(x), np.ravel(model.sf(x)), rtol=1e-12) + assert rbd.mean() == pytest.approx(float(model.mean()), rel=1e-6) + + +@pytest.mark.parametrize("make", [wear, drift]) +def test_a_repaired_degradation_life_is_exact(make): + model = make() + rbd = RepairableRBD( + EDGES, {"a": {"reliability": model, "repairability": REPAIR}} + ) + life = float(model.mean()) + assert rbd.mean_availability() == pytest.approx( + life / (life + 20.0), rel=1e-6 + ) + run = rbd.availability( + t_simulation=20_000.0, mc_samples=200, seed=2, control_variate=False + ) + interval = run.mean_availability_interval(0.9999) + assert interval.lower < rbd.mean_availability() < interval.upper + + +def test_an_accelerated_process_is_refused(): + g = np.random.default_rng(1) + hours = np.tile(np.arange(1.0, 6.0) * 10.0, 6) + units = np.repeat(np.arange(6), 5) + stress = np.repeat([1.0, 1.0, 2.0, 2.0, 3.0, 3.0], 5) + worn = np.concatenate( + [np.cumsum(g.gamma(s, 0.25, 5)) for s in (1, 1, 2, 2, 3, 3)] + ) + model = surv.GammaProcess.fit( + hours, worn, units, threshold=8.0, Z=stress[:, None] + ) + with pytest.raises(ValueError, match="stress covariates"): + NonRepairableRBD(EDGES, {"a": model}) + + +# The level after an interval, given no failure ------------------------------ + + +def test_a_gamma_level_is_the_truncated_increment(): + model = wear() + alpha, beta = (float(p) for p in model.params) + y, t, threshold = 3.0, 90.0, float(model.threshold) + g = np.random.default_rng(4) + drawn = y + g.gamma(alpha * t, 1.0 / beta, 200_000) + kept = drawn[drawn < threshold] + ours = [level_after(model, y, t, u) for u in g.random(4000)] + assert stats.ks_2samp(kept, ours).pvalue > 1e-3 + assert max(ours) < threshold + + +def test_a_wiener_level_is_the_path_killed_at_the_threshold(): + model = drift() + mu, sigma = (float(p) for p in model.params) + y, t, threshold = 5.0, 60.0, float(model.threshold) + g = np.random.default_rng(5) + # Paths on a fine grid, killed where a step or its bridge crosses. + n, steps = 40_000, 600 + dt = t / steps + path = np.full(n, y) + alive = np.ones(n, bool) + for _ in range(steps): + step = path + mu * dt + sigma * np.sqrt(dt) * g.standard_normal(n) + after = np.maximum(threshold - step, 0.0) + bridge = np.exp(-2.0 * (threshold - path) * after / (sigma**2 * dt)) + alive &= (step < threshold) & (g.random(n) >= bridge) + path = step + ours = [level_after(model, y, t, u) for u in g.random(4000)] + assert stats.ks_2samp(path[alive], ours).pvalue > 1e-3 + assert max(ours) < threshold + + +# Replacement on condition by level ----------------------------------------- + + +@pytest.mark.parametrize("make, level", [(wear, 7.0), (drift, 6.0)]) +def test_level_replacement_agrees_with_the_process(make, level): + model = make() + horizon, n = 2000.0, 150 + failures, replacements = reference(model, level, 60.0, horizon, 8000) + for counted, costs in ( + (failures, {"repair_cost": 1.0}), + (replacements, {"cost": 1.0}), + ): + run = levelled(model, level, **costs).cost( + t_simulation=horizon, mc_samples=n, seed=1 + ) + spread = np.sqrt(counted.var() / counted.size + run.samples.var() / n) + assert run.samples.mean() == pytest.approx( + counted.mean(), abs=4.0 * spread + ) + + +def test_a_lower_level_replaces_more_and_fails_less(): + model = wear() + counts = [ + levelled(model, level, repair_cost=1.0, cost=1000.0) + .cost(t_simulation=3000.0, mc_samples=60, seed=3) + .samples + for level in (7.5, 5.0) + ] + failures = [np.mean(c % 1000.0) for c in counts] + replacements = [np.mean(c // 1000.0) for c in counts] + assert failures[1] < failures[0] + assert replacements[1] > replacements[0] + + +def test_a_level_is_kept_by_saving(): + rbd = levelled(wear(), 6.5, repair_cost=3.0, cost=1.0) + again = rbd_from_json(rbd_to_json(rbd)) + first = rbd.cost(t_simulation=1000.0, mc_samples=20, seed=7) + assert again.cost( + t_simulation=1000.0, mc_samples=20, seed=7 + ).samples.tolist() == (first.samples.tolist()) + + +def test_level_replacement_is_simulated_only(): + rbd = levelled(wear(), 6.0, repair_cost=1.0, cost=1.0) + for exact in ( + rbd.mean_availability, + lambda: rbd.point_availability([100.0]), + lambda: rbd.expected_cost(500.0), + ): + with pytest.raises(NotImplementedError, match="measured degradation"): + exact() + with pytest.raises(NotImplementedError, match="leave it out"): + rbd.availability( + t_simulation=100.0, + mc_samples=2, + seed=0, + state={"a": NodeState(age=50.0)}, + ) + + +@pytest.mark.parametrize( + "preventive, life, match", + [ + ({"policy": "age", "level": 6.0}, wear, "only to the 'condition'"), + ({"policy": "condition", "level": 6.0}, None, "Wiener or gamma"), + ( + {"policy": "condition", "level": 6.0, "threshold": 0.1}, + wear, + "not both", + ), + ({"policy": "condition", "level": 8.0}, wear, "below the"), + ({"policy": "condition", "level": "6"}, wear, "below the"), + ], +) +def test_a_level_is_checked(preventive, life, match): + model = life() if life is not None else surv.Weibull.from_params([500, 2]) + spec = { + "reliability": model, + "repairability": REPAIR, + "preventive": {"interval": 60.0, **preventive}, + } + with pytest.raises(ValueError, match=match): + RepairableRBD(EDGES, {"a": spec}) diff --git a/repyability/tests/test_differential_importance.py b/repyability/tests/test_differential_importance.py index 60c85213..4c97fbb7 100644 --- a/repyability/tests/test_differential_importance.py +++ b/repyability/tests/test_differential_importance.py @@ -256,6 +256,16 @@ def test_a_repairable_diagram_s_components_over_time(): ) total = sum(birnbaum.values()) assert state == pytest.approx({n: v / total for n, v in birnbaum.items()}) + # Several windows: one share each, as the other measures give. + windows = np.array([50.0, 100.0]) + several = rbd.differential_importance(window=windows) + for node, share in several.items(): + assert share.shape == windows.shape + for k, length in enumerate(windows): + assert share[k] == pytest.approx( + rbd.differential_importance(window=float(length))[node], + rel=1e-12, + ) def test_a_repairable_diagram_s_lives_against_its_repairs(): diff --git a/repyability/tests/test_engine_plugins.py b/repyability/tests/test_engine_plugins.py index 28f8876d..fe34da7c 100644 --- a/repyability/tests/test_engine_plugins.py +++ b/repyability/tests/test_engine_plugins.py @@ -12,7 +12,7 @@ import pytest -from repyability.rbd import _compiled, engines, repairable_rbd +from repyability.rbd import _compiled, _event_loop, engines from repyability.tests.test_simulation_engines import ( identical, needs_numba, @@ -64,9 +64,7 @@ def runner( plan.entropy, plan.antithetic, ) - return repairable_rbd._PythonRunner( - rbd, tally, progress, context, jobs - ) + return _event_loop._PythonRunner(rbd, tally, progress, context, jobs) @pytest.fixture(autouse=True) diff --git a/repyability/tests/test_evaluation_costs.py b/repyability/tests/test_evaluation_costs.py index 2226baa8..1a0543df 100644 --- a/repyability/tests/test_evaluation_costs.py +++ b/repyability/tests/test_evaluation_costs.py @@ -30,9 +30,11 @@ from repyability import network as network_module from repyability.rbd import ( _ccf_chain, + _ccf_groups, _hidden_tests, + _intervals, + _long_run, _ordered_bdd, - repairable_rbd, ) from repyability.rbd._mean_lifetime import ( mean_lifetime, @@ -249,13 +251,13 @@ def unit(): def test_a_chain_s_kept_steps_give_what_its_series_gives(): rbd = two_out_of_three() group = rbd.ccf_groups[0] - times, _ = rbd._long_run_grid() - kept = rbd._group_states(group, times).probabilities + times, _ = _long_run._long_run_grid(rbd) + kept = _ccf_groups._group_states(rbd, group, times).probabilities # The same chain with each step's series summed for the vector alone. original = _ccf_chain._Hidden.evolve try: _ccf_chain._Hidden.evolve = _ccf_chain._Hidden._uniformized - again = rbd._group_states(group, times).probabilities + again = _ccf_groups._group_states(rbd, group, times).probabilities finally: _ccf_chain._Hidden.evolve = original np.testing.assert_allclose(kept, again, rtol=1e-11, atol=1e-15) @@ -263,12 +265,12 @@ def test_a_chain_s_kept_steps_give_what_its_series_gives(): def test_a_plan_keeps_its_groups_tables_and_its_copies_do_not(): rbd = two_out_of_three() - plan = rbd._with_intervals(inspection={"a": 4380.0}) + plan = _intervals._with_intervals(rbd, inspection={"a": 4380.0}) cost = plan.expected_cost_rate() - assert "_ccf_tables" in plan.__dict__ + assert plan._cache.ccf_tables is not None availability = plan.mean_availability() - other = plan._with_intervals(inspection={"b": 26280.0}) - assert "_ccf_tables" not in other.__dict__ + other = _intervals._with_intervals(plan, inspection={"b": 26280.0}) + assert other._cache.ccf_tables is None fresh = RepairableRBD(**{**rbd._init_args}).with_intervals({"a": 4380.0}) assert cost == pytest.approx(fresh.expected_cost_rate(), rel=1e-12) assert availability == pytest.approx(fresh.mean_availability(), rel=1e-12) @@ -294,10 +296,10 @@ def test_the_walk_s_convolutions_are_scipy_s(n, m): @pytest.fixture(params=["every combination", "local search"]) def search(request, monkeypatch): if request.param == "local search": - monkeypatch.setattr(repairable_rbd, "_MAX_COMBINATIONS", 1) + monkeypatch.setattr(_intervals, "_MAX_COMBINATIONS", 1) def choose(options, evaluate, **target): - return repairable_rbd._choose_from( + return _intervals._choose_from( list(options), options, evaluate, diff --git a/repyability/tests/test_exact_mttf.py b/repyability/tests/test_exact_mttf.py index fb33b553..6e3bbf08 100644 --- a/repyability/tests/test_exact_mttf.py +++ b/repyability/tests/test_exact_mttf.py @@ -157,9 +157,9 @@ def test_the_exact_mttf_does_not_touch_the_global_rng(): def test_simulation_options_without_simulate_are_refused(): rbd = series(E([0.01])) - with pytest.raises(TypeError, match="mc_samples, seed"): + with pytest.raises(ValueError, match="mc_samples, seed"): rbd.mean(1000, seed=1) - with pytest.raises(TypeError, match="tolerance"): + with pytest.raises(ValueError, match="tolerance"): rbd.mean_time_to_failure(tolerance=0.1) with warnings.catch_warnings(): warnings.simplefilter("error") diff --git a/repyability/tests/test_expected_window.py b/repyability/tests/test_expected_window.py index fd21d956..0702397b 100644 --- a/repyability/tests/test_expected_window.py +++ b/repyability/tests/test_expected_window.py @@ -17,7 +17,7 @@ from scipy.integrate import quad from repyability import ExpectedCost, ExpectedEvents, RepairableRBD -from repyability.rbd import routes +from repyability.rbd import _long_run, _spares, routes E = surv.Exponential.from_params W = surv.Weibull.from_params @@ -207,7 +207,9 @@ def test_a_replacement_due_at_the_window_s_end_falls_after_it(): jump = np.diff(events.node_preventive["c"])[0] survive = float(np.ravel(life.sf(500.0))[0]) assert jump == pytest.approx(survive**2, rel=1e-6) - simulated = rbd._simulated_replacements(1000.0, ["c"], 2000, 3)["c"] + simulated = _spares._simulated_replacements(rbd, 1000.0, ["c"], 2000, 3)[ + "c" + ] exact = events.node_failures["c"][0] + events.node_preventive["c"][0] assert simulated.mean() == pytest.approx(exact, abs=0.04) assert rbd.spares_demand(1000.0)["c"].mean == pytest.approx( @@ -294,13 +296,42 @@ def systems(): } +def test_a_test_a_rounding_short_of_the_period_s_end_is_counted_once(): + # A staggered pair's tests of 2 hours every 731.3: the period the + # counts are extended by ends at 248.642 + 5 * 731.3 + 731.3, and the + # test there, worked out as 248.642 + 6 * 731.3, falls a rounding + # short of it, in the period, and was counted again in the extension + # (a quarter more planned outages). The planned outages are the + # system's falls at the tests' instants, no failure falling there. + interval, offset = 731.3, 0.34 * 731.3 + + def tested(**more): + return { + "reliability": E([3e-4]), + "repairability": X([8.0]), + "inspection": {"interval": interval, "duration": X([2.0]), **more}, + } + + rbd = RepairableRBD(PARALLEL, {"a": tested(), "b": tested(offset=offset)}) + T = 20.5 * interval + tests = np.concatenate( + [interval * np.arange(1.0, 21.0), offset + interval * np.arange(21.0)] + ) + tests = np.sort(tests[tests < T]) + before = rbd.point_availability(np.nextafter(tests, -np.inf)) + falls = np.sum(before - rbd.point_availability(tests)) + events = rbd.expected_events(T) + assert events.system_planned_outages == pytest.approx(falls, rel=1e-6) + assert events.system_planned_outages < 3.4 + + @pytest.mark.parametrize("name", ["block replacement", "tested", "nested"]) def test_the_counts_settle_at_the_long_run_rates(name): rbd = systems()[name] t = np.array([1e6, 2e6]) cost = rbd.expected_cost(t) events = rbd.expected_events(t) - failures, planned = rbd._outage_frequencies() + failures, planned = _long_run._outage_frequencies(rbd) def slope(values): return np.diff(values)[0] / 1e6 @@ -375,12 +406,22 @@ def test_a_nested_rbd_counts_as_the_flat_one_does(): np.testing.assert_allclose( left.system_downtime, right.system_downtime, rtol=1e-7 ) - # The nested RBD's failures are its own system's; its replacements are - # not this RBD's to count. + # The nested RBD's failures are its own system's, and its preventive + # actions its own system's planned outages, as the simulation counts + # them for the node (its components' replacements are its own + # expected_events' to count). + own = inner.expected_events(t) + np.testing.assert_allclose( + left.node_failures["n"], own.system_failures, rtol=1e-6 + ) + np.testing.assert_allclose( + left.node_corrective["n"], own.system_failures, rtol=1e-6 + ) np.testing.assert_allclose( - left.node_failures["n"], inner.expected_failures(t), rtol=1e-12 + left.node_preventive["n"], own.system_planned_outages, rtol=1e-6 ) - assert not np.any(left.node_preventive["n"]) + assert np.all(left.node_preventive["n"] > 0.0) + assert not np.any(left.node_inspections["n"]) def test_forced_nodes_do_nothing(): @@ -436,7 +477,7 @@ def test_the_cost_result_adds_up(): assert dict(cost)["mean"] is cost.mean -def test_nothing_priced_costs_nothing_without_any_checks(): +def test_an_acquisition_cost_alone_costs_nothing_without_any_checks(): rbd = RepairableRBD( PARALLEL, { @@ -451,6 +492,15 @@ def test_nothing_priced_costs_nothing_without_any_checks(): assert cost.acquisition_cost == 100.0 np.testing.assert_array_equal(cost.total, [100.0, 100.0]) assert rbd.analysis_routes()["expected_cost"].route == routes.EXACT + # With nothing priced at all, there is no cost model: it refuses. + unpriced = RepairableRBD( + PARALLEL, {n: unit(0.01, 0.5) for n in "ab"}, repair_crews=1 + ) + with pytest.raises(ValueError, match="Nothing is priced"): + unpriced.expected_cost([10.0, 20.0]) + assert unpriced.analysis_routes()["expected_cost"].route == ( + routes.REFUSED + ) # The counts themselves come from the crews' chain (#146; see # test_chains_over_time.py). route = rbd.analysis_routes()["expected_events"] @@ -479,7 +529,7 @@ def test_the_routes_report_the_counts(): for name in ("expected_failures", "expected_events"): assert report[name].route == routes.NUMERICAL assert "Birnbaum/Vesely" in report[name].reason - assert report["expected_cost"].route == routes.EXACT + assert report["expected_cost"].route == routes.REFUSED # unpriced priced = RepairableRBD( PARALLEL, {n: unit(0.01, 0.5, repair_cost=1.0) for n in "ab"}, @@ -518,3 +568,58 @@ def test_the_routes_report_the_counts(): with pytest.raises(NotImplementedError) as error: slow.expected_failures(100.0) assert str(error.value) == route.reason + + +def test_a_nested_node_counts_as_the_simulation_counts_it(): + # A nested RBD maintained inside: the outer expected_events counts + # the node's failures and preventive actions as the simulation's + # node counts do (its system's failures and planned outages), where + # its preventive count was 0. + sub = RepairableRBD( + [("s", "a"), ("a", "t")], + { + "a": { + "reliability": W([10.0, 2.0]), + "repairability": L([1.5, 0.5]), + "preventive": { + "interval": 8.0, + "duration": E([2.0]), + "cost": 10.0, + }, + "repair_cost": 100.0, + } + }, + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") # its costs are not counted + top = RepairableRBD( + [("s", "sub"), ("sub", "C"), ("C", "t")], + { + "sub": sub, + "C": { + "reliability": W([50.0, 2.0]), + "repairability": L([1.5, 0.5]), + }, + }, + ) + T = 40.0 + events = top.expected_events(T) + assert ( + events.node_preventive["C"] == 0.0 + and events.node_preventive["sub"] > 1.0 + ) + timelines = top.simulate_timelines(T, mc_samples=3000, seed=3) + for name, counts in ( + ("node_failures", timelines.components["sub"].failures), + ("node_preventive", timelines.components["sub"].planned_outages), + ): + exact = getattr(events, name)["sub"] + error = np.std(counts) / np.sqrt(len(counts)) + assert abs(exact - np.mean(counts)) < 4 * error, (name, exact) + own = sub.expected_events(T) + assert events.node_preventive["sub"] == pytest.approx( + own.system_planned_outages, rel=1e-6 + ) + assert events.node_corrective["sub"] == pytest.approx( + own.system_failures, rel=1e-6 + ) diff --git a/repyability/tests/test_fault_tree.py b/repyability/tests/test_fault_tree.py index 2f0212ad..f20b6b6d 100644 --- a/repyability/tests/test_fault_tree.py +++ b/repyability/tests/test_fault_tree.py @@ -10,9 +10,11 @@ import itertools import math +import warnings import numpy as np import pytest +import scipy.stats as st import surpyval as surv from surpyval import FixedEventProbability @@ -720,19 +722,25 @@ def same_groups(a, b) -> bool: "mgl", "mgl-independent", "mgl-rate", + "alpha", + "alpha-rate", ] def group_model(kind, size): - from repyability import MGL, BetaFactor + from repyability import MGL, AlphaFactor, BetaFactor letters = (0.15, 0.4, 0.5)[: size - 1] + alphas = [0.85, 0.1, 0.03, 0.02][:size] + alphas[0] = 1.0 - sum(alphas[1:]) return { "beta": lambda: BetaFactor(0.1), "beta-rate": lambda: BetaFactor(0.2, basis="rate"), "mgl": lambda: MGL(*letters), "mgl-independent": lambda: MGL(*letters, shocks="independent"), "mgl-rate": lambda: MGL(*letters, basis="rate"), + "alpha": lambda: AlphaFactor(alphas, testing="non-staggered"), + "alpha-rate": lambda: AlphaFactor(alphas, basis="rate"), }[kind]() @@ -1003,3 +1011,694 @@ def test_a_diagrams_groups_become_the_trees(): tree = FaultTree.from_rbd(unaffected) assert tree.ccf_groups == [] and set(tree.events) == {"a"} assert tree.top_event_probability() == pytest.approx(unaffected.ff()) + + +# -- the lifetime analyses and the remaining measures (#314) ----------------- +# +# Each against the diagram of the same system (to_rbd), which works them out +# on its own structure, and against closed forms and the definitions where +# the tree is small enough to enumerate. + + +def lifetime_tree(seed, probability=False): + """A random tree over lifetime events (Weibull, exponential and + log-normal models, some shared between events) and, with + ``probability``, an event given as a probability.""" + rng = np.random.default_rng(seed) + gates, probabilities, top = random_tree( + rng, int(rng.integers(3, 6)), int(rng.integers(2, 4)) + ) + models = [ + W([float(rng.uniform(50, 200)), float(rng.uniform(1.2, 3.0))]), + surv.Exponential.from_params([float(rng.uniform(0.002, 0.02))]), + surv.LogNormal.from_params( + [float(rng.uniform(3.5, 5.0)), float(rng.uniform(0.3, 0.8))] + ), + ] + events = {e: models[i % 3] for i, e in enumerate(probabilities)} + if probability: + last = list(events)[-1] + events[last] = float(rng.uniform(0.02, 0.2)) + return FaultTree(gates, events, top=top) + + +def lifetime_trees(): + from repyability.tests.catalogue import fault_tree_kinds + + trees = {name: tree for name, tree in fault_tree_kinds().items()} + trees.update({f"random {seed}": lifetime_tree(seed) for seed in range(8)}) + trees.update( + {f"random {seed}, mixed": lifetime_tree(seed, True) for seed in (8, 9)} + ) + return trees + + +LIFETIME_TREES = lifetime_trees() + + +@pytest.mark.filterwarnings("ignore:Common-cause group:UserWarning") +@pytest.mark.parametrize("name", sorted(LIFETIME_TREES)) +def test_the_lifetime_analyses_agree_with_the_diagram(name): + tree = LIFETIME_TREES[name] + rbd = tree.to_rbd() + routes = tree.analysis_routes() + t = np.array([20.0, 60.0, 150.0]) + if tree.is_fixed: + t = np.array([1.0]) + # The probability's shape, density and hazard. + np.testing.assert_allclose(tree.Hf(t), rbd.Hf(t), rtol=1e-12) + np.testing.assert_allclose(tree.df(t), rbd.df(t), rtol=1e-8, atol=1e-18) + np.testing.assert_allclose(tree.hf(t), rbd.hf(t), rtol=1e-8, atol=1e-18) + assert isinstance(tree.df(float(t[0])), float) + # The measures from the structure and the probabilities. + assert tree.structural_importance() == pytest.approx( + rbd.structural_importance(), rel=1e-12 + ) + ours, theirs = tree.improvement_potential(t), rbd.improvement_potential(t) + for e in tree.events: + np.testing.assert_allclose(ours[e], theirs[e], rtol=1e-10, atol=1e-16) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", RuntimeWarning) + ours, theirs = tree.parameter_sensitivity( + t + ), rbd.parameter_sensitivity(t) + assert set(ours) == set(theirs) + for key, values in ours.items(): + assert list(values) == list(theirs[key]) + for name_, value in values.items(): + np.testing.assert_allclose( + value, theirs[key][name_], rtol=1e-7, atol=1e-12 + ) + if routes["mean"].route == "refused": + with pytest.raises((ValueError, NotImplementedError)): + tree.mean() + with pytest.raises((ValueError, NotImplementedError)): + rbd.mean() + return + # The lifetime: its mean, the B-lives (to reliabilities the tree + # reaches: an event given as a probability keeps some from the start, + # or for good), and who causes the top event. + assert tree.mean() == pytest.approx(rbd.mean(), rel=1e-8) + high, low = tree.sf(0.0), tree.sf(1e7) + targets = high - np.array([0.05, 0.1, 0.5]) * (high - low) + np.testing.assert_allclose( + tree.time_to_reliability(targets), + rbd.time_to_reliability(targets), + rtol=1e-9, + ) + percent = 100.0 * (1.0 - targets[1]) + assert tree.bx_life(percent) == pytest.approx( + rbd.bx_life(percent), rel=1e-9 + ) + ours, theirs = ( + tree.barlow_proschan_importance(), + rbd.barlow_proschan_importance(), + ) + assert set(ours) == set(theirs) + for key in ours: + assert ours[key] == pytest.approx(theirs[key], rel=1e-6, abs=1e-12) + ours, theirs = ( + tree.barlow_proschan_importance(t), + rbd.barlow_proschan_importance(t), + ) + for key in ours: + np.testing.assert_allclose( + ours[key], theirs[key], rtol=1e-6, atol=1e-12 + ) + + +def test_exponential_events_have_closed_forms(): + E = surv.Exponential.from_params + rates = {"a": 0.01, "b": 0.03, "c": 0.005} + either = FaultTree( + {"top": ("or", list(rates))}, {e: E([r]) for e, r in rates.items()} + ) + total = sum(rates.values()) + # Any of them: a failure rate of their sum. + assert either.mean() == pytest.approx(1.0 / total, rel=1e-10) + t = np.array([10.0, 50.0, 200.0]) + np.testing.assert_allclose(either.Hf(t), total * t, rtol=1e-12) + np.testing.assert_allclose(either.hf(t), total, rtol=1e-6) + np.testing.assert_allclose( + either.df(t), total * np.exp(-total * t), rtol=1e-6 + ) + for x in (1.0, 10.0, 50.0): + assert either.bx_life(x) == pytest.approx( + -np.log1p(-x / 100.0) / total, rel=1e-9 + ) + assert either.time_to_reliability(0.5) == pytest.approx( + np.log(2.0) / total, rel=1e-9 + ) + # Each causes the top event in proportion to its rate, at any time. + shares = either.barlow_proschan_importance() + for e, r in rates.items(): + assert shares[e] == pytest.approx(r / total, rel=1e-8) + np.testing.assert_allclose( + either.barlow_proschan_importance(t)[e], r / total, rtol=1e-8 + ) + # Both of two: the sum of the means less the mean of the first failure. + both = FaultTree( + {"top": ("and", ["a", "b"])}, {"a": E([0.01]), "b": E([0.03])} + ) + assert both.mean() == pytest.approx(100.0 + 100.0 / 3 - 25.0, rel=1e-10) + # The system's sensitivity to a rate: dR/d rate = -t exp(-total t) + # (the Birnbaum importance of an event under an OR gate is the + # others' survival). + sens = either.parameter_sensitivity(t) + for e in rates: + np.testing.assert_allclose( + sens[e]["failure_rate"], -t * np.exp(-total * t), rtol=1e-7 + ) + + +def pivotal_fraction(gates, events, top, e) -> float: + """The fraction of the other events' states in which ``e`` is + pivotal: the top event occurs with it and not without.""" + others = [x for x in events if x != e] + count = 0 + for state in itertools.product((False, True), repeat=len(others)): + occurred = frozenset(x for x, s in zip(others, state) if s) + count += occurs_by_hand(gates, top, occurred | {e}) and not ( + occurs_by_hand(gates, top, occurred) + ) + return count / 2 ** len(others) + + +@pytest.mark.parametrize("seed", range(25)) +def test_structural_importance_counts_the_pivotal_states(seed): + rng = np.random.default_rng(300 + seed) + gates, probabilities, top = random_tree(rng) + tree = FaultTree(gates, probabilities, top=top) + importance = tree.structural_importance() + for e in probabilities: + assert importance[e] == pytest.approx( + pivotal_fraction(gates, probabilities, top, e), abs=1e-12 + ) + # The gates alone: the same with any models, or common-cause groups. + from repyability import BetaFactor, CCFGroup + + first = list(probabilities)[:2] + timed = FaultTree( + gates, + {e: W([100.0 * (i + 1), 2.0]) for i, e in enumerate(probabilities)}, + top=top, + ) + assert timed.structural_importance() == importance + grouped = FaultTree( + gates, + {**probabilities, **{e: 0.05 for e in first}}, + top=top, + ccf_groups=[CCFGroup(first, BetaFactor(0.1))], + ) + assert grouped.structural_importance() == importance + + +@pytest.mark.parametrize("seed", range(25)) +def test_improvement_potential_is_the_fall_from_preventing_the_event(seed): + rng = np.random.default_rng(400 + seed) + gates, probabilities, top = random_tree(rng) + tree = FaultTree(gates, probabilities, top=top) + total, _, _ = enumerated(gates, probabilities, top) + potential = tree.improvement_potential() + birnbaum = tree.birnbaum_importance() + for e, q in probabilities.items(): + prevented, _, _ = enumerated(gates, probabilities, top, {e: False}) + assert potential[e] == pytest.approx(total - prevented, rel=1e-12) + assert potential[e] == pytest.approx(birnbaum[e] * q, rel=1e-12) + + +def test_improvement_potential_with_a_group_conditions_the_member(): + from repyability import BetaFactor + + # A member's potential takes the shared cause with it: preventing a + # pump's own failure leaves the cause, which the diagram's potential + # counts the same way. + tree = cooling_with_ccf(BetaFactor(0.1)) + rbd = tree.to_rbd() + ours, theirs = tree.improvement_potential(), rbd.improvement_potential() + for e in tree.events: + assert ours[e] == pytest.approx(theirs[e], rel=1e-12) + # It is the criticality's numerator, the Birnbaum importance times the + # member's marginal probability of having occurred. + criticality = tree.criticality_importance() + top = tree.top_event_probability() + for e in tree.events: + assert ours[e] == pytest.approx(criticality[e] * top, rel=1e-12) + + +def test_parameter_sensitivity_matches_differences_of_the_probability(): + from repyability import MGL, CCFGroup + + unit = W([100.0, 2.0]) + tree = FaultTree( + { + "top": ("or", ["vote", "d", "e"]), + "vote": ("vote", 2, ["a", "b", "c"]), + }, + { + "a": unit, + "b": unit, + "c": unit, + "d": surv.LogNormal.from_params([4.5, 0.5]), + "e": 0.02, + }, + ccf_groups=[CCFGroup(["a", "b", "c"], MGL(0.1, 0.3, basis="rate"))], + ) + t = np.array([30.0, 90.0]) + sens = tree.parameter_sensitivity(t) + assert list(sens) == [("a", "b", "c"), "d", "e"] + assert list(sens[("a", "b", "c")]) == [ + "alpha", + "beta", + "ccf_beta", + "ccf_gamma", + ] + assert list(sens["e"]) == ["p"] + h = 1e-6 + for lever in tree.levers(): + key, name, value = lever.key, lever.name, lever.value + up = tree.with_levers({lever: value * (1 + h)}).sf(t) + down = tree.with_levers({lever: value * (1 - h)}).sf(t) + np.testing.assert_allclose( + sens[key][name], + (up - down) / (2 * h * value), + rtol=1e-5, + atol=1e-10, + ) + # A probability's lever moves the top event by its Birnbaum importance. + np.testing.assert_allclose( + sens["e"]["p"], -tree.birnbaum_importance(t)["e"], rtol=1e-9 + ) + # The same as the diagram's, under the same keys. + theirs = tree.to_rbd().parameter_sensitivity(t) + for key, values in sens.items(): + for name, value in values.items(): + np.testing.assert_allclose(value, theirs[key][name], rtol=1e-7) + + +def test_bx_life_inverts_the_probability(): + tree = LIFETIME_TREES["lifetimes"] + for x in (1.0, 10.0, 63.2): + assert tree.sf(tree.bx_life(x)) == pytest.approx(1 - x / 100, abs=1e-9) + assert tree.ff(tree.bx_life(x)) == pytest.approx(x / 100, abs=1e-9) + times = tree.bx_life(np.array([[1.0, 10.0], [50.0, 90.0]])) + assert times.shape == (2, 2) + np.testing.assert_allclose( + tree.sf(times.ravel()), 1 - np.array([1.0, 10.0, 50.0, 90.0]) / 100 + ) + assert tree.sf(tree.time_to_reliability(0.75)) == pytest.approx(0.75) + assert tree.time_to_reliability(0.75, upper_bound=1e4) == pytest.approx( + tree.time_to_reliability(0.75), rel=1e-9 + ) + with pytest.raises(ValueError, match="percentage"): + tree.bx_life(0.0) + with pytest.raises(ValueError, match="in \\(0, 1\\)"): + tree.time_to_reliability(1.0) + with pytest.raises(ValueError, match="never reached"): + FaultTree(tree.gates, {**tree.events, "c": 0.5}).time_to_reliability( + 0.6 + ) + + +def test_a_tree_of_probabilities_has_no_lifetime(): + tree = LIFETIME_TREES["fixed"] + for call in ( + tree.mean, + lambda: tree.bx_life(10), + lambda: tree.time_to_reliability(0.5), + ): + with pytest.raises(ValueError, match="does not vary with time"): + call() + with pytest.raises(ValueError, match="does not vary with time"): + tree.mean_uncertainty({"a": {"p": st.uniform(0.05, 0.1)}}, n_draws=3) + # Its density is 0, its hazard too, and nothing causes the top event + # over time. + assert tree.df() == 0.0 + assert tree.hf() == 0.0 + assert tree.Hf() == pytest.approx(-np.log(tree.sf())) + with pytest.raises(ValueError, match="no lifetimes"): + tree.barlow_proschan_importance() + # Its sensitivities are to the probabilities, each the Birnbaum + # importance, negated: the levers move the probabilities themselves. + sens = tree.parameter_sensitivity() + birnbaum = tree.birnbaum_importance() + for e in tree.events: + assert sens[e] == {"p": pytest.approx(-birnbaum[e], rel=1e-9)} + moved = tree.with_levers({("a", "p"): 0.3}) + assert moved.events == {**tree.events, "a": 0.3} + assert tree.events["a"] == 0.1 + + +@pytest.mark.filterwarnings("ignore:Common-cause group:UserWarning") +def test_a_group_splitting_a_probability_has_no_mean(): + tree = LIFETIME_TREES["common cause"] + message = "split a failure probability" + with pytest.raises(NotImplementedError, match=message): + tree.mean() + with pytest.raises(NotImplementedError, match=message): + tree.mean_uncertainty( + {"c": {"failure_rate": st.uniform(0.001, 0.002)}} + ) + # The B-lives, density and hazard hold at a time, as the diagram's. + rbd = tree.to_rbd() + assert tree.bx_life(10) == pytest.approx(rbd.bx_life(10), rel=1e-9) + assert tree.hf(30.0) == pytest.approx(rbd.hf(30.0), rel=1e-8) + by_rate = LIFETIME_TREES["common cause by rate"] + assert by_rate.mean() == pytest.approx(by_rate.to_rbd().mean(), rel=1e-8) + + +def test_the_uncertainty_draws_are_the_events_models(): + from repyability import BetaFactor, CCFGroup + + tree = LIFETIME_TREES["lifetimes"] + t = np.array([20.0, 60.0]) + plausible = [W([80.0, 2.0]), W([100.0, 2.0]), W([120.0, 2.0])] + result = tree.sf_uncertainty(t, {"a": plausible}, n_draws=12, seed=4) + assert result.samples.shape == (12, 2) + np.testing.assert_allclose(result.nominal, tree.sf(t), rtol=1e-15) + # Each draw's probability is that of the tree with the drawn model. + values = [ + FaultTree(tree.gates, {**tree.events, "a": model}).sf(t) + for model in plausible + ] + for row in result.samples: + assert any(np.allclose(row, value, rtol=1e-14) for value in values) + assert len({tuple(row) for row in result.samples}) == 3 + # The same draws as the diagram's with the seed, keyed by the events. + theirs = tree.to_rbd().sf_uncertainty( + t, {"a": plausible}, n_draws=12, seed=4 + ) + np.testing.assert_array_equal(result.samples, theirs.samples) + # The mean time too, each draw's the exact mean with the drawn model. + means = tree.mean_uncertainty({"a": plausible}, n_draws=6, seed=4) + assert means.nominal == pytest.approx(tree.mean(), rel=1e-10) + exact = [ + FaultTree(tree.gates, {**tree.events, "a": model}).mean() + for model in plausible + ] + for sample in means.samples: + assert any(sample == pytest.approx(value, rel=1e-8) for value in exact) + # A fitted event is drawn from its fit by default, a probability is + # not; events of one population share their draws. + fitted = LIFETIME_TREES["fitted"] + result = fitted.sf_uncertainty(t, n_draws=5, seed=1) + assert result.samples.shape == (5, 2) + assert np.all(result.samples != fitted.sf(t)) + with pytest.raises(ValueError, match="No node's model is a surpyval fit"): + LIFETIME_TREES["fixed"].sf_uncertainty(n_draws=3) + alike = FaultTree(tree.gates, {**tree.events, "b": tree.events["a"]}) + shared = alike.sf_uncertainty( + t, {("a", "b"): {"beta": st.uniform(1.5, 1.0)}}, n_draws=4, seed=2 + ) + assert shared.samples.shape == (4, 2) + with pytest.raises(ValueError, match="must have the same model"): + tree.sf_uncertainty( + t, {("a", "b"): {"beta": st.uniform(1.5, 1.0)}}, n_draws=4 + ) + # A group's members together, and its model drawn too. + grouped = FaultTree( + tree.gates, + {"a": W([100, 2]), "b": W([100, 2]), "c": tree.events["c"]}, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1, basis="rate"))], + ) + group = grouped.ccf_groups[0] + drawn = grouped.sf_uncertainty( + t, + { + ("a", "b"): {"alpha": st.uniform(80, 40)}, + group: {"beta": st.uniform(0.05, 0.1)}, + }, + n_draws=5, + seed=3, + ) + theirs = grouped.to_rbd().sf_uncertainty( + t, + { + ("a", "b"): {"alpha": st.uniform(80, 40)}, + group: {"beta": st.uniform(0.05, 0.1)}, + }, + n_draws=5, + seed=3, + ) + np.testing.assert_array_equal(drawn.samples, theirs.samples) + with pytest.raises(ValueError, match="common-cause group"): + grouped.sf_uncertainty( + t, {"a": {"alpha": st.uniform(80, 40)}}, n_draws=2 + ) + + +@pytest.mark.parametrize("name", sorted(LIFETIME_TREES)) +def test_the_levers_are_those_the_sensitivities_report(name): + from repyability import Lever + + tree = LIFETIME_TREES[name] + levers = tree.levers() + assert all(isinstance(lever, Lever) for lever in levers) + assert all(not lever.discrete and not lever.calendar for lever in levers) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", RuntimeWarning) + reported = tree.parameter_sensitivity(30.0) + assert [(lever.key, lever.name) for lever in levers] == [ + (key, name_) for key, names in reported.items() for name_ in names + ] + for lever in levers: + low, high = lever.bounds + assert low <= lever.value <= high + # Each lever moved builds a tree with it moved, and the rest as they + # were; a value past the range is refused. + for lever in levers: + value = lever.value * 1.5 if lever.value else 0.5 + if value > lever.bounds[1]: + value = 0.5 * (lever.value + lever.bounds[1]) + moved = tree.with_levers({lever: value}) + assert isinstance(moved, FaultTree) + assert moved.gates == tree.gates and moved.top == tree.top + values = {(x.key, x.name): x.value for x in moved.levers()} + before = {(x.key, x.name): x.value for x in levers} + assert values.pop((lever.key, lever.name)) == pytest.approx(value) + before.pop((lever.key, lever.name)) + assert values == before + copy = tree.with_levers({}) + assert copy is not tree and copy.to_dict() == tree.to_dict() + + +def test_with_levers_keeps_what_each_event_is(): + from repyability import BetaFactor, CCFGroup + + tree = FaultTree( + {"top": ("or", ["pair", "c", "d"]), "pair": ("and", ["a", "b"])}, + { + "a": W([100, 2]), + "b": W([100, 2]), + "c": surv.Exponential.from_params([0.01]), + "d": 0.05, + }, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1, basis="rate"))], + ) + moved = tree.with_levers( + { + (("a", "b"), "alpha"): 150.0, + (("a", "b"), "ccf_beta"): 0.2, + ("d", "p"): 0.1, + } + ) + # The members' one model for every member, the group's own beta. + assert moved.events["a"].params[0] == 150.0 + assert moved.events["b"].params[0] == 150.0 + assert moved.events["a"] is moved.events["b"] + assert moved.ccf_groups[0].model.beta == 0.2 + assert moved.ccf_groups[0].model.basis == "rate" + assert moved.events["c"] is tree.events["c"] + # A probability stays one. + assert moved.events["d"] == 0.1 and isinstance(moved.events["d"], float) + assert FaultTree.from_json(moved.to_json()).events["d"] == 0.1 + assert tree.events["d"] == 0.05 and tree.ccf_groups[0].model.beta == 0.1 + with pytest.raises(ValueError, match="cannot be"): + tree.with_levers({(("a", "b"), "ccf_beta"): 1.5}) + with pytest.raises(ValueError, match="no lever 'gamma'"): + tree.with_levers({("c", "gamma"): 1.0}) + with pytest.raises(ValueError, match="has no levers"): + tree.with_levers({("pair", "alpha"): 1.0}) + with pytest.raises(TypeError, match="number"): + tree.with_levers({("d", "p"): "0.1"}) + + +# -- the 0.14 persona review -------------------------------------------------- + +BRIDGE_EDGES = [ + ("s", "a"), + ("s", "b"), + ("a", "c"), + ("b", "c"), + ("c", "d"), + ("c", "e"), + ("a", "d"), + ("b", "e"), + ("d", "t"), + ("e", "t"), +] + + +def bridge_models(): + return { + "a": W([100, 2]), + "b": W([120, 1.5]), + "c": W([80, 3]), + "d": W([150, 1.2]), + "e": W([90, 2.5]), + } + + +def same(found: dict, expected: dict, rel=1e-9) -> None: + assert set(found) == set(expected) + for key, value in expected.items(): + assert found[key] == pytest.approx(value, rel=rel, nan_ok=True), key + + +def test_kind_and_fv_type_agree_with_the_diagram(): + from repyability import BetaFactor, CCFGroup + + diagram = NonRepairableRBD(BRIDGE_EDGES, bridge_models()) + tree = FaultTree.from_rbd(diagram) + for kind in ("failure", "success"): + same( + tree.criticality_importance(50.0, kind=kind), + diagram.criticality_importance(50.0, kind=kind), + ) + same( + tree.differential_importance( + 50.0, change="proportional", kind=kind + ), + diagram.differential_importance( + 50.0, change="proportional", kind=kind + ), + ) + for fv_type in ("c", "p"): + for method in ("exact", "rare_event"): + same( + tree.fussell_vesely(50.0, fv_type=fv_type, method=method), + diagram.fussell_vesely(50.0, fv_type=fv_type, method=method), + ) + # A series system has one path set, every event: each event's path-set + # share is the probability that both have occurred, over the top + # event's. + chain = FaultTree( + {"top": ("or", ["a", "b"])}, {"a": W([100, 2]), "b": W([80, 1.5])} + ) + both = float(W([100, 2]).ff(30.0) * W([80, 1.5]).ff(30.0)) + assert chain.fussell_vesely(30.0, fv_type="p") == pytest.approx( + {"a": both / chain.ff(30.0), "b": both / chain.ff(30.0)} + ) + # The success-oriented criticality is 1 for every event in series. + assert chain.criticality_importance(30.0, kind="success") == ( + pytest.approx({"a": 1.0, "b": 1.0}) + ) + # With a common-cause group, as the diagram conditions on it. + grouped = NonRepairableRBD( + [ + ("s", "a"), + ("s", "b"), + ("s", "c"), + ("a", "j"), + ("b", "j"), + ("c", "j"), + ("j", "t"), + ], + {n: W([100, 2]) for n in "abc"} | {"j": PerfectReliability}, + k={"j": 2}, + ccf_groups=[CCFGroup(["a", "b", "c"], BetaFactor(0.1, basis="rate"))], + ) + grouped_tree = FaultTree.from_rbd(grouped) + for method in ("exact", "rare_event"): + same( + grouped_tree.fussell_vesely(50.0, fv_type="p", method=method), + grouped.fussell_vesely(50.0, fv_type="p", method=method), + ) + same( + grouped_tree.criticality_importance(50.0, kind="success"), + grouped.criticality_importance(50.0, kind="success"), + ) + with pytest.raises(ValueError, match="kind must be"): + tree.criticality_importance(50.0, kind="bogus") + with pytest.raises(ValueError, match="kind must be"): + tree.differential_importance(50.0, kind="bogus") + with pytest.raises(ValueError, match="fv_type must be"): + tree.fussell_vesely(50.0, fv_type="bogus") + + +def test_mean_by_simulation_and_random_are_the_diagram_s(): + tree = FaultTree( + {"top": ("or", ["a", "g"]), "g": ("and", ["b", "c"])}, + {"a": W([100, 2]), "b": W([80, 1.5]), "c": W([120, 2.5])}, + ) + diagram = tree.to_rbd() + assert np.array_equal(tree.random(50, seed=3), diagram.random(50, seed=3)) + estimate = tree.mean(method="simulate", mc_samples=4000, seed=1) + assert estimate == diagram.mean(method="simulate", mc_samples=4000, seed=1) + lifetimes = tree.random(4000, seed=1) + assert estimate == pytest.approx(lifetimes.mean()) + assert abs(estimate - tree.mean()) < 3 * lifetimes.std() / np.sqrt(4000) + for call in ( + lambda: tree.mean(mc_samples=5), + lambda: tree.mean(seed=1), + ): + with pytest.raises(ValueError, match="only with method='simulate'"): + call() + with pytest.raises(ValueError, match="'exact' or 'simulate'"): + tree.mean(method="guess") + with pytest.raises(ValueError, match="size must be a positive"): + tree.random(0) + report = tree.analysis_routes() + assert report["random"].route == "simulated" + assert "method='simulate'" in report["mean"].reason + + +def test_from_rbd_warns_of_capacities_left_out(): + capped = NonRepairableRBD( + [("s", "p1"), ("s", "p2"), ("p1", "t"), ("p2", "t")], + {"p1": W([100, 2]), "p2": W([120, 1.5])}, + capacity={"p1": 60, "p2": 40}, + ) + with pytest.warns(UserWarning, match="capacities are left out") as caught: + tree = FaultTree.from_rbd(capped) + assert caught[0].filename == __file__ + assert tree.sf(50.0) == pytest.approx(capped.sf(50.0)) + with warnings.catch_warnings(): + warnings.simplefilter("error") + FaultTree.from_rbd(NonRepairableRBD(BRIDGE_EDGES, bridge_models())) + + +def test_times_of_any_shape(): + tree = FaultTree.from_rbd(NonRepairableRBD(BRIDGE_EDGES, bridge_models())) + grid = np.array([[10.0, 20.0, 30.0], [40.0, 50.0, 60.0]]) + flat = grid.ravel() + for name in ("sf", "ff", "top_event_probability", "Hf", "df", "hf"): + assert np.array_equal( + getattr(tree, name)(grid), getattr(tree, name)(flat).reshape(2, 3) + ) + for name in ("birnbaum_importance", "fussell_vesely", "joint_importance"): + shaped, rows = getattr(tree, name)(grid), getattr(tree, name)(flat) + for key in shaped: + assert np.array_equal(shaped[key], rows[key].reshape(2, 3)), name + assert isinstance(tree.sf(np.array(10.0)), float) + assert tree.sf(np.array(10.0)) == tree.sf(10.0) + with pytest.raises(ValueError, match="this fault tree is time-varying"): + tree.sf() + with pytest.raises(TypeError, match="x must be numbers, got '10'"): + tree.bx_life("10") + + +def test_every_event_a_probability_has_no_failures_over_time(): + tree = FaultTree({"top": ("and", ["a", "b"])}, {"a": 0.1, "b": 0.2}) + with pytest.raises(ValueError, match="no lifetimes") as error: + tree.barlow_proschan_importance() + report = tree.analysis_routes()["barlow_proschan_importance"] + assert report.route == "refused" and report.reason == str(error.value) + # One event with a life keeps the measure: the probability has no part. + mixed = FaultTree( + {"top": ("and", ["a", "b"])}, {"a": 0.1, "b": W([100, 2])} + ) + assert mixed.barlow_proschan_importance() == {"a": 0.0, "b": 1.0} + assert mixed.analysis_routes()["barlow_proschan_importance"].route == ( + "numerical" + ) diff --git a/repyability/tests/test_fitted_processes.py b/repyability/tests/test_fitted_processes.py new file mode 100644 index 00000000..b938fa0d --- /dev/null +++ b/repyability/tests/test_fitted_processes.py @@ -0,0 +1,236 @@ +"""surpyval's fitted repairable-unit models as components (#269): a +Poisson process (CrowAMSAA, Duane, HPP) is minimal repair of the life +whose cumulative hazard is its cumulative intensity, and a generalised +renewal process (Kijima I or II) its life repaired by its Kijima model and +restoration factor.""" + +import numpy as np +import pytest +import surpyval as surv + +from repyability import NodeState, RepairableRBD + +EDGES = [("s", "a"), ("a", "t")] +TIMES = np.cumsum(np.random.default_rng(0).weibull(1.5, 60) * 10) + + +def one(life, **more): + return RepairableRBD( + EDGES, {"a": {"reliability": life, "repairability": "instant", **more}} + ) + + +@pytest.mark.parametrize("fitter", [surv.CrowAMSAA, surv.Duane, surv.HPP]) +def test_a_poisson_process_has_its_intensity(fitter): + # Minimal repair in no time: the expected number of failures by T is + # the process's cumulative intensity, worked out exactly. + process = fitter.fit(TIMES) + rbd = one(process) + life = rbd._init_args["components"]["a"]["reliability"] + t = np.array([1.0, 10.0, 100.0, 500.0]) + np.testing.assert_allclose(life.Hf(t), process.cif(t), rtol=1e-12) + for horizon in (100.0, 500.0): + assert rbd.expected_failures(horizon) == pytest.approx( + float(process.cif(horizon)), rel=1e-9 + ) + + +@pytest.mark.parametrize("kijima", ["i", "ii"]) +def test_a_generalised_renewal_process_is_its_life_and_kijima_repair(kijima): + process = surv.GeneralizedRenewal.fit( + TIMES, dist=surv.Weibull, kijima=kijima + ) + repair = surv.Exponential.from_params([1.0]) + given = RepairableRBD( + EDGES, {"a": {"reliability": process, "repairability": repair}} + ) + spec = given._init_args["components"]["a"] + assert spec["reliability"] is process.model + assert spec["repair"] == { + "model": f"kijima{len(kijima)}", + "q": float(process.restoration), + } + # The same as the spec written out, simulation for simulation. + written = RepairableRBD( + EDGES, + { + "a": { + "reliability": process.model, + "repairability": repair, + "repair": spec["repair"], + } + }, + ) + runs = [ + rbd.availability(t_simulation=300.0, mc_samples=200, seed=3) + for rbd in (given, written) + ] + assert runs[0].mean_availability == runs[1].mean_availability + assert runs[0].system_failures == runs[1].system_failures + + +@pytest.mark.parametrize( + "fitter", [surv.GeneralizedOneRenewal, surv.ARA], ids=["G1", "ARA"] +) +def test_renewal_models_that_are_not_kijimas_are_refused(fitter): + process = fitter.fit(TIMES, dist=surv.Weibull) + with pytest.raises(ValueError, match="not Kijima's.*SurPyval#833"): + one(process) + + +def test_a_process_brings_its_own_repair(): + with pytest.raises(ValueError, match="brings its own repair"): + one(surv.CrowAMSAA.fit(TIMES), repair={"model": "kijima1", "q": 0.5}) + + +def test_a_process_is_saved_as_the_life_and_repair_it_is(): + rbd = one(surv.CrowAMSAA.fit(TIMES)) + back = RepairableRBD.from_dict(rbd.to_dict()) + assert back.expected_failures(200.0) == pytest.approx( + rbd.expected_failures(200.0), rel=1e-12 + ) + + +# -- starting from a virtual age (#269) -------------------------------------- + +LIFE = surv.Weibull.from_params([100.0, 2.5]) + + +def H(t): + return (np.asarray(t) / 100.0) ** 2.5 + + +def kijima(model, q, **more): + return RepairableRBD( + EDGES, + { + "a": { + "reliability": LIFE, + "repairability": "instant", + "repair": {"model": model, "q": q}, + **more, + } + }, + ) + + +def reference_failures(model, q, virtual, age, horizon, n, seed): + """The mean failures by ``horizon`` of a unit at virtual age + ``virtual`` at its last repair and ``age`` since, repaired in no time, + written out from Kijima's definition: each life ``X`` from virtual age + ``v`` has ``H(v + X) = H(v) + E``; a repair after operating ``x`` + since the last takes ``v`` to ``v + q x`` (I) or ``q (v + x)`` (II).""" + rng = np.random.default_rng(seed) + total = 0 + for _ in range(n): + v, since, t = virtual, age, 0.0 + while True: + now = v + since + left = 100.0 * (H(now) + rng.exponential()) ** (1 / 2.5) - now + t += left + if t > horizon: + break + total += 1 + x = since + left + v = v + q * x if model == "kijima1" else q * (v + x) + since = 0.0 + return total / n + + +@pytest.mark.parametrize("model", ["kijima1", "kijima2"]) +@pytest.mark.parametrize( + "virtual, age", [(0.0, 60.0), (150.0, 0.0), (90.0, 40.0)] +) +def test_a_unit_starts_from_its_virtual_age(model, virtual, age): + n = 20_000 + run = kijima(model, 0.5).availability( + t_simulation=200.0, + mc_samples=n, + seed=5, + state={"a": NodeState(age=age, virtual_age=virtual or None)}, + ) + expected = reference_failures(model, 0.5, virtual, age, 200.0, n, 6) + # Two estimates of a mean of about 3-6 failures (sd about 1.5), each + # from 20,000 runs. + assert run.system_failures / n == pytest.approx(expected, abs=0.06) + + +def test_minimal_repair_from_a_virtual_age_is_exact(): + # q = 1: the failures by T are H(v + a + T) - H(v + a). + n = 20_000 + run = kijima("kijima1", 1.0).availability( + t_simulation=50.0, + mc_samples=n, + seed=7, + state={"a": NodeState(age=30.0, virtual_age=120.0)}, + ) + exact = float(H(200.0) - H(150.0)) + assert run.system_failures / n == pytest.approx(exact, abs=0.05) + + +def test_a_unit_down_returns_at_its_virtual_age(): + # Down, its repair all but over (a mean of 1e-6): then as a unit up at + # its virtual age, age 0. + rbd = RepairableRBD( + EDGES, + { + "a": { + "reliability": LIFE, + "repairability": surv.Exponential.from_params([1e6]), + "repair": {"model": "kijima1", "q": 1.0}, + } + }, + ) + n = 20_000 + run = rbd.availability( + t_simulation=50.0, + mc_samples=n, + seed=8, + state={"a": NodeState(alive=False, virtual_age=150.0)}, + ) + assert run.system_failures / n == pytest.approx( + float(H(200.0) - H(150.0)), abs=0.05 + ) + + +def test_a_fitted_units_state_from_unit_states(): + # surpyval's unit_states gives the virtual age now and the time since + # the last failure: the state is age=since_failure, + # virtual_age=virtual_age - since_failure. + # The fitted unit, at the end of its history: its last failure just + # repaired, at a virtual age above 0 (Kijima II, q above 0). + process = surv.GeneralizedRenewal.fit( + TIMES, dist=surv.Weibull, kijima="ii" + ) + now = process.unit_states().iloc[0] + since = float(now["since_failure"]) + state = NodeState(age=since, virtual_age=float(now["virtual_age"]) - since) + assert state.virtual_age > 0.0 + rbd = one(process) + started, new = ( + rbd.availability( + t_simulation=100.0, mc_samples=4000, seed=1, state=given + ) + for given in ({"a": state}, None) + ) + # Older than new, it fails more often (its life is a wearing Weibull). + assert started.system_failures > new.system_failures + + +def test_a_virtual_age_needs_imperfect_repair_and_a_simulation(): + with pytest.raises(ValueError, match="no virtual age"): + one(LIFE).availability( + t_simulation=10.0, + mc_samples=2, + state={"a": NodeState(age=5.0, virtual_age=3.0)}, + ) + with pytest.raises(NotImplementedError, match="taken by the simulations"): + kijima("kijima1", 1.0).point_availability( + 10.0, state={"a": NodeState(age=5.0, virtual_age=3.0)} + ) + with pytest.raises(NotImplementedError, match="since it was renewed"): + kijima("kijima1", 0.5, replace_after=3).availability( + t_simulation=10.0, + mc_samples=2, + state={"a": NodeState(age=5.0, virtual_age=3.0)}, + ) diff --git a/repyability/tests/test_fussell_vesely_exact.py b/repyability/tests/test_fussell_vesely_exact.py index 1211dc86..ca313d32 100644 --- a/repyability/tests/test_fussell_vesely_exact.py +++ b/repyability/tests/test_fussell_vesely_exact.py @@ -12,7 +12,7 @@ from surpyval import Exponential, FixedEventProbability, Weibull from repyability import FaultTree, NonRepairableRBD, RepairableRBD -from repyability.rbd import modular +from repyability.rbd import _repairable_importance, modular from repyability.rbd.helper_classes import PerfectReliability F = FixedEventProbability.from_params @@ -299,7 +299,9 @@ def test_a_repairable_diagram_averages_the_union_over_its_inspections(): **{n: repairable(0.2) for n in "bcde"}, }, ) - p, q, weights, _ = rbd._importance_probabilities(None, None) + p, q, weights, _ = _repairable_importance._importance_probabilities( + rbd, None, None + ) size = len(weights) cuts = rbd.get_min_cut_sets() nodes = list("abcde") diff --git a/repyability/tests/test_help.py b/repyability/tests/test_help.py index 1fda1893..d8f09170 100644 --- a/repyability/tests/test_help.py +++ b/repyability/tests/test_help.py @@ -26,7 +26,10 @@ def test_a_cross_reference_reads_as_its_name(): text = f"see [`point_availability`][{target}]." assert readable(text) == "see ``point_availability``." assert readable(None) is None - assert readable("a [link](guide/greeks.md)") == "a [link](guide/greeks.md)" + assert ( + readable("a [link](../guide/greeks.md)") + == "a [link](../guide/greeks.md)" + ) @pytest.mark.parametrize("cls", [NonRepairableRBD, RepairableRBD, FaultTree]) diff --git a/repyability/tests/test_hidden_failures_any_life.py b/repyability/tests/test_hidden_failures_any_life.py index cc2e3766..466d38d3 100644 --- a/repyability/tests/test_hidden_failures_any_life.py +++ b/repyability/tests/test_hidden_failures_any_life.py @@ -13,7 +13,7 @@ from surpyval import Exponential, Gamma, LogNormal, Weibull from repyability import NodeState, RepairableRBD -from repyability.rbd import _hidden_life, routes +from repyability.rbd import _hidden_life, _intervals, routes from repyability.rbd._point_availability import InspectionCurve E = Exponential.from_params @@ -184,7 +184,7 @@ def test_the_long_run_profile_by_its_definition(name): atol=1e-15, ) # Its last test 0.3 intervals before 0. - later = {"a": NodeState(stationary=True, phase=0.3 * interval)} + later = {"a": NodeState(stationary=True, since_scheduled=0.3 * interval)} np.testing.assert_allclose( rbd.point_availability(u, state=later), profile(model, interval, np.mod(u + 0.3 * interval, interval)), @@ -330,8 +330,8 @@ def cost_rate(interval): assert plan.intervals["a"] == pytest.approx(best, rel=2e-3) assert plan.cost_rate == pytest.approx(cost_rate(best), rel=1e-6) assert plan.cost_rate == pytest.approx( - rbd._with_intervals( - inspection={"a": plan.intervals["a"]} + _intervals._with_intervals( + rbd, inspection={"a": plan.intervals["a"]} ).expected_cost_rate(), rel=1e-12, ) @@ -402,7 +402,7 @@ def test_from_a_state(age, phase): # or, younger than that, put into service since, and not yet tested. model, interval = W([200.0, 2.5]), 50.0 rbd = RepairableRBD(SINGLE, {"a": hidden(model, interval)}) - state = {"a": NodeState(age=age, phase=phase)} + state = {"a": NodeState(age=age, since_scheduled=phase)} known = max(age - phase, 0.0) first = interval - phase if phase else interval reference = Units( diff --git a/repyability/tests/test_hidden_failures_timed.py b/repyability/tests/test_hidden_failures_timed.py index a0972380..5ecc3b5a 100644 --- a/repyability/tests/test_hidden_failures_timed.py +++ b/repyability/tests/test_hidden_failures_timed.py @@ -20,7 +20,7 @@ ) from repyability.rbd import _hidden_life from repyability.rbd import _hidden_tests as ht -from repyability.rbd import routes +from repyability.rbd import _long_run, routes from repyability.rbd._point_availability import ( InspectionCurve, PartialTestCurve, @@ -267,11 +267,11 @@ def unit(): ), "in a repair": ( hidden(W([4.0, 2.0]), E([5.0]), X([0.05])), - NodeState(alive=False, down_for=0.1, phase=0.6), + NodeState(alive=False, down_for=0.1, since_scheduled=0.6), ), "up at an age": ( hidden(W([4.0, 2.0]), E([5.0]), X([0.05])), - NodeState(age=2.0, phase=0.4), + NodeState(age=2.0, since_scheduled=0.4), ), "exponential, missing": ( hidden( @@ -362,9 +362,9 @@ def test_tests_at_once_take_the_function_off_line(): assert both.mean_unavailability() == pytest.approx(4.28047e-4, rel=1e-5) assert apart.mean_unavailability() == pytest.approx(7.11935e-5, rel=1e-5) # Each test of both takes the function down, unless one is failed. - planned = both._outage_frequencies()[1] * 8760.0 + planned = _long_run._outage_frequencies(both)[1] * 8760.0 assert planned == pytest.approx(0.9997, abs=1e-4) - assert apart._outage_frequencies()[1] * 8760.0 < 0.02 + assert _long_run._outage_frequencies(apart)[1] * 8760.0 < 0.02 def test_a_system_s_long_run_is_its_settled_values(): @@ -384,7 +384,7 @@ def test_a_system_s_long_run_is_its_settled_values(): assert availability + rbd.mean_unavailability() == pytest.approx( 1.0, abs=1e-12 ) - failures, planned = rbd._outage_frequencies() + failures, planned = _long_run._outage_frequencies(rbd) ends = np.array([60.0, 80.0]) mission = np.ravel(rbd.mission_availability(ends)) late = (mission[1] * ends[1] - mission[0] * ends[0]) / 20.0 diff --git a/repyability/tests/test_importance_over_time.py b/repyability/tests/test_importance_over_time.py index 2e92ad62..bb58537e 100644 --- a/repyability/tests/test_importance_over_time.py +++ b/repyability/tests/test_importance_over_time.py @@ -20,7 +20,14 @@ import scipy.linalg import surpyval as surv -from repyability import BetaFactor, CCFGroup, NodeState, RepairableRBD +from repyability import ( + BetaFactor, + CCFGroup, + NodeState, + PerfectReliability, + RepairableRBD, +) +from repyability.rbd import _ccf_groups, _crews, _curves from repyability.rbd.rbd import RBD E, W = surv.Exponential.from_params, surv.Weibull.from_params @@ -141,7 +148,7 @@ def enumerated(rbd, availability: dict): def test_criticality_and_fussell_vesely_by_enumeration(): rbd = bridge() t = 40.0 - curves = rbd._availability_curves(t, set()) + curves = _curves._availability_curves(rbd, t, set()) availability = { n: float(c.at(np.array([t]))[0]) for n, c in curves.items() } @@ -264,9 +271,9 @@ def test_held_nodes_over_time(): def test_with_crews_the_chain_against_its_matrix_exponential(): rbd = pair_then_c(repair_crews=1) times = np.array([0.5, 4.0, 40.0]) - chain = rbd._crew_chain(frozenset()) - start = rbd._crew_start(chain, {}) - p, _ = rbd._chain_probabilities(set(), set()) + chain = _crews._crew_chain(rbd, frozenset()) + start = _curves._crew_start(rbd, chain, {}) + p, _ = _crews._chain_probabilities(rbd, set(), set()) q = {n: 1.0 - v for n, v in p.items()} generator = np.asarray( chain.generator.toarray() @@ -313,10 +320,10 @@ def test_with_common_causes_a_member_is_conditioned_on_its_state(): ) # A member's: the system given it up less given it down, over the # group's joint states at each time (a with b, through the cause). - groups = rbd._groups_over_time() + groups = _ccf_groups._groups_over_time(rbd) (group,) = groups chances = group.probabilities(times) - c_up = rbd._groups_curve(200.0).curves["c"].at(times) + c_up = _ccf_groups._groups_curve(rbd, 200.0).curves["c"].at(times) position = list(group.members).index("a") a_up = ~group.down[:, position] b_up = ~group.down[:, 1 - position] @@ -351,13 +358,199 @@ def test_what_it_refuses(): kijima = RepairableRBD([("s", "a"), ("a", "t")], {"a": imperfect}) with pytest.raises(NotImplementedError): kijima.birnbaum_importance(x=1.0) - # Crews around a nested RBD: the chain's states take no nested curve. - inner = RepairableRBD([("s", "u"), ("u", "t")], {"u": unit()}) - crewed = RepairableRBD( - [("s", "a"), ("s", "b"), ("a", "n"), ("b", "n"), ("n", "t")], - {"a": unit(), "b": unit(), "n": inner}, + + +# -- limited repair crews around a nested RBD (#312) ------------------------- + + +def nested_in_series(): + """x, a nested RBD with a crew of its own, in series with y and z in + parallel, sharing one crew: the outer system, the pair alone with the + crew, and the nested RBD. The nested RBD is independent of the pair's + chain, so each measure has a closed form from the two halves' + curves.""" + inner = RepairableRBD( + [("s", "u"), ("s", "v"), ("u", "t"), ("v", "t")], + {"u": unit(E([0.05]), E([0.2])), "v": unit(E([0.08]), E([0.3]))}, + repair_crews=1, + ) + y, z = unit(E([0.03]), E([0.1])), unit(E([0.04]), E([0.15])) + outer = RepairableRBD( + [("s", "x"), ("x", "y"), ("x", "z"), ("y", "t"), ("z", "t")], + {"x": inner, "y": y, "z": z}, + repair_crews=1, + ) + pair = RepairableRBD( + [("s", "y"), ("s", "z"), ("y", "t"), ("z", "t")], + {"y": y, "z": z}, + repair_crews=1, + ) + return outer, pair, inner + + +def halves(pair, inner, times): + """The closed forms' parts at ``times``: the nested RBD's availability + ``a``, the pair's ``A``, and the pair's joint states from its chain + (through its own success criticality: ``y up, z down`` is y's times + ``A``), all of which the outer measures are products of.""" + a = inner.point_availability(times) + A = pair.point_availability(times) + success = pair.criticality_importance(times, kind="success") + y_up_z_down = success["y"] * A + z_up_y_down = success["z"] * A + both_down = 1.0 - A + both_up = A - y_up_z_down - z_up_y_down + return a, A, y_up_z_down, z_up_y_down, both_down, both_up + + +def closed_forms(pair, inner, times) -> dict: + """Each measure's numerator over the system's unavailability: x is + critical and down when the pair is up; y when x is up and z down.""" + a, A, y_up_z_down, z_up_y_down, both_down, both_up = halves( + pair, inner, times + ) + down = 1.0 - a * A + return { + "criticality": { + "x": (1.0 - a) * A / down, + "y": a * both_down / down, + "z": a * both_down / down, + }, + "success": { + "x": np.ones_like(a), + "y": y_up_z_down / A, + "z": z_up_y_down / A, + }, + # Cut sets {x} and {y, z}; path sets {x, y} and {x, z}. + "cuts": { + "x": (1.0 - a) / down, + "y": both_down / down, + "z": both_down / down, + }, + "paths": { + "x": (1.0 - a) * (1.0 - both_up) / down, + "y": (1.0 - a) * (z_up_y_down + both_down) / down, + "z": (1.0 - a) * (y_up_z_down + both_down) / down, + }, + } + + +def test_with_crews_a_nested_rbd_enters_by_its_own_curve(): + outer, pair, inner = nested_in_series() + times = np.array([0.5, 5.0, 30.0, 200.0]) + expected = closed_forms(pair, inner, times) + values = { + "criticality": outer.criticality_importance(times), + "success": outer.criticality_importance(times, kind="success"), + "cuts": outer.fussell_vesely(times), + "paths": outer.fussell_vesely(times, fv_type="p"), + } + for measure, nodes in expected.items(): + for node, value in nodes.items(): + np.testing.assert_allclose(values[measure][node], value, rtol=1e-9) + # Each node is in one cut set, so the rare-event sum is the same. + rare = outer.fussell_vesely(times, method="rare_event") + for node, value in expected["cuts"].items(): + np.testing.assert_allclose(rare[node], value, rtol=1e-9) + # From the long run, every time's value is the long-run one. + long_run = outer.criticality_importance() + stationary = outer.criticality_importance( + [0.0, 3.0, 50.0], state="stationary" + ) + for node, value in long_run.items(): + np.testing.assert_allclose(stationary[node], value, rtol=1e-9) + + +def test_with_crews_a_nested_rbd_over_a_window(): + from scipy.integrate import simpson + + outer, pair, inner = nested_in_series() + end = 20.0 + # The numerators and the system's unavailability, integrated over the + # window on a fine grid: the measure is the ratio of the integrals. + grid = np.linspace(0.0, end, 4001) + a, A, _, _, both_down, _ = halves(pair, inner, grid) + down = simpson(1.0 - a * A, x=grid) + criticality = outer.criticality_importance(window=end) + assert criticality["x"] == pytest.approx( + simpson((1.0 - a) * A, x=grid) / down, rel=1e-6 + ) + assert criticality["y"] == pytest.approx( + simpson(a * both_down, x=grid) / down, rel=1e-6 + ) + cuts = outer.fussell_vesely(window=end) + assert cuts["x"] == pytest.approx( + simpson(1.0 - a, x=grid) / down, rel=1e-6 + ) + assert cuts["z"] == pytest.approx( + simpson(both_down, x=grid) / down, rel=1e-6 + ) + # A long window from the long run is the long-run value. + long_run = outer.fussell_vesely() + window = outer.fussell_vesely(window=30.0, state="stationary") + for node, value in long_run.items(): + assert window[node] == pytest.approx(value, rel=1e-9) + + +def test_with_crews_a_nested_rbd_against_simulated_timelines(): + """w in series with two out of x (a nested RBD with a crew of its + own), y and z, one crew for w, y and z: x's measures depend on the + joint states of the chain's components, so they are checked against + the histories of seeded simulations.""" + inner = RepairableRBD( + [("s", "u"), ("s", "v"), ("u", "t"), ("v", "t")], + {"u": unit(E([0.05]), E([0.2])), "v": unit(E([0.08]), E([0.3]))}, repair_crews=1, ) - with pytest.raises(NotImplementedError, match="nested RBD"): - crewed.criticality_importance(x=1.0) - assert set(crewed.birnbaum_importance(x=1.0)) == {"a", "b", "n"} + edges = [("s", "x"), ("s", "y"), ("s", "z"), ("x", "j"), ("y", "j")] + edges += [("z", "j"), ("j", "w"), ("w", "t")] + rbd = RepairableRBD( + edges, + { + "x": inner, + "y": unit(E([0.03]), E([0.1])), + "z": unit(E([0.04]), E([0.15])), + "w": unit(E([0.02]), E([0.2])), + "j": PerfectReliability, + }, + k={"j": 2}, + repair_crews=1, + ) + times = np.array([4.0, 12.0, 30.0]) + count = 8000 + simulated = rbd.simulate_timelines(30.0, mc_samples=count, seed=5) + up = {n: simulated.components[n].state(times) for n in "xyzw"} + + def works(up): + votes = up["x"].astype(int) + up["y"] + up["z"] + return up["w"] & (votes >= 2) + + system = works(up) + assert np.array_equal(system, simulated.system.state(times)) + unavailability = 1.0 - rbd.point_availability(times) + criticality = rbd.criticality_importance(times) + cuts = rbd.fussell_vesely(times) + down = {n: ~v for n, v in up.items()} + cut_sets = { + "x": down["x"] & (down["y"] | down["z"]), + "y": down["y"] & (down["x"] | down["z"]), + "z": down["z"] & (down["x"] | down["y"]), + "w": down["w"], + } + + def within(estimate: np.ndarray, value: np.ndarray) -> None: + # Each a mean of 0s and 1s: within four standard errors. + error = np.sqrt(estimate * (1.0 - estimate) / count) + assert np.all(np.abs(estimate - value) < 4.0 * error + 1e-12) + + within(1.0 - system.mean(axis=0), unavailability) + for node in "xyzw": + held_up = works({**up, node: np.ones_like(system)}) + held_down = works({**up, node: np.zeros_like(system)}) + critical = held_up & ~held_down & down[node] + within(critical.mean(axis=0), criticality[node] * unavailability) + within(cut_sets[node].mean(axis=0), cuts[node] * unavailability) + # The measures change with time, and x's is no product of the two + # halves: with w up and y down, say, z's state decides whether x is + # critical. + assert np.ptp(criticality["x"]) > 0.01 diff --git a/repyability/tests/test_input_checks.py b/repyability/tests/test_input_checks.py index e8228ecb..237f4cbf 100644 --- a/repyability/tests/test_input_checks.py +++ b/repyability/tests/test_input_checks.py @@ -106,6 +106,7 @@ def _repairable(): spec = { "reliability": surv.Exponential.from_params([0.01]), "repairability": surv.Exponential.from_params([0.5]), + "repair_cost": 1.0, # priced, so cost() reaches its window check } return RepairableRBD([("s", "a"), ("a", "t")], {"a": spec}) @@ -147,3 +148,251 @@ def test_no_spares_are_used_in_no_time_however_it_is_worked_out(): simulated = rbd.spares_demand(0.0, method="simulate", mc_samples=4)["a"] np.testing.assert_array_equal(exact.probabilities, [1.0]) np.testing.assert_array_equal(simulated.probabilities, [1.0]) + + +# -- the 0.14 persona review -------------------------------------------------- + + +def test_counts_of_lifetimes_are_checked(): + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + for bad in (0, -1, 1.5, True, "10"): + with pytest.raises(ValueError, match="mc_samples must be a positive"): + rbd.mean(method="simulate", mc_samples=bad, seed=1) + with pytest.raises(ValueError, match="size must be a positive"): + rbd.random(bad, seed=1) + with pytest.raises(ValueError, match="mc_samples must be at least 2"): + rbd.mean_time_to_failure_interval(mc_samples=1, seed=1) + with pytest.raises(ValueError, match="mc_samples must be at least 4"): + rbd.mean_time_to_failure_interval( + mc_samples=2, seed=1, antithetic=True + ) + interval = rbd.mean_time_to_failure_interval(mc_samples=2, seed=1) + assert interval.method == "simulated" + assert np.isfinite(interval.standard_error) + + +def test_simulation_options_on_an_exact_call_are_value_errors(): + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with pytest.raises(ValueError, match="only with method='simulate'"): + rbd.mean(mc_samples=100) + with pytest.raises(ValueError, match="seed only with method='simulate'"): + rbd.mean(seed=1) + + +def test_a_misspelt_time_keyword_is_a_type_error(): + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with pytest.raises(TypeError, match="unexpected keyword argument 't'"): + rbd.sf(t=50) + with pytest.raises(ValueError, match="x is required: this RBD"): + rbd.sf() + assert rbd.sf(x=50.0) == rbd.sf(50.0) + + +def test_the_held_nodes_must_be_a_collection(): + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with pytest.raises(TypeError, match="working_nodes takes a collection"): + rbd.sf(50.0, working_nodes=0) + with pytest.raises(TypeError, match="broken_nodes takes a collection"): + rbd.birnbaum_importance(50.0, broken_nodes=1.5) + repairable = RepairableRBD( + PARALLEL, {n: {"reliability": W, "repairability": E} for n in "ab"} + ) + with pytest.raises(TypeError, match="working_nodes takes a collection"): + repairable.mission_availability(0, 100) + with pytest.raises(TypeError, match="working_nodes takes a collection"): + repairable.point_availability(10.0, working_nodes=1.5) + assert rbd.sf(50.0, working_nodes=["a"]) == 1.0 + assert rbd.sf(50.0, working_nodes={"a"}) == 1.0 + + +def test_bx_life_takes_a_percentage(): + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with pytest.raises(TypeError, match="x must be numbers, got '10'"): + rbd.bx_life("10") + with pytest.raises(TypeError, match="x must be numbers"): + rbd.bx_life(True) + with pytest.raises(ValueError, match="percentage in \\(0, 100\\)"): + rbd.bx_life(0.0) + assert rbd.sf(rbd.bx_life(10)) == pytest.approx(0.9) + # A fixed survivor in parallel keeps the reliability at 0.9 for good. + kept = NonRepairableRBD( + PARALLEL, {"a": surv.FixedEventProbability.from_params(0.1), "b": W} + ) + with pytest.raises(ValueError, match="never reached") as error: + kept.bx_life(20) + assert "never falls below 0.9" in str(error.value) + assert kept.bx_life(5) == pytest.approx(kept.time_to_reliability(0.95)) + + +def test_from_dict_needs_a_type(): + from repyability import RBD + + with pytest.raises(ValueError, match='has no "type"'): + RBD.from_dict({"edges": [["s", "a"], ["a", "t"]]}) + with pytest.raises(ValueError, match='has no "type"'): + NonRepairableRBD.from_dict({}) + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + assert RBD.from_dict(rbd.to_dict()).sf(50.0) == rbd.sf(50.0) + + +def test_path_set_probabilities_pair_with_the_path_sets(): + edges = [("s", "a"), ("s", "b"), ("a", "c"), ("b", "c"), ("c", "t")] + rbd = NonRepairableRBD(edges, {"a": W, "b": W, "c": E}) + probabilities = {"a": 0.9, "b": 0.8, "c": 0.5} + found = rbd.path_set_probabilities(probabilities) + expected = [ + np.prod([probabilities[n] for n in path]) + for path in rbd.minimal_path_sets() + ] + assert np.allclose(found, expected) + assert [sorted(p) for p in rbd.minimal_path_sets()] == [ + ["a", "c"], + ["b", "c"], + ] + with pytest.raises(ValueError, match="none given for 'b', 'c'"): + rbd.path_set_probabilities({"a": 0.9}) + with pytest.raises(TypeError, match="not a time"): + rbd.path_set_probabilities(50.0) + + +def test_an_irrelevant_node_behind_a_junction_is_found(): + from repyability import RBD, PerfectReliability + + # x is in parallel with a path through the junction j alone, which + # never fails: x is in no minimal path set and cannot matter. + rbd = NonRepairableRBD( + [("s", "a"), ("a", "j"), ("a", "x"), ("j", "t"), ("x", "t")], + {"a": W, "x": E, "j": PerfectReliability}, + ) + assert rbd.find_irrelevant_components() == {"x"} + assert rbd.structure_check["has_irrelevant_nodes"] + assert rbd.structure_check["irrelevant_nodes"] == {"x"} + assert rbd.birnbaum_importance(50.0) == {"a": pytest.approx(1.0), "x": 0.0} + assert rbd.minimal_path_sets() == [frozenset({"a"})] + # A junction is no component: never listed. With a vote it is relevant + # to the structure, and every member is relevant. + vote = NonRepairableRBD( + [("s", "a"), ("s", "b"), ("a", "j"), ("b", "j"), ("j", "t")], + {"a": W, "b": W, "j": PerfectReliability}, + k={"j": 2}, + ) + assert vote.find_irrelevant_components() == set() + assert not vote.structure_check["has_irrelevant_nodes"] + assert RBD( + [("s", "a"), ("a", "t"), ("a", "b"), ("b", "t")] + ).find_irrelevant_components() == {"b"} + + +def test_a_group_s_levers_name_its_uncertainty(): + import scipy.stats as st + + from repyability import BetaFactor, CCFGroup, FaultTree + + rbd = NonRepairableRBD( + PARALLEL, + {"a": W, "b": W}, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1, basis="rate"))], + ) + group = rbd.ccf_groups[0] + assert {(lever.key, lever.name) for lever in rbd.levers()} >= { + (("a", "b"), "ccf_beta") + } + by_lever = { + ("a", "b"): {"alpha": st.uniform(90, 20), "ccf_beta": st.beta(2, 18)} + } + by_group = { + ("a", "b"): {"alpha": st.uniform(90, 20)}, + group: {"beta": st.beta(2, 18)}, + } + for diagram in (rbd, FaultTree.from_rbd(rbd)): + ours = diagram.sf_uncertainty(50.0, by_lever, n_draws=40, seed=1) + theirs = diagram.sf_uncertainty(50.0, by_group, n_draws=40, seed=1) + assert np.array_equal(ours.samples, theirs.samples) + own = rbd.sf_uncertainty( + 50.0, {("a", "b"): {"ccf_beta": st.beta(2, 18)}}, n_draws=40, seed=1 + ) + assert np.array_equal( + own.samples, + rbd.sf_uncertainty( + 50.0, {group: {"beta": st.beta(2, 18)}}, n_draws=40, seed=1 + ).samples, + ) + with pytest.raises(ValueError, match="given an uncertainty twice"): + rbd.sf_uncertainty( + 50.0, + { + ("a", "b"): {"ccf_beta": st.beta(2, 18)}, + group: {"beta": st.beta(2, 18)}, + }, + n_draws=4, + seed=1, + ) + with pytest.raises(ValueError, match=r"\['ccf_gamma'\] are not levers"): + rbd.sf_uncertainty( + 50.0, + {("a", "b"): {"ccf_gamma": st.beta(2, 18)}}, + n_draws=4, + seed=1, + ) + repairable = RepairableRBD( + PARALLEL, + {n: {"reliability": E, "repairability": E} for n in "ab"}, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], + ) + assert np.array_equal( + repairable.mean_availability_uncertainty( + {("a", "b"): {"ccf_beta": st.beta(2, 18)}}, n_draws=10, seed=1 + ).samples, + repairable.mean_availability_uncertainty( + {repairable.ccf_groups[0]: {"beta": st.beta(2, 18)}}, + n_draws=10, + seed=1, + ).samples, + ) + + +def test_the_measures_at_zero_are_nan_without_a_warning(): + import warnings + + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with warnings.catch_warnings(): + warnings.simplefilter("error") + for name in ( + "risk_achievement_worth", + "fussell_vesely", + "criticality_importance", + ): + found = getattr(rbd, name)(0.0) + assert all(np.isnan(v) for v in found.values()), name + assert np.isnan(rbd.fussell_vesely(0.0, method="rare_event")["a"]) + + +def test_with_levers_names_the_kind_of_diagram(): + from repyability import FaultTree + + rbd = NonRepairableRBD(PARALLEL, {"a": W, "b": W}) + with pytest.raises( + ValueError, match="no node with a model or common-cause group\\. " + ): + rbd.with_levers({("zz", "alpha"): 5.0}) + with pytest.raises( + ValueError, match="no event with a model or common-cause group\\. " + ): + FaultTree.from_rbd(rbd).with_levers({("zz", "alpha"): 5.0}) + repairable = RepairableRBD( + PARALLEL, {n: {"reliability": W, "repairability": E} for n in "ab"} + ) + with pytest.raises(ValueError, match="or the repair crews \\(None\\)"): + repairable.with_levers({("zz", "reliability.alpha"): 5.0}) + + +def test_a_non_repairable_prints_its_models_and_costs(): + from repyability import NonRepairable + + unit = NonRepairable(W) + assert repr(unit) == "NonRepairable(Weibull, replaced in ExactEventTime)" + priced = NonRepairable(W, E, planned_cost=10.0, unplanned_cost=100.0) + assert repr(priced) == ( + "NonRepairable(Weibull, replaced in Exponential, planned_cost=10, " + "unplanned_cost=100)" + ) diff --git a/repyability/tests/test_inspection.py b/repyability/tests/test_inspection.py index c9a63dc3..1011510f 100644 --- a/repyability/tests/test_inspection.py +++ b/repyability/tests/test_inspection.py @@ -14,7 +14,8 @@ from scipy.integrate import quad from repyability import RBD, RepairableRBD -from repyability.rbd.repairable_rbd import Event +from repyability.rbd import _long_run +from repyability.rbd._events import Event X = surv.ExactEventTime.from_params E = surv.Exponential.from_params @@ -299,8 +300,9 @@ def test_importance_is_averaged_over_the_interval(): # Every system failure has both failed and both critical. assert rbd.criticality_importance()["a"] == pytest.approx(1.0) assert rbd.fussell_vesely()["a"] == pytest.approx(1.0) - # Held working, "a" makes the system perfect. - assert rbd.mean_availability(working_nodes=["a"]) == 1.0 + # Held working, "a" makes the system perfect (to rounding: the mean is + # worked out numerically, and some numpy builds sum it a unit short). + assert rbd.mean_availability(working_nodes=["a"]) == pytest.approx(1.0) def test_the_cost_rate_trades_tests_against_hidden_downtime(): @@ -379,7 +381,7 @@ def test_tests_and_repairs_that_take_time_are_numerical(): (1.0 - kept) / interval, rel=1e-8 ) # Each test takes the unit, working then, off line: planned outages. - assert timed._outage_frequencies()[1] == pytest.approx( + assert _long_run._outage_frequencies(timed)[1] == pytest.approx( kept / interval, rel=1e-8 ) timed.birnbaum_importance() diff --git a/repyability/tests/test_inspection_offsets.py b/repyability/tests/test_inspection_offsets.py index b6225827..643ca7c3 100644 --- a/repyability/tests/test_inspection_offsets.py +++ b/repyability/tests/test_inspection_offsets.py @@ -185,23 +185,6 @@ def test_a_single_share_is_a_number_too(): assert plan.offsets == {"v1": 0.0, "v2": 4380.0} -def test_offsets_is_the_old_name_of_offset_shares(): - # Its values are shares, where with_intervals' offsets are times (#222). - pair = valves([(8760.0, 0.0), (8760.0, 0.0)]) - plan = pair.optimal_inspection_intervals( - allowed=[8760.0], min_availability=0.99, offset_shares=[0.0, 0.5] - ) - with pytest.warns(FutureWarning, match="renamed offset_shares"): - old = pair.optimal_inspection_intervals( - allowed=[8760.0], min_availability=0.99, offsets=[0.0, 0.5] - ) - assert old == plan - with pytest.raises(ValueError, match="offset_shares alone"): - pair.optimal_inspection_intervals( - allowed=[8760.0], offsets=[0.0], offset_shares=[0.0] - ) - - def test_unknown_nodes_are_named(): pair = valves([(8760.0, 0.0), (8760.0, 0.0)]) with pytest.raises(ValueError, match=r"names \['V3'\]"): diff --git a/repyability/tests/test_joint_importance.py b/repyability/tests/test_joint_importance.py index 47a53e03..1739a239 100644 --- a/repyability/tests/test_joint_importance.py +++ b/repyability/tests/test_joint_importance.py @@ -5,7 +5,10 @@ failed, against its meaning (the system's change when both improve, less the changes when each does, is the joint importance times the two improvements, exactly, as the structure is multilinear), and across the -classes: a fault tree's equals its diagram's.""" +classes: a fault tree's equals its diagram's. With common-cause groups +(#317) each pair is conditioned on its states: checked against +conditioning on every combination of every group's outcomes, and with +the shared causes at zero against the diagram without groups.""" import itertools @@ -15,12 +18,14 @@ from surpyval import FixedEventProbability from repyability import ( + MGL, BetaFactor, CCFGroup, FaultTree, NonRepairableRBD, RepairableRBD, ) +from repyability.rbd.ccf import shock_outcomes E, W = surv.Exponential.from_params, surv.Weibull.from_params F = FixedEventProbability.from_params @@ -137,14 +142,108 @@ def test_a_repeated_node_is_one_component(): assert pairs[frozenset({"a", "b"})] == pytest.approx(-0.95) -def test_common_cause_groups_are_refused(): - grouped = NonRepairableRBD( - [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], - {"a": F(0.1), "b": F(0.1)}, - ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], - ) - with pytest.raises(NotImplementedError, match="common-cause"): - grouped.joint_importance() +def conditioned_pairs(rbd, x, held=()): + """The joint importance of each pair by conditioning on every + combination of every group's outcomes: the system given each of the + pair working and failed (a member's state through the outcomes, a + node outside the groups held), ``R(1, 1) - R(1, 0) - R(0, 1) + R(0, + 0)``; 0 for a pair with a node held.""" + p = {n: np.atleast_1d(rbd.reliabilities[n].sf(x)) for n in rbd.nodes} + q = {n: np.atleast_1d(rbd.reliabilities[n].ff(x)) for n in rbd.nodes} + for node in held: # held working + p[node], q[node] = np.ones_like(p[node]), np.zeros_like(q[node]) + grouped = {m for g in rbd.ccf_groups for m in g.members} + out = {} + for i, j in itertools.combinations(rbd.nodes, 2): + if i in held or j in held: + out[i, j] = np.zeros_like(p[i]) + continue + sums = {} + for w, given, failing in shock_outcomes(rbd.ccf_groups, p, q): + for a, b in itertools.product((True, False), repeat=2): + chance = w + hold_p, hold_q = dict(given), dict(failing) + for node, up in ((i, a), (j, b)): + if node in grouped: + chance = chance * (given if up else failing)[node] + hold_p[node] = (1.0 if up else 0.0) * np.ones_like(p[i]) + hold_q[node] = 1.0 - hold_p[node] + up_, _ = rbd._system_probabilities(hold_p, hold_q) + was = sums.get((a, b), (0.0, 0.0)) + sums[a, b] = (was[0] + chance, was[1] + chance * up_) + R = {combo: up_ / chance for combo, (chance, up_) in sums.items()} + out[i, j] = ( + R[True, True] - R[True, False] - R[False, True] + R[False, False] + ) + return out + + +GROUPED_BRIDGE = { + "a": E([0.01]), + "b": E([0.01]), + "c": W([80, 1.5]), + "d": E([0.01]), + "e": E([0.01]), +} + + +@pytest.mark.parametrize( + "groups", + [ + [CCFGroup(["a", "b"], BetaFactor(0.2))], + [CCFGroup(["a", "b"], BetaFactor(0.1, basis="rate"))], + [ + CCFGroup(["a", "b"], BetaFactor(0.2)), + CCFGroup(["d", "e"], BetaFactor(0.3, basis="rate")), + ], + [CCFGroup(["a", "d", "e"], MGL(0.1, 0.3))], + [CCFGroup(["a", "d", "e"], MGL(0.1, 0.3, shocks="independent"))], + ], +) +def test_with_common_cause_groups_each_pair_is_conditioned_on(groups): + # Pairs within a group, across groups, of a member and a node outside + # the groups, and of two outside, at times and at one; with nodes + # held, their pairs are 0 and the rest conditioned on them. + rbd = NonRepairableRBD(BRIDGE, GROUPED_BRIDGE, ccf_groups=groups) + x = np.array([5.0, 40.0, 150.0]) + joint = rbd.joint_importance(x) + expected = conditioned_pairs(rbd, x) + assert set(joint) == set(expected) + for pair, value in expected.items(): + np.testing.assert_allclose(joint[pair], value, rtol=1e-9, atol=1e-14) + at = rbd.joint_importance(40.0) + for pair, value in at.items(): + assert value == pytest.approx(joint[pair][1], abs=1e-14) + held = rbd.joint_importance(x, working_nodes=["c"]) + for pair, value in conditioned_pairs(rbd, x, held={"c"}).items(): + np.testing.assert_allclose(held[pair], value, rtol=1e-9, atol=1e-14) + assert np.all(held[("a", "c")] == 0.0) + # A member cannot be held, as for the other measures. + with pytest.raises(NotImplementedError, match="CCF group member"): + rbd.joint_importance(x, broken_nodes=["a"]) + # A fault tree of the diagram has the same pairs. + tree = FaultTree.from_rbd(rbd) + theirs = tree.joint_importance(x) + for pair, value in joint.items(): + np.testing.assert_allclose(theirs[pair], value, rtol=1e-9, atol=1e-14) + + +def test_shared_causes_at_zero_give_the_pairs_without_groups(): + x = np.array([5.0, 40.0, 150.0]) + plain = NonRepairableRBD(BRIDGE, GROUPED_BRIDGE).joint_importance(x) + for model in (BetaFactor(0.0), BetaFactor(0.0, basis="rate")): + grouped = NonRepairableRBD( + BRIDGE, + GROUPED_BRIDGE, + ccf_groups=[ + CCFGroup(["a", "b"], model), + CCFGroup(["d", "e"], model), + ], + ).joint_importance(x) + for pair, value in plain.items(): + np.testing.assert_allclose( + grouped[pair], value, rtol=1e-10, atol=1e-14 + ) def unit(rate, repair=1.0): @@ -201,13 +300,59 @@ def test_with_crews_each_pair_is_held_in_the_chain(): for (i, j), value in joint.items(): expected = held_pairs(lambda w, b: rbd.mean_availability(w, b), i, j) assert value == pytest.approx(expected, abs=1e-12) - grouped = RepairableRBD( + + +def test_a_repairable_diagram_with_groups(): + # With the shared cause at zero, the pairs without the group, in the + # long run, at times and over a window (test_ccf_repairable_modules + # checks a shared cause against conditioning on every combination). + units = {"a": unit(0.1), "b": unit(0.1), "c": unit(0.02, 0.5)} + plain = RepairableRBD(PAIR_THEN_C, units) + none = RepairableRBD( + PAIR_THEN_C, + units, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.0))], + ) + x = np.array([2.0, 20.0]) + for kwargs, rel in ( + ({}, 1e-9), + ({"x": x}, 1e-5), + ({"window": 10.0}, 1e-5), + ): + # Over time the groups' chains are followed numerically. + ours = none.joint_importance(**kwargs) + theirs = plain.joint_importance(**kwargs) + assert set(ours) == set(theirs) + for pair, value in theirs.items(): + np.testing.assert_allclose(ours[pair], value, rtol=rel) + # A shared cause: with a down, b is likelier down too, so c is worth + # less then, and more of a complement of a; the pair a, b is as + # before, c alone remaining given their states. + shared = RepairableRBD( + PAIR_THEN_C, + units, + ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], + ) + joint, plain_joint = shared.joint_importance(), plain.joint_importance() + assert joint[("a", "c")] > plain_joint[("a", "c")] > 0 + assert joint[("a", "b")] == pytest.approx(plain_joint[("a", "b")]) + assert joint[("a", "b")] == pytest.approx( + -float(shared.node_availability()["c"]), rel=1e-9 + ) + held = shared.joint_importance(working_nodes=["c"]) + assert held[("a", "c")] == 0.0 and held[("a", "b")] == pytest.approx(-1.0) + with pytest.raises(NotImplementedError, match="common-cause group"): + shared.joint_importance(broken_nodes=["a"]) + # With limited repair crews too, the groups refuse as the other + # measures do. + crews = RepairableRBD( PAIR_THEN_C, - {n: unit(0.1) for n in "abc"}, + units, ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.1))], + repair_crews=1, ) - with pytest.raises(NotImplementedError, match="common-cause"): - grouped.joint_importance() + with pytest.raises(NotImplementedError, match="repair crews"): + crews.joint_importance() def test_a_fault_tree_s_is_its_diagram_s(): @@ -258,13 +403,31 @@ def test_a_fault_tree_over_time_and_with_groups(): np.testing.assert_allclose( joint[("x", "y")], -tree.events["z"].ff([10.0, 100.0]), rtol=1e-12 ) + # With a group, as its diagram's, and from the top event with each + # pair's states conditioned on: an AND of the pair, so the top event + # given both occurring is 1, and else 0. grouped = FaultTree( - {"top": ("and", ["x", "y"])}, - {"x": 0.1, "y": 0.1}, + {"top": ("and", ["x", "y", "z"])}, + {"x": 0.1, "y": 0.1, "z": 0.3}, ccf_groups=[CCFGroup(["x", "y"], BetaFactor(0.1))], ) - with pytest.raises(NotImplementedError, match="common-cause"): - grouped.joint_importance() + joint = grouped.joint_importance() + assert joint[("x", "y")] == pytest.approx(-0.3) + # Given x occurred, so has y with (0.01 + 0.99 * 0.09 ** 2) / 0.0991. + assert joint[("x", "z")] == pytest.approx(-0.018019 / 0.0991, rel=1e-9) + as_diagram = grouped.to_rbd().joint_importance() + for pair, value in joint.items(): + assert value == pytest.approx(as_diagram[pair], abs=1e-14) + none = FaultTree( + {"top": ("and", ["x", "y", "z"])}, + {"x": 0.1, "y": 0.1, "z": 0.3}, + ccf_groups=[CCFGroup(["x", "y"], BetaFactor(0.0))], + ).joint_importance() + plain = FaultTree( + {"top": ("and", ["x", "y", "z"])}, {"x": 0.1, "y": 0.1, "z": 0.3} + ).joint_importance() + for pair, value in plain.items(): + assert none[pair] == pytest.approx(value, abs=1e-14) def test_pairs_are_found_either_way_round(): diff --git a/repyability/tests/test_junctions.py b/repyability/tests/test_junctions.py index d367410e..c5a5c10a 100644 --- a/repyability/tests/test_junctions.py +++ b/repyability/tests/test_junctions.py @@ -28,6 +28,8 @@ E = surv.Exponential.from_params W = surv.Weibull.from_params +# A probability of failing of 0: a unit that never fails, so a junction. +NEVER = surv.FixedEventProbability.from_params([0.0]) TRAINS = ["a", "b", "c"] CAPACITY = {"a": 50.0, "b": 50.0, "c": 50.0, "v": 120.0, "p": 200.0} @@ -208,8 +210,9 @@ def stage(names): [ PerfectReliability, {"reliability": PerfectReliability, "repairability": E([0.1])}, + {"reliability": NEVER, "repairability": E([0.1])}, ], - ids=["itself", "as a life"], + ids=["itself", "as a life", "a probability of 0"], ) def test_a_part_that_never_fails_changes_nothing(perfect): # #175: PerfectReliability as a repairable component's life. @@ -250,6 +253,7 @@ def pumps(capacity): [ PerfectReliability, {"reliability": PerfectReliability, "repairability": E([0.1])}, + {"reliability": NEVER, "repairability": E([0.1])}, ], ) def test_a_junction_is_saved(junction): @@ -463,3 +467,16 @@ def test_a_nonrepairable_junction_has_no_mttf(): assert rbd.analysis_routes()["node_mttf"].route == "exact" mean = float(u.mean()) assert rbd.node_mttf() == pytest.approx({"a": mean, "b": mean, "c": mean}) + + +def test_a_life_that_never_fails_makes_a_junction(): + # A probability of failing of 0 is a junction, as PerfectReliability + # is: every method's value is the station's with PerfectReliability + # there; anything only a part that fails has is refused, naming it. + never = station({"reliability": NEVER, "repairability": E([0.1])}) + perfect = station() + assert never._junctions() == {"h"} + assert never.mean_availability() == perfect.mean_availability() + assert never.analysis_routes()["mean_availability"].route == "exact" + with pytest.raises(ValueError, match="probability of failing of 0"): + station({"reliability": NEVER, "repair_cost": 1.0}) diff --git a/repyability/tests/test_levers.py b/repyability/tests/test_levers.py index fe5e3982..d118a8bd 100644 --- a/repyability/tests/test_levers.py +++ b/repyability/tests/test_levers.py @@ -243,6 +243,74 @@ def test_with_levers_moves_several_and_leaves_the_diagram_as_it_was(): assert type(rbd.with_levers({})) is RepairableRBD +@pytest.mark.parametrize("coverage", [1.0, 0.8]) +def test_a_group_s_members_keep_their_own_offsets(coverage): + # A common-cause group's members share their life and repair models, + # which are the group's levers, moved in each member's own spec; a + # member's test offset (its own place on the calendar) is its own + # lever, as without the group, and is kept when the group's levers + # move. The sensitivities are then the staggered pair's: central + # differences through with_levers. + def tested(**more): + inspection = {"interval": 100.0, "cost": 1.0, **more} + if coverage < 1.0: + inspection.update(coverage=coverage, full_test=300.0) + return { + "reliability": E([1e-3]), + "repairability": "instant", + "inspection": inspection, + } + + group = CCFGroup(["a", "b"], BetaFactor(0.1)) + staggered = RepairableRBD( + PAIR, {"a": tested(), "b": tested(offset=50.0)}, ccf_groups=[group] + ) + together = RepairableRBD( + PAIR, {"a": tested(), "b": tested()}, ccf_groups=[group] + ) + levers = named(staggered.levers()) + assert (("a", "b"), "inspection.interval") in levers + assert ("b", "inspection.offset") in levers + assert ("a", "inspection.offset") not in levers + assert levers.index((("a", "b"), "inspection.interval")) < levers.index( + ("b", "inspection.offset") + ) + offset = next( + lever + for lever in staggered.levers() + if (lever.key, lever.name) == ("b", "inspection.offset") + ) + assert (offset.value, offset.bounds) == (50.0, (0.0, 100.0)) + # The interval's floor is the members' offsets'. + interval = next( + lever + for lever in staggered.levers() + if (lever.key, lever.name) == (("a", "b"), "inspection.interval") + ) + assert interval.bounds == (50.0, math.inf) + moved = staggered.with_levers({interval: 100.0}) + assert moved._inspection["b"].offset == 50.0 + assert moved.mean_unavailability() == staggered.mean_unavailability() + assert staggered.mean_unavailability() < together.mean_unavailability() + step = 1e-5 + reported = staggered.parameter_sensitivity(rel_step=step) + assert reported[("a", "b")]["inspection.interval"] != pytest.approx( + together.parameter_sensitivity(rel_step=step)[("a", "b")][ + "inspection.interval" + ] + ) + for lever in staggered.levers(): + h = step * abs(lever.value) + up = staggered.with_levers({lever: lever.value + h}) + down = staggered.with_levers({lever: lever.value - h}) + change = -(up.mean_unavailability() - down.mean_unavailability()) + change /= 2.0 * h + assert reported[lever.key][lever.name] == pytest.approx( + change, rel=1e-9, abs=1e-12 + ), lever + assert abs(reported[("a", "b")]["inspection.interval"]) < 1.0 + + def test_a_test_interval_takes_its_full_tests_with_it(): tested = unit( inspection={ @@ -339,26 +407,3 @@ def test_non_repairable_levers_move_as_the_sensitivity_does(): down = rbd.with_levers({lever: lever.value - h}).sf(30.0) reported = rbd.parameter_sensitivity(30.0, rel_step=step)["a"]["alpha"] assert (up - down) / (2.0 * h) == pytest.approx(reported, rel=1e-6) - - -# -- the private names, kept for 0.13 ---------------------------------------- - - -def test_the_private_names_callers_used_still_work(): - rbd = RepairableRBD( - SERIES, - { - "a": unit(preventive={"policy": "block", "interval": 50.0}), - "b": unit(preventive={"policy": "block", "interval": 50.0}), - }, - ) - old = _sensitivity.levers(rbd) - assert all(isinstance(lever, _sensitivity.Lever) for lever in old) - assert [(lever.key, lever.name, lever.value) for lever in old] == [ - (lever.key, lever.name, lever.value) for lever in rbd.levers() - ] - assert all(callable(lever.build) for lever in old) - calendar = [_sensitivity._calendar_lever(rbd, lever) for lever in old] - assert calendar == [lever.calendar for lever in rbd.levers()] - spec = _sensitivity._as_spec(rbd._init_args["components"]["a"]) - assert spec["preventive"]["interval"] == 50.0 diff --git a/repyability/tests/test_limited_failure_population.py b/repyability/tests/test_limited_failure_population.py index 7a8c8db4..7e7c1efd 100644 --- a/repyability/tests/test_limited_failure_population.py +++ b/repyability/tests/test_limited_failure_population.py @@ -30,6 +30,7 @@ RepeatedStandbyNode, StandbyModel, ) +from repyability.rbd import _event_loop from repyability.rbd._model_utils import ( is_exponential, lfp_p, @@ -390,7 +391,7 @@ def test_repairable_draws_come_from_their_streams(): } }, ) - specs, complete = maintained._stream_specs(200.0) + specs, complete = _event_loop._stream_specs(maintained, 200.0) assert complete and (("c",), DURATION) in specs assert maintained.availability( 200.0, mc_samples=100, seed=17 diff --git a/repyability/tests/test_maintenance_calendar.py b/repyability/tests/test_maintenance_calendar.py new file mode 100644 index 00000000..1c4116a0 --- /dev/null +++ b/repyability/tests/test_maintenance_calendar.py @@ -0,0 +1,734 @@ +"""The long-run optimal replacement of a part over a calendar of +opportunities (``NonRepairable.optimal_calendar_policy``; #319, UC-13 of +``repyability/maintenance/README.md``): the calendar, and phase 1, dynamic +programming on a grid (the brief's section 3).""" + +import json +import math +import warnings + +import numpy as np +import pytest +import surpyval as surv + +from repyability import Calendar, CalendarPolicy, NonRepairable, Opportunity +from repyability.maintenance import _replacement +from repyability.maintenance._calendar import _Grid + + +def solve(part, calendar, **options): + """The calendar policy, on a grid of a given step where a check needs + one (the public method sizes its own).""" + return _replacement.optimal_calendar_policy(part, calendar, **options) + + +def price(part, calendar, rule, **options): + """A rule's cost rate, likewise.""" + return _replacement.calendar_policy_cost(part, calendar, rule, **options) + + +LIFE = surv.Weibull.from_params([1000.0, 2.5]) +BEARING = NonRepairable(LIFE) + +# The plant: weekly stops at 3, quarterly shuts at 2, an annual one at 1, +# a planned replacement on any other day at 4, a failure at 20. +PLANT = Calendar( + [ + Opportunity("weekly", 3.0, period=7.0), + Opportunity("quarterly", 2.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, + anytime=4.0, +) + + +@pytest.fixture(scope="module") +def plant(): + return solve(BEARING, PLANT, step=1.0) + + +# The calendar. + + +def test_an_opportunity_on_a_period_or_dates(): + weekly = Opportunity("weekly", 3, period=7, offset=3) + assert weekly.cost == 3.0 and weekly.period == 7.0 + assert weekly.times(0.0, 20.0).tolist() == [3.0, 10.0, 17.0] + # Both ways from the offset. + assert weekly.times(-10.0, 0.0).tolist() == [-4.0] + plan = Opportunity("turnaround", 1.0, dates=[1130.0, 400.0, 400.0]) + assert plan.dates == (400.0, 1130.0) + assert not plan.periodic + + +def test_the_step_is_not_an_argument(): + # A numerical setting, sized from the models: the calendar says when + # decisions are made. The default here is 7/8 of a day, on the shuts. + with pytest.raises(TypeError): + BEARING.optimal_calendar_policy(PLANT, step=1.0) + with pytest.raises(TypeError): + BEARING.calendar_policy_cost(PLANT, {}, step=1.0) + policy = BEARING.optimal_calendar_policy(PLANT) + assert policy.step == 0.875 + daily = solve(BEARING, PLANT, step=1.0) + assert policy.cost_rate == pytest.approx(daily.cost_rate, rel=1e-6) + + +@pytest.mark.parametrize( + "kwargs, error, match", + [ + ({"name": "", "cost": 1.0, "period": 7.0}, TypeError, "name"), + ({"name": "anytime", "cost": 1.0, "period": 7.0}, ValueError, "any"), + ({"name": "a", "cost": -1.0, "period": 7.0}, ValueError, "cost"), + ({"name": "a", "cost": True, "period": 7.0}, ValueError, "cost"), + ({"name": "a", "cost": 1.0}, ValueError, "one of"), + ( + {"name": "a", "cost": 1.0, "period": 7.0, "dates": [1.0]}, + ValueError, + "one of", + ), + ({"name": "a", "cost": 1.0, "period": 0.0}, ValueError, "period"), + ({"name": "a", "cost": 1.0, "period": math.inf}, ValueError, "period"), + ( + {"name": "a", "cost": 1.0, "period": 7.0, "offset": math.nan}, + ValueError, + "offset", + ), + ( + {"name": "a", "cost": 1.0, "dates": [1.0], "offset": 2.0}, + ValueError, + "offset", + ), + ({"name": "a", "cost": 1.0, "dates": []}, ValueError, "dates"), + ({"name": "a", "cost": 1.0, "dates": [math.inf]}, ValueError, "dates"), + ], +) +def test_an_opportunity_is_checked(kwargs, error, match): + with pytest.raises(error, match=match): + Opportunity(**kwargs) + + +def test_the_calendar_period_and_prices(): + assert PLANT.periodic and PLANT.period == 364.0 + assert PLANT.planned_cost([0.0, 1.0, 7.0, 91.0]).tolist() == [ + 1.0, + 4.0, + 3.0, + 2.0, + ] + # Laid on a grid, an opportunity counts at the grid time nearest it. + assert PLANT.planned_cost([7.4], step=1.0).tolist() == [3.0] + assert PLANT.planned_cost([7.4]).tolist() == [4.0] + leap = Calendar([Opportunity("a", 1.0, period=365.25)], 9.0, None) + assert leap.period == 365.25 + mixed = Calendar( + [ + Opportunity("w", 1.0, period=7.0), + Opportunity("a", 1.0, period=365.25), + ], + 9.0, + None, + ) + assert mixed.period == 10227.0 + plan = Calendar([Opportunity("t", 1.0, dates=[400.0, 1130.0])], 9.0) + assert not plan.periodic and plan.period is None + assert plan.plan_end == 1130.0 + assert Calendar(failure=9.0, anytime=None).planned_cost([5.0])[0] == ( + math.inf + ) + + +def test_the_calendar_is_checked(): + with pytest.raises(ValueError, match="differ"): + Calendar( + [ + Opportunity("a", 1.0, period=7.0), + Opportunity("a", 2.0, period=9.0), + ] + ) + with pytest.raises(TypeError, match="Opportunity"): + Calendar([("a", 1.0)]) + with pytest.raises(ValueError, match="failure"): + Calendar(failure=-1.0) + with pytest.raises(ValueError, match="anytime"): + Calendar(anytime="cheap") + # The prices taken from the part must be there. + with pytest.raises(ValueError, match="failure price"): + solve(BEARING, Calendar(), step=10.0) + + +def test_the_part_gives_the_failure_and_anytime_prices(): + unit = NonRepairable(LIFE, planned_cost=4.0, unplanned_cost=20.0) + by_part = solve( + unit, Calendar([Opportunity("annual", 1.0, period=364.0)]), step=7.0 + ) + given = solve( + BEARING, + Calendar([Opportunity("annual", 1.0, period=364.0)], 20.0, 4.0), + step=7.0, + ) + assert by_part.cost_rate == given.cost_rate + + +def test_dates_off_the_grid_are_moved_with_a_warning(): + calendar = Calendar([Opportunity("a", 1.0, period=365.25)], 20.0, None) + with pytest.warns(UserWarning, match="nearest"): + grid = _Grid(calendar, None, 1.0) + # Each date to its nearest grid time: no drift over the cycle. + costs, _ = grid.prices(0, 4 * 1461) + assert np.flatnonzero(np.isfinite(costs))[:5].tolist() == [ + 0, + 365, + 730, + 1096, + 1461, + ] + + +def test_the_default_step_puts_the_opportunities_on_the_grid(): + step = _replacement._default_step(PLANT, 887.0) + assert 7.0 / step == round(7.0 / step) + assert step <= 887.0 / _replacement.STEPS_PER_LIFE * 1.0001 + + +# The step is sized from the periods: an offset off the grid (a few hours +# on a period of days, as dates turned into model time give) is laid at +# the grid time nearest it, and the calendar keeps its period. + +SHUT = NonRepairable( + surv.Weibull.from_params([365.0, 2.0]), + planned_cost=150.0, + unplanned_cost=400.0, +) + + +def shut_every_180(offset): + return Calendar([Opportunity("shut", 100.0, period=180.0, offset=offset)]) + + +def test_an_offset_off_the_grid_keeps_the_period(): + with warnings.catch_warnings(): + warnings.simplefilter("error") + aligned = SHUT.optimal_calendar_policy(shut_every_180(0.0)) + with pytest.warns(UserWarning, match="nearest"): + moved = SHUT.optimal_calendar_policy(shut_every_180(0.1234)) + assert (aligned.method, aligned.basis) == ("dp", "exact") + assert (moved.method, moved.basis, moved.period) == ("dp", "exact", 180.0) + assert aligned.period == 180.0 + # The step is a whole fraction of the period, the same either way + # (about the mean life, 323.5 days, over 1,000), so the shut at + # 0.1234 + k 180 is laid at k 180, within half a step, and the two + # policies are one and the same: the cost rate exactly, each shut's + # threshold and long-run chance of a replacement. + assert moved.step == aligned.step + assert 180.0 / moved.step == round(180.0 / moved.step) + assert 0.1234 < moved.step / 2 + assert moved.cost_rate == aligned.cost_rate + assert moved.schedule() == aligned.schedule() + shuts = moved.schedule(until=400.0) + assert [t for t, _, _, _ in shuts] == [0.0, 180.0, 360.0] + assert len({chance for _, _, _, chance in shuts}) == 1 + assert shuts[0][3] == pytest.approx( + moved.replacement_rates["shut"] * 180.0, rel=1e-9 + ) + assert _replacement._default_step(shut_every_180(0.1234), 323.5) == ( + _replacement._default_step(shut_every_180(0.0), 323.5) + ) + + +def test_an_offset_of_whole_steps_shifts_the_policy(): + aligned = SHUT.optimal_calendar_policy(shut_every_180(0.0)) + offset = 10 * aligned.step + with warnings.catch_warnings(): + warnings.simplefilter("error") + shifted = SHUT.optimal_calendar_policy(shut_every_180(offset)) + assert shifted.step == aligned.step and shifted.period == 180.0 + assert shifted.cost_rate == pytest.approx(aligned.cost_rate, rel=1e-9) + moved = shifted.schedule(until=400.0) + kept = aligned.schedule(until=400.0 - offset) + assert [t - offset for t, _, _, _ in moved] == pytest.approx( + [t for t, _, _, _ in kept], abs=1e-9 + ) + assert [(k, a) for _, k, a, _ in moved] == [(k, a) for _, k, a, _ in kept] + times = np.arange(0.0, 360.0, 7.0) + assert np.array_equal( + shifted._threshold(times + offset), aligned._threshold(times) + ) + + +def test_a_unit_past_the_oldest_age_is_replaced_at_the_next_opportunity(): + # The thresholds are worked out up to the age a unit outlives with a + # chance of 1e-9; a unit given as in service past it is past every + # threshold, and goes at the next shut, or on any day now. + shuts = SHUT.optimal_calendar_policy( + Calendar([Opportunity("shut", 100.0, period=180.0)], anytime=None) + ) + oldest = shuts.settings["max_age"] + assert 5000.0 > oldest + assert tuple(shuts.next_replacement(5000.0, now=0.0).values()) == ( + 0.0, + "shut", + 100.0, + ) + assert tuple(shuts.next_replacement(5000.0, now=10.0).values()) == ( + 180.0, + "shut", + 100.0, + ) + assert shuts.next_replacement(oldest, now=10.0).time == 180.0 + any_day = SHUT.optimal_calendar_policy(shut_every_180(0.0)) + assert any_day.next_replacement(5000.0, now=10.0).kind == "anytime" + assert any_day.next_replacement(5000.0, now=10.0).time == pytest.approx( + 10.0, abs=any_day.step + ) + integrated = solve( + SHUT, + Calendar([Opportunity("shut", 100.0, period=180.0)], anytime=None), + method="integration", + ) + assert tuple(integrated.next_replacement(5000.0, now=10.0).values()) == ( + 180.0, + "shut", + 100.0, + ) + + +def test_a_calendar_with_no_planned_replacement_is_refused(): + # As optimal_replacement_policy refuses with no costs: there is no + # policy to find but running to failure. + unplanned = NonRepairable( + surv.Weibull.from_params([365.0, 2.0]), unplanned_cost=400.0 + ) + with pytest.raises(ValueError, match="no planned replacement"): + unplanned.optimal_calendar_policy(Calendar()) + with pytest.raises(ValueError, match="anytime is None"): + SHUT.optimal_calendar_policy(Calendar(anytime=None)) + with pytest.raises(ValueError, match="no planned replacement"): + SHUT.optimal_calendar_policy( + Calendar(anytime=None), method="integration" + ) + # An opportunity, or a price on any day, is a planned replacement. + assert ( + unplanned.optimal_calendar_policy( + Calendar([Opportunity("shut", 100.0, period=180.0)]) + ).replacement_rates["shut"] + > 0.0 + ) + assert ( + SHUT.optimal_calendar_policy(Calendar()).replacement_rates["anytime"] + > 0.0 + ) + + +def test_the_policy_carries_the_life_it_was_worked_out_for(): + from repyability.rbd.serialisation import serialise_model + + policy = SHUT.optimal_calendar_policy(shut_every_180(0.0)) + assert policy.life == serialise_model(SHUT.reliability) + assert json.loads(json.dumps(policy.to_dict()))["life"] == policy.life + + +# Phase 1, test 1: with no opportunities and a planned replacement on any +# day at the planned price, it is age replacement. + + +def test_it_collapses_to_age_replacement(): + unit = NonRepairable( + surv.Weibull.from_params([1000, 2.5]), + planned_cost=1.0, + unplanned_cost=5.0, + ) + classical = unit.optimal_replacement_policy() + policy = solve(unit, Calendar(), step=1.0) + assert policy.method == "dp" and policy.basis == "exact" + assert abs(policy._threshold(10.0) - classical.interval) <= 1.0 + assert policy.cost_rate == pytest.approx(classical.cost_rate, rel=1e-5) + + +# Test 2: relative value iteration agrees with differencing a long finite +# horizon over whole cycles. + + +def test_the_cycle_rate_is_the_long_horizons(): + ages = _replacement._Ages(BEARING, 1.0, None) + grid = _Grid(PLANT, 4.0, 1.0) + costs, _ = grid.prices(0, grid.cycle) + cycle = _replacement._relative_values(costs, ages, 20.0) + record, _, _, _ = _replacement._window(grid, 0, 40 * 364, ages, 20.0) + differenced = _replacement._differenced(record, 364, 11 * 364, 1.0) + assert differenced == pytest.approx(cycle.rate, rel=1e-6) + assert cycle.low <= cycle.rate <= cycle.high + + +# Test 3: the policy simulated plainly costs g a unit of time. + + +def simulated(policy, calendar, step, years, n=4000, seed=0): + """The mean cost a day of ``policy`` over ``years`` cycles of the + plant, after five to settle, each unit's life drawn whole and a failed + unit replaced at once; and its standard error.""" + rng = np.random.default_rng(seed) + days = int(364 * (years + 5) / step) + burn = int(364 * 5 / step) + times = np.arange(days) * step + prices = calendar.planned_cost(times, step=step) + limits = policy._threshold(times) + scale, shape = 1000.0, 2.5 + age = np.zeros(n) + life = scale * rng.weibull(shape, n) + cost = np.zeros(n) + for d in range(days): + counted = d >= burn + # A planned replacement at the start of the step. + if np.isfinite(prices[d]): + replaced = age >= limits[d] - 1e-9 + if counted: + cost[replaced] += prices[d] + age[replaced] = 0.0 + life[replaced] = scale * rng.weibull(shape, replaced.sum()) + # Failures within the step, each replaced at once. + end = age + step + failing = life < end + while failing.any(): + if counted: + cost[failing] += calendar.failure + age[failing] = age[failing] - life[failing] + life[failing] = scale * rng.weibull(shape, failing.sum()) + failing = life < end + end = age + step + failing &= life < end + age += step + rate = cost / (364 * years) + return rate.mean(), rate.std() / math.sqrt(n) + + +def test_the_simulated_policy_costs_its_rate(plant): + mean, error = simulated(plant, PLANT, 1.0, 6) + assert abs(mean - plant.cost_rate) < 2 * error + + +# What the policy does, by kind: its rates and costs. + + +def test_the_costs_by_kind_sum_to_the_rate(plant): + assert sum(plant.cost_rates.values()) == pytest.approx( + plant.cost_rate, rel=1e-9 + ) + for kind, rate in plant.replacement_rates.items(): + price = {"failure": 20.0, "anytime": 4.0}.get(kind) + if price is None: + price = {o.name: o.cost for o in PLANT.opportunities}[kind] + assert plant.cost_rates[kind] == pytest.approx(rate * price) + # At these prices the annual shut does all the planned work. + year = {k: v * 364 for k, v in plant.replacement_rates.items()} + assert year["annual"] == pytest.approx(0.9619, abs=1e-4) + assert year["failure"] == pytest.approx(0.0783, abs=1e-4) + assert year["weekly"] == year["quarterly"] == year["anytime"] == 0.0 + + +def counted(policy, calendar, years=20, n=4000, seed=1): + """Each kind's replacements a day, simulated plainly as ``simulated`` + does, after five years to settle.""" + rng = np.random.default_rng(seed) + times = np.arange(364 * (years + 5), dtype=float) + burn = 364 * 5 + prices = calendar.planned_cost(times, step=1.0) + limits = policy._threshold(times) + names = {o.cost: o.name for o in calendar.opportunities} + names[calendar.anytime] = "anytime" + age, life = np.zeros(n), 1000.0 * rng.weibull(2.5, n) + count = dict.fromkeys(list(names.values()) + ["failure"], 0.0) + for d, t in enumerate(times): + replaced = age >= limits[d] - 1e-9 + if d >= burn and np.isfinite(prices[d]): + count[names[prices[d]]] += replaced.sum() + age[replaced] = 0.0 + life[replaced] = 1000.0 * rng.weibull(2.5, replaced.sum()) + failing = life < age + 1.0 + while failing.any(): + if d >= burn: + count["failure"] += failing.sum() + age[failing] -= life[failing] + life[failing] = 1000.0 * rng.weibull(2.5, failing.sum()) + failing = life < age + 1.0 + age += 1.0 + return {k: v / (n * 364 * years) for k, v in count.items()} + + +def test_the_rates_by_kind_are_simulated(): + calendar = Calendar( + [ + Opportunity("quarterly", 2.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=60.0, + anytime=4.0, + ) + policy = solve(BEARING, calendar, step=1.0) + simulated_rates = counted(policy, calendar) + for kind, rate in policy.replacement_rates.items(): + assert simulated_rates[kind] == pytest.approx(rate, rel=0.03, abs=2e-6) + # Dearer failures: the quarterly shuts are used too. + assert policy.replacement_rates["quarterly"] > 0.0 + + +def test_the_schedule_says_which_shuts_are_used(plant): + by_kind = {} + for _, kind, _, chance in plant.schedule(): + by_kind.setdefault(kind, []).append(chance) + # The long-run chance of a replacement at each shut: the annual + # shut's is its rate over the year; the others are never used. + assert by_kind["annual"] == [ + pytest.approx(plant.replacement_rates["annual"] * 364, rel=1e-9) + ] + assert max(by_kind["quarterly"] + by_kind["weekly"]) == 0.0 + + +def test_the_next_replacement_is_checked(plant): + with pytest.raises(ValueError, match="age"): + plant.next_replacement(-1.0) + with pytest.raises(ValueError, match="age"): + plant.next_replacement(True) + with pytest.raises(ValueError, match="now"): + plant.next_replacement(10.0, now=math.inf) + + +# Test 4: the phase effect, and the dearer day never used. + + +def test_a_shut_shortly_before_the_annual_waits_longer(plant): + quarterly = {t: a for t, _, a, _ in plant.schedule(kinds=("quarterly",))} + # The quarterly shut a quarter before the annual waits for it longer + # than the one three quarters before it. + assert quarterly[273.0] > quarterly[91.0] + weekly = [a for t, _, a, _ in plant.schedule(kinds=("weekly",))] + assert weekly[-1] > weekly[0] + + +def test_the_dearer_day_is_never_used(plant): + # Any day at 4, a weekly stop at 3 within a week: no unit the policy + # keeps reaches the age the dearer day would replace it at. The + # oldest is one the annual shut kept, a year on. + annual = plant._threshold(0.0) + oldest = annual + 364.0 + days = np.arange(364.0) + kinds = PLANT.planned_cost(days, step=1.0) + anytime = days[kinds == 4.0] + assert np.all(plant._threshold(anytime) > oldest) + + +# Test 5: skipping the annual shut for a young unit. + + +def test_a_young_unit_skips_the_annual_shut(plant): + # 60 days old at the annual shut: kept to the next; 300: replaced. + kept = plant.next_replacement(age=60.0, now=364.0) + assert (kept.time, kept.kind, kept.price) == (728.0, "annual", 1.0) + replaced = plant.next_replacement(age=300.0, now=364.0) + assert (replaced.time, replaced.kind) == (364.0, "annual") + # Just put in at the shut: not replaced again there. + assert plant.next_replacement(age=0.0, now=364.0).time == 728.0 + # Put in on day 200 after a failure: 164 days old at the next shut. + assert plant.next_replacement(age=100.0, now=300.0).time == 364.0 + + +# Test 6: the policy does not depend on the horizon, but near its end. + + +def test_the_policy_does_not_depend_on_the_horizon(): + ages = _replacement._Ages(BEARING, 1.0, None) + grid = _Grid(PLANT, 4.0, 1.0) + third = slice(2 * 364, 3 * 364) + + def year_three(years): + _, first, _, _ = _replacement._window(grid, 0, years * 364, ages, 20.0) + return first[third] + + long = year_three(20) + assert np.array_equal(year_three(10), long) + assert np.array_equal(year_three(40), long) + # Five years: the end reaches back into year three, by a few days. + short = year_three(5) + finite = (short >= 0) & (long >= 0) + assert np.abs(short[finite] - long[finite]).max() <= 10 + + +# Test 7: it beats the simple policies, over ten years of the plant. + +TEN_YEARS = 3640.0 + + +def test_it_beats_the_simple_policies(plant): + def ten_years(rule): + return price(BEARING, PLANT, rule, step=1.0) * TEN_YEARS + + age = NonRepairable( + LIFE, planned_cost=4.0, unplanned_cost=20.0 + ).optimal_replacement_policy() + optimal = plant.cost_rate * TEN_YEARS + every_annual = ten_years({"annual": 0.0}) + any_day = ten_years({"anytime": age.interval}) + run_to_failure = ten_years({}) + assert optimal == pytest.approx(25.27, abs=0.01) + assert every_annual == pytest.approx(25.48, abs=0.01) + assert any_day == pytest.approx(50.41, abs=0.01) + assert run_to_failure == pytest.approx(20.0 / 887.3 * TEN_YEARS, rel=1e-3) + assert optimal < every_annual < any_day < run_to_failure + # The optimal policy itself, as a rule, costs its rate. + rule = {"annual": plant._threshold(0.0)} + assert ten_years(rule) >= optimal + + +def test_running_to_failure_costs_the_failures(): + rate = price(BEARING, PLANT, {}, step=1.0) + assert rate == pytest.approx(20.0 / LIFE.mean(), rel=1e-5) + + +# Test 8: a life whose replacing ages are not all those from the youngest. + + +def test_a_set_that_is_no_threshold_is_flagged(): + unit = NonRepairable(surv.LogNormal.from_params([6.0, 1.0])) + calendar = Calendar([Opportunity("monthly", 1.0, period=30.0)], 20.0) + with pytest.warns(UserWarning, match="youngest"): + policy = solve(unit, calendar, step=1.0, max_age=20000.0) + assert not policy.threshold_only + assert np.isfinite(policy._threshold(0.0)) + + +def test_a_threshold_is_not_flagged(plant): + assert plant.threshold_only + + +# Plans of given dates. + +PLAN = Calendar( + [ + Opportunity("weekly", 5.0, period=7.0), + Opportunity("turnaround", 1.0, dates=[400.0, 1130.0, 1860.0]), + ], + failure=40.0, + anytime=None, +) + + +def test_a_repeated_plan_is_the_periodic_calendar(): + plan = Calendar( + [Opportunity("t", 1.0, dates=[364.0, 728.0, 1092.0])], 20.0, None + ) + annual = Calendar([Opportunity("t", 1.0, period=364.0)], 20.0, None) + dated = solve(BEARING, plan, step=1.0) + periodic = solve(BEARING, annual, step=1.0) + assert dated.cost_rate == pytest.approx(periodic.cost_rate, rel=1e-12) + assert dated.settings["repeat"] == 1092.0 + # Repeated forward only: nothing at 0, before the plan. + assert dated._threshold(0.0) == math.inf + assert dated._threshold(364.0) == periodic._threshold(364.0) + + +def test_a_plan_says_until_when_its_tail_does_not_matter(): + repeat = solve(BEARING, PLAN, step=1.0) + anytime = solve(BEARING, PLAN, step=1.0, tail="anytime") + assert repeat.determined_until == anytime.determined_until + until = repeat.determined_until + assert 0.0 < until <= PLAN.plan_end + before = np.arange(0.0, until) + assert np.array_equal( + repeat._threshold(before), anytime._threshold(before) + ) + assert repeat._threshold(until) != anytime._threshold(until) + assert repeat.cost_rate == anytime.cost_rate + assert [t for t, _, _, _ in repeat.schedule(kinds=("turnaround",))] == [ + 400.0, + 1130.0, + 1860.0, + ] + + +# The policy result. + + +def test_the_policy_reads_and_goes_through_json(plant): + assert isinstance(plant, CalendarPolicy) + assert plant["cost_rate"] == plant.cost_rate + assert plant.period == 364.0 + with pytest.raises(AttributeError): + plant.cost_rate = 0.0 + plain = json.loads(json.dumps(plant.to_dict())) + assert plain["method"] == "dp" and plain["basis"] == "exact" + thresholds = plant._threshold(np.array([0.0, 7.0, 3.0])) + assert thresholds.shape == (3,) + assert thresholds[0] == plant._threshold(0.0) + # The thresholds repeat with the calendar. + assert plant._threshold(364.0 * 7 + 91.0) == plant._threshold(91.0) + + +def test_the_schedule(plant): + shuts = plant.schedule() + assert len(shuts) == 52 + assert shuts[0][:3] == (0.0, "annual", plant._threshold(0.0)) + days = plant.schedule(kinds=("anytime",)) + assert len(days) == 364 - 52 + with pytest.raises(ValueError, match="Unknown"): + plant.schedule(kinds=("monthly",)) + with pytest.raises(ValueError, match="until"): + plant.schedule(until=-1.0) + + +def test_the_answer_is_the_same_in_hours(): + # The default step is sized from the life and the calendar, never in + # absolute time. + days = BEARING.optimal_calendar_policy(PLANT) + hourly = NonRepairable(surv.Weibull.from_params([24000.0, 2.5])) + hours = hourly.optimal_calendar_policy( + Calendar( + [ + Opportunity("weekly", 3.0, period=168.0), + Opportunity("quarterly", 2.0, period=91 * 24.0), + Opportunity("annual", 1.0, period=364 * 24.0), + ], + failure=20.0, + anytime=4.0, + ) + ) + assert hours.step == pytest.approx(days.step * 24, rel=1e-12) + assert hours.cost_rate == pytest.approx(days.cost_rate / 24, rel=1e-9) + assert hours._threshold(91 * 24.0) == pytest.approx( + days._threshold(91.0) * 24, rel=1e-9 + ) + + +@pytest.mark.parametrize( + "kwargs, error, match", + [ + ({"method": "fast"}, ValueError, "method"), + ({"tail": "loop"}, ValueError, "tail"), + ({"step": 0.0}, ValueError, "step"), + ({"step": "1"}, ValueError, "step"), + ({"max_age": -1.0}, ValueError, "max_age"), + ({"method": "sampling", "mc_samples": 1}, ValueError, "mc_samples"), + ], +) +def test_the_arguments_are_checked(kwargs, error, match): + with pytest.raises(error, match=match): + solve(BEARING, PLANT, **kwargs) + with pytest.raises(TypeError, match="Calendar"): + solve(BEARING, [Opportunity("a", 1.0, period=7.0)]) + + +def test_a_rule_is_checked(): + with pytest.raises(ValueError, match="Unknown kind"): + price(BEARING, PLANT, {"monthly": 0.0}, step=7.0) + with pytest.raises(ValueError, match="at least 0"): + price(BEARING, PLANT, {"annual": -1.0}, step=7.0) + with pytest.raises(TypeError, match="dict"): + price(BEARING, PLANT, [0.0], step=7.0) + shut_only = Calendar(PLANT.opportunities, 20.0, None) + with pytest.raises(ValueError, match="any day"): + price(BEARING, shut_only, {"anytime": 1.0}, step=7.0) + with warnings.catch_warnings(): + warnings.simplefilter("error") + assert price( + BEARING, shut_only, {"annual": math.inf}, step=7.0 + ) == pytest.approx(20.0 / LIFE.mean(), rel=1e-4) diff --git a/repyability/tests/test_maintenance_calendar_methods.py b/repyability/tests/test_maintenance_calendar_methods.py new file mode 100644 index 00000000..9d23da27 --- /dev/null +++ b/repyability/tests/test_maintenance_calendar_methods.py @@ -0,0 +1,239 @@ +"""The calendar policy's other methods (#319, UC-13): integration between +opportunities in continuous time (phase 2, the brief's section 4), +sampling for a long cycle (phase 3, section 5), and how ``method="auto"`` +chooses.""" + +import math +import warnings + +import numpy as np +import pytest +import surpyval as surv + +from repyability import Calendar, NonRepairable, Opportunity +from repyability.maintenance import _replacement + + +def solve(part, calendar, **options): + """The calendar policy, on a grid of a given step where a check needs + one (the public method sizes its own).""" + return _replacement.optimal_calendar_policy(part, calendar, **options) + + +def price(part, calendar, rule, **options): + """A rule's cost rate, likewise.""" + return _replacement.calendar_policy_cost(part, calendar, rule, **options) + + +LIFE = surv.Weibull.from_params([1000.0, 2.5]) +BEARING = NonRepairable(LIFE) + +# Shuts only (no replacement on any day): quarterly at 3, annual at 1, a +# failure at 20. +SHUTS = Calendar( + [ + Opportunity("quarterly", 3.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, + anytime=None, +) + +# Weekly stops at 5 too, and a failure at 40: every kind is used. +NESTED = Calendar( + [ + Opportunity("weekly", 5.0, period=7.0), + Opportunity("quarterly", 3.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=40.0, + anytime=None, +) + +# Co-prime periods: the calendar repeats only after 338,780,400 days. +COPRIME = Calendar( + [ + Opportunity("weekly", 6.0, period=7.0, offset=3.0), + Opportunity("p93", 4.0, period=93.0, offset=10.0), + Opportunity("p400", 2.0, period=400.0, offset=50.0), + Opportunity("p1301", 1.0, period=1301.0, offset=200.0), + ], + failure=40.0, + anytime=None, +) + + +@pytest.fixture(scope="module") +def nested_exact(): + return solve(BEARING, NESTED, step=1.0) + + +# Phase 2, test 1: it agrees with the grid, ever closer as the grid's step +# shrinks. + + +def test_integration_agrees_with_the_grid(): + integrated = solve(BEARING, NESTED, method="integration") + assert integrated.method == "integration" and integrated.step is None + gaps = {} + for step in (7.0, 1.0): + grid = solve(BEARING, NESTED, step=step) + gaps[step] = abs(grid.cost_rate - integrated.cost_rate) + assert grid.cost_rate == pytest.approx(integrated.cost_rate, rel=1e-3) + assert gaps[1.0] < gaps[7.0] + grid = solve(BEARING, NESTED, step=1.0) + for time, kind, age, _ in integrated.schedule( + kinds=("quarterly", "annual") + ): + assert abs(grid._threshold(time) - age) <= 2.0, (time, kind) + + +# Test 2: opportunities every day at the planned price are age replacement. + + +def test_daily_opportunities_are_age_replacement(): + unit = NonRepairable(LIFE, planned_cost=1.0, unplanned_cost=5.0) + classical = unit.optimal_replacement_policy() + daily = Calendar([Opportunity("daily", 1.0, period=1.0)], anytime=None) + policy = solve(unit, daily, method="integration") + # Deciding once a day costs a hair more than at any moment. + assert policy.cost_rate == pytest.approx(classical.cost_rate, rel=1e-6) + assert policy.cost_rate >= classical.cost_rate * (1 - 1e-9) + assert abs(policy._threshold(0.0) - classical.interval) <= 1.0 + + +# Test 3: it refuses a replacement on any day. + + +def test_integration_refuses_a_replacement_on_any_day(): + calendar = Calendar(SHUTS.opportunities, failure=20.0, anytime=4.0) + with pytest.raises(NotImplementedError, match="method='dp'"): + solve(BEARING, calendar, method="integration") + + +# Test 4: long gaps, an annual shut and a life of about three years. + + +def test_long_gaps_need_few_nodes(): + unit = NonRepairable(surv.Weibull.from_params([1100.0, 3.0])) + annual = Calendar([Opportunity("annual", 1.0, period=365.0)], 10.0, None) + integrated = solve(unit, annual, method="integration") + grid = solve(unit, annual, step=1.0) + assert integrated.settings["nodes"] < 365 + assert integrated.cost_rate == pytest.approx(grid.cost_rate, rel=1e-4) + assert abs(integrated._threshold(0.0) - grid._threshold(0.0)) <= 1.0 + + +def test_integration_repeats_a_plan(): + plan = Calendar( + [Opportunity("t", 1.0, dates=[364.0, 728.0, 1092.0])], 20.0, None + ) + annual = Calendar([Opportunity("t", 1.0, period=364.0)], 20.0, None) + dated = solve(BEARING, plan, method="integration") + periodic = solve(BEARING, annual, method="integration") + assert dated.cost_rate == pytest.approx(periodic.cost_rate, rel=1e-9) + assert dated._threshold(0.0) == math.inf + assert dated._threshold(364.0) == pytest.approx(periodic._threshold(0.0)) + with pytest.raises(NotImplementedError, match="method='dp'"): + solve(BEARING, plan, method="integration", tail="anytime") + + +def test_integration_refuses_a_calendar_with_no_planned_replacement(): + # No opportunities and none on any day: nothing to find but running + # to failure, refused as the other methods refuse it. + with pytest.raises(ValueError, match="no planned replacement"): + solve( + BEARING, Calendar(failure=20.0, anytime=None), method="integration" + ) + + +# Phase 3, test 1: sampling reproduces the exact rate. + + +def test_sampling_reproduces_the_exact_rate(nested_exact): + sampled = solve( + BEARING, NESTED, step=1.0, method="sampling", mc_samples=32, seed=1 + ) + assert sampled.basis == "sampled" + assert abs(sampled.cost_rate - nested_exact.cost_rate) < ( + 2 * sampled.cost_rate_se + ) + fewer = solve( + BEARING, NESTED, step=1.0, method="sampling", mc_samples=8, seed=1 + ) + # The error falls as one over the root of the samples. + assert 1.2 < fewer.cost_rate_se / sampled.cost_rate_se < 3.5 + + +# Test 2: a co-prime calendar, in seconds a sample. + + +def test_a_coprime_calendar_is_sampled(monkeypatch): + policy = solve(BEARING, COPRIME, step=1.0, mc_samples=8, seed=3) + assert policy.method == "sampling" + assert np.isfinite(policy.cost_rate) and policy.cost_rate_se > 0.0 + assert policy.period is None + # A window twice as long changes the rate by less than its error. + monkeypatch.setattr( + _replacement, "SETTLE_SPANS", 2 * _replacement.SETTLE_SPANS + ) + longer = solve(BEARING, COPRIME, step=1.0, mc_samples=8, seed=3) + assert abs(longer.cost_rate - policy.cost_rate) < policy.cost_rate_se + + +# Test 3: the policy itself is never sampled. + + +def test_the_sampled_policy_is_deterministic(): + one = solve(BEARING, COPRIME, step=1.0, mc_samples=2, seed=1) + other = solve(BEARING, COPRIME, step=1.0, mc_samples=2, seed=2) + times = np.array([10.0, 50.0, 200.0, 1450.0]) + assert np.array_equal(one._threshold(times), other._threshold(times)) + assert np.array_equal(one._threshold(times), one._threshold(times)) + shuts = one.schedule(until=500.0, kinds=("p400", "p1301")) + assert [(t, k) for t, k, _, _ in shuts] == [ + (50.0, "p400"), + (200.0, "p1301"), + (450.0, "p400"), + ] + for time, _, age, _ in shuts: + assert one._threshold(time) == age + with pytest.raises(ValueError, match="until"): + one.schedule() + + +# Test 4: which method "auto" takes. + + +def test_auto_chooses_the_method(nested_exact): + assert nested_exact.method == "dp" and nested_exact.basis == "exact" + # Falling to sampling is said: the cycle is too long, and mc_samples + # sets the cost. + with pytest.warns(UserWarning, match="mc_samples") as record: + sampled = solve(BEARING, COPRIME, step=1.0, mc_samples=2, seed=0) + assert sampled.method == "sampling" + assert f"more than {_replacement.MAX_CYCLE:,}" in str(record[0].message) + with warnings.catch_warnings(): + warnings.simplefilter("error") + asked = solve( + BEARING, COPRIME, step=1.0, method="sampling", mc_samples=2, seed=0 + ) + assert asked.cost_rate == sampled.cost_rate + plan = Calendar([Opportunity("t", 1.0, dates=[400.0, 1130.0])], 20.0, None) + assert solve(BEARING, plan, step=10.0).method == "dp" + # Never integration, which is asked for. + assert all( + solve(BEARING, c, step=step, mc_samples=2).method != "integration" + for c, step in ((SHUTS, 7.0), (plan, 10.0)) + ) + with pytest.raises(NotImplementedError, match="method='dp'"): + solve(BEARING, plan, method="sampling") + + +def test_dp_on_a_long_cycle_differences_a_long_horizon(): + policy = solve(BEARING, COPRIME, step=1.0, method="dp") + assert policy.basis == "differenced" + sampled = solve(BEARING, COPRIME, step=1.0, mc_samples=16, seed=5) + assert abs(policy.cost_rate - sampled.cost_rate) < 4 * sampled.cost_rate_se + times = np.array([10.0, 200.0, 450.0]) + assert np.array_equal(policy._threshold(times), sampled._threshold(times)) diff --git a/repyability/tests/test_maintenance_calendar_rbd.py b/repyability/tests/test_maintenance_calendar_rbd.py new file mode 100644 index 00000000..abfe1c85 --- /dev/null +++ b/repyability/tests/test_maintenance_calendar_rbd.py @@ -0,0 +1,510 @@ +"""A ``RepairableRBD`` component replaced on a calendar policy (#319, +UC-13, the brief's deliverable 6): a ``"preventive"`` spec with +``"policy": "calendar"`` and the ``CalendarPolicy`` of +``NonRepairable.optimal_calendar_policy``, simulated by the Python event +loop, each replacement charged at its opportunity's price.""" + +import copy +import math + +import numpy as np +import pytest +import surpyval as surv + +from repyability import ( + Calendar, + NodeState, + NonRepairable, + Opportunity, + RepairableRBD, +) +from repyability.maintenance import _replacement + + +def solve(part, calendar, **options): + """The calendar policy, on a grid of a given step where a check needs + one (the public method sizes its own).""" + return _replacement.optimal_calendar_policy(part, calendar, **options) + + +def price(part, calendar, rule, **options): + """A rule's cost rate, likewise.""" + return _replacement.calendar_policy_cost(part, calendar, rule, **options) + + +LIFE = surv.Weibull.from_params([1000.0, 2.5]) +PLANT = Calendar( + [ + Opportunity("weekly", 3.0, period=7.0), + Opportunity("quarterly", 2.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, + anytime=4.0, +) +SHUTS = Calendar( + [ + Opportunity("quarterly", 3.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, + anytime=None, +) + + +@pytest.fixture(scope="module") +def plant(): + return solve(NonRepairable(LIFE), PLANT, step=1.0) + + +def bearing(policy, **more): + spec = { + "reliability": LIFE, + "repairability": "instant", + "replace_cost": 20.0, + "preventive": {"policy": "calendar", "calendar": policy}, + } + return RepairableRBD([("s", "b"), ("b", "t")], {"b": {**spec, **more}}) + + +@pytest.mark.parametrize("method", ["dp", "integration"]) +def test_a_unit_is_next_replaced_where_its_threshold_is_reached(method): + policy = solve(NonRepairable(LIFE), SHUTS, step=1.0, method=method) + # New at 0: the next annual shut. Put in at an annual shut (300 days + # on, it is young at the next): the one after. Old: at once. + annual = 1.0, "annual" + assert policy._next_planned(0.0, 0.0) == (364.0, *annual) + assert policy._next_planned(300.0, 300.0) == (728.0, *annual) + assert policy._next_planned(-500.0, 0.0) == (0.0, *annual) + assert policy._next_planned(100.0, 150.0) == (364.0, *annual) + + +def test_the_diagram_costs_the_policys_rate(plant): + # The brief's test 3 against the real engine: over 40 years, the mean + # cost a day is the policy's rate (from new, a few units' worth of + # start-up, well inside the error). + rbd = bearing(plant) + years = 40 + result = rbd.cost( + t_simulation=364.0 * years, mc_samples=300, seed=2, engine="python" + ) + days = 364.0 * years + assert abs(result.mean / days - plant.cost_rate) < 3 * ( + result.mean_se / days + ) + + +def test_the_compiled_engine_leaves_it_to_python(plant): + rbd = bearing(plant) + route = rbd.analysis_routes()["availability"] + assert route.engine == "python" + assert "calendar policy" in route.engine_reason + with pytest.raises(NotImplementedError, match="calendar policy"): + rbd.availability(t_simulation=100.0, mc_samples=2, engine="numba") + + +def test_the_exact_methods_refuse_it(plant): + rbd = bearing(plant) + with pytest.raises(NotImplementedError, match="calendar policy"): + rbd.expected_cost_rate() + with pytest.raises(NotImplementedError, match="calendar policy"): + rbd.mean_availability() + assert rbd.has_costs + + +def test_a_calendar_policy_is_not_saved(plant): + with pytest.raises(NotImplementedError, match="calendar"): + bearing(plant).to_json() + + +def test_its_times_are_absolute(plant): + rbd = bearing(plant) + with pytest.raises(ValueError, match="no since_scheduled"): + rbd.availability( + t_simulation=100.0, + mc_samples=2, + state={"b": NodeState(age=10.0, since_scheduled=5.0)}, + ) + # An age is taken: an old unit is replaced at the first chance. + rbd.availability( + t_simulation=100.0, mc_samples=2, state={"b": NodeState(age=900.0)} + ) + + +@pytest.mark.parametrize( + "preventive, match", + [ + ({"policy": "calendar"}, "needs a calendar"), + ({"policy": "calendar", "calendar": "weekly"}, "needs a calendar"), + ({"policy": "calendar", "interval": 30.0, "calendar": 0}, "interval"), + ({"policy": "calendar", "cost": 3.0, "calendar": 0}, "no cost"), + ({"interval": 30.0, "calendar": 0}, "only to the 'calendar'"), + ( + {"policy": "calendar", "opportunity": 3.0, "calendar": 0}, + "no maintenance group", + ), + ( + {"policy": "calendar", "opportunity": -1.0, "calendar": 0}, + "an age at least 0", + ), + ], +) +def test_the_spec_is_checked(plant, preventive, match): + # 0 stands for the policy itself. + if preventive.get("calendar") == 0: + preventive = {**preventive, "calendar": plant} + with pytest.raises(ValueError, match=match): + RepairableRBD( + [("s", "b"), ("b", "t")], + { + "b": { + "reliability": LIFE, + "repairability": "instant", + "preventive": preventive, + } + }, + ) + + +def test_a_policy_worked_out_for_another_life_is_refused(plant): + # The policy carries the saved form of the life it was worked out + # for: a component with another life is refused, naming both. + other = surv.Weibull.from_params([365.0, 2.0]) + with pytest.raises(ValueError, match=r"Weibull \[1000.0, 2.5\]") as found: + bearing(plant, reliability=other) + assert "Weibull [365.0, 2.0]" in str(found.value) + assert "another life" in str(found.value) + # The same life, given again, passes; so does a policy with no saved + # form (a life that does not save). + bearing(plant, reliability=surv.Weibull.from_params([1000.0, 2.5])) + assert unsaved(plant).life is None + bearing(unsaved(plant), reliability=other) + + +def test_a_failure_cost_is_the_diagrams(plant): + # The calendar's failure price prices the policy; the diagram charges + # its own replace_cost at a failure, as for any component. + free = bearing(plant, replace_cost=0.0) + priced = bearing(plant) + options = dict(t_simulation=3640.0, mc_samples=50, seed=4) + assert free.cost(**options).mean < priced.cost(**options).mean + assert math.isfinite(free.cost(**options).mean) + + +# -- in a maintenance group (#349) -------------------------------------------- +# +# A calendar component is a member like any other: its planned +# replacements are stops of its group, and a stop renews it early from its +# opportunity age, charged the calendar's price for a planned replacement +# on the day (its replace_cost where the calendar allows none), the new +# unit's next replacement counted from then. The policy is not +# re-optimised for the group's stops. + +X = surv.ExactEventTime.from_params +SERIES = [("s", "a"), ("a", "b"), ("b", "t")] + + +@pytest.fixture(scope="module") +def shuts(): + """The policy over SHUTS (no replacement on any day): a new unit is + replaced at the annual shut, from 88 days old there.""" + return solve(NonRepairable(LIFE), SHUTS, step=1.0) + + +def states(rbd, until): + """The system's changes of state, stepped through one by one.""" + rbd.initialize_event_queue(until) + changes = [rbd.next_event()] + while changes[-1][0] < until: + changes.append(rbd.next_event()) + return changes + + +def unsaved(policy): + """The policy with no saved life (as of a life that does not save), so + that the timelines below may give it to a unit that never fails in + the window, not the bearing it was worked out for.""" + policy = copy.copy(policy) + object.__setattr__(policy, "life", None) + return policy + + +def members(policy, opportunity, life=300.0, repair_cost=100.0): + """A train's members: "a" fails at ``life`` (back 5 later), each + failure a stop of the train; "b", never failing in the window, is on + ``policy`` and renewed early from ``opportunity`` old, at 20 a + failure.""" + return { + "a": { + "reliability": X(life), + "repairability": X(5.0), + "repair_cost": repair_cost, + "group": "train", + }, + "b": { + "reliability": X(1000.0), + "repairability": X(5.0), + "replace_cost": 20.0, + "preventive": { + "policy": "calendar", + "calendar": unsaved(policy), + "opportunity": opportunity, + }, + "group": "train", + }, + } + + +def grouped(policy, opportunity, **more): + """The train of ``members`` in series, set up at 500 a stop.""" + return RepairableRBD( + SERIES, + members(policy, opportunity, **more), + maintenance_groups={"train": {"setup_cost": 500.0}}, + ) + + +def test_a_calendar_replacement_is_a_stop_of_its_group(shuts): + # "a", never failing in the window, is replaced on its calendar at the + # annual shuts, 364 and 728, at 1 each; "b", never failing, replaced + # at 1000 old on its own (taking 2) and from 20 old at a stop, is + # renewed at each, down 2: one set-up a stop. + assert shuts._next_planned(0.0, 0.0) == (364.0, 1.0, "annual") + rbd = RepairableRBD( + SERIES, + { + "a": { + "reliability": X(1000.0), + "repairability": X(5.0), + "replace_cost": 20.0, + "preventive": { + "policy": "calendar", + "calendar": unsaved(shuts), + }, + "group": "train", + }, + "b": { + "reliability": X(1000.0), + "repairability": X(5.0), + "preventive": { + "interval": 1000.0, + "opportunity": 20.0, + "duration": X(2.0), + "cost": 20.0, + }, + "group": "train", + }, + }, + maintenance_groups={"train": {"setup_cost": 500.0}}, + ) + assert states(rbd, 800.0) == [ + (364.0, False), + (366.0, True), + (728.0, False), + (730.0, True), + (800.0, True), + ] + runs = rbd.simulate_timelines(800.0, mc_samples=1, seed=1) + assert list(runs.components["a"][0].changes) == [] + assert list(runs.components["b"][0].changes) == [ + 364.0, + 366.0, + 728.0, + 730.0, + ] + assert list(runs.system[0].causes) == ["b", "b", "b", "b"] + result = rbd.availability( + 800.0, mc_samples=2, seed=1, control_variate=False + ) + assert result.mean_availability_interval().estimate == pytest.approx( + 796.0 / 800.0 + ) + # Renewals summed over the two simulations; costs a simulation. + assert result.opportunistic_renewals == {"a": 0, "b": 4} + assert result.cost.by_category == { + "repair": 0.0, + "replace": 0.0, + "preventive": 2 * 1.0 + 2 * 20.0, + "inspection": 0.0, + "component_downtime": 0.0, + "system_downtime": 0.0, + "setup": 2 * 500.0, + } + + +def test_a_stop_renews_the_calendar_component_from_its_opportunity_age( + shuts, +): + # "a" fails at 300 and 605. Alone, "b" is replaced at the annual shuts, + # 364 (364 old) and 728. From 250 old at a stop, it is renewed at 300 + # and 605 (300 and 305 old), days the shuts price no replacement on, + # so each is charged its replace_cost; and its next planned + # replacement counts from its new start: not 364 (64 old, under the + # threshold) but 728. From 320 old it is renewed at neither. + assert shuts._planned_price(300.0) == math.inf + assert shuts._next_planned(300.0, 300.0)[0] == 728.0 + assert shuts._next_planned(605.0, 605.0)[0] == 728.0 + for opportunity, renewed, preventive, stops in ( + (250.0, 2, 20.0 + 20.0 + 1.0, 3), + (320.0, 0, 1.0 + 1.0, 4), + ): + rbd = grouped(shuts, opportunity) + assert states(rbd, 800.0) == [ + (300.0, False), + (305.0, True), + (605.0, False), + (610.0, True), + (800.0, True), + ] + result = rbd.availability( + 800.0, mc_samples=2, seed=1, control_variate=False + ) + assert result.opportunistic_renewals == {"a": 0, "b": 2 * renewed} + assert result.cost.by_category["preventive"] == preventive + assert result.cost.by_category["repair"] == 2 * 100.0 + assert result.cost.by_category["setup"] == stops * 500.0 + + +def test_an_early_renewal_is_charged_the_calendars_price_on_the_day(plant): + # "a" fails every 15 (at 10, 25, 40, ...) and "b", from 0 old, is + # renewed at each stop: at the plant's price on the day, a weekly shut + # (70, 175) at 3, any other day at 4. Never 20, its failure price. + rbd = grouped(plant, 0.0, life=10.0, repair_cost=0.0) + stops = np.arange(10.0, 200.0, 15.0) + prices = PLANT.planned_cost(stops, step=plant.step) + assert set(prices) == {3.0, 4.0} and prices[4] == 3.0 + result = rbd.availability( + 200.0, mc_samples=2, seed=1, control_variate=False + ) + assert result.opportunistic_renewals == {"a": 0, "b": 2 * stops.size} + assert result.cost.by_category["preventive"] == prices.sum() + assert result.cost.by_category["setup"] == stops.size * 500.0 + + +def test_the_grouped_rule_against_a_plain_simulation(): + # Two instant units in a train with a set-up of 50 a stop: "a" + # (Weibull, 50 by 3) repaired at 100, replaced at 60 old for 20, and + # from 30 old at a stop; "b" (Weibull, 100 by 2.5) replaced at 10 at + # a failure, on a policy over shuts every 70 at 2 (4 on any day), and + # from 40 old at a stop, at the calendar's price on the day. The rules + # written out plainly, with draws of their own, give the same mean + # cost and early renewals. + a_life = surv.Weibull.from_params([50.0, 3.0]) + b_life = surv.Weibull.from_params([100.0, 2.5]) + calendar = Calendar( + [Opportunity("shut", 2.0, period=70.0)], failure=10.0, anytime=4.0 + ) + policy = NonRepairable(b_life).optimal_calendar_policy(calendar) + rbd = RepairableRBD( + SERIES, + { + "a": { + "reliability": a_life, + "repairability": "instant", + "repair_cost": 100.0, + "preventive": { + "interval": 60.0, + "cost": 20.0, + "opportunity": 30.0, + }, + "group": "train", + }, + "b": { + "reliability": b_life, + "repairability": "instant", + "replace_cost": 10.0, + "preventive": { + "policy": "calendar", + "calendar": policy, + "opportunity": 40.0, + }, + "group": "train", + }, + }, + maintenance_groups={"train": {"setup_cost": 50.0}}, + ) + horizon, n = 2000.0, 200 + + def plain(rng): + put, life = {}, {} + + def renew(node, t): + put[node] = t + life[node] = ( + 50.0 * rng.weibull(3.0) + if node == "a" + else 100.0 * rng.weibull(2.5) + ) + + renew("a", 0.0) + renew("b", 0.0) + cost, early = 0.0, {"a": 0, "b": 0} + while True: + failure = {node: put[node] + life[node] for node in "ab"} + next_b = policy.next_replacement(0.0, put["b"]) + planned = {"a": put["a"] + 60.0, "b": next_b.time} + t, node = min((min(failure[n], planned[n]), n) for n in "ab") + if t >= horizon: + return cost, early + cost += 50.0 + failed = failure[node] <= planned[node] + if node == "a": + cost += 100.0 if failed else 20.0 + renew("a", t) + if t - put["b"] >= 40.0: + cost += calendar.planned_cost([t], step=policy.step)[0] + renew("b", t) + early["b"] += 1 + else: + cost += 10.0 if failed else next_b.price + renew("b", t) + if t - put["a"] >= 30.0: + cost += 20.0 + renew("a", t) + early["a"] += 1 + + rng = np.random.default_rng(7) + runs = [plain(rng) for _ in range(n)] + costs = np.array([cost for cost, _ in runs]) + result = rbd.availability( + horizon, mc_samples=n, seed=3, control_variate=False + ) + se = math.hypot(result.cost.mean_se, costs.std(ddof=1) / math.sqrt(n)) + assert abs(result.cost.mean - costs.mean()) < 3 * se + for node in "ab": + counts = np.array([early[node] for _, early in runs]) + assert counts.mean() > 5 + se = math.sqrt(2) * counts.std(ddof=1) / math.sqrt(n) + simulated = result.opportunistic_renewals[node] / n + assert abs(simulated - counts.mean()) < 3 * se + + +def test_the_group_is_simulated_whole_in_python(shuts): + rbd = grouped(shuts, 250.0) + route = rbd.analysis_routes()["availability"] + assert route.engine == "python" + assert "maintenance groups" in route.engine_reason + with pytest.raises(NotImplementedError, match="maintenance groups"): + rbd.availability(t_simulation=100.0, mc_samples=2, engine="numba") + # A conditional run simulates its modules alone: a calendar member + # goes with its group, here every component. + with pytest.raises(NotImplementedError, match="every component"): + rbd.availability(t_simulation=100.0, mc_samples=2, conditional=True) + three = RepairableRBD( + [("s", "a"), ("a", "b"), ("b", "c"), ("c", "t")], + { + **members(shuts, 250.0), + "c": {"reliability": LIFE, "repairability": "instant"}, + }, + maintenance_groups={"train": {"setup_cost": 500.0}}, + ) + assert three._conditional_modules(set(), set()) == ["a", "b"] + with pytest.raises(NotImplementedError, match="held"): + three._conditional_modules({"a"}, set()) + for method in (rbd.expected_cost_rate, rbd.mean_availability): + with pytest.raises(NotImplementedError, match="calendar policy"): + method() + with pytest.raises(NotImplementedError, match="calendar policy"): + rbd.spares_demand(100.0) + for name in ("expected_cost_rate", "mean_availability", "spares_demand"): + assert rbd.analysis_routes()[name].route == "refused" diff --git a/repyability/tests/test_maintenance_intervals.py b/repyability/tests/test_maintenance_intervals.py index da634fde..9909d146 100644 --- a/repyability/tests/test_maintenance_intervals.py +++ b/repyability/tests/test_maintenance_intervals.py @@ -16,6 +16,7 @@ import surpyval as surv from repyability import NonRepairable, RepairableRBD +from repyability.rbd import _intervals W = surv.Weibull.from_params E = surv.Exponential.from_params @@ -53,7 +54,9 @@ def grid_best(plant, feasible=lambda rbd: True): for a, b in itertools.product( np.arange(500.0, 700.0, 10.0), np.arange(420.0, 700.0, 10.0) ): - rbd = plant._with_intervals(preventive={"a": a, "b1": b, "b2": b}) + rbd = _intervals._with_intervals( + plant, preventive={"a": a, "b1": b, "b2": b} + ) if feasible(rbd): best = min(best, (rbd.expected_cost_rate(), (a, b))) return best @@ -96,7 +99,7 @@ def test_the_cheapest_intervals(plant): # standby: its replacements stop the plant too. assert plan.intervals["a"] > plan.intervals["b1"] + 50.0 assert plan.intervals["b1"] == pytest.approx(plan.intervals["b2"]) - again = plant._with_intervals(preventive=plan.intervals) + again = _intervals._with_intervals(plant, preventive=plan.intervals) assert again.expected_cost_rate() == pytest.approx(plan.cost_rate) assert again.mean_availability() == pytest.approx(plan.availability) @@ -131,7 +134,9 @@ def test_a_cost_cap(plant): best = max( rbd.mean_availability() for rbd in ( - plant._with_intervals(preventive={"a": a, "b1": b, "b2": b}) + _intervals._with_intervals( + plant, preventive={"a": a, "b1": b, "b2": b} + ) for a, b in itertools.product( np.arange(560.0, 660.0, 10.0), np.arange(460.0, 660.0, 10.0) ) @@ -161,7 +166,9 @@ def test_a_memoryless_unit_is_never_replaced(): def test_only_the_named_components_change(plant): plan = plant.optimal_replacement_intervals(nodes=["a"]) assert set(plan.intervals) == {"a"} - fixed = plant._with_intervals(preventive={"a": plan.intervals["a"]}) + fixed = _intervals._with_intervals( + plant, preventive={"a": plan.intervals["a"]} + ) assert fixed.expected_cost_rate() == pytest.approx(plan.cost_rate) @@ -205,7 +212,7 @@ def calendar_best(plant, allowed, feasible=lambda rbd: True): best = (math.inf, None) for combination in itertools.product(allowed, repeat=3): intervals = dict(zip(["a", "b1", "b2"], combination)) - rbd = plant._with_intervals(preventive=intervals) + rbd = _intervals._with_intervals(plant, preventive=intervals) if feasible(rbd): best = min(best, (rbd.expected_cost_rate(), combination)) return best @@ -337,7 +344,7 @@ def test_every_combination_is_tried(): ) best = (math.inf, None) for a, b in itertools.product(CALENDAR, CALENDAR): - design = rbd._with_intervals(inspection={"v1": a, "v2": b}) + design = _intervals._with_intervals(rbd, inspection={"v1": a, "v2": b}) if design.mean_availability() >= target: best = min(best, (design.expected_cost_rate(), (a, b))) assert (plan.intervals["v1"], plan.intervals["v2"]) == best[1] @@ -346,7 +353,7 @@ def test_every_combination_is_tried(): def test_the_local_search_on_many_combinations(monkeypatch): # Force the local search on a problem small enough to enumerate. - import repyability.rbd.repairable_rbd as module + import repyability.rbd._intervals as module rbd = two_valves() exhaustive = rbd.optimal_inspection_intervals( @@ -371,7 +378,9 @@ def test_a_cost_cap_on_the_tests(): ) assert plan.cost_rate <= 0.1 for a, b in itertools.product(CALENDAR, CALENDAR): - design = two_valves()._with_intervals(inspection={"v1": a, "v2": b}) + design = _intervals._with_intervals( + two_valves(), inspection={"v1": a, "v2": b} + ) if design.expected_cost_rate() <= 0.1: assert design.mean_availability() <= plan.availability + 1e-15 diff --git a/repyability/tests/test_maintenance_next.py b/repyability/tests/test_maintenance_next.py new file mode 100644 index 00000000..b21ff747 --- /dev/null +++ b/repyability/tests/test_maintenance_next.py @@ -0,0 +1,377 @@ +"""When to replace a part in service next +(``NonRepairable.optimal_next_replacement``; #319, UC-12 of +``repyability/maintenance/README.md``).""" + +import json +import math + +import numpy as np +import pytest +import surpyval as surv + +from repyability import ( + Calendar, + MaintenancePolicy, + NextReplacement, + NonRepairable, + Opportunity, +) + +LIFE = surv.Weibull.from_params([365.0, 2.0]) +BEARING = NonRepairable(LIFE, planned_cost=150.0, unplanned_cost=400.0) +OUTAGES = [(150.0, 100.0), (330.0, 100.0)] +# The shuts that keep coming: every 180 days, at $100 (#348). +SHUTS = Calendar([Opportunity("shut", 100.0, period=180.0)]) + + +@pytest.fixture(scope="module") +def uc12(): + return BEARING.optimal_next_replacement(200.0, scheduled_outages=OUTAGES) + + +@pytest.fixture(scope="module") +def shuts_policy(): + return BEARING.optimal_calendar_policy(SHUTS) + + +def by_kind(found): + return { + (choice.kind, choice.time if choice.kind == "outage" else None): ( + choice + ) + for choice in found.choices + } + + +def test_the_bearing_at_200_days(uc12): + # UC-12: valued at the baseline, the replacement policy's cost rate. + assert isinstance(uc12, NextReplacement) + assert uc12.cost_rate == BEARING.optimal_replacement_policy().cost_rate + assert uc12.cost_rate == pytest.approx(1.117, abs=5e-4) + choices = by_kind(uc12) + assert choices["now", None].cost == 150.0 + assert choices["outage", 150.0].cost == pytest.approx(110.0, abs=0.05) + assert choices["outage", 330.0].cost == pytest.approx(156.5, abs=0.05) + assert choices["planned", None].cost == pytest.approx(133.9, abs=0.05) + assert choices["planned", None].time == pytest.approx(97.6, abs=0.05) + assert choices["failure", None].cost == pytest.approx(186.1, abs=0.05) + assert not uc12.now + assert uc12.best is choices["outage", 150.0] + evens = {key: choice.break_even_age for key, choice in choices.items()} + assert evens["outage", 150.0] == pytest.approx(288.4, abs=0.05) + assert evens["outage", 330.0] == pytest.approx(189.3, abs=0.05) + assert evens["planned", None] == pytest.approx(297.6, abs=0.05) + assert evens["failure", None] == pytest.approx(133.3, abs=0.05) + assert uc12.replace_from_age == evens["planned", None] + + +@pytest.mark.parametrize("age", [0.0, 100.0, 200.0, 290.0]) +def test_with_no_outages_it_is_the_age_replacement(age): + # Valued at the age replacement's own cost rate, the best planned + # replacement of a part is at the optimal age, and replacing now is the + # best from that age. + policy = BEARING.optimal_replacement_policy() + found = BEARING.optimal_next_replacement(age) + assert found.cost_rate == policy.cost_rate + planned = by_kind(found)["planned", None] + assert planned.time == pytest.approx(policy.interval - age, abs=1e-3) + assert found.replace_from_age == pytest.approx(policy.interval, rel=1e-4) + assert found.best is planned + older = BEARING.optimal_next_replacement(policy.interval + 10.0) + assert older.now + + +def test_the_age_replacement_policy_given_is_the_default(uc12): + policy = BEARING.optimal_replacement_policy() + assert isinstance(policy, MaintenancePolicy) + given = BEARING.optimal_next_replacement( + 200.0, scheduled_outages=OUTAGES, policy=policy + ) + assert given.to_dict() == uc12.to_dict() + + +def test_a_calendar_with_no_opportunities_gives_the_default_baseline(uc12): + # A calendar with no opportunities, replacement on any day at the + # planned price and a failure at the unplanned one, is age + # replacement. Its policy's cost rate is exact for its grid (a step + # of about the mean life over 1,000: 0.32 days here), within about + # the square of the step of the continuous one, so the baselines + # agree to 1e-6 relative, and with them the costs and break-even + # ages to 1e-3 (they agree to 1e-4). + policy = BEARING.optimal_calendar_policy(Calendar()) + found = BEARING.optimal_next_replacement( + 200.0, scheduled_outages=OUTAGES, policy=policy + ) + assert found.cost_rate == policy.cost_rate + assert found.cost_rate == pytest.approx(uc12.cost_rate, rel=1e-6) + assert found.best.kind == uc12.best.kind + assert found.best.time == uc12.best.time + assert found.replace_from_age == pytest.approx( + uc12.replace_from_age, abs=1e-3 + ) + for mine, theirs in zip(found.choices, uc12.choices): + assert (mine.kind, mine.price) == (theirs.kind, theirs.price) + assert mine.time == pytest.approx(theirs.time, abs=1e-3) + assert mine.cost == pytest.approx(theirs.cost, abs=1e-3) + if theirs.break_even_age is None: + assert mine.break_even_age is None + else: + assert mine.break_even_age == pytest.approx( + theirs.break_even_age, abs=1e-3 + ) + + +def test_the_shuts_policy_keeps_the_bearing_longer(uc12, shuts_policy): + # UC-12 with the shuts' policy as the baseline (#348): a day of the + # bearing's service is worth 0.990 rather than 1.117, so keeping it is + # dearer and replacing now beats each later choice from a younger + # age; the outage in 150 days from 256.1 days old, not 288.4. + found = BEARING.optimal_next_replacement( + 200.0, scheduled_outages=OUTAGES, policy=shuts_policy + ) + assert found.cost_rate == shuts_policy.cost_rate + assert found.cost_rate == pytest.approx(0.990, abs=5e-4) + choices = by_kind(found) + assert choices["now", None].cost == 150.0 + assert choices["outage", 150.0].cost == pytest.approx(124.6, abs=0.05) + assert choices["outage", 330.0].cost == pytest.approx(178.5, abs=0.05) + assert choices["planned", None].cost == pytest.approx(142.8, abs=0.05) + assert choices["planned", None].time == pytest.approx(63.9, abs=0.05) + assert choices["failure", None].cost == pytest.approx(210.4, abs=0.05) + assert not found.now + assert found.best is choices["outage", 150.0] + evens = {key: choice.break_even_age for key, choice in choices.items()} + assert evens["outage", 150.0] == pytest.approx(256.13, abs=0.05) + assert evens["outage", 330.0] == pytest.approx(152.95, abs=0.05) + assert evens["planned", None] == pytest.approx(263.86, abs=0.05) + assert evens["failure", None] == pytest.approx(86.35, abs=0.05) + assert found.replace_from_age == evens["planned", None] + # Every later choice costs more, and is beaten from a younger age. + for key, theirs in by_kind(uc12).items(): + if key != ("now", None): + assert choices[key].cost > theirs.cost + 1.0 + assert evens[key] < theirs.break_even_age - 1.0 + # A break-even age is where the choice costs what replacing now does. + at_even = BEARING.optimal_next_replacement( + evens["outage", 150.0], scheduled_outages=OUTAGES, policy=shuts_policy + ) + assert by_kind(at_even)["outage", 150.0].cost == pytest.approx( + 150.0, abs=1e-6 + ) + assert by_kind(at_even)["outage", 150.0].break_even_age == pytest.approx( + evens["outage", 150.0], abs=1e-6 + ) + + +def simulated(age, until, price, rate, n=400_000, seed=0): + """A choice's cost simulated plainly: keep the Weibull(365, 2) part from + ``age`` to ``until`` from now, then replace it at ``price``, or at 400 + if it fails first, less the time it serves at ``rate``; the mean and its + standard error.""" + rng = np.random.default_rng(seed) + held = (age / 365.0) ** 2 + left = 365.0 * np.sqrt(held - np.log(rng.random(n))) - age + cost = np.where(left > until, price, 400.0) + cost = cost - rate * np.minimum(left, until) + return cost.mean(), cost.std() / math.sqrt(n) + + +@pytest.mark.parametrize("baseline", ["age replacement", "shuts"]) +@pytest.mark.parametrize("age", [50.0, 250.0]) +def test_each_choice_is_its_simulated_cost(age, baseline, shuts_policy): + policy = shuts_policy if baseline == "shuts" else None + found = BEARING.optimal_next_replacement( + age, scheduled_outages=OUTAGES, policy=policy + ) + for choice in found.choices[1:]: + mean, error = simulated( + age, choice.time, choice.price, found.cost_rate + ) + assert abs(choice.cost - mean) < 4 * error, choice + + +def test_an_exponential_life_runs_to_failure(): + # Memoryless: keeping to u costs price * exp(-rate u), running to + # failure nothing, and replacing now never pays at any age. + rate = 1 / 200.0 + unit = NonRepairable( + surv.Exponential.from_params([rate]), + planned_cost=150.0, + unplanned_cost=400.0, + ) + found = unit.optimal_next_replacement(300.0, scheduled_outages=OUTAGES) + assert found.cost_rate == pytest.approx(400.0 * rate, rel=1e-12) + for time, price in OUTAGES: + assert by_kind(found)["outage", time].cost == pytest.approx( + price * math.exp(-rate * time), rel=1e-6 + ) + assert found.best.kind == "failure" + assert found.best.cost == pytest.approx(0.0, abs=1e-6) + assert by_kind(found)["planned", None].time == math.inf + assert found.replace_from_age == math.inf + + +def planned_at_once(found): + """The planned choice, as replacing now: at time 0, at now_cost.""" + planned = by_kind(found)["planned", None] + now = by_kind(found)["now", None] + return (planned.time, planned.price, planned.cost) == ( + 0.0, + now.price, + now.cost, + ) + + +@pytest.mark.parametrize("now_cost", [500.0, 100.0]) +def test_a_planned_replacement_at_once_is_replacing_now(now_cost): + # A part past its optimal age (297.6): no later time beats replacing + # it at once, which is replacing now, at now_cost, whatever the + # planned price. Dearer than the planned price: with no outage to + # wait for, running to failure (254.5) is the best, replacing now is + # not, and no age makes it so; cheaper: replacing now is the best. + found = BEARING.optimal_next_replacement(400.0, now_cost=now_cost) + assert planned_at_once(found) + assert found.now == (now_cost < 150.0) + assert found.best.time == (0.0 if found.now else math.inf) + choices = by_kind(found) + assert choices["failure", None].cost == pytest.approx(254.5, abs=0.05) + assert choices["planned", None].break_even_age == pytest.approx( + 297.6 if now_cost > 150.0 else 122.7, abs=0.05 + ) + if now_cost > 150.0: + assert found.best is choices["failure", None] + assert found.replace_from_age == math.inf + else: + assert found.best is choices["now", None] + assert found.replace_from_age == pytest.approx(122.7, abs=0.05) + # A younger part: a planned replacement later, at the planned price, + # whatever replacing now costs. + young = BEARING.optimal_next_replacement(100.0, now_cost=now_cost) + assert not planned_at_once(young) + planned = by_kind(young)["planned", None] + assert planned.price == 150.0 and planned.time == pytest.approx( + 197.6, abs=0.05 + ) + assert young.now is (young.best.time == 0.0) + + +def test_now_is_true_exactly_when_the_best_choice_is_now(): + # Dearer now than planned, far past the optimal age: replacing now + # (200) beats running to failure (329.8), and is the best. + found = BEARING.optimal_next_replacement(1000.0, now_cost=200.0) + assert planned_at_once(found) and found.now + assert found.best.kind == "now" and found.best.cost == 200.0 + for age, now_cost in ((100.0, 200.0), (400.0, 500.0), (1000.0, 200.0)): + found = BEARING.optimal_next_replacement(age, now_cost=now_cost) + assert found.now is (found.best.time == 0.0) + + +def test_a_cheap_chance_now_replaces_a_younger_part(): + # Replacing now in an outage today ($100) beats waiting for one in 150 + # days from a younger age than at the planned price. + dear = BEARING.optimal_next_replacement(100.0, scheduled_outages=OUTAGES) + cheap = BEARING.optimal_next_replacement( + 100.0, scheduled_outages=OUTAGES, now_cost=100.0 + ) + assert ( + by_kind(cheap)["outage", 150.0].break_even_age + < by_kind(dear)["outage", 150.0].break_even_age + ) + + +def test_the_outages_come_in_time_order(): + found = BEARING.optimal_next_replacement( + 100.0, scheduled_outages=OUTAGES[::-1] + ) + assert [c.time for c in found.choices if c.kind == "outage"] == [ + 150.0, + 330.0, + ] + + +def test_the_answer_is_the_same_in_hours(): + days = BEARING.optimal_next_replacement(200.0, scheduled_outages=OUTAGES) + hourly = NonRepairable( + surv.Weibull.from_params([365.0 * 24, 2.0]), + planned_cost=150.0, + unplanned_cost=400.0, + ) + hours = hourly.optimal_next_replacement( + 200.0 * 24, scheduled_outages=[(t * 24, p) for t, p in OUTAGES] + ) + assert hours.cost_rate == pytest.approx(days.cost_rate / 24, rel=1e-6) + assert hours.replace_from_age == pytest.approx( + days.replace_from_age * 24, rel=1e-6 + ) + for day, hour in zip(days.choices, hours.choices): + assert hour.cost == pytest.approx(day.cost, rel=1e-6) + assert hour.time == pytest.approx(day.time * 24, rel=1e-6) + + +def test_it_goes_through_json(): + found = BEARING.optimal_next_replacement(200.0, scheduled_outages=OUTAGES) + plain = json.loads(json.dumps(found.to_dict())) + assert plain["best"]["kind"] == "outage" + assert plain["choices"][0]["break_even_age"] is None + assert len(plain["choices"]) == 5 + + +@pytest.mark.parametrize("age", [-1.0, math.inf, True, "200"]) +def test_an_age_is_at_least_0(age): + with pytest.raises(ValueError, match="age"): + BEARING.optimal_next_replacement(age) + + +@pytest.mark.parametrize( + "outages, error, match", + [ + ("150", TypeError, "pairs"), + ([(150.0,)], TypeError, "pair"), + ([(0.0, 100.0)], ValueError, "time"), + ([("150", 100.0)], ValueError, "time"), + ([(150.0, -1.0)], ValueError, "price"), + ([(150.0, math.inf)], ValueError, "price"), + ], +) +def test_the_outages_are_checked(outages, error, match): + with pytest.raises(error, match=match): + BEARING.optimal_next_replacement(100.0, scheduled_outages=outages) + + +def test_the_other_inputs_are_checked(): + with pytest.raises(TypeError): + BEARING.optimal_next_replacement(100.0, shuts=OUTAGES) + with pytest.raises(ValueError, match="now_cost"): + BEARING.optimal_next_replacement(100.0, now_cost=-1.0) + with pytest.raises(ValueError, match="costs not set"): + NonRepairable(LIFE, unplanned_cost=400.0).optimal_next_replacement(1.0) + with pytest.raises(TypeError): + BEARING.optimal_next_replacement(100.0, None) + + +@pytest.mark.parametrize( + "policy", [0.99, "0.99", SHUTS, {"cost_rate": 0.99}, (297.6, 1.117)] +) +def test_a_policy_is_a_policy(policy): + with pytest.raises(TypeError, match="policy must be the policy"): + BEARING.optimal_next_replacement(100.0, policy=policy) + + +def test_a_bare_cost_rate_says_to_give_the_policy(): + with pytest.raises(TypeError, match="not its cost rate"): + BEARING.optimal_next_replacement(100.0, policy=0.99) + + +@pytest.mark.parametrize("rate", [-1.0, math.inf, math.nan]) +def test_a_policy_has_a_cost_rate(rate): + with pytest.raises(ValueError, match="cost rate of policy"): + BEARING.optimal_next_replacement( + 100.0, policy=MaintenancePolicy(297.6, rate) + ) + + +def test_a_life_with_no_closed_form_is_refused(): + estimate = surv.KaplanMeier.fit(LIFE.random(50)) + unit = NonRepairable(estimate, planned_cost=150.0, unplanned_cost=400.0) + with pytest.raises(NotImplementedError, match="closed form"): + unit.optimal_next_replacement(100.0) diff --git a/repyability/tests/test_maintenance_skip.py b/repyability/tests/test_maintenance_skip.py new file mode 100644 index 00000000..849221aa --- /dev/null +++ b/repyability/tests/test_maintenance_skip.py @@ -0,0 +1,394 @@ +"""Whether to skip the next planned replacement right after a failure +(``NonRepairable.skip_next_replacement``; #319, UC-6 of +``repyability/maintenance/README.md``).""" + +import json +import math +import warnings + +import numpy as np +import pytest +import surpyval as surv +from scipy.integrate import quad + +from repyability import NonRepairable, SkipValue + +LIFE = surv.Weibull.from_params([365.0, 2.0]) +BEARING = NonRepairable(LIFE, planned_cost=150.0, unplanned_cost=400.0) +YEAR = 365.25 + + +def per_day(rate): + """The continuous rate a day of a rate ``rate`` a year.""" + return math.log1p(rate) / YEAR + + +def weibull_lives(rng, size): + return 365.0 * np.sqrt(-np.log(rng.random(size))) + + +def simulated_fixed(days, every, planned, unplanned, rate, n, seed): + """The NPV of skipping on a plan on fixed times, simulated plainly: + done, a new unit at ``days`` and its failures to ``days + every``; + skipped, the failures of the unit in now (and its replacements) from + ``days`` to ``days + every``. The rest is the same both ways.""" + rng, r = np.random.default_rng(seed), per_day(rate) + + def failures(start, keep_from): + time, value = np.full(n, start), np.zeros(n) + while True: + time = time + weibull_lives(rng, n) + hit = time < days + every + if not hit.any(): + return value + counted = hit & (time >= keep_from) + value[counted] += unplanned * np.exp(-r * time[counted]) + time = np.where(hit, time, np.inf) + + done = planned * math.exp(-r * days) + failures(days, days) + skipped = failures(0.0, days) + difference = done - skipped + return difference.mean(), difference.std() / math.sqrt(n) + + +def simulated_resetting(every, planned, unplanned, rate, n, seed, years): + """The present values of each choice on a plan that resets, simulated + plainly over ``years``, the first cycle cut off at ``every`` (done) or + ``2 every`` (skipped): their means, and the error of their + difference.""" + rng, r = np.random.default_rng(seed), per_day(rate) + + def value(first): + time, total = np.zeros(n), np.zeros(n) + cut = np.full(n, first) + live = np.ones(n, dtype=bool) + while live.any(): + life = weibull_lives(rng, live.sum()) + failed = life < cut[live] + time[live] += np.minimum(life, cut[live]) + costs = np.where(failed, unplanned, planned) + total[live] += costs * np.exp(-r * time[live]) + cut[live] = every + live = time < years * YEAR + return total + + done, skipped = value(every), value(2.0 * every) + error = math.sqrt((done.var() + skipped.var()) / n) + return done.mean(), skipped.mean(), error + + +def test_the_bearing_on_a_plan_that_resets(): + # UC-6: every 180 days, 150 planned, 400 at a failure, 8% a year. + value = BEARING.skip_next_replacement( + 180, resets=True, discount_rate=per_day(0.08) + ) + assert isinstance(value, SkipValue) + assert value.time_to_next == 180.0 + assert value.discount_rate == per_day(0.08) + assert value.npv == pytest.approx(26.558, abs=1e-3) + assert value.npv == value.replace - value.skip + assert value.best == "skip" + done, skipped, error = simulated_resetting( + 180.0, 150.0, 400.0, 0.08, 40_000, 3, 300 + ) + assert abs(value.npv - (done - skipped)) < 4 * error + # Each is the present value of the whole future (300 years of it + # simulated: what is left is discounted away). + assert value.replace == pytest.approx(done, rel=2e-3) + assert value.skip == pytest.approx(skipped, rel=2e-3) + + +@pytest.mark.parametrize("days", [1.0, 30.0, 120.0, 180.0]) +def test_the_bearing_on_a_plan_on_fixed_times(days): + value = BEARING.skip_next_replacement( + 180, resets=False, time_to_next=days, discount_rate=per_day(0.08) + ) + assert value.time_to_next == days + assert value.npv == value.replace - value.skip + mean, error = simulated_fixed( + days, 180.0, 150.0, 400.0, 0.08, 300_000, int(days) + ) + assert abs(value.npv - mean) < 4 * error + + +def test_the_fixed_answer_by_the_time_to_the_next_replacement(): + # On a 180-day plan, too frequent for the bearing (the best age is + # near 298 days), skipping always pays, the more the sooner the next + # replacement; on a 365-day plan it stops paying some 200 to 300 days + # out. + r = per_day(0.08) + + def npv(every, days): + return BEARING.skip_next_replacement( + every, resets=False, time_to_next=days, discount_rate=r + ) + + values = [npv(180, d).npv for d in (1, 30, 90, 180)] + assert values == sorted(values, reverse=True) + assert values[0] == pytest.approx(149.06, abs=0.01) + assert values[-1] == pytest.approx(35.26, abs=0.01) + assert npv(365, 10).npv > 0 > npv(365, 300).npv + assert npv(365, 10).best == "skip" + assert npv(365, 300).best == "replace" + + +def test_the_fixed_costs_are_those_to_the_following_replacement(): + # Done: the replacement at d, the new unit's failures to d + T, and + # the failures before d; skipped: the failures to d + T. Each from + # the part's failures over a window (UC-5, discounted by hand), the + # failures after d from a new unit of the same life. + r, every, days = per_day(0.08), 180.0, 30.0 + value = BEARING.skip_next_replacement( + every, resets=False, time_to_next=days, discount_rate=r + ) + from repyability import RepairableRBD + + unit = RepairableRBD( + [("s", "u"), ("u", "t")], + { + "u": { + "reliability": LIFE, + "repairability": "instant", + "replace_cost": 400.0, + } + }, + ) + failures = unit.expected_cost( + np.array([every, days, days + every]), discount_rate=r + ).mean + later = math.exp(-r * days) + assert value.replace == pytest.approx( + 150.0 * later + later * failures[0] + failures[1], rel=1e-12 + ) + assert value.skip == pytest.approx(failures[2], rel=1e-12) + + +def exponential_closed_forms(rate, every, planned, unplanned, r, days): + """The two plans' NPVs for an exponential life, written out: its + failures come at ``rate`` whatever its age.""" + total = r + rate + + def length(cut): + return -math.expm1(-total * cut) / total + + def cost(cut): + failed = rate * length(cut) + return unplanned * failed + planned * math.exp(-total * cut) + + once, twice = length(every), length(2 * every) + resetting = ( + cost(every) - cost(2 * every) + cost(every) * (twice - once) / once + ) + + def failures(t): + # A new unit and its replacements fail at ``rate`` throughout. + return unplanned * rate * (-math.expm1(-r * t) / r if r else t) + + later = math.exp(-r * days) + fixed = ( + planned * later + + later * failures(every) + - (failures(days + every) - failures(days)) + ) + return resetting, fixed + + +@pytest.mark.parametrize("yearly", [0.0, 0.08, 0.5]) +def test_an_exponential_life_against_its_closed_forms(yearly): + rate, every, days = 1 / 200.0, 90.0, 40.0 + part = NonRepairable( + surv.Exponential.from_params([rate]), + planned_cost=150.0, + unplanned_cost=400.0, + ) + resetting, fixed = exponential_closed_forms( + rate, every, 150.0, 400.0, per_day(yearly), days + ) + r = per_day(yearly) + value = part.skip_next_replacement(every, resets=True, discount_rate=r) + assert value.npv == pytest.approx(resetting, rel=1e-8) + value = part.skip_next_replacement( + every, resets=False, time_to_next=days, discount_rate=r + ) + assert value.npv == pytest.approx(fixed, rel=1e-6) + # With no ageing, a planned replacement only costs: skipping saves. + assert resetting > 0 and fixed > 0 + + +def test_a_resetting_plan_with_discounting_values_the_whole_future(): + # Done, the plan from new, A(T) / (r M(T)); skipped, the longer first + # cycle and then the plan from new: written out for an exponential + # life, whose cycle values have closed forms. + rate, every, r = 1 / 200.0, 90.0, per_day(0.08) + part = NonRepairable( + surv.Exponential.from_params([rate]), + planned_cost=150.0, + unplanned_cost=400.0, + ) + total = r + rate + + def length(cut): + return -math.expm1(-total * cut) / total + + def cost(cut): + return 400.0 * rate * length(cut) + 150.0 * math.exp(-total * cut) + + whole = cost(every) / (r * length(every)) + skipped = cost(2 * every) + (1 - r * length(2 * every)) * whole + value = part.skip_next_replacement(every, resets=True, discount_rate=r) + assert value.replace == pytest.approx(whole, rel=1e-8) + assert value.skip == pytest.approx(skipped, rel=1e-8) + + +def test_with_no_discounting_a_resetting_plan_is_the_classic_comparison(): + # Undiscounted, skipping lengthens the first cycle: it saves its cost + # difference, less the long-run cost rate over the time it adds. Each + # choice is its first cycle's cost less its length at that rate. + every, planned, unplanned = 180.0, 150.0, 400.0 + + def mean_length(cut): + return quad(lambda x: float(LIFE.sf(x)), 0, cut, epsrel=1e-13)[0] + + def cycle_cost(cut): + kept = float(LIFE.sf(cut)) + return unplanned * (1 - kept) + planned * kept + + rate_of_cost = cycle_cost(every) / mean_length(every) + classic = ( + cycle_cost(every) + - cycle_cost(2 * every) + + rate_of_cost * (mean_length(2 * every) - mean_length(every)) + ) + value = BEARING.skip_next_replacement(every, resets=True) + assert value.discount_rate == 0.0 + assert value.npv == pytest.approx(classic, rel=1e-9) + # Each choice's whole future is endless: only the difference is + # finite (with a rate the same fields hold the present values). + assert value.replace == value.skip == math.inf + assert value.best == "skip" + assert json.loads(json.dumps(value.to_dict()))["npv"] == value.npv + # And a tiny rate is next to it. + tiny = BEARING.skip_next_replacement( + every, resets=True, discount_rate=1e-9 + ) + assert tiny.npv == pytest.approx(classic, rel=1e-6) + + +@pytest.mark.parametrize("resets", [True, False]) +def test_a_rate_that_discounts_the_costs_away_is_warned_of(resets): + # A rate a year given with a life in days (#231): both plans warn as + # a RepairableRBD's costs do, naming the life's mean. + with pytest.warns(UserWarning, match="shortest mean life") as record: + value = BEARING.skip_next_replacement( + 180.0, + resets=resets, + time_to_next=None if resets else 30.0, + discount_rate=0.08, + ) + assert "exp(-25.9)" in str(record[0].message) + assert math.isfinite(value.npv) + with warnings.catch_warnings(): + warnings.simplefilter("error") + BEARING.skip_next_replacement( + 180.0, + resets=resets, + time_to_next=None if resets else 30.0, + discount_rate=per_day(0.08), + ) + + +def test_the_rate_is_a_unit_of_the_life_and_the_answer_the_same_in_hours(): + # The same bearing with its life in hours, and the rate an hour. + in_hours = NonRepairable( + surv.Weibull.from_params([365.0 * 24.0, 2.0]), + planned_cost=150.0, + unplanned_cost=400.0, + ) + r = per_day(0.08) + for resets, time_to_next in ((True, None), (False, 30.0)): + days = BEARING.skip_next_replacement( + 180.0, resets=resets, time_to_next=time_to_next, discount_rate=r + ) + hours = in_hours.skip_next_replacement( + 180.0 * 24.0, + resets=resets, + time_to_next=None if time_to_next is None else 24.0 * time_to_next, + discount_rate=r / 24.0, + ) + assert hours.npv == pytest.approx(days.npv, rel=1e-6) + assert hours.replace == pytest.approx(days.replace, rel=1e-6) + assert hours.skip == pytest.approx(days.skip, rel=1e-6) + + +def test_the_value_goes_through_json(): + value = BEARING.skip_next_replacement( + 180, resets=False, time_to_next=30, discount_rate=per_day(0.08) + ) + plain = json.loads(json.dumps(value.to_dict())) + assert plain["resets"] is False + assert plain["time_to_next"] == 30.0 + assert plain["npv"] == pytest.approx(123.08, abs=0.01) + assert plain["best"] == "skip" + assert set(plain) == { + "resets", + "every", + "time_to_next", + "discount_rate", + "replace", + "skip", + "npv", + "best", + } + + +def test_the_time_to_the_next_replacement_goes_with_a_fixed_plan_only(): + with pytest.raises(ValueError, match="give no time_to_next"): + BEARING.skip_next_replacement(180, resets=True, time_to_next=30) + with pytest.raises(ValueError, match="needs time_to_next"): + BEARING.skip_next_replacement(180, resets=False) + with pytest.raises(ValueError, match="at most every"): + BEARING.skip_next_replacement(180, resets=False, time_to_next=181) + for time_to_next in (0, -5, math.inf, True, "30"): + with pytest.raises(ValueError, match="time_to_next"): + BEARING.skip_next_replacement( + 180, resets=False, time_to_next=time_to_next + ) + + +@pytest.mark.parametrize("rate", [-0.01, math.inf, math.nan, True, "8%"]) +def test_a_discount_rate_is_a_rate_at_least_0(rate): + with pytest.raises(ValueError, match="discount_rate"): + BEARING.skip_next_replacement(180, resets=True, discount_rate=rate) + + +@pytest.mark.parametrize("every", [0, -1, math.inf, True, "180"]) +def test_every_is_a_time(every): + with pytest.raises(ValueError, match="every"): + BEARING.skip_next_replacement(every, resets=True) + + +def test_resets_is_true_or_false(): + with pytest.raises(TypeError, match="resets"): + BEARING.skip_next_replacement(180, resets=1) + + +def test_the_costs_are_the_part_s_own(): + with pytest.raises(ValueError, match="costs not set"): + NonRepairable(LIFE).skip_next_replacement(180, resets=True) + with pytest.raises(ValueError, match="costs not set"): + NonRepairable(LIFE, planned_cost=150.0).skip_next_replacement( + 180, resets=True + ) + part = NonRepairable(LIFE) + part.set_costs_planned_and_unplanned(150.0, 400.0) + value = part.skip_next_replacement( + 180, resets=True, discount_rate=per_day(0.08) + ) + assert value.npv == pytest.approx(26.558, abs=1e-3) + + +def test_a_life_with_no_closed_form_is_refused(): + estimate = surv.KaplanMeier.fit(LIFE.random(50)) + part = NonRepairable(estimate, planned_cost=150.0, unplanned_cost=400.0) + with pytest.raises(NotImplementedError, match="closed form"): + part.skip_next_replacement(180, resets=True) diff --git a/repyability/tests/test_minimal_repair_timed.py b/repyability/tests/test_minimal_repair_timed.py new file mode 100644 index 00000000..b0b79d58 --- /dev/null +++ b/repyability/tests/test_minimal_repair_timed.py @@ -0,0 +1,450 @@ +"""Minimal repair whose repairs take time (#318): a component repaired +minimally (Kijima, ``q = 1``) fails at its life's hazard at its operating +age, the time it has been up, and each failure's repair takes time. Its +availability over time and its expected failures are worked out on a grid +of its operating age (``MinimalRepairCurve``, by +``minimal_repair_lattice``), and the values over a window, the importance +over time and the expected cost follow for any system with it. Its +long-run values stay refused: its rate of failures need not settle.""" + +import math + +import numpy as np +import pytest +import surpyval as surv + +from repyability import RepairableRBD +from repyability.rbd import _curves +from repyability.rbd import routes as r +from repyability.rbd._point_availability import ( + MinimalRepairCurve, + _cells, + _lattice, + cumulative_hazard, +) +from repyability.rbd._point_availability import knots as point_knots +from repyability.rbd._point_availability import minimal_repair_lattice +from repyability.tests import catalogue + +W = surv.Weibull.from_params +E = surv.Exponential.from_params +L = surv.LogNormal.from_params +EDGES = [("s", "c"), ("c", "t")] +MINIMAL = {"model": "kijima1", "q": 1.0} + + +def single(spec): + return RepairableRBD(EDGES, {"c": spec}) + + +def minimal(life, repairs, **more): + """A unit minimally repaired, its repairs taking ``repairs``.""" + return { + "reliability": life, + "repairability": repairs, + "repair": MINIMAL, + **more, + } + + +def closed_forms(rate: float, mu: float, t: np.ndarray): + """An exponential unit (failure rate ``rate``, repair rate ``mu``): + its point availability and expected failures by ``t``, the two-state + Markov chain's.""" + total = rate + mu + up = mu / total + rate / total * np.exp(-total * t) + failures = rate * ( + mu / total * t + rate / total**2 * (1.0 - np.exp(-total * t)) + ) + return up, failures + + +# -- the lattice ------------------------------------------------------------- + + +def test_the_lattice_against_the_two_state_chain(): + # An exponential life keeps no age, so the sum over the powers of an + # exponential repair time is the two-state Markov chain, in closed form. + rate, mu = 0.02, 0.2 + t = 0.05 * np.arange(4001) + life, repair = E([rate]), E([mu]) + lattice = _lattice( + *_cells( + lambda s: 1.0 - np.asarray(repair.sf(s), float), + t, + point_knots(repair), + ) + ) + up, failures = minimal_repair_lattice(cumulative_hazard(life, t), lattice) + exact_up, exact_failures = closed_forms(rate, mu, t) + np.testing.assert_allclose(up, exact_up, atol=1e-6) + np.testing.assert_allclose(failures, exact_failures, atol=1e-6) + + +def test_the_lattice_error_falls_as_the_square_of_the_step(): + rate, mu = 0.02, 0.2 + life, repair = E([rate]), E([mu]) + errors = [] + for n in (500, 1000, 2000): + t = (100.0 / n) * np.arange(n + 1) + lattice = _lattice( + *_cells( + lambda s: 1.0 - np.asarray(repair.sf(s), float), + t, + point_knots(repair), + ) + ) + up, _ = minimal_repair_lattice(cumulative_hazard(life, t), lattice) + errors.append(np.abs(up - closed_forms(rate, mu, t)[0]).max()) + assert errors[0] / errors[1] > 3.5 + assert errors[1] / errors[2] > 3.5 + + +def test_repairs_in_no_time_on_the_lattice_give_the_closed_form(): + # The atom at 0 for every power: the sums are 1 and H(t), which the + # curve takes in closed form. + life = W([100.0, 2.0]) + t = 0.1 * np.arange(2001) + lattice = np.zeros(len(t)) + lattice[0] = 1.0 + up, failures = minimal_repair_lattice(cumulative_hazard(life, t), lattice) + np.testing.assert_allclose(up, 1.0, atol=1e-13) + np.testing.assert_allclose(failures, (t / 100.0) ** 2.0, atol=1e-11) + + +def test_the_curve_keeps_the_interface(): + life, repair = W([100.0, 2.0]), L([2.0, 0.5]) + curve = MinimalRepairCurve( + life, + 100.0, + 1000, + True, + repair_sf=lambda s: np.asarray(repair.sf(s), float), + repair_splits=point_knots(repair), + step=0.1, + n=1000, + ) + assert curve.settle == np.inf and curve.period is None + assert curve.grids() == [(0.1, 100.0)] + np.testing.assert_array_equal(curve.breaks(0.0, 100.0), [0.0, 100.0]) + assert len(curve.knots(0.0, 100.0)) == 1003 + x = np.array([0.0, 0.05, 50.0, 100.0]) + values = curve.at(x) + assert values[0] == 1.0 and np.all(np.diff(values) < 0.0) + events = curve.events(x) + assert events["failures"][0] == 0.0 + np.testing.assert_array_equal(events["corrective"], events["failures"]) + assert np.all(events["preventive"] == 0.0) + assert curve.atoms(100.0).times.size == 0 + + +# -- against the exact cases -------------------------------------------------- + + +def test_repairs_of_no_length_are_minimal_repair_in_no_time(): + # A repair of 1e-8 on average: up throughout, failing H(t) times, as + # the closed form for repairs in no time has it. + life = W([100.0, 2.5]) + timed = single(minimal(life, E([1e8]), repair_cost=1.0)) + instant = single(minimal(life, "instant", repair_cost=1.0)) + t = np.array([10.0, 100.0, 250.0]) + np.testing.assert_allclose(timed.point_availability(t), 1.0, atol=1e-6) + # Linear between its grid's points, where the closed form is convex: + # to 1e-7 or so, as any curve on a grid. + np.testing.assert_allclose( + timed.expected_failures(t), + instant.expected_failures(t), + rtol=1e-6, + atol=1e-6, + ) + np.testing.assert_allclose( + timed.expected_failures(t), + cumulative_hazard(life, t), + rtol=1e-6, + atol=1e-6, + ) + np.testing.assert_allclose( + timed.expected_cost(t).mean, + instant.expected_cost(t).mean, + rtol=1e-6, + atol=1e-6, + ) + + +@pytest.mark.parametrize( + "repair", + [E([0.2]), L([2.0, 0.5]), W([3.0, 3.0])], + ids=["exponential", "lognormal", "weibull"], +) +def test_with_an_exponential_life_minimal_repair_is_renewal(repair): + # A memoryless unit is as good as new after any repair, so its minimal + # repair is the plain renewal unit, worked out by its renewal equation. + life = E([0.02]) + minimally = single(minimal(life, repair, repair_cost=3.0)) + renewed = single( + {"reliability": life, "repairability": repair, "repair_cost": 3.0} + ) + t = np.array([5.0, 25.0, 100.0, 300.0]) + np.testing.assert_allclose( + minimally.point_availability(t), + renewed.point_availability(t), + atol=1e-6, + ) + np.testing.assert_allclose( + minimally.expected_failures(t), renewed.expected_failures(t), atol=1e-6 + ) + np.testing.assert_allclose( + minimally.mission_availability(t), + renewed.mission_availability(t), + atol=1e-6, + ) + np.testing.assert_allclose( + minimally.expected_cost(t).mean, + renewed.expected_cost(t).mean, + rtol=1e-6, + ) + one, other = minimally.expected_events(t), renewed.expected_events(t) + np.testing.assert_allclose( + one.node_downtime["c"], other.node_downtime["c"], atol=1e-5 + ) + + +def test_an_exponential_unit_against_its_two_state_chain(): + rate, mu = 0.02, 0.2 + rbd = single(minimal(E([rate]), E([mu]))) + t = np.array([5.0, 25.0, 100.0, 300.0]) + up, failures = closed_forms(rate, mu, t) + np.testing.assert_allclose(rbd.point_availability(t), up, atol=1e-6) + np.testing.assert_allclose(rbd.expected_failures(t), failures, atol=1e-6) + + +# -- against the simulation -------------------------------------------------- + + +def simulated(rbd, horizon: float, mc_samples: int, seed: int): + """A plain run (its means its own, not the exact methods'): the + system's availability on a grid, its mean, and its failures.""" + return rbd.availability( + horizon, + mc_samples=mc_samples, + seed=seed, + curve_points=10, + control_variate=False, + ) + + +@pytest.mark.parametrize( + "life, repair", + [ + (W([100.0, 2.0]), L([2.0, 0.5])), + (W([100.0, 3.0]), L([2.5, 1.0])), + (W([100.0, 0.8]), E([0.1])), + ], + ids=["shape 2, lognormal", "shape 3, lognormal", "wearing in"], +) +def test_a_timed_unit_against_the_simulation(life, repair): + rbd = single(minimal(life, repair, repair_cost=1.0)) + horizon, n = 300.0, 4000 + sim = simulated(rbd, horizon, n, 2) + # The curve: the fraction up at each grid time, binomial. + grid = sim.timeline[1:-1] + up = rbd.point_availability(grid) + error = np.sqrt(up * (1.0 - up) / n) + assert np.all(np.abs(sim.availability[1:-1] - up) < 4.0 * error + 1e-3) + # The failures: only the repairs are priced, at 1 each. + failures = rbd.expected_failures(horizon) + error = float(np.std(sim.cost.samples)) / math.sqrt(n) + assert abs(sim.cost.mean - failures) < 4.0 * error + assert sim.system_failures / n == pytest.approx(failures, rel=0.05) + # The mean availability over the window. + error = float(np.std(sim.uptimes)) / math.sqrt(n) / horizon + assert ( + abs(sim.sample_mean_availability - rbd.mission_availability(horizon)) + < 4.0 * error + ) + + +def test_a_fitted_crow_amsaa_process_against_the_simulation(): + rbd = catalogue.repairable_kinds()["fitted Crow-AMSAA process"] + horizon, n = 1000.0, 4000 + sim = simulated(rbd, horizon, n, 5) + counted = rbd.expected_events(horizon) + error = float(np.std(sim.cost.samples)) / math.sqrt(n) + assert abs(sim.cost.mean - counted.node_failures["a"]) < 4.0 * error + assert sim.system_failures / n == pytest.approx( + counted.system_failures, rel=0.05 + ) + error = float(np.std(sim.uptimes)) / math.sqrt(n) / horizon + assert ( + abs(sim.sample_mean_availability - rbd.mission_availability(horizon)) + < 4.0 * error + ) + + +def pair(pump, **options): + """A pump minimally repaired and a plain spare, in parallel.""" + return RepairableRBD( + [("s", "pump"), ("s", "spare"), ("pump", "t"), ("spare", "t")], + { + "pump": pump, + "spare": { + "reliability": W([150.0, 1.5]), + "repairability": E([0.1]), + "repair_cost": 2.0, + }, + }, + **options, + ) + + +def test_a_system_with_a_timed_unit_against_the_simulation(): + pump = minimal(W([100.0, 2.5]), L([2.0, 0.5]), repair_cost=1.0) + rbd = pair(pump, downtime_cost_rate=5.0) + horizon, n = 300.0, 4000 + sim = simulated(rbd, horizon, n, 3) + grid = sim.timeline[1:-1] + up = rbd.point_availability(grid) + error = np.sqrt(up * (1.0 - up) / n) + assert np.all(np.abs(sim.availability[1:-1] - up) < 4.0 * error + 1e-3) + error = float(np.std(sim.uptimes)) / math.sqrt(n) / horizon + assert ( + abs(sim.sample_mean_availability - rbd.mission_availability(horizon)) + < 4.0 * error + ) + expected = rbd.expected_cost(horizon) + error = float(np.std(sim.cost.samples)) / math.sqrt(n) + assert abs(sim.cost.mean - expected.mean) < 4.0 * error + assert sim.system_failures / n == pytest.approx( + rbd.expected_events(horizon).system_failures, rel=0.1 + ) + + +def test_importance_over_time_with_a_timed_unit(): + # In parallel, the spare's Birnbaum importance at t is the pump's + # unavailability then: against the pump simulated alone. + pump = minimal(W([100.0, 2.5]), L([2.0, 0.5])) + rbd = pair(pump) + alone = single(pump) + n = 4000 + sim = simulated(alone, 300.0, n, 4) + grid = sim.timeline[1:-1] + importance = rbd.birnbaum_importance(x=grid) + down = 1.0 - sim.availability[1:-1] + error = np.sqrt(down * (1.0 - down) / n) + assert np.all(np.abs(importance["spare"] - down) < 4.0 * error + 1e-3) + # And the pump's is the spare's unavailability, from its renewal + # equation: the exact structure at the curves. + spare = single({"reliability": W([150.0, 1.5]), "repairability": E([0.1])}) + np.testing.assert_allclose( + importance["pump"], 1.0 - spare.point_availability(grid), atol=1e-6 + ) + + +# -- the routes and the refusals --------------------------------------------- + + +def test_the_routes_say_numerical_over_a_window_and_refuse_the_long_run(): + rbd = single(minimal(W([100.0, 2.0]), E([0.5]), repair_cost=1.0)) + report = rbd.analysis_routes() + for name in ( + "point_availability", + "mission_availability", + "expected_failures", + "expected_events", + "expected_cost", + "availability_rate", + ): + assert report[name].route == r.NUMERICAL, name + route, how = _curves._node_over_time(rbd, "c") + assert route == r.NUMERICAL and "operating age followed on a grid" in how + for name in ( + "mean_availability", + "expected_cost_rate", + "birnbaum_importance", + ): + assert report[name].route == r.REFUSED, name + assert "numerical, its operating age" in report["mean_availability"].reason + with pytest.raises(NotImplementedError, match="operating age followed"): + rbd.mean_availability() + assert report["availability"].route == r.SIMULATED + # And the run's means are the numerical values by default (#187). + assert "need none" in report["availability"].reason + + +def test_the_other_blockers_still_refuse(): + for spec, blocker in [ + ( + minimal( + W([100.0, 2.0]), + E([0.5]), + repair={"model": "kijima1", "q": 0.9}, + ), + "q below 1", + ), + ( + minimal(W([100.0, 2.0]), E([0.5]), replace_after=3), + "replaced at its failure 3", + ), + ( + minimal(W([100.0, 2.0]), E([0.5]), preventive={"interval": 50.0}), + "preventive maintenance", + ), + (minimal(W([100.0, 2.0], f0=0.1), E([0.5])), "dead on arrival"), + ]: + rbd = single(spec) + with pytest.raises(NotImplementedError, match=blocker): + rbd.point_availability([50.0]) + assert rbd.analysis_routes()["expected_failures"].route == r.REFUSED + + +def test_too_long_a_window_is_refused(): + # A repair far shorter than the step costs nothing, but a window of + # thousands of lives would need more grid points than are followed. + rbd = single(minimal(W([1.0, 2.0]), E([100.0]))) + with pytest.raises(NotImplementedError, match="grid points"): + rbd.point_availability([1e6]) + + +def test_a_tested_minimally_repaired_unit_is_refused_as_the_route_says(): + # A component with hidden failures repaired minimally: the values over + # time are refused (its failures are found only by its tests), as + # analysis_routes says, by every method, with the route's message; + # they were computed as if every repair renewed it. A run's default + # means then fall back to the simulations' own, not a refused value. + spec = { + "reliability": W([10.0, 1.5]), + "repairability": surv.LogNormal.from_params([1.5, 0.5]), + "repair": {"model": "kijima1", "q": 1.0}, + "inspection": {"interval": 10.0}, + } + rbd = RepairableRBD([("s", "A"), ("A", "t")], {"A": spec}) + report = rbd.analysis_routes() + calls = { + "point_availability": lambda: rbd.point_availability([5.0, 40.0]), + "mission_availability": lambda: rbd.mission_availability(40.0), + "expected_events": lambda: rbd.expected_events(40.0), + "expected_failures": lambda: rbd.expected_failures(40.0), + } + for name, call in calls.items(): + assert report[name].route == r.REFUSED + with pytest.raises(NotImplementedError) as error: + call() + assert str(error.value) == report[name].reason + assert "found only by its tests" in report[name].reason + result = rbd.availability(40.0, mc_samples=200, seed=3) + assert result.mean_availability_interval().method != "exact" + assert result.mean_availability == pytest.approx( + result.sample_mean_availability, rel=1e-12 + ) + assert result.control_variate is None + # The renewed unit (q = 0) is what was computed: its values differ. + renewed = RepairableRBD( + [("s", "A"), ("A", "t")], + {"A": {**spec, "repair": {"model": "kijima1", "q": 0.0}}}, + ) + assert renewed.analysis_routes()["mission_availability"].route != ( + r.REFUSED + ) + assert renewed.mission_availability(40.0) != pytest.approx( + result.mean_availability, abs=0.02 + ) diff --git a/repyability/tests/test_mixture_lives.py b/repyability/tests/test_mixture_lives.py index 274cd162..bcccd180 100644 --- a/repyability/tests/test_mixture_lives.py +++ b/repyability/tests/test_mixture_lives.py @@ -1,13 +1,14 @@ """A surpyval MixtureModel as a repairable component's life (#227). A mixture (two modes, such as infant mortality and wear-out) has the -survival function, distribution, density and mean the analyses take, but -no quantile function (surpyval #651): the simulations' streams draw its -lives by inverting its distribution function (``mixture_quantile``), one -uniform a draw, and a life given an age through its cumulative hazard -(``MixtureLife``). Every analysis on diagrams with a mixture life is -checked in ``test_analysis_routes`` (the catalogue's "mixture life" -diagrams), and the engines against each other in ``test_catalogue``. +survival function, distribution, density and mean the analyses take, and +from surpyval 0.24 a quantile function, which loses the upper tail and is +slow (surpyval #821): the simulations' streams draw its lives by inverting +its distribution function (``mixture_quantile``), one uniform a draw, and +a life given an age through its cumulative hazard (``MixtureLife``). +Every analysis on diagrams with a mixture life is checked in +``test_analysis_routes`` (the catalogue's "mixture life" diagrams), and +the engines against each other in ``test_catalogue``. """ import json @@ -17,13 +18,13 @@ import surpyval as surv from repyability import NonRepairable, NonRepairableRBD, RepairableRBD +from repyability.rbd._event_loop import _aged_life from repyability.rbd._sampling import ( MixtureLife, inverse_sampler, mixture_quantile, stream_sampler, ) -from repyability.rbd.repairable_rbd import _aged_life from repyability.tests.catalogue import mixture_life REPAIR = surv.Exponential.from_params([1.0]) @@ -47,9 +48,9 @@ def test_the_issue_s_diagram(mix): def survival(mix, x): - """The mixture's survival from its components' (its own ``sf``, - ``1 - ff``, loses its precision in the tail, surpyval #671).""" - return sum(w * mix.dist.sf(x, *p) for w, p in zip(mix.w, mix.params)) + """The mixture's survival function (precise in the tail from surpyval + 0.24, #671).""" + return mix.sf(x) def test_the_quantile_inverts_the_distribution(mix): @@ -205,3 +206,80 @@ def test_a_standalone_unit_prices_age_replacement(mix): assert unit.cost_rate(age) <= unit.cost_rate(0.8 * age) assert unit.cost_rate(age) <= unit.cost_rate(1.25 * age) assert isinstance(MixtureLife(mix).qf(np.array([0.5]))[0], float) + + +ISSUE_295 = { + "model": "MixtureModel", + "dist": "Weibull", + "m": 2, + "params": [[60.0, 0.8], [2000.0, 3.5]], + "w": [0.25, 0.75], +} + + +def exact_life_left(age, u): + # The root of H(age + x) = H(age) - log(u) for the issue's mixture, + # in extended precision, by bisection. + L = np.longdouble + + def H(x): + x = L(x) + return -np.log( + L(0.25) * np.exp(-((x / L(60)) ** L(0.8))) + + L(0.75) * np.exp(-((x / L(2000)) ** L(3.5))) + ) + + target = H(age) - np.log(L(u)) + lo, hi = L(age), L(age) + 1 + while H(hi) < target: + hi = 2 * hi + 1 + for _ in range(200): + mid = (lo + hi) / 2 + lo, hi = (mid, hi) if H(mid) < target else (lo, mid) + return float((lo + hi) / 2 - L(age)) + + +def test_a_life_given_an_age_is_the_root_to_the_last_bits(): + # #295: the mixture's life left at an age, by Newton's steps on its + # cumulative hazard, is the root to a few bits of age + x, from new to + # deep in the tail (where conditional_gaps root-found with some 150 + # evaluations), and agrees with surpyval's draw. + from surpyval.recurrent.renewal.renewal_model import conditional_gaps + + life = MixtureLife.of(surv.MixtureModel.from_dict(ISSUE_295)) + rng = np.random.default_rng(4) + for age in (0.0, 5.0, 100.0, 1000.0, 3000.0, 9000.0, 15000.0): + for u in np.r_[rng.uniform(size=8), 1e-6, 0.999]: + left = life.aged(age, u) + exact = exact_life_left(age, u) + assert abs(left - exact) <= 16 * np.finfo(float).eps * ( + age + exact + ), (age, u) + theirs = conditional_gaps(life, np.array([age]), np.array([u])) + assert left == pytest.approx(theirs[0], rel=1e-7) + assert life.aged(100.0, 1.0) == 0.0 + assert life.aged(100.0, 0.0) == np.inf + + +def test_a_life_given_an_age_takes_a_few_evaluations(monkeypatch): + # #295: a handful of evaluations of the cumulative hazard a draw, at + # any age (the issue's 15,000 h run went from ~53 s to a few). + life = MixtureLife.of(surv.MixtureModel.from_dict(ISSUE_295)) + calls = [] + plain = MixtureLife._cumulative + + def counted(self, x): + calls.append(len(x)) + return plain(self, x) + + life.aged(100.0, 0.5) # its table, made once + monkeypatch.setattr(MixtureLife, "_cumulative", counted) + rng = np.random.default_rng(0) + counts = [] + for age in (0.0, 100.0, 1000.0, 2000.0, 8000.0, 30000.0): + for u in rng.uniform(size=50): + calls.clear() + life.aged(age, u) + counts.append(len(calls)) + assert max(counts) <= 8 + assert np.mean(counts) <= 4.5 diff --git a/repyability/tests/test_model_utils.py b/repyability/tests/test_model_utils.py index fce188f6..2b495d4b 100644 --- a/repyability/tests/test_model_utils.py +++ b/repyability/tests/test_model_utils.py @@ -4,6 +4,8 @@ silently changing reliability results. """ +import numpy as np +import pytest import surpyval as surv from surpyval import FixedEventProbability @@ -12,6 +14,7 @@ is_exponential, is_fixed_probability, ) +from repyability.rbd._point_availability import knots from repyability.rbd.helper_classes import PerfectReliability from repyability.rbd.standby_node import StandbyModel @@ -51,3 +54,20 @@ def test_is_exponential(): assert is_exponential(surv.Exponential.from_params([0.01])) assert not is_exponential(surv.Weibull.from_params([100, 2])) assert not is_exponential(PerfectReliability) + + +def test_a_fault_in_a_model_probe_is_raised_not_swallowed(): + # A model that cannot take the quantiles asked gives no knots; a fault + # (here a NameError, as a bug would raise) is not taken for that. + class CannotTake: + def qf(self, p): + raise ValueError("p out of range") + + class Faulty: + def qf(self, p): + raise NameError("name 'q' is not defined") + + assert knots(CannotTake()).size == 0 + with pytest.raises(NameError): + knots(Faulty()) + assert np.all(knots(surv.Weibull.from_params([10.0, 2.0])) > 0) diff --git a/repyability/tests/test_monte_carlo.py b/repyability/tests/test_monte_carlo.py index a8967667..6bee7099 100644 --- a/repyability/tests/test_monte_carlo.py +++ b/repyability/tests/test_monte_carlo.py @@ -19,6 +19,7 @@ from scipy.stats import norm from repyability import NonRepairableRBD, RepairableRBD, StandbyModel +from repyability.rbd import _event_loop from repyability.rbd import _montecarlo as montecarlo from repyability.rbd import _streams, non_repairable_rbd from repyability.rbd.non_repairable_rbd import _check_lifetimes @@ -436,16 +437,8 @@ def expected(mttr): def keyed_uptimes(rbd, n, key): - tally = rbd._run( - T, - set(), - set(), - "p", - n, - False, - None, - entropy=key, - common=True, + tally = _event_loop._run( + rbd, T, set(), set(), "p", n, False, None, entropy=key, common=True ) return np.asarray(tally.uptimes) diff --git a/repyability/tests/test_network.py b/repyability/tests/test_network.py index 7e26d974..f0646cc9 100644 --- a/repyability/tests/test_network.py +++ b/repyability/tests/test_network.py @@ -2,15 +2,19 @@ nodes (#104).""" import itertools +import re +import warnings import numpy as np import pytest +import scipy.stats as st import surpyval as surv from surpyval import FixedEventProbability from repyability import Network, NonRepairableRBD from repyability import network as network_module from repyability.rbd.shannon import _shannon_value_and_gradient +from repyability.tests.catalogue import BRIDGE E, W = surv.Exponential.from_params, surv.Weibull.from_params F = FixedEventProbability.from_params @@ -574,8 +578,568 @@ def test_the_methods_are_checked(): net.sf() with pytest.raises(ValueError, match="'exact' or 'simulate'"): net.sf(1.0, method="guess") - with pytest.raises(ValueError, match="only to method='simulate'"): + with pytest.raises(ValueError, match="only with method='simulate'"): net.sf(1.0, mc_samples=10) with pytest.raises(ValueError, match="no lifetime"): bridge({n: F(0.1) for n in "abcde"}).mean() assert repr(net) == "Network(5 links, 0 failing nodes, 's' to 't')" + + +# -- the measures, lives, uncertainty, saving and routes (#315) ------------ + + +def as_diagram(models, nodes=None): + """The bridge as a block diagram: the links as nodes in the same + roles (the catalogue's ``BRIDGE``).""" + return NonRepairableRBD(BRIDGE, {**models, **(nodes or {})}) + + +def each_agrees(a, b, rtol=1e-12): + assert set(a) == set(b) + for key in b: + np.testing.assert_allclose(a[key], b[key], rtol=rtol, atol=1e-300) + + +#: The measures a network and a diagram define alike, with their options. +MEASURES = [ + ("improvement_potential", {}), + ("risk_achievement_worth", {}), + ("risk_reduction_worth", {}), + ("criticality_importance", {}), + ("criticality_importance", {"kind": "success"}), + ("fussell_vesely", {}), + ("fussell_vesely", {"fv_type": "p"}), + ("fussell_vesely", {"method": "rare_event"}), + ("fussell_vesely", {"fv_type": "p", "method": "rare_event"}), + ("differential_importance", {}), + ("differential_importance", {"change": "proportional"}), + ( + "differential_importance", + {"change": "proportional", "kind": "success"}, + ), + ("differential_importance", {"groups": {"ends": "abde", "mid": "c"}}), + ("joint_importance", {}), +] + + +@pytest.mark.parametrize( + "models", + [ + {n: F(0.1 + 0.05 * i) for i, n in enumerate("abcde")}, + {n: W([100.0 + 10 * i, 1.2 + 0.2 * i]) for i, n in enumerate("abcde")}, + ], + ids=["fixed", "Weibull"], +) +def test_the_measures_are_the_diagram_s(models): + # Each measure of the bridge network is the diagram's of the same + # system, at each time: the measures are its definitions over the + # same values (#315). + net, rbd = bridge(models), as_diagram(models) + fixed = net.is_fixed + times = [None] if fixed else [10.0, np.array([10.0, 50.0, 120.0])] + for x in times: + for name, options in MEASURES: + each_agrees( + getattr(net, name)(x, **options), + getattr(rbd, name)(x, **options), + ) + each_agrees(net.structural_importance(), rbd.structural_importance()) + if fixed: + return + each_agrees( + net.barlow_proschan_importance(), rbd.barlow_proschan_importance() + ) + each_agrees( + net.barlow_proschan_importance([20.0, 80.0]), + rbd.barlow_proschan_importance([20.0, 80.0]), + ) + assert net.time_to_reliability(0.5) == pytest.approx( + rbd.time_to_reliability(0.5), rel=1e-12 + ) + np.testing.assert_allclose( + net.bx_life([5, 50]), rbd.bx_life([5, 50]), rtol=1e-12 + ) + assert net.bx_life(10, upper_bound=200.0) == pytest.approx( + net.time_to_reliability(0.9), rel=1e-12 + ) + + +def test_failing_nodes_are_nodes_in_series_with_their_links(): + # A network whose nodes fail, against the diagram with those nodes + # drawn in series with their links: a node between two links, and a + # failing terminal. + models = { + "a": W([100.0, 1.5]), + "b": W([120.0, 2.0]), + "c": E([0.004]), + "d": W([80.0, 1.2]), + } + nodes = {"m": E([0.003]), "t": W([300.0, 3.0])} + net = Network( + { + "a": ("s", "m", models["a"]), + "b": ("m", "t", models["b"]), + "c": ("s", "t", models["c"]), + "d": ("m", "t", models["d"]), + }, + "s", + "t", + nodes=nodes, + ) + rbd = NonRepairableRBD( + [ + ("s", "a"), + ("a", "m"), + ("m", "b"), + ("m", "d"), + ("b", "t"), + ("d", "t"), + ("s", "c"), + ("c", "t"), + ("t", "end"), + ], + {**models, **nodes}, + ) + for x in (10.0, np.array([5.0, 60.0, 200.0])): + np.testing.assert_allclose(net.sf(x), rbd.sf(x), rtol=1e-12) + each_agrees(net.birnbaum_importance(x), rbd.birnbaum_importance(x)) + for name, options in MEASURES: + if "groups" in options: + continue + each_agrees( + getattr(net, name)(x, **options), + getattr(rbd, name)(x, **options), + ) + each_agrees(net.structural_importance(), rbd.structural_importance()) + each_agrees( + net.barlow_proschan_importance(), rbd.barlow_proschan_importance() + ) + assert net.bx_life(10) == pytest.approx(rbd.bx_life(10), rel=1e-12) + + +def test_small_measures_keep_their_precision(): + # Two parallel links in series with a third: ff = q^2 + q3 - q^2 q3, + # and each measure in closed form from it. + q, q3 = 1e-9, 1e-12 + net = Network( + { + "a": ("s", "m", F(q)), + "b": ("s", "m", F(q)), + "c": ("m", "t", F(q3)), + }, + "s", + "t", + ) + ff = q**2 + q3 - q**2 * q3 + assert net.criticality_importance()["c"] == pytest.approx( + (1 - q**2) * q3 / ff, rel=1e-12 + ) + assert net.criticality_importance()["a"] == pytest.approx( + q * (1 - q3) * q / ff, rel=1e-12 + ) + assert net.fussell_vesely()["c"] == pytest.approx(q3 / ff, rel=1e-12) + assert net.fussell_vesely()["a"] == pytest.approx(q**2 / ff, rel=1e-12) + assert net.risk_achievement_worth()["c"] == pytest.approx( + 1 / ff, rel=1e-12 + ) + assert net.risk_reduction_worth()["c"] == pytest.approx( + ff / q**2, rel=1e-12 + ) + assert net.risk_reduction_worth()["a"] == pytest.approx(ff / q3, rel=1e-12) + assert net.improvement_potential()["c"] == pytest.approx( + (1 - q**2) * q3, rel=1e-12 + ) + joint = net.joint_importance() + assert joint["a", "b"] == pytest.approx(-(1 - q3), rel=1e-12) + assert joint["a", "c"] == pytest.approx(q, rel=1e-12) + + +def fitted_bridge(): + """The bridge with fitted link models: two of one type (one fitted + object), two of another, and one given.""" + rng = np.random.default_rng(0) + first = surv.Weibull.fit(rng.weibull(1.5, 60) * 100) + second = surv.Weibull.fit(rng.weibull(2.0, 40) * 120) + return { + "a": first, + "b": first, + "c": second, + "d": W([90, 1.4]), + "e": second, + } + + +@pytest.mark.parametrize("sampling", ["random", "sobol"]) +def test_the_uncertainty_is_drawn_as_the_diagram_s(sampling): + # The draws of the links' fitted models are a diagram's with the same + # seed and sources (the same machinery): the same samples, to the + # rounding of the two engines (#315). + models = fitted_bridge() + net, rbd = bridge(models), as_diagram(models) + assert net._uncertainty_sources(None) == [ + (("a", "b"), "fit"), + (("c", "e"), "fit"), + ] + t = np.array([10.0, 50.0, 120.0]) + ours = net.sf_uncertainty(t, n_draws=100, seed=3, sampling=sampling) + theirs = rbd.sf_uncertainty(t, n_draws=100, seed=3, sampling=sampling) + assert ours.n_draws == 100 and ours.samples.shape == (100, 3) + np.testing.assert_allclose(ours.samples, theirs.samples, rtol=1e-13) + np.testing.assert_allclose(ours.nominal, theirs.nominal, rtol=1e-13) + given = {("a", "b"): "fit", "d": {"alpha": st.uniform(80, 20)}} + ours = net.sf_uncertainty( + 50.0, given, n_draws=50, seed=1, sampling=sampling + ) + theirs = rbd.sf_uncertainty( + 50.0, given, n_draws=50, seed=1, sampling=sampling + ) + assert isinstance(ours.nominal, float) and ours.samples.shape == (50,) + np.testing.assert_allclose(ours.samples, theirs.samples, rtol=1e-13) + ours = net.mean_uncertainty(given, n_draws=10, seed=1, sampling=sampling) + theirs = rbd.mean_uncertainty(given, n_draws=10, seed=1, sampling=sampling) + np.testing.assert_allclose(ours.samples, theirs.samples, rtol=1e-9) + assert ours.nominal == pytest.approx(net.mean(), rel=1e-12) + # Times of any shape, a draw's row in their shape. + shaped = net.sf_uncertainty(t.reshape(3, 1), given, n_draws=4, seed=1) + assert shaped.samples.shape == (4, 3, 1) and shaped.nominal.shape == (3, 1) + + +def test_the_uncertainty_of_a_fixed_network_needs_no_time(): + # Every draw a fixed probability: the network's reliability with the + # drawn probabilities, each sample one of the combinations. + models = {n: F(0.1 + 0.05 * i) for i, n in enumerate("abcde")} + models["b"] = models["a"] + net = bridge(models, nodes={"x": F(0.2)}) + given = {"x": [F(0.2), F(0.4)], ("a", "b"): [F(0.1), F(0.3)]} + ours = net.sf_uncertainty(None, given, n_draws=30, seed=2) + assert ours.samples.shape == (30,) and ours.nominal == net.sf() + possible = { + round(bridge({**models, "a": F(p), "b": F(p)}, {"x": F(q)}).sf(), 12) + for p in (0.1, 0.3) + for q in (0.2, 0.4) + } + assert set(np.round(ours.samples, 12)) == possible + with pytest.raises(ValueError, match="x is required"): + bridge({**models, "a": W([100, 2])}).sf_uncertainty( + None, {"a": [W([80, 2])]} + ) + + +def test_the_uncertainty_is_checked(): + models = fitted_bridge() + net = bridge(models) + with pytest.warns(UserWarning, match="drawn without 'b'"): + net.sf_uncertainty(50.0, {"a": "fit"}, n_draws=3, seed=1) + for given, message in [ + ({}, "Give the uncertain elements"), + ({"z": "fit"}, "not a link or a failing node"), + ({"a": "fit", ("a", "b"): "fit"}, "given an uncertainty twice"), + ({("a", "c"): "fit"}, "must have the same model"), + ({"d": "fit"}, "parameter covariance"), + ]: + with pytest.raises(ValueError, match=message): + net.sf_uncertainty(50.0, given, n_draws=3, seed=1) + with pytest.raises(ValueError, match="n_draws must be"): + net.sf_uncertainty(50.0, {"a": "fit"}, n_draws=0) + with pytest.raises(ValueError, match="No link's or node's model"): + bridge({n: W([100, 2]) for n in "abcde"}).sf_uncertainty(50.0) + with pytest.raises(ValueError, match="no lifetime"): + bridge({n: F(0.1) for n in "abcde"}).mean_uncertainty({"a": [F(0.2)]}) + + +def test_the_network_saves_and_loads(): + models = fitted_bridge() + net = bridge(models, nodes={"x": E([0.002]), "t": 0.01}) + t = np.array([10.0, 50.0, 120.0]) + # A fit's covariance round-trips, so its uncertainty can be drawn (the + # links of one fitted object are separate objects once loaded, so they + # are given together). + given = {("a", "b"): "fit", ("c", "e"): "fit"} + for clone in ( + Network.from_dict(net.to_dict()), + Network.from_json(net.to_json(indent=2)), + ): + assert clone.links == net.links + assert list(clone.nodes) == list(net.nodes) + assert (clone.source, clone.target) == (net.source, net.target) + np.testing.assert_array_equal(clone.sf(t), net.sf(t)) + np.testing.assert_array_equal( + clone.sf_uncertainty(t, given, n_draws=5, seed=1).samples, + net.sf_uncertainty(t, given, n_draws=5, seed=1).samples, + ) + saved = net.to_dict() + assert saved["type"] == "Network" and saved["source"] == "s" + assert [link["link"] for link in saved["links"]] == list("abcde") + assert saved["links"][0]["nodes"] == ["s", "x"] + assert [node["node"] for node in saved["nodes"]] == ["x", "t"] + assert saved["nodes"][1]["model"]["kind"] == "surpyval" + # Tuple names, which JSON turns into lists, come back as tuples. + grid = built(lattice(3, 3), lambda name: E([0.01]), nodes={(1, 1): F(0.1)}) + clone = Network.from_json(grid.to_json()) + assert clone.links == grid.links and set(clone.nodes) == {(1, 1)} + assert clone.source == (0, 0) and clone.target == (2, 2) + assert clone.sf(10.0) == grid.sf(10.0) + with pytest.raises(ValueError, match="Not a network"): + Network.from_dict({"type": "FaultTree"}) + + class Own: + def sf(self, x): + return np.exp(-np.asarray(x) / 100) + + def ff(self, x): + return 1 - self.sf(x) + + with pytest.raises(NotImplementedError, match="Cannot serialise"): + Network({"a": ("s", "t", Own())}, "s", "t").to_dict() + + +#: A call of each analysis the routes report. +CALLS = { + **{ + name: (lambda name: lambda net: getattr(net, name)(10.0))(name) + for name in ( + "sf", + "ff", + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "fussell_vesely", + "differential_importance", + "joint_importance", + "barlow_proschan_importance", + ) + }, + "structural_importance": lambda net: net.structural_importance(), + "cut_sets": lambda net: net.cut_sets(), + "path_sets": lambda net: net.path_sets(), + "minimal_cut_sets": lambda net: net.minimal_cut_sets(), + "minimal_path_sets": lambda net: net.minimal_path_sets(), + "mean": lambda net: net.mean(), + "time_to_reliability": lambda net: net.time_to_reliability(0.5), + "bx_life": lambda net: net.bx_life(10), + "random": lambda net: net.random(10, seed=1), + "sf_uncertainty": lambda net: net.sf_uncertainty( + 10.0, {"a": [W([90, 2])]}, n_draws=2, seed=1 + ), + "mean_uncertainty": lambda net: net.mean_uncertainty( + {"a": [W([90, 2])]}, n_draws=2, seed=1 + ), +} + + +def test_the_routes_say_how_each_analysis_is_worked_out(monkeypatch): + net = bridge({n: W([100.0, 1.5]) for n in "abcde"}, nodes={"x": E([0.01])}) + report = net.analysis_routes() + public = { + name + for name in dir(Network) + if not name.startswith("_") and callable(getattr(Network, name)) + } + assert set(report) == set(CALLS) + assert public - set(report) == { + "analysis_routes", + "from_dict", + "from_json", + "to_dict", + "to_json", + } + assert report["sf"].route == report["fussell_vesely"].route == "exact" + assert report["joint_importance"].route == "exact" + assert report["mean"].route == report["bx_life"].route == "numerical" + assert report["barlow_proschan_importance"].route == "numerical" + assert report["random"].route == "simulated" + assert report["sf_uncertainty"].route == "simulated" + assert report["mean_uncertainty"].route == "simulated" + fixed = bridge({n: F(0.1) for n in "abcde"}) + report = fixed.analysis_routes() + assert report["sf"].route == "exact" + for name in ("mean", "mean_uncertainty", "barlow_proschan_importance"): + assert report[name].route == "refused" + assert "no lifetime" in report[name].reason + assert report["bx_life"].route == "refused" + with pytest.raises(ValueError, match=re.escape(report["bx_life"].reason)): + fixed.bx_life(10) + # Too large a diagram: every exact value refused, with the message + # the methods raise, and the simulation (and the paths) left. + monkeypatch.setattr(network_module, "MAX_STATES", 3) + large = bridge({n: W([100.0, 1.5]) for n in "abcde"}) + report = large.analysis_routes() + for name, route in report.items(): + if name in ("random", "path_sets", "minimal_path_sets"): + assert route.route != "refused" + CALLS[name](large) + continue + assert route.route == "refused", name + assert "method='simulate'" in route.reason + with pytest.raises(NotImplementedError) as error: + CALLS[name](large) + assert str(error.value) == route.reason + + +def test_the_new_methods_are_checked(): + net = bridge({n: W([100.0, 1.5]) for n in "abcde"}) + with pytest.raises(ValueError, match="kind must be"): + net.criticality_importance(10.0, kind="other") + with pytest.raises(ValueError, match="fv_type must be"): + net.fussell_vesely(10.0, fv_type="x") + with pytest.raises(ValueError, match="'exact' or 'rare_event'"): + net.fussell_vesely(10.0, method="guess") + with pytest.raises(ValueError, match="change must be"): + net.differential_importance(10.0, change="sideways") + with pytest.raises(ValueError, match="names"): + net.differential_importance(10.0, groups={"g": ["z"]}) + with pytest.raises(ValueError, match="percentage"): + net.bx_life(100) + with pytest.raises(ValueError, match=r"in \(0, 1\)"): + net.time_to_reliability(1.0) + with pytest.raises(ValueError, match="non-negative"): + net.barlow_proschan_importance(-1.0) + with pytest.raises(TypeError, match="must be numbers"): + net.bx_life("10") + with pytest.raises(ValueError, match="no lifetime"): + bridge({n: F(0.1) for n in "abcde"}).barlow_proschan_importance() + with pytest.raises(ValueError, match="does not vary"): + bridge({n: F(0.1) for n in "abcde"}).time_to_reliability(0.5) + + +# -- the 0.14 persona review -------------------------------------------------- + + +def test_a_distribution_itself_is_refused(): + with pytest.raises(TypeError, match="Weibull distribution itself"): + Network({"a": ("s", "t", surv.Weibull)}, "s", "t") + with pytest.raises(TypeError, match="Node 'x' is surpyval's Weibull"): + Network( + {"a": ("s", "x", E([0.1])), "b": ("x", "t", E([0.1]))}, + "s", + "t", + nodes={"x": surv.Weibull}, + ) + + +def test_a_fixed_element_among_lives_has_no_lifetime_of_its_own(): + # The link fails at the start with probability 0.1, or never: the + # chain lasts 0 then, else as long as the Weibull link, so its mean is + # 0.9 of the Weibull's, its failures over time all the Weibull's. + life = W([100.0, 2.0]) + chain = Network({"a": ("s", 1, 0.1), "b": (1, "t", life)}, "s", "t") + assert chain.mean() == pytest.approx(0.9 * float(life.mean()), rel=1e-9) + assert chain.mean(method="simulate", mc_samples=20_000, seed=1) == ( + pytest.approx(chain.mean(), rel=0.03) + ) + assert chain.barlow_proschan_importance() == {"a": 0.0, "b": 1.0} + assert chain.barlow_proschan_importance(50.0) == {"a": 0.0, "b": 1.0} + assert chain.mean_uncertainty( + {"b": {"alpha": st.uniform(90, 20)}}, n_draws=5, seed=1 + ).nominal == pytest.approx(chain.mean()) + diagram = NonRepairableRBD( + [("s", "a"), ("a", "b"), ("b", "t")], {"a": F(0.1), "b": life} + ) + assert chain.mean() == pytest.approx(diagram.mean(), rel=1e-9) + # A fixed survivor in parallel keeps the terminals joined for good. + pair = Network({"a": ("s", "t", 0.1), "b": ("s", "t", life)}, "s", "t") + assert pair.mean() == np.inf + assert pair.barlow_proschan_importance() == {"a": 0.0, "b": 1.0} + assert pair.analysis_routes()["mean"].route == "numerical" + # With every element fixed there are no lifetimes at all. + fixed = Network({"a": ("s", "t", 0.1), "b": ("s", "t", 0.2)}, "s", "t") + report = fixed.analysis_routes() + for name, call in ( + ("mean", fixed.mean), + ("barlow_proschan_importance", fixed.barlow_proschan_importance), + ( + "mean_uncertainty", + lambda: fixed.mean_uncertainty( + {"a": {"p": st.uniform(0.05, 0.1)}}, n_draws=5, seed=1 + ), + ), + ): + with pytest.raises(ValueError, match="no lifetimes") as error: + call() + assert report[name].route == "refused" + assert report[name].reason == str(error.value) + + +def test_minimal_sets_are_listed_smallest_first(): + models = {n: W([100.0 + 10 * i, 1.5]) for i, n in enumerate("abcde")} + net = bridge(models) + diagram = NonRepairableRBD(BRIDGE, models) + assert net.minimal_cut_sets() == diagram.minimal_cut_sets() + assert net.minimal_path_sets() == diagram.minimal_path_sets() + assert set(net.minimal_cut_sets()) == net.cut_sets() + assert set(net.minimal_path_sets()) == net.path_sets() + sizes = [len(s) for s in net.minimal_cut_sets()] + assert sizes == sorted(sizes) + report = net.analysis_routes() + assert report["minimal_cut_sets"] == report["cut_sets"] + assert report["minimal_path_sets"] == report["path_sets"] + + +def test_simulation_options_need_method_simulate(): + net = bridge({n: E([0.01]) for n in "abcde"}) + for call in ( + lambda: net.sf(1.0, seed=1), + lambda: net.ff(1.0, mc_samples=10), + lambda: net.mean(seed=1), + lambda: net.mean(mc_samples=10), + ): + with pytest.raises(ValueError, match="only with method='simulate'"): + call() + assert net.sf(1.0, method="simulate", mc_samples=10, seed=1) <= 1.0 + + +def test_times_of_any_shape(): + net = bridge({n: W([100.0 + 10 * i, 1.5]) for i, n in enumerate("abcde")}) + grid = np.array([[10.0, 20.0, 30.0], [40.0, 50.0, 60.0]]) + flat = grid.ravel() + for name in ("sf", "ff"): + assert np.array_equal( + getattr(net, name)(grid), getattr(net, name)(flat).reshape(2, 3) + ) + assert np.array_equal( + getattr(net, name)(grid, method="simulate", mc_samples=50, seed=1), + getattr(net, name)( + flat, method="simulate", mc_samples=50, seed=1 + ).reshape(2, 3), + ) + for name in ( + "birnbaum_importance", + "fussell_vesely", + "criticality_importance", + "barlow_proschan_importance", + ): + shaped, rows = getattr(net, name)(grid), getattr(net, name)(flat) + for key in shaped: + assert np.array_equal(shaped[key], rows[key].reshape(2, 3)), name + assert isinstance(net.sf(np.array(10.0)), float) + + +def test_no_negative_zero_and_no_warning_at_zero(): + # A link in no path has an importance of 0, printed so, and at t = 0 + # the ratios over an unreliability of 0 are nan silently. + net = Network( + {"a": ("s", "t", E([0.01])), "b": ("s", "x", E([0.01]))}, + "s", + "t", + ) + assert str(net.birnbaum_importance(10.0)["b"]) == "0.0" + assert str(net.improvement_potential(10.0)["b"]) == "0.0" + assert str(net.birnbaum_importance(0.0)["b"]) == "0.0" + with warnings.catch_warnings(): + warnings.simplefilter("error") + at_zero = net.risk_achievement_worth(0.0) + assert np.isinf(at_zero["a"]) + assert np.isnan(net.fussell_vesely(0.0)["a"]) + assert np.isnan(net.criticality_importance(0.0)["a"]) + + +def test_x_is_required_names_the_network(): + net = bridge({n: E([0.01]) for n in "abcde"}) + with pytest.raises(ValueError, match="this network is time-varying"): + net.sf() + with pytest.raises(TypeError, match="unexpected keyword argument 't'"): + net.sf(t=1.0) + with pytest.raises(TypeError, match="x must be numbers, got '10'"): + net.bx_life("10") diff --git a/repyability/tests/test_non_repairable_failures.py b/repyability/tests/test_non_repairable_failures.py new file mode 100644 index 00000000..5055994d --- /dev/null +++ b/repyability/tests/test_non_repairable_failures.py @@ -0,0 +1,180 @@ +"""The failures of a part replaced when it fails, over a window from its +current unit's age (``NonRepairable.failures``; #319, UC-5 of +``repyability/maintenance/README.md``), and its costs given to the +constructor.""" + +import json +import math + +import numpy as np +import pytest +import surpyval as surv + +from repyability import FailureForecast, NonRepairable + +LIFE = surv.Weibull.from_params([365.0, 2.0]) +BEARING = NonRepairable(LIFE, unplanned_cost=400.0) + + +def simulated_failures(scale, shape, age, days, n=400_000, seed=0, fix=0.0): + """The failures of a Weibull unit of ``age`` and its replacements in + ``days``, each replacement taking ``fix``, simulated plainly: the mean + count and its standard error.""" + rng = np.random.default_rng(seed) + held = (age / scale) ** shape + left = scale * (held - np.log(rng.random(n))) ** (1 / shape) - age + count, time = np.zeros(n), left + running = time < days + while running.any(): + count[running] += 1 + fresh = scale * (-np.log(rng.random(running.sum()))) ** (1 / shape) + time[running] += fix + fresh + running = time < days + return count.mean(), count.std() / math.sqrt(n) + + +def test_the_bearing_over_82_days_from_224(): + # UC-5: a bearing 224 days old, over the next 82 days. + forecast = BEARING.failures(82.0, age=224.0) + assert isinstance(forecast, FailureForecast) + held, reached = (224 / 365) ** 2, (306 / 365) ** 2 + assert forecast.probability == pytest.approx( + -math.expm1(held - reached), rel=1e-12 + ) + mean, error = simulated_failures(365.0, 2.0, 224.0, 82.0) + assert abs(forecast.expected_failures - mean) < 4 * error + assert forecast.expected_failures == pytest.approx(0.2830, abs=5e-4) + assert forecast.expected_cost == pytest.approx( + 400.0 * forecast.expected_failures, rel=1e-12 + ) + assert forecast.window == 82.0 and forecast.age == 224.0 + + +def test_an_exponential_life_fails_at_its_rate_whatever_its_age(): + rate = 1 / 200.0 + unit = NonRepairable(surv.Exponential.from_params([rate])) + for age in (0.0, 180.0): + forecast = unit.failures(365.0, age=age) + assert forecast.probability == pytest.approx( + -math.expm1(-rate * 365.0), rel=1e-10 + ) + assert forecast.expected_failures == pytest.approx( + rate * 365.0, rel=1e-6 + ) + + +@pytest.mark.parametrize("age, days", [(0, 365), (500, 120), (100, 700)]) +def test_the_failures_are_the_renewal_count_simulated(age, days): + unit = NonRepairable(surv.Weibull.from_params([300.0, 3.0])) + mean, error = simulated_failures(300.0, 3.0, age, days, seed=age) + forecast = unit.failures(float(days), age=float(age)) + assert abs(forecast.expected_failures - mean) < 4 * error + + +def test_a_replacement_that_takes_time_delays_the_next_failures(): + # Ten days to replace each failed unit: fewer failures fit in the + # window, as simulated, and the chance of the first is unchanged. + fix = surv.ExactEventTime.from_params([10.0]) + slow = NonRepairable(surv.Weibull.from_params([100.0, 1.5]), fix) + quick = NonRepairable(surv.Weibull.from_params([100.0, 1.5])) + mean, error = simulated_failures(100.0, 1.5, 50.0, 400.0, seed=3, fix=10.0) + slow_forecast = slow.failures(400.0, age=50.0) + assert abs(slow_forecast.expected_failures - mean) < 4 * error + assert ( + slow_forecast.expected_failures + < quick.failures(400.0, 50.0).expected_failures + ) + assert slow_forecast.probability == quick.failures(400.0, 50.0).probability + + +def test_windows_at_once(): + windows = np.array([0.0, 30.0, 82.0, 365.0]) + forecast = BEARING.failures(windows, age=224.0) + assert forecast.expected_failures.shape == (4,) + assert forecast.expected_failures[0] == 0.0 + assert np.all(np.diff(forecast.expected_failures) > 0) + for window, failures, chance in zip( + windows, forecast.expected_failures, forecast.probability + ): + one = BEARING.failures(window, age=224.0) + # The renewal count is worked out to about 1e-7 (expected_events), + # on a grid that depends on the windows asked for together. + assert one.expected_failures == pytest.approx(failures, rel=1e-6) + assert one.probability == pytest.approx(chance, rel=1e-12, abs=1e-15) + + +def test_a_short_window_has_about_one_failure_at_most(): + # Over a day, a new unit failing as well is negligible: the expected + # failures are the chance of the first. + forecast = BEARING.failures(1.0, age=224.0) + assert forecast.expected_failures == pytest.approx( + forecast.probability, rel=1e-4 + ) + + +def test_no_unplanned_cost_is_none(): + unit = NonRepairable(LIFE) + assert unit.failures(82.0, age=224.0).expected_cost is None + + +def test_the_forecast_goes_through_json(): + forecast = BEARING.failures(82.0, age=224.0) + plain = json.loads(json.dumps(forecast.to_dict())) + assert plain["window"] == 82.0 + assert plain["age"] == 224.0 + assert plain["expected_cost"] == pytest.approx(forecast.expected_cost) + + +def test_a_unit_that_cannot_be_working_at_its_age_is_refused(): + fixed = NonRepairable(surv.ExactEventTime.from_params([10.0])) + with pytest.raises(ValueError, match="no chance"): + fixed.failures(30.0, age=20.0) + + +@pytest.mark.parametrize("t", [-1.0, float("inf"), "82", [1.0, -2.0]]) +def test_a_window_is_at_least_0(t): + with pytest.raises((ValueError, TypeError), match="t must"): + BEARING.failures(t, age=0.0) + + +@pytest.mark.parametrize("age", [-1.0, float("inf"), True, "224"]) +def test_an_age_is_at_least_0(age): + with pytest.raises(ValueError, match="age"): + BEARING.failures(82.0, age=age) + + +def test_the_costs_given_to_the_constructor(): + both = NonRepairable(LIFE, planned_cost=150, unplanned_cost=400) + assert (both.cp, both.cu) == (150.0, 400.0) + set_after = NonRepairable(LIFE) + set_after.set_costs_planned_and_unplanned(150, 400) + assert both.find_optimal_replacement() == pytest.approx( + set_after.find_optimal_replacement(), rel=1e-12 + ) + # One alone: failures need only the unplanned cost; the age + # replacement needs both. + assert NonRepairable(LIFE, unplanned_cost=400).cu == 400.0 + with pytest.raises(ValueError, match="costs not set"): + NonRepairable(LIFE, unplanned_cost=400).find_optimal_replacement() + with pytest.raises(ValueError, match="costs not set"): + NonRepairable(LIFE, planned_cost=150).find_optimal_replacement() + + +@pytest.mark.parametrize( + "costs, match", + [ + ({"planned_cost": -1.0}, "planned_cost"), + ({"unplanned_cost": float("inf")}, "unplanned_cost"), + ({"unplanned_cost": True}, "unplanned_cost"), + ({"planned_cost": "150"}, "planned_cost"), + ({"planned_cost": 400, "unplanned_cost": 150}, "less than"), + ], +) +def test_the_costs_are_checked(costs, match): + with pytest.raises(ValueError, match=match): + NonRepairable(LIFE, **costs) + + +def test_the_costs_are_given_by_name(): + with pytest.raises(TypeError): + NonRepairable(LIFE, None, 150, 400) diff --git a/repyability/tests/test_opportunistic_maintenance.py b/repyability/tests/test_opportunistic_maintenance.py index 8fda9dd1..837d59f9 100644 --- a/repyability/tests/test_opportunistic_maintenance.py +++ b/repyability/tests/test_opportunistic_maintenance.py @@ -9,6 +9,7 @@ import surpyval as surv from repyability import RepairableRBD +from repyability.rbd import _long_run from repyability.rbd import routes as r E, W = surv.Exponential.from_params, surv.Weibull.from_params @@ -353,7 +354,7 @@ def test_the_exact_cost_rate_charges_a_set_up_per_stop(): # stop, charged one set-up. priced = pair(setup=500.0) plain = pair(group=None) - frequencies = [priced._node_frequencies(node) for node in "ab"] + frequencies = [_long_run._node_frequencies(priced, node) for node in "ab"] actions = sum( failure + maintained for failure, maintained, _ in frequencies ) @@ -375,7 +376,8 @@ def test_the_exact_cost_rate_charges_a_set_up_per_stop(): maintenance_groups={"train": {"setup_cost": setup}}, ) assert rbd.has_costs == bool(setup) - assert rbd.expected_cost_rate() == 0.0 + with pytest.raises(ValueError, match="Nothing is priced"): + rbd.expected_cost_rate() def test_replacements_on_a_clock_share_a_set_up(): @@ -468,7 +470,7 @@ def test_groups_are_checked(components, groups, message): ({"interval": 60.0, "opportunity": "soon"}, "from 0 to the interval"), ( {"interval": 60.0, "policy": "block", "opportunity": 30.0}, - "only to the 'age' policy", + "only to the 'age' and 'calendar' policies", ), ], ) diff --git a/repyability/tests/test_parameter_sensitivity.py b/repyability/tests/test_parameter_sensitivity.py index 22731fcf..8784d661 100644 --- a/repyability/tests/test_parameter_sensitivity.py +++ b/repyability/tests/test_parameter_sensitivity.py @@ -162,3 +162,31 @@ def test_x_optional_for_fixed(rbd_parallel: NonRepairableRBD): def test_unknown_node_override_raises(rbd_series: NonRepairableRBD): with pytest.raises(ValueError): rbd_series.parameter_sensitivity(10.0, broken_nodes=["nope"]) + + +def test_the_shares_that_never_fail_and_are_dead_are_parameters(): + # A limited failure population and zero inflation (#267). In + # surpyval's terms lfp_p is the share that ever fails, the share dead + # on arrival f0 among it: R(t) = 1 - f0 - (lfp_p - f0) F(t), with F + # the Weibull's, so dR/dlfp_p = -F(t) and dR/df0 = -(1 - F(t)). They + # are levers, in the order of surpyval's covariance(), with the range + # (0, 1). + model = surv.Weibull.from_params([100, 2], lfp_p=0.6, f0=0.1) + rbd = NonRepairableRBD([("s", "a"), ("a", "t")], {"a": model}) + t = np.array([50.0, 150.0]) + f = 1 - np.exp(-((t / 100) ** 2)) + sens = rbd.parameter_sensitivity(t)["a"] + assert list(sens) == ["alpha", "beta", "lfp_p", "f0"] + np.testing.assert_allclose(model.sf(t), 1 - 0.1 - 0.5 * f, rtol=1e-12) + np.testing.assert_allclose(sens["lfp_p"], -f, rtol=1e-6) + np.testing.assert_allclose(sens["f0"], -(1 - f), rtol=1e-6) + levers = {lever.name: lever for lever in rbd.levers()} + assert levers["lfp_p"].value == 0.6 and levers["lfp_p"].bounds == (0, 1) + moved = rbd.with_levers({("a", "lfp_p"): 0.5}) + assert moved.reliabilities["a"].lfp_p == 0.5 + assert moved.reliabilities["a"].f0 == 0.1 + # A model with neither has no such levers. + plain = NonRepairableRBD( + [("s", "a"), ("a", "t")], {"a": surv.Weibull.from_params([100, 2])} + ) + assert list(plain.parameter_sensitivity(t)["a"]) == ["alpha", "beta"] diff --git a/repyability/tests/test_performance_equivalence.py b/repyability/tests/test_performance_equivalence.py index 2663286b..2fecd177 100644 --- a/repyability/tests/test_performance_equivalence.py +++ b/repyability/tests/test_performance_equivalence.py @@ -58,10 +58,11 @@ ) from repyability.non_repairable import NonRepairable from repyability.rbd import ( + _event_loop, + _events, _sampling, _streams, non_repairable_rbd, - repairable_rbd, standby_node, ) from repyability.rbd.helper_classes import ( @@ -76,6 +77,7 @@ from repyability.rbd.repeated_node import RepeatedNode from repyability.rbd.repeated_standby_node import RepeatedStandbyNode from repyability.rbd.results import CostResult +from repyability.tests import timeline_reference from repyability.utils.wrappers import numpy_seed W = surv.Weibull.from_params @@ -97,7 +99,7 @@ def no_fast_path(monkeypatch): monkeypatch.setattr(non_repairable_rbd, "row_sampler", lambda model: None) monkeypatch.setattr(standby_node, "inverse_sampler", lambda model: None) # A repairable diagram's streams sample through stream_sampler (#227). - monkeypatch.setattr(repairable_rbd, "stream_sampler", lambda model: None) + monkeypatch.setattr(_event_loop, "stream_sampler", lambda model: None) def assert_same(a, b, path="result"): @@ -928,9 +930,11 @@ def next_event(self, method="p"): "sub": component, }, ) - plan, complete = rbd._stream_plan(200.0, 1, False) + plan, complete = _event_loop._stream_plan(rbd, 200.0, 1, False) assert not complete - sources = rbd._streamed_components(_streams.Run(plan, reseed=True)) + sources = _event_loop._streamed_components( + rbd, _streams.Run(plan, reseed=True) + ) assert sources["sub"] is rbd.components["sub"] calls.clear() rbd.availability(200.0, mc_samples=5, seed=30) @@ -965,11 +969,11 @@ def test_nested_rbds_step_by_hand_as_before_after_a_simulation(): def test_event_queue_pops_in_priority_queue_order(): # Equal times are where two heaps could disagree; use many. rng = np.random.default_rng(27) - ours = repairable_rbd._EventQueue() + ours = _events._EventQueue() theirs: queue.PriorityQueue = queue.PriorityQueue() for step in range(2000): if rng.random() < 0.6 or theirs.empty(): - event = repairable_rbd.Event( + event = _events.Event( float(rng.integers(0, 20)), step, bool(rng.random() < 0.5) ) ours.put(event) @@ -1193,16 +1197,16 @@ def overlaps_one_at_a_time(starts, events, system_timeline, t_end): for c, start in enumerate(starts): own = [(t, e) for t, node, e in events if node == c] timeline = [(0.0, start)] + own + [(t_end, 0)] - joint_t, joint = repairable_rbd.combined_timeline( + joint_t, joint = timeline_reference.combined_timeline( timeline, system_timeline ) out.append( ( - repairable_rbd.time_at_status(timeline, 1), - repairable_rbd.intersection(joint_t, joint), - repairable_rbd.intersection(joint_t, 2 - joint), - repairable_rbd.union(joint_t, joint), - repairable_rbd.union(joint_t, 2 - joint), + timeline_reference.time_at_status(timeline, 1), + timeline_reference.intersection(joint_t, joint), + timeline_reference.intersection(joint_t, 2 - joint), + timeline_reference.union(joint_t, joint), + timeline_reference.union(joint_t, 2 - joint), ) ) return out diff --git a/repyability/tests/test_phased_mission.py b/repyability/tests/test_phased_mission.py index 3a6acf1f..3e8b72b1 100644 --- a/repyability/tests/test_phased_mission.py +++ b/repyability/tests/test_phased_mission.py @@ -389,7 +389,7 @@ def test_the_methods_are_checked(): mission = PhasedMission([("up", 1.0, SERIES)]) with pytest.raises(ValueError, match="'exact' or 'simulate'"): mission.reliability(method="guess") - with pytest.raises(ValueError, match="only to method='simulate'"): + with pytest.raises(ValueError, match="only with method='simulate'"): mission.reliability(mc_samples=10) with pytest.raises(ValueError): mission.reliability_interval(0) @@ -416,3 +416,470 @@ def test_series_and_parallel_phases_in_closed_form(): assert either.phase_failure_probabilities()["parallel"] == pytest.approx( (1.0 - ra[0]) * (1.0 - rb[0]), rel=1e-10 ) + + +# -- importance and cut sets (#316) ---------------------------------------- + + +MEASURES = ( + "birnbaum_importance", + "improvement_potential", + "risk_achievement_worth", + "risk_reduction_worth", + "criticality_importance", + "fussell_vesely", +) + + +def fails(mission, status): + """Whether the mission fails with its components failing in the phases + ``status`` gives (numbered from 0; ``len(phases)`` for never).""" + count = len(mission.phases) + for j, phase in enumerate(mission.phases): + works = {n: status.get(n, count) > j for n in phase.rbd.G.nodes} + works.update({n: status[o] > j for n, o in phase.rbd.repeated.items()}) + works[phase.rbd.input_node] = works[phase.rbd.output_node] = True + if not phase.rbd.is_system_working(works, "p"): + return True + return False + + +def by_brute_force(mission): + """Every measure, and the minimal cut sets, by conditioning on every + combination of the phases the components fail in: each assignment's + probability from the components' models at the phases' ends, a + component held by keeping the assignments with it so (its own + probability left out).""" + nodes = list(mission.components) + ends = np.array([phase.end for phase in mission.phases]) + count = len(ends) + chances, failed = {}, {} + for node, model in mission.components.items(): + sf = np.concatenate(([1.0], np.ravel(model.sf(ends)))) + chances[node] = np.append(sf[:-1] - sf[1:], sf[-1]) + failed[node] = np.ravel(model.ff(ends)) + assignments = [ + dict(zip(nodes, phases)) + for phases in itertools.product(range(count + 1), repeat=len(nodes)) + ] + outcome = { + tuple(status.values()): fails(mission, status) + for status in assignments + } + + def probabilities(held=None): + success = failure = 0.0 + for status in assignments: + if held is not None and status[held[0]] != held[1]: + continue + chance = np.prod( + [ + chances[n][f] + for n, f in status.items() + if held is None or n != held[0] + ] + ) + if outcome[tuple(status.values())]: + failure += chance + else: + success += chance + return success, failure + + # The minimal cut sets: the failing assignments' events (each failed + # component, by its phase), less those implied by another's: the same + # components failed by those phases or earlier ones, or fewer of them. + cuts = [ + {n: f for n, f in status.items() if f < count} + for status in assignments + if outcome[tuple(status.values())] + ] + minimal = [ + cut + for cut in cuts + if not any( + other != cut + and all(n in cut and cut[n] <= j for n, j in other.items()) + for other in cuts + ) + ] + cut_sets = { + frozenset((n, mission.phases[j].name) for n, j in cut.items()) + for cut in minimal + } + success, failure = probabilities() + out: dict = {name: {} for name in MEASURES} + out["criticality_success"] = {} + out["rare_event"] = {} + # The sums of products above are good to about 1e-13 of 1, and the + # ratios to that over the unreliability; the mission's own values + # keep a small measure's relative precision, which these do not. + out["tolerance"] = 1e-13 / failure + for node in nodes: + r_works, f_works = probabilities((node, count)) + r_failed, f_failed = probabilities((node, 0)) + out["birnbaum_importance"][node] = r_works - r_failed + out["improvement_potential"][node] = r_works - success + out["risk_achievement_worth"][node] = f_failed / failure + out["risk_reduction_worth"][node] = failure / f_works + out["criticality_importance"][node] = (r_works - success) / failure + out["criticality_success"][node] = (success - r_failed) / success + naming = [cut for cut in minimal if node in cut] + occurred = 0.0 + for status in assignments: + if any( + all(status[n] <= j for n, j in cut.items()) for cut in naming + ): + occurred += np.prod([chances[n][f] for n, f in status.items()]) + out["fussell_vesely"][node] = occurred / failure + out["rare_event"][node] = ( + sum( + np.prod([failed[n][j] for n, j in cut.items()]) + for cut in naming + ) + / failure + ) + return out, cut_sets + + +def check_against(mission, expected, cut_sets, rel=1e-12): + close = {"rel": rel, "abs": expected["tolerance"]} + for name in MEASURES: + got = getattr(mission, name)() + assert list(got) == list(mission.components) + for node, value in expected[name].items(): + assert got[node] == pytest.approx(value, **close), (name, node) + for node, value in expected["criticality_success"].items(): + got = mission.criticality_importance(kind="success")[node] + assert got == pytest.approx(value, **close), node + for node, value in expected["rare_event"].items(): + got = mission.fussell_vesely(method="rare_event")[node] + assert got == pytest.approx(value, **close), node + assert set(mission.minimal_cut_sets()) == cut_sets + + +@pytest.mark.parametrize("name", DIAGRAMS) +def test_a_one_phase_mission_has_the_diagram_s_measures(name): + diagram = DIAGRAMS[name] + mission = PhasedMission([("only", 40.0, diagram)]) + for measure in MEASURES: + got = getattr(mission, measure)() + expected = getattr(diagram, measure)(40.0) + assert set(got) == set(expected) + for node, value in expected.items(): + assert got[node] == pytest.approx(float(value), rel=1e-12) + for kind in ("failure", "success"): + got = mission.criticality_importance(kind=kind) + expected = diagram.criticality_importance(40.0, kind=kind) + for node, value in expected.items(): + assert got[node] == pytest.approx(float(value), rel=1e-12) + got = mission.fussell_vesely(method="rare_event") + expected = diagram.fussell_vesely(40.0, method="rare_event") + for node, value in expected.items(): + assert got[node] == pytest.approx(float(value), rel=1e-12) + assert mission.minimal_cut_sets() == [ + frozenset((node, "only") for node in cut) + for cut in diagram.minimal_cut_sets() + ] + + +def two_phase_mission(): + """Take-off on both engines and the gear, then cruise on either engine + and the navigation unit.""" + engine, gear, nav = W([200.0, 1.8]), W([500.0, 2.5]), E([0.002]) + takeoff = NonRepairableRBD( + [("s", "e1"), ("e1", "e2"), ("e2", "g"), ("g", "t")], + {"e1": engine, "e2": engine, "g": gear}, + ) + cruise = NonRepairableRBD( + [("s", "e1"), ("s", "e2"), ("e1", "n"), ("e2", "n"), ("n", "t")], + {"e1": engine, "e2": engine, "n": nav}, + ) + return PhasedMission( + [("take-off", 0.5, takeoff), ("cruise", 20.0, cruise)] + ) + + +def test_a_two_phase_mission_by_enumerating_the_failure_phases(): + mission = two_phase_mission() + expected, cut_sets = by_brute_force(mission) + check_against(mission, expected, cut_sets) + # The gear is needed only to take off: a cut set at the take-off, + # and a failure in the cruise does no harm. + assert mission.minimal_cut_sets() == [ + frozenset({("e1", "take-off")}), + frozenset({("e2", "take-off")}), + frozenset({("g", "take-off")}), + frozenset({("n", "cruise")}), + frozenset({("e1", "cruise"), ("e2", "cruise")}), + ] + assert mission.fussell_vesely()["g"] < mission.fussell_vesely()["n"] + for name in MEASURES: + assert getattr(mission, name)()["g"] > 0.0 + + +def test_missions_of_more_phases_and_kinds_by_enumerating(): + # Three phases: the landing's two-of-three engines with an auxiliary, + # and the navigation unit unused after the cruise. + engine, gear, nav, aux = ( + W([200.0, 1.8]), + W([500.0, 2.5]), + E([0.002]), + E([0.01]), + ) + models = {"e1": engine, "e2": engine, "g": gear, "n": nav, "x": aux} + takeoff = NonRepairableRBD( + [("s", "e1"), ("e1", "e2"), ("e2", "g"), ("g", "t")], + {k: models[k] for k in ("e1", "e2", "g")}, + ) + cruise = NonRepairableRBD( + [("s", "e1"), ("s", "e2"), ("e1", "n"), ("e2", "n"), ("n", "t")], + {k: models[k] for k in ("e1", "e2", "n")}, + ) + landing = NonRepairableRBD( + [ + ("s", "g"), + ("g", "e1"), + ("g", "e2"), + ("g", "x"), + ("e1", "v"), + ("e2", "v"), + ("x", "v"), + ("v", "t"), + ], + {**{k: models[k] for k in ("g", "e1", "e2", "x")}, "v": F(0.0)}, + k={"v": 2}, + ) + mission = PhasedMission( + [ + ("take-off", 0.5, takeoff), + ("cruise", 20.0, cruise), + ("landing", 0.5, landing), + ] + ) + expected, cut_sets = by_brute_force(mission) + check_against(mission, expected, cut_sets, rel=1e-11) + # A repeated node, a nested diagram and a fixed probability. + inner = NonRepairableRBD( + [("s", "p"), ("s", "q"), ("p", "t"), ("q", "t")], + {"p": E([0.02]), "q": E([0.02])}, + ) + first = NonRepairableRBD( + [ + ("s", "ps"), + ("ps", "a"), + ("a", "t"), + ("s", "ps2"), + ("ps2", "sub"), + ("sub", "t"), + ], + {"ps": W([300.0, 2.0]), "ps2": "ps", "a": UNITS["a"], "sub": inner}, + ) + check = NonRepairableRBD( + [("s", "sw"), ("sw", "a"), ("a", "t")], + {"sw": F(0.05), "a": UNITS["a"]}, + ) + mixed = PhasedMission([("first", 10.0, first), ("check", 0.0, check)]) + expected, cut_sets = by_brute_force(mixed) + check_against(mixed, expected, cut_sets) + + +@pytest.mark.parametrize( + "phases", + [ + [("up", 30.0, BRIDGE), ("on", 30.0, VOTE)], + [("a", 10.0, SERIES), ("b", 20.0, BRIDGE), ("c", 5.0, PARALLEL)], + ], + ids=["bridge then vote", "series, bridge, parallel"], +) +def test_the_measures_are_the_same_from_the_path_sets(phases, monkeypatch): + diagram = PhasedMission(phases) + monkeypatch.setattr(phased_mission, "METHOD", "paths") + paths = PhasedMission(phases) + for name in MEASURES: + got, expected = getattr(diagram, name)(), getattr(paths, name)() + assert list(got) == list(expected) + for node in got: + assert got[node] == pytest.approx(expected[node], rel=1e-10) + assert diagram.minimal_cut_sets() == paths.minimal_cut_sets() + assert diagram.analysis_routes()["fussell_vesely"].route == "exact" + assert paths.analysis_routes()["fussell_vesely"].route == "exact" + + +def test_a_reliable_mission_keeps_its_measures_precise(): + # As in ``test_a_reliable_mission_keeps_its_unreliability_precise``: + # the mission succeeds if "a" lasts 20 hours, whatever "b" does. + tiny = E([1e-9]) + pair = NonRepairableRBD( + [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], + {"a": tiny, "b": tiny}, + ) + one = NonRepairableRBD([("s", "a"), ("a", "t")], {"a": tiny}) + mission = PhasedMission([("both", 10.0, pair), ("one", 10.0, one)]) + failure = -np.expm1(-2e-8) + assert mission.birnbaum_importance() == {"a": 1.0, "b": 0.0} + potential = mission.improvement_potential() + assert potential["a"] == pytest.approx(failure, rel=1e-9) + assert potential["b"] == 0.0 + raw = mission.risk_achievement_worth() + assert raw["a"] == pytest.approx(1.0 / failure, rel=1e-9) + assert raw["b"] == pytest.approx(1.0, rel=1e-9) + rrw = mission.risk_reduction_worth() + assert rrw["a"] == np.inf and rrw["b"] == pytest.approx(1.0, rel=1e-9) + assert mission.criticality_importance() == pytest.approx( + {"a": 1.0, "b": 0.0}, rel=1e-9 + ) + # Given a success, "a" failed from the start would have failed it. + assert mission.criticality_importance(kind="success") == pytest.approx( + {"a": 1.0, "b": 0.0}, rel=1e-9 + ) + # "a" failed by the first phase fails the pair only with "b", but by + # the second phase alone: the later, weaker event is the cut set. + assert mission.minimal_cut_sets() == [frozenset({("a", "one")})] + for method in ("exact", "rare_event"): + assert mission.fussell_vesely(method=method) == pytest.approx( + {"a": 1.0, "b": 0.0}, rel=1e-9 + ) + + +def test_the_rare_event_sum_passes_the_exact_union(): + # On a bridge whose units are likely to fail, the cut sets overlap: + # the sum of their probabilities passes the union's, and 1. + likely = NonRepairableRBD(BRIDGE.G.edges, {n: F(0.8) for n in "abcde"}) + mission = PhasedMission([("up", 1.0, likely), ("on", 1.0, likely)]) + exact = mission.fussell_vesely() + summed = mission.fussell_vesely(method="rare_event") + assert exact["a"] == pytest.approx( + float(likely.fussell_vesely(1.0)["a"]), rel=1e-12 + ) + assert summed["a"] == pytest.approx( + float(likely.fussell_vesely(1.0, method="rare_event")["a"]), rel=1e-12 + ) + assert exact["a"] < 1.0 < summed["a"] + + +def test_the_measures_are_checked_and_refuse_a_large_diagram(monkeypatch): + mission = two_phase_mission() + with pytest.raises(ValueError, match="'failure' or 'success'"): + mission.criticality_importance(kind="both") + with pytest.raises(ValueError, match="'exact' or 'rare_event'"): + mission.fussell_vesely(method="simulate") + monkeypatch.setattr(phased_mission, "MAX_NODES", 3) + large = two_phase_mission() + for name in MEASURES: + with pytest.raises(NotImplementedError, match="too large"): + getattr(large, name)() + # The cut sets need no diagram; the sum of their probabilities needs + # the mission's unreliability, which does. + assert len(large.minimal_cut_sets()) == 5 + with pytest.raises(NotImplementedError, match="too large"): + large.fussell_vesely(method="rare_event") + + +def test_the_cut_sets_diagram_has_its_own_limit(monkeypatch): + # A mission whose own diagram fits in the limit but whose cut sets' + # does not refuses the exact Fussell-Vesely importance alone. + mission = PhasedMission([("up", 30.0, PARALLEL), ("on", 30.0, SERIES)]) + mission.reliability() + monkeypatch.setattr(phased_mission, "MAX_NODES", 4) + with pytest.raises(NotImplementedError, match="rare_event"): + mission.fussell_vesely() + assert mission.analysis_routes()["fussell_vesely"].route == "refused" + assert mission.analysis_routes()["birnbaum_importance"].route == "exact" + assert mission.fussell_vesely(method="rare_event")["a"] > 0.0 + + +# -- the 0.14 persona review -------------------------------------------------- + + +def test_a_junction_is_no_component_of_a_mission(): + # A PerfectReliability vote point never fails: it is folded out of the + # structure, so it is no component and has no importance. A mission of + # one phase is the diagram at the phase's end. + from repyability import PerfectReliability + + vote = NonRepairableRBD( + [ + ("s", "a"), + ("s", "b"), + ("s", "c"), + ("a", "j"), + ("b", "j"), + ("c", "j"), + ("j", "d"), + ("d", "t"), + ], + {"a": UNITS["a"], "b": UNITS["b"], "c": UNITS["c"], "d": UNITS["d"]} + | {"j": PerfectReliability}, + k={"j": 2}, + ) + mission = PhasedMission([("p", 50.0, vote)]) + assert list(mission.components) == ["a", "b", "c", "d"] + assert mission.sf() == pytest.approx(vote.sf(50.0), rel=1e-12) + assert mission.sf(method="simulate", mc_samples=4000, seed=1) == ( + pytest.approx(vote.sf(50.0), abs=0.03) + ) + checks = { + "birnbaum_importance": vote.birnbaum_importance(50.0), + "improvement_potential": vote.improvement_potential(50.0), + "risk_achievement_worth": vote.risk_achievement_worth(50.0), + "risk_reduction_worth": vote.risk_reduction_worth(50.0), + "criticality_importance": vote.criticality_importance(50.0), + "fussell_vesely": vote.fussell_vesely(50.0), + } + for name, expected in checks.items(): + found = getattr(mission, name)() + assert set(found) == {"a", "b", "c", "d"}, name + for node, value in expected.items(): + assert found[node] == pytest.approx(value, rel=1e-9), (name, node) + assert set(mission.minimal_cut_sets()) == { + frozenset((node, "p") for node in cut) + for cut in vote.minimal_cut_sets() + } + + +def test_a_component_with_no_reliability_refuses_the_exact_analyses(): + from repyability import StandbyModel + + warm = StandbyModel( + [W([100, 2]), W([120, 1.5]), W([90, 3])], k=2, dormancy_factor=0.3 + ) + assert warm.is_simulated + diagram = NonRepairableRBD( + [("s", "g"), ("g", "v"), ("v", "t")], {"g": warm, "v": W([200, 1.5])} + ) + mission = PhasedMission([("p", 60.0, diagram)]) + report = mission.analysis_routes() + with pytest.raises(NotImplementedError) as error: + mission.sf() + for name in ("sf", "birnbaum_importance", "fussell_vesely"): + assert report[name].route == "refused" + assert report[name].reason == str(error.value) + assert report[name].nodes == ("g",) + assert report["reliability_interval"].route == "simulated" + estimate = mission.reliability_interval(4000, seed=1) + assert estimate.method == "simulated" + simulated = mission.sf(method="simulate", mc_samples=4000, seed=1) + assert estimate.lower < simulated < estimate.upper or ( + simulated == pytest.approx(estimate.estimate) + ) + + +def test_an_interval_needs_two_missions(): + mission = PhasedMission([("up", 10.0, SERIES)]) + with pytest.raises(ValueError, match="mc_samples must be at least 2"): + mission.reliability_interval(1, seed=1) + with pytest.raises(ValueError, match="mc_samples must be at least 4"): + mission.reliability_interval(2, seed=1, antithetic=True) + assert mission.reliability_interval(2, seed=1).method == "simulated" + + +def test_simulation_options_need_method_simulate(): + mission = PhasedMission([("up", 10.0, SERIES)]) + for name in ("reliability", "unreliability", "sf", "ff"): + with pytest.raises(ValueError, match="only with method='simulate'"): + getattr(mission, name)(seed=1) + with pytest.raises(ValueError, match="only with method='simulate'"): + getattr(mission, name)(mc_samples=10) + with pytest.raises(ValueError, match="only with method='simulate'"): + mission.phase_failure_probabilities(seed=1) + assert mission.sf(method="simulate", mc_samples=10, seed=1) <= 1.0 diff --git a/repyability/tests/test_point_availability.py b/repyability/tests/test_point_availability.py index 95815ca7..87353fbe 100644 --- a/repyability/tests/test_point_availability.py +++ b/repyability/tests/test_point_availability.py @@ -20,8 +20,8 @@ from scipy.special import gamma as gamma_function import repyability.rbd._point_availability as point_availability -import repyability.rbd.repairable_rbd as repairable_rbd from repyability import NodeState, RepairableRBD +from repyability.rbd import _curves from repyability.tests.test_performance_equivalence import ( instrument_air, repairable_rbds, @@ -288,7 +288,7 @@ def test_units_replaced_together_go_down_together(monkeypatch): rbd = pair(spec, dict(spec)) windows = [10_000.0, 40_000.0] default = rbd.mission_availability(windows) - monkeypatch.setattr(repairable_rbd, "_POINT_STEPS", 4000) + monkeypatch.setattr(_curves, "_POINT_STEPS", 4000) fine = rbd.mission_availability(windows) assert np.abs(default - fine).max() < 2e-8 # Without the maintenance of units that each reach their age kept off @@ -409,9 +409,7 @@ def test_mission_repeats_its_calendar(monkeypatch): ) windows = np.array([333.0, 2000.0, 4321.5]) repeated = rbd.mission_availability(windows) - monkeypatch.setattr( - repairable_rbd, "_settling", lambda curves: (np.inf, None) - ) + monkeypatch.setattr(_curves, "_settling", lambda curves: (np.inf, None)) assert rbd.mission_availability(windows) == pytest.approx( repeated, abs=1e-12 ) @@ -493,7 +491,9 @@ def test_matches_the_simulation(name): def test_instant_repair_is_always_up(): rbd = alone(unit(W([100.0, 2.0]), "instant")) assert rbd.point_availability([0.0, 10.0, 1e4]).tolist() == [1, 1, 1] - assert rbd.mission_availability(100.0) == 1.0 + # To rounding: the integral of a constant 1 sums the quadrature's + # weights, exact to a few units of rounding in some numpy versions. + assert rbd.mission_availability(100.0) == pytest.approx(1.0, abs=1e-15) def test_units_dead_on_arrival_and_units_that_never_fail(): @@ -527,7 +527,7 @@ def test_the_error_falls_as_the_square_of_the_step(monkeypatch): windows = np.array([50.0, 300.0, 5000.0]) points, missions = [], [] for steps in [500, 2000, 8000]: - monkeypatch.setattr(repairable_rbd, "_POINT_STEPS", steps) + monkeypatch.setattr(_curves, "_POINT_STEPS", steps) points.append(rbd.point_availability(t)) missions.append(rbd.mission_availability(windows)) # Against the finest: the default grid is some 16 times closer than @@ -615,13 +615,13 @@ def spec(scale=100.0): {"a": spec(), "b": spec(), "c": spec(120.0)}, ) built = [] - plain = RepairableRBD._unit_curve + plain = _curves._unit_curve - def counted(self, node, *args, **kwargs): + def counted(rbd, node, *args, **kwargs): built.append(node) - return plain(self, node, *args, **kwargs) + return plain(rbd, node, *args, **kwargs) - monkeypatch.setattr(RepairableRBD, "_unit_curve", counted) + monkeypatch.setattr(_curves, "_unit_curve", counted) x = np.array([0.0, 30.0, 300.0]) shared = ( rbd.point_availability(x), @@ -633,7 +633,7 @@ def counted(self, node, *args, **kwargs): built.clear() rbd.point_availability(x, state={"b": NodeState(age=50.0)}) assert built == ["a", "b", "c"] - monkeypatch.setattr(RepairableRBD, "_curve_twin", lambda *a: None) + monkeypatch.setattr(_curves, "_curve_twin", lambda *a: None) apart = ( rbd.point_availability(x), rbd.mission_availability(1000.0), @@ -648,7 +648,7 @@ def test_a_curve_is_followed_about_as_far_as_it_takes_to_settle(): # is followed until it has kept within 1e-10 for a quarter of its # length, not four times as far (as before) when it has not yet. rbd = alone(unit(W([80.0, 1.6]), LN([0.5, 0.6]))) - curve = rbd._unit_curve("c", 5000.0) + curve = _curves._unit_curve(rbd, "c", 5000.0) off = np.abs(curve.at(curve.times) - curve.long_run) settled = curve.times[np.flatnonzero(off >= 1e-10)[-1]] tail = curve.times[len(curve.times) - len(curve.times) // 4 :] diff --git a/repyability/tests/test_preventive_maintenance.py b/repyability/tests/test_preventive_maintenance.py index 7edb5b48..7b4a2291 100644 --- a/repyability/tests/test_preventive_maintenance.py +++ b/repyability/tests/test_preventive_maintenance.py @@ -15,7 +15,8 @@ from scipy.special import gammainc from repyability import NonRepairable, RepairableRBD -from repyability.rbd.repairable_rbd import Event +from repyability.rbd import _long_run +from repyability.rbd._events import Event X = surv.ExactEventTime.from_params E = surv.Exponential.from_params @@ -137,16 +138,19 @@ def test_maintenance_that_takes_time_is_a_planned_outage(): assert per_run(result, result.system_restorations) == 9 assert result.system_failures == 0 assert result.mean_up_time == 40.0 - assert result.cost.by_category == { - "repair": 0.0, - "replace": 0.0, - "preventive": 50.0, - "inspection": 0.0, - "component_downtime": 60.0, - "system_downtime": 140.0, - "setup": 0.0, - } - assert result.cost.by_component == {"c": 110.0} + # The costs are the exact means, integrated: to rounding. + assert result.cost.by_category == pytest.approx( + { + "repair": 0.0, + "replace": 0.0, + "preventive": 50.0, + "inspection": 0.0, + "component_downtime": 60.0, + "system_downtime": 140.0, + "setup": 0.0, + } + ) + assert result.cost.by_component == pytest.approx({"c": 110.0}) fci = result.criticalities.failure_criticality_index # Planned outages are not failures. assert fci.per_system_failure.get("c", 0) == 0 @@ -259,7 +263,7 @@ def test_age_replacement_rate_is_the_non_repairable_rate(): # Maintained in zero time: never down, so no planned outages; the # instant repairs are zero-length failures. assert rbd.mean_availability() == 1.0 - assert rbd._outage_frequencies()[1] == 0.0 + assert _long_run._outage_frequencies(rbd)[1] == 0.0 assert rbd.mean_up_time() == pytest.approx( 1 / rbd.system_failure_frequency() ) @@ -364,7 +368,7 @@ def test_the_exact_long_run_values_match_a_long_simulation(): assert result.system_uptime / window == pytest.approx( rbd.mean_availability(), abs=0.002 ) - failures, planned = rbd._outage_frequencies() + failures, planned = _long_run._outage_frequencies(rbd) assert failures == rbd.system_failure_frequency() assert result.system_failures / window == pytest.approx(failures, rel=0.1) assert result.system_planned_outages / window == pytest.approx( @@ -482,9 +486,10 @@ def test_forced_nodes_are_not_maintained(): ({"interval": float("nan")}, "positive number"), ({"interval": "ten"}, "positive number"), ( - {"interval": 10, "policy": "calendar"}, - "'age', 'block' or 'condition'", + {"interval": 10, "policy": "weekly"}, + "'age', 'block', 'condition' or 'calendar'", ), + ({"policy": "calendar"}, "needs a calendar"), ({"interval": 10, "duration": "quick"}, "time-to-maintain model"), ({"interval": 10, "duration": 2.0}, "time-to-maintain model"), ({"interval": 10, "cost": -1.0}, "non-negative"), @@ -513,7 +518,8 @@ def test_a_zero_cost_prices_nothing(): } rbd = single(spec) assert not rbd.has_costs - assert rbd.cost(100.0, mc_samples=2, seed=1) is None + with pytest.raises(ValueError, match="Nothing is priced"): + rbd.cost(100.0, mc_samples=2, seed=1) spec["preventive"]["cost"] = 1.0 assert single(spec).has_costs diff --git a/repyability/tests/test_quadrature.py b/repyability/tests/test_quadrature.py index 429756ab..bd99f768 100644 --- a/repyability/tests/test_quadrature.py +++ b/repyability/tests/test_quadrature.py @@ -14,7 +14,7 @@ import surpyval as surv from repyability import RepairableRBD -from repyability.rbd import _quadrature +from repyability.rbd import _curves, _quadrature from repyability.rbd._point_availability import ChainDips E = surv.Exponential.from_params @@ -111,7 +111,7 @@ def systems(): def every_knot(rbd, t): """The mission availability over ``[0, t]`` summed between every knot of every curve (``t`` before the curves settle), as before #164.""" - curves = rbd._availability_curves(t, set()) + curves = _curves._availability_curves(rbd, t, set()) edges = np.unique( np.concatenate( [[0.0, t]] + [curve.knots(0.0, t) for curve in curves.values()] @@ -120,7 +120,7 @@ def every_knot(rbd, t): edges = edges[(edges >= 0.0) & (edges <= t)] a, b = edges[:-1], edges[1:] x, half = _quadrature.points(a, b) - up = rbd._curves_at(curves, x, set(), set(), "p") + up = _curves._curves_at(rbd, curves, x, set(), set(), "p") return float(_quadrature.summed(up, half).sum()) / t @@ -166,7 +166,7 @@ def counted(*args, **kwargs): monkeypatch.setattr(_quadrature, "refined", counted) value = rbd.mission_availability(t) - curves = rbd._availability_curves(t, set()) + curves = _curves._availability_curves(rbd, t, set()) knots = sum(len(c.knots(0.0, t)) for c in curves.values()) assert knots > 200_000 assert seen and seen[0] < 20_000 diff --git a/repyability/tests/test_rates.py b/repyability/tests/test_rates.py index d77d24b9..b0901e3f 100644 --- a/repyability/tests/test_rates.py +++ b/repyability/tests/test_rates.py @@ -13,7 +13,6 @@ import surpyval as surv import repyability.rbd._block_replacement as block_replacement -import repyability.rbd.repairable_rbd as repairable_rbd from repyability import ( BetaFactor, CCFGroup, @@ -21,7 +20,7 @@ NonRepairableRBD, RepairableRBD, ) -from repyability.rbd import _rates +from repyability.rbd import _crews, _curves, _rates E, W = surv.Exponential.from_params, surv.Weibull.from_params @@ -319,10 +318,10 @@ def test_a_maintenance_shorter_than_a_step_is_differentiated_exactly( rbd = maintained_briefly(policy) x = np.array([80.3, 81.0, 82.0, 160.6]) rate = rbd.availability_rate(x).rate - curve = rbd._availability_curves(200.0, set())["v"] + curve = _curves._availability_curves(rbd, 200.0, set())["v"] smoothed = _rates.differences(curve, x, 200.0) assert np.max(np.abs(smoothed / rate - 1.0)) > 0.005 - monkeypatch.setattr(repairable_rbd, "_POINT_STEPS", 16_000) + monkeypatch.setattr(_curves, "_POINT_STEPS", 16_000) monkeypatch.setattr(block_replacement, "_MIN_STEPS", 32_000) monkeypatch.setattr(block_replacement, "_MAX_STEPS", 64_000) finer = rbd.availability_rate(x).rate @@ -338,8 +337,8 @@ def test_a_nested_rbd_s_rate_is_its_nodes_by_their_importance(): {"a": inner, "b": unit(E([0.1])), "c": unit(W([30, 1.5]))}, ) x = np.array([80.3, 81.0, 160.6]) - nested = outer._availability_curves(200.0, set())["a"] - own = inner._availability_curves(200.0, set())["v"] + nested = _curves._availability_curves(outer, 200.0, set())["a"] + own = _curves._availability_curves(inner, 200.0, set())["v"] exact = _rates.derivative(own, x, 200.0) np.testing.assert_allclose( _rates.derivative(nested, x, 200.0), exact, rtol=1e-12 @@ -422,7 +421,7 @@ def test_with_one_crew_each_part_is_its_own_transitions(): ) x = np.array([0.0, 0.4, 2.0, 30.0]) rate = rbd.availability_rate(x) - chain = rbd._crew_chain() + chain = _crews._crew_chain(rbd) generator = chain.generator.toarray() a, b = (chain.nodes.index(n) for n in "ab") state = {s: k for k, s in enumerate(chain.states)} diff --git a/repyability/tests/test_rbd_imperfect_repair.py b/repyability/tests/test_rbd_imperfect_repair.py index 58a1055a..54596466 100644 --- a/repyability/tests/test_rbd_imperfect_repair.py +++ b/repyability/tests/test_rbd_imperfect_repair.py @@ -12,7 +12,7 @@ from repyability import RepairableRBD from repyability.rbd import routes as r -from repyability.rbd.repairable_rbd import _aged_life +from repyability.rbd._event_loop import _aged_life E, W, L = ( surv.Exponential.from_params, @@ -479,10 +479,6 @@ def test_minimal_repair_in_a_system_agrees_with_the_simulation(): minimal(W([100.0, 2.0]), replace_after=3), "replaced at its failure 3", ), - ( - dict(minimal(W([100.0, 2.0])), repairability=E([0.5])), - "repairs take time", - ), ( minimal(W([100.0, 2.0]), preventive={"interval": 50.0}), "preventive maintenance", @@ -492,12 +488,13 @@ def test_minimal_repair_in_a_system_agrees_with_the_simulation(): ids=[ "q below 1", "replaced", - "timed repairs", "maintained", "dead on arrival", ], ) def test_other_imperfect_repair_is_simulated_over_time(spec, blocker): + # Minimal repair with repairs that take time is numerical over a + # window too (#318): see test_minimal_repair_timed.py. rbd = single(spec) for call in ( lambda: rbd.expected_failures(50.0), diff --git a/repyability/tests/test_readme_capabilities.py b/repyability/tests/test_readme_capabilities.py new file mode 100644 index 00000000..942d2668 --- /dev/null +++ b/repyability/tests/test_readme_capabilities.py @@ -0,0 +1,39 @@ +"""The README's capability matrices are the code's (see +``repyability.tests.capabilities``, which makes them): a method added, +removed or made exact, or a refusal lifted, shows there.""" + +import pathlib + +from repyability.tests import capabilities + + +def test_the_readme_s_capability_matrices_are_current(): + readme = pathlib.Path(__file__).resolve().parents[2] / "README.md" + text = readme.read_text() + start = text.index(capabilities.START) + end = text.index(capabilities.END) + len(capabilities.END) + assert text[start:end] == capabilities.matrix(), ( + "The README's capability matrices are out of date: run " + "`python -m repyability.tests.capabilities` to write them." + ) + + +def test_every_public_method_has_a_row_or_is_listed_as_none(): + # Each model type's public analyses appear in the matrix, so that a new + # one is not left out of it. + named = { + method + for _, rows in capabilities.CAPABILITIES + for _, methods in rows + for method in methods + } + for name, cls in capabilities.MODELS.items(): + missing = sorted( + method + for method in dir(cls) + if not method.startswith("_") + and callable(getattr(cls, method)) + and method not in named + and method not in capabilities.NOT_ANALYSES + ) + assert not missing, (name, missing) diff --git a/repyability/tests/test_refused_inputs.py b/repyability/tests/test_refused_inputs.py index bb0ccac5..4a181c21 100644 --- a/repyability/tests/test_refused_inputs.py +++ b/repyability/tests/test_refused_inputs.py @@ -208,12 +208,12 @@ def degrading(): @pytest.mark.parametrize("model", MODELS) -def test_an_exact_mean_warns_of_options_it_ignores(model): +def test_an_exact_mean_refuses_options_it_would_ignore(model): node = model() exact = node.mean() - with pytest.warns(FutureWarning, match="ignores mc_samples, seed.*0.14"): - assert node.mean(mc_samples=10, seed=1) == exact - with pytest.raises(TypeError, match="only with method='simulate'"): + with pytest.raises(ValueError, match=r"mean\(\) takes mc_samples, seed"): + node.mean(mc_samples=10, seed=1) + with pytest.raises(ValueError, match="only with method='simulate'"): node.mean(method="exact", mc_samples=10) with warnings.catch_warnings(): warnings.simplefilter("error") diff --git a/repyability/tests/test_regression_node.py b/repyability/tests/test_regression_node.py index e508ffc3..d5675e0d 100644 --- a/repyability/tests/test_regression_node.py +++ b/repyability/tests/test_regression_node.py @@ -78,6 +78,50 @@ def test_mean_and_random(models): assert a.mean() == pytest.approx(node.mean(), rel=0.05) +def _nodes(models): + """A node of each kind with a proper curve: fixed covariates (AFT and + PO) and a schedule that raises the load at 50.""" + schedule = StepSchedule.from_changepoints([0, 50], [[0.0], [0.5]]) + return [ + RegressionNode(models["aft"], covariates=[0.2]), + RegressionNode(models["po"], covariates=[-0.4]), + RegressionNode(models["aft"], schedule=schedule), + ] + + +def test_the_mean_is_the_integral_of_the_survival_function(models): + # Integrated to the end of the tail, not cut off where the survival + # falls to 1e-4 as the tabulated grid was. + from scipy.integrate import quad + + for node in _nodes(models): + integral = sum( + quad( + lambda t: float(node.sf(t)), + a, + b, + epsabs=0.0, + epsrel=1e-12, + limit=500, + )[0] + for a, b in ((0.0, 50.0), (50.0, np.inf)) + ) + assert node.mean() == pytest.approx(integral, rel=1e-9) + + +def test_draws_invert_the_distribution(models): + # Each uniform u is the lifetime t with F(t) = u: the model's quantile + # at fixed covariates, the cumulative hazard inverted along a schedule + # (no grid, so the tail is drawn too). + u = np.array([1e-9, 0.01, 0.3, 0.7, 0.99, 1.0 - 1e-9]) + for node in _nodes(models): + t = node._draw(u) + np.testing.assert_allclose(node.ff(t), u, rtol=1e-6) + schedule = _nodes(models)[2] + exact = schedule._draw(u) + np.testing.assert_allclose(schedule.ff(exact), u, rtol=1e-12) + + def test_semiparametric_mean_random_raise_clearly(models): # A semiparametric Cox baseline has no proper tail, so MTTF is undefined; # mean/random must raise rather than return a wrong number; sf works. @@ -272,3 +316,89 @@ def test_schedule_mean_and_random(models): np.random.seed(0) b = node.random(2000) assert np.allclose(a, b) and (a > 0).all() + + +def test_a_covariate_path_is_integrated_and_drawn(models): + # surpyval's CovariatePath moves linearly between its points: its mean + # and draws work as a step schedule's, and saving it is refused by name. + from surpyval.univariate.regression import CovariatePath + + path = CovariatePath.from_points([0, 50, 100], [[0.0], [0.5], [0.2]]) + node = RegressionNode(models["aft"], schedule=path) + assert node.mean() == pytest.approx( + float(models["aft"].mean_tvc(path)), rel=1e-6 + ) + u = np.array([0.01, 0.5, 0.99]) + np.testing.assert_allclose(node.ff(node._draw(u)), u, rtol=1e-12) + with pytest.raises(NotImplementedError, match="CovariatePath"): + node.to_dict() + + +# -- covariates as levers (#272) -------------------------------------------- + + +def test_the_covariates_are_levers(): + # An exponential AFT model: life 100 at load 1 and 25 at load 2, so + # theta(z) = 100 * 4 ** -(z - 1), R = exp(-t / theta) and dR/dz = + # -R * t * ln(4) / theta. + x = np.array([50.0, 100.0, 150.0, 12.5, 25.0, 37.5]) + load = np.array([[1.0], [1.0], [1.0], [2.0], [2.0], [2.0]]) + model = surv.ExponentialAFT.fit(x, Z=load) + rbd = NonRepairableRBD( + [("s", "a"), ("a", "t")], + {"a": RegressionNode(model, covariates=[1.0])}, + ) + (lever,) = rbd.levers() + assert (lever.key, lever.name, lever.value) == ("a", "covariate.0", 1.0) + assert lever.bounds == (-np.inf, np.inf) + t = np.array([20.0, 50.0]) + theta = float(model.mean(np.array([[1.0]]))) + expected = -np.exp(-t / theta) * t * np.log(4.0) / theta + np.testing.assert_allclose( + rbd.parameter_sensitivity(t)["a"]["covariate.0"], expected, rtol=1e-5 + ) + moved = rbd.with_levers({("a", "covariate.0"): 2.0}) + np.testing.assert_allclose( + moved.sf(t), model.sf(t, np.array([[2.0], [2.0]])), rtol=1e-12 + ) + + +def test_covariates_are_named_by_the_models_features(): + import pandas as pd + + rng = np.random.default_rng(3) + frame = pd.DataFrame( + { + "x": rng.weibull(2.0, 200) * 100, + "load": rng.uniform(0.5, 1.5, 200), + "temp": rng.uniform(20.0, 60.0, 200), + } + ) + model = surv.WeibullPH.fit_from_df( + frame, x_col="x", Z_cols=["load", "temp"] + ) + rbd = NonRepairableRBD( + [("s", "a"), ("a", "t")], + {"a": RegressionNode(model, covariates=[1.0, 40.0])}, + ) + assert [lever.name for lever in rbd.levers()] == [ + "covariate.load", + "covariate.temp", + ] + assert list(rbd.parameter_sensitivity(50.0)["a"]) == [ + "covariate.load", + "covariate.temp", + ] + + +def test_a_node_along_a_schedule_has_no_levers(): + x = np.array([50.0, 100.0, 150.0, 12.5, 25.0, 37.5]) + load = np.array([[1.0], [1.0], [1.0], [2.0], [2.0], [2.0]]) + model = surv.ExponentialAFT.fit(x, Z=load) + schedule = StepSchedule.from_changepoints([0, 50], [[1.0], [2.0]]) + rbd = NonRepairableRBD( + [("s", "a"), ("a", "t")], + {"a": RegressionNode(model, schedule=schedule)}, + ) + assert rbd.levers() == [] + assert rbd.parameter_sensitivity(10.0) == {} diff --git a/repyability/tests/test_removed_in_0_12.py b/repyability/tests/test_removed_in_0_12.py index dd03067c..796f0afb 100644 --- a/repyability/tests/test_removed_in_0_12.py +++ b/repyability/tests/test_removed_in_0_12.py @@ -104,13 +104,13 @@ def test_repairables_old_name_raises(): def test_simulation_options_need_method_simulate(): rbd = NonRepairableRBD(ONE, {"a": W([100, 2])}) for method in ("mean", "mean_time_to_failure"): - with pytest.raises(TypeError, match="only with method='simulate'"): + with pytest.raises(ValueError, match="only with method='simulate'"): getattr(rbd, method)(mc_samples=100) - with pytest.raises(TypeError, match="seed"): + with pytest.raises(ValueError, match="seed"): getattr(rbd, method)(seed=1) assert rbd.mean(method="simulate", mc_samples=100, seed=1) > 0 node = RepeatedNode(W([100, 2]), 2, "series") - with pytest.raises(TypeError, match="only with method='simulate'"): + with pytest.raises(ValueError, match="only with method='simulate'"): node.mean(mc_samples=100) diff --git a/repyability/tests/test_removed_in_0_14.py b/repyability/tests/test_removed_in_0_14.py new file mode 100644 index 00000000..28c04c57 --- /dev/null +++ b/repyability/tests/test_removed_in_0_14.py @@ -0,0 +1,89 @@ +"""What 0.13 deprecated is gone in 0.14: ``optimal_inspection_intervals``' +``offsets``, renamed ``offset_shares`` (#222); calling +``CapacityDistribution.mean()``, a property since 0.13 (#235); the +simulation options given to the exact ``mean`` of a ``StandbyModel``, +``LoadSharingModel`` or ``DegradingNode`` (#233); and ``rbd._sensitivity``'s +``levers`` and ``Lever``, the names callers used before the levers were +public (#244).""" + +import numpy as np +import pytest +import surpyval as surv + +from repyability import ( + BetaFactor, + CCFGroup, + DegradingNode, + LoadSharingModel, + NonRepairableRBD, + RepairableRBD, + StandbyModel, +) +from repyability.rbd import _sensitivity + +W, E = surv.Weibull.from_params, surv.Exponential.from_params +PAIR = [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")] + + +def valve(): + return { + "reliability": E([2e-6]), + "repairability": "instant", + "inspection": {"interval": 8760.0, "cost": 500.0}, + } + + +def test_offsets_is_refused_for_offset_shares(): + pair = RepairableRBD( + [("s", "v1"), ("s", "v2"), ("v1", "t"), ("v2", "t")], + {"v1": valve(), "v2": valve()}, + ccf_groups=[CCFGroup(["v1", "v2"], BetaFactor(0.1))], + ) + with pytest.raises(TypeError, match="offsets"): + pair.optimal_inspection_intervals(allowed=[8760.0], offsets=[0.0, 0.5]) + plan = pair.optimal_inspection_intervals( + allowed=[8760.0], offset_shares=[0.0, 0.5] + ) + assert plan.offsets == {"v1": 0.0, "v2": 4380.0} + + +def test_the_capacity_mean_is_not_called(): + rbd = NonRepairableRBD( + PAIR, {"a": W([100, 2]), "b": W([90, 2])}, capacity={"a": 1, "b": 2} + ) + capacity = rbd.capacity_distribution(50.0) + assert type(capacity.mean) is float + with pytest.raises(TypeError, match="not callable"): + capacity.mean() + over_time = rbd.capacity_distribution([10.0, 50.0]) + assert type(over_time.mean) is np.ndarray + with pytest.raises(TypeError, match="not callable"): + over_time.mean() + + +def aft(): + x = np.array([50.0, 100.0, 150.0, 12.5, 25.0, 37.5]) + load = np.array([[1.0], [1.0], [1.0], [2.0], [2.0], [2.0]]) + return surv.ExponentialAFT.fit(x, Z=load) + + +@pytest.mark.parametrize( + "build", + [ + lambda: StandbyModel([E([0.01]), E([0.01])]), + lambda: LoadSharingModel([aft(), aft()], load=2.0, k=1), + lambda: DegradingNode([(100, W([1000, 2])), (50, E([1 / 500]))]), + ], + ids=["standby", "load sharing", "degrading"], +) +@pytest.mark.parametrize("name", ["mc_samples", "seed"]) +def test_an_exact_means_simulation_options_are_refused(build, name): + node = build() + with pytest.raises(ValueError, match=f"takes {name} only with"): + node.mean(**{name: 10}) + assert node.mean(method="simulate", mc_samples=200, seed=1) > 0.0 + + +def test_the_levers_private_names_are_gone(): + assert not hasattr(_sensitivity, "levers") + assert not hasattr(_sensitivity, "Lever") diff --git a/repyability/tests/test_repair_crews.py b/repyability/tests/test_repair_crews.py index 201573c0..597bd15d 100644 --- a/repyability/tests/test_repair_crews.py +++ b/repyability/tests/test_repair_crews.py @@ -9,7 +9,7 @@ import surpyval as surv from repyability import RBD, RepairableRBD -from repyability.rbd import _compiled +from repyability.rbd import _compiled, _crews, _event_loop from repyability.rbd import routes as r E, W, L = ( @@ -218,7 +218,7 @@ def test_the_report_and_the_engines_follow_the_crews(monkeypatch): assert report["availability"].route == r.SIMULATED # numba's own loop simulates the crews (#155); an engine of the # interface's version is not given them. - plan = system(1)._stream_plan(1.0, 0, False)[0] + plan = _event_loop._stream_plan(system(1), 1.0, 0, False)[0] assert _compiled.unsupported(system(1), plan, None) == "repair crews" assert _compiled.unsupported(system(1), plan, None, numba=True) is None monkeypatch.setattr(_compiled, "available", lambda: True) @@ -241,7 +241,7 @@ def test_a_nested_rbd_has_crews_of_its_own(): {"p": inner, "q": dict(unit)}, repair_crews=1, ) - assert not outer._crews_limited() + assert not _crews._crews_limited(outer) with pytest.raises(NotImplementedError, match="repair crew"): outer.mean_availability() result = outer.availability( diff --git a/repyability/tests/test_repairable_ccf.py b/repyability/tests/test_repairable_ccf.py index 3b2b1028..366131fa 100644 --- a/repyability/tests/test_repairable_ccf.py +++ b/repyability/tests/test_repairable_ccf.py @@ -618,6 +618,38 @@ def components(): ) +@pytest.mark.parametrize("interval", [99.9, 100.0, 100.001]) +def test_a_test_at_the_period_s_end_is_not_rounded_past_it(interval): + # Three tests of 99.9 end at the period of the full tests, 299.7, + # where the chain's last test, worked out as 3 * 99.9, rounds just + # past it, and would be skipped: the full test is lost, and the + # unavailability doubles. With no shared cause the group is the + # independent members, at every interval alike. + def components(): + return { + "a": hidden(1e-3, interval, coverage=0.8, full_test=3 * interval), + "b": hidden( + 1e-3, + interval, + offset=50.0, + coverage=0.8, + full_test=3 * interval, + ), + } + + grouped = RepairableRBD( + PAIR, components(), ccf_groups=[CCFGroup(["a", "b"], BetaFactor(0.0))] + ) + plain = RepairableRBD(PAIR, components()) + assert grouped.mean_unavailability() == pytest.approx( + plain.mean_unavailability(), rel=1e-6 + ) + x = np.array([30.0 * 3 * interval + 20.0]) + assert grouped.point_unavailability(x) == pytest.approx( + plain.point_unavailability(x), rel=1e-6 + ) + + def test_a_shared_cause_raises_the_members_importance(): # Tested members: a shared cause makes the pair fail together, so a # member down says more of the other being down. diff --git a/repyability/tests/test_repairable_cost.py b/repyability/tests/test_repairable_cost.py index 32258cf1..e18913e4 100644 --- a/repyability/tests/test_repairable_cost.py +++ b/repyability/tests/test_repairable_cost.py @@ -9,12 +9,15 @@ """ import json +import warnings import numpy as np import pytest import surpyval as surv from repyability import RepairableRBD +from repyability.rbd import routes +from repyability.rbd.routes import Refused # MTTF = 10, MTTR = 1 -> A = 10/11, omega = 1/11 RELIABILITY = surv.Exponential.from_params([0.1]) @@ -66,17 +69,19 @@ def test_terms_sum(): # -- everything is optional ------------------------------------------------ -def test_no_costs_is_zero_and_reports_unpriced(): +def test_no_costs_reports_unpriced_and_refuses(): rbd = one_component() assert rbd.has_costs is False - assert rbd.expected_cost_rate() == 0.0 + with pytest.raises(Refused, match="Nothing is priced"): + rbd.expected_cost_rate() def test_zero_costs_are_not_a_cost_model(): # Explicit zeros price nothing, so there is still no cost model to run. rbd = one_component({"repair_cost": 0.0}, downtime_cost_rate=0.0) assert rbd.has_costs is False - assert rbd.expected_cost_rate() == 0.0 + with pytest.raises(Refused, match="Nothing is priced"): + rbd.expected_cost_rate() def test_any_single_cost_is_enough(): @@ -189,7 +194,8 @@ def test_uncosted_rbd_still_round_trips(): rbd = one_component() restored = RepairableRBD.from_json(rbd.to_json()) assert restored.has_costs is False - assert restored.expected_cost_rate() == 0.0 + with pytest.raises(Refused, match="Nothing is priced"): + restored.expected_cost_rate() # -- instant repair (zero repair time) ------------------------------------- @@ -347,9 +353,27 @@ def test_cost_rides_along_on_availability(): assert result.cost.mean > 0.0 -def test_cost_is_none_when_nothing_is_priced(): +def test_the_cost_analyses_refuse_when_nothing_is_priced(): + # A diagram with nothing priced has no cost model to evaluate: the + # cost analyses refuse, as every analysis refuses a diagram with + # nothing it works on (routes.Refused), and the routes say so. rbd = one_component() - assert rbd.cost(t_simulation=100.0, mc_samples=10, seed=0) is None + assert not rbd.has_costs + report = rbd.analysis_routes() + calls = { + "cost": lambda: rbd.cost(t_simulation=100.0, mc_samples=10, seed=0), + "expected_cost": lambda: rbd.expected_cost(100.0), + "expected_cost_rate": lambda: rbd.expected_cost_rate(), + "total_cost": lambda: rbd.total_cost(100.0), + } + for name, call in calls.items(): + with pytest.raises(Refused, match="Nothing is priced") as error: + call() + assert report[name].route == routes.REFUSED + assert report[name].reason == str(error.value) + with pytest.raises(Refused, match="Nothing is priced"): + rbd.compare(rbd, 100.0, mc_samples=10, seed=0, quantity="cost") + # The availability simulation still runs, with no cost result. assert ( rbd.availability(t_simulation=100.0, mc_samples=10, seed=0).cost is None @@ -452,3 +476,25 @@ def test_cost_distributions_survive_a_json_round_trip(): a = rbd.cost(t_simulation=100.0, mc_samples=20, seed=1) b = restored.cost(t_simulation=100.0, mc_samples=20, seed=1) assert np.array_equal(a.samples, b.samples) + + +def test_a_nested_rbd_that_carries_costs_is_warned_of(): + # The outer diagram does not count a nested diagram's costs: it says + # so once, as it is built, naming the node; a nested diagram with + # nothing priced is not warned of, and a diagram rebuilt with the + # same nested one (a lever moved) says nothing new. + inner = one_component({"repair_cost": 100.0}, downtime_cost_rate=7.0) + plain = one_component() + edges = [("s", "x"), ("x", "C"), ("C", "t")] + spec = {"reliability": RELIABILITY, "repairability": REPAIRABILITY} + with pytest.warns( + UserWarning, match="Node 'x' is a nested RBD that" + ) as record: + outer = RepairableRBD(edges, {"x": inner, "C": spec}) + assert len(record) == 1 + assert not outer.has_costs + with warnings.catch_warnings(): + warnings.simplefilter("error") + RepairableRBD(edges, {"x": plain, "C": spec}) + outer.with_levers({("C", "reliability.failure_rate"): 0.2}) + outer.parameter_sensitivity(x=[10.0]) diff --git a/repyability/tests/test_repairable_metrics.py b/repyability/tests/test_repairable_metrics.py index 8c3684e7..9fde0cd8 100644 --- a/repyability/tests/test_repairable_metrics.py +++ b/repyability/tests/test_repairable_metrics.py @@ -123,3 +123,16 @@ def test_result_derived_properties_zero_guards(): assert result.mean_up_time == float("inf") assert result.failure_frequency == 0.0 assert np.isfinite(result.mean_down_time) + + +def test_node_availability_lists_the_components_alone(): + # As node_sf and the importance measures do: the input and output + # nodes are no components. + rbd = RepairableRBD( + [("s", "a"), ("a", "b"), ("b", "t")], + {"a": _comp(0.2, 1.0), "b": _comp(0.5, 2.0)}, + ) + availability = rbd.node_availability() + assert list(availability) == ["a", "b"] + assert availability == pytest.approx({"a": 1 / 1.2, "b": 2 / 2.5}) + assert set(rbd.node_availability()) == set(rbd.birnbaum_importance()) diff --git a/repyability/tests/test_repairable_rbd.py b/repyability/tests/test_repairable_rbd.py index 3b2afde8..4b238cdf 100644 --- a/repyability/tests/test_repairable_rbd.py +++ b/repyability/tests/test_repairable_rbd.py @@ -481,3 +481,22 @@ def get_next_event(): assert pytest.approx(exp_t) == actual_t[:-1] assert pytest.approx(exp_availability) == actual_availability[:-1] + + +def test_one_nested_rbd_object_for_two_nodes_is_refused_when_built(): + # A nested diagram carries its simulation state, so one object given + # for two nodes is refused as the diagram is built, naming the nodes + # (it failed late, in a simulation); a deep copy is its own. + import copy + + E = surv.Exponential.from_params + inner = RepairableRBD( + [("s", "a"), ("a", "t")], + {"a": {"reliability": E([0.2]), "repairability": E([1.0])}}, + ) + edges = [("s", "x"), ("s", "y"), ("x", "t"), ("y", "t")] + with pytest.raises(ValueError, match="'x' and 'y' are given one"): + RepairableRBD(edges, {"x": inner, "y": inner}) + twice = RepairableRBD(edges, {"x": inner, "y": copy.deepcopy(inner)}) + assert twice.mean_availability() == pytest.approx(1 - (1 / 6) ** 2) + twice.availability(10.0, mc_samples=10, seed=0) diff --git a/repyability/tests/test_repairable_sensitivity.py b/repyability/tests/test_repairable_sensitivity.py index 55ef606f..efd92820 100644 --- a/repyability/tests/test_repairable_sensitivity.py +++ b/repyability/tests/test_repairable_sensitivity.py @@ -397,3 +397,36 @@ def test_what_it_refuses(): ) with pytest.raises(NotImplementedError): imperfect.parameter_sensitivity() + + +def test_the_q_of_a_minimally_repaired_unit_is_left_out_over_time(): + # Minimal repair (q = 1) is the one imperfect repair the values over + # time take, so the derivative in q does not exist there: the + # sensitivities over time leave the lever out (they reported nan), + # while levers() still lists it for with_levers. + W = surv.Weibull.from_params + rbd = a_b_then_c( + a={ + "reliability": W([10.0, 1.5]), + "repairability": surv.LogNormal.from_params([1.5, 0.5]), + "repair": {"model": "kijima1", "q": 1.0}, + }, + b=exponential(0.01, 0.1), + c=exponential(0.02, 0.1), + ) + for asked in ({"x": [20.0]}, {"window": 50.0}): + report = rbd.parameter_sensitivity(**asked) + assert "repair.q" not in report["a"] + assert set(report["a"]) == { + "reliability.alpha", + "reliability.beta", + "repairability.mu", + "repairability.sigma", + } + assert all( + np.isfinite(np.ravel(v)).all() for v in report["a"].values() + ) + levers = [lever for lever in rbd.levers() if lever.key == "a"] + assert "repair.q" in {lever.name for lever in levers} + moved = rbd.with_levers({("a", "repair.q"): 0.9}) + assert moved.analysis_routes()["point_availability"].route == "refused" diff --git a/repyability/tests/test_repairable_standby.py b/repyability/tests/test_repairable_standby.py index 557df066..c08d0cab 100644 --- a/repyability/tests/test_repairable_standby.py +++ b/repyability/tests/test_repairable_standby.py @@ -11,7 +11,7 @@ import surpyval as surv from repyability import RBD, RepairableRBD -from repyability.rbd import _compiled +from repyability.rbd import _compiled, _event_loop from repyability.rbd import routes as r E, W, L = ( @@ -321,7 +321,7 @@ def test_runs_with_groups_are_reproducible_and_compiled(monkeypatch): # numba's own loop runs them (#155), not an engine of the interface's # version. alone = group(0.02, 0.25, units=3) - plan = alone._stream_plan(1.0, 0, False)[0] + plan = _event_loop._stream_plan(alone, 1.0, 0, False)[0] assert _compiled.unsupported(alone, plan, None) == "standby groups" assert _compiled.unsupported(alone, plan, None, numba=True) is None with monkeypatch.context() as patched: diff --git a/repyability/tests/test_repairable_uncertainty.py b/repyability/tests/test_repairable_uncertainty.py index 02e44d6e..f0751ddd 100644 --- a/repyability/tests/test_repairable_uncertainty.py +++ b/repyability/tests/test_repairable_uncertainty.py @@ -235,3 +235,51 @@ def test_quasi_random_draws_in_a_nonrepairable_diagram(): sampling="sobol", ) assert importance.first_order[("p1", "p2")] == pytest.approx(1.0, abs=0.02) + + +def test_no_fit_refuses_the_default_and_the_routes_say_so(): + # With no fitted model the default uncertainty has nothing to draw: a + # Refused, which the routes quote (the methods stay simulated: given + # the uncertain nodes, they run). The fits inside a nested RBD are + # not drawn, and the message says so. + from repyability.rbd.routes import Refused + + rbd = RepairableRBD( + SERIES, + { + "pump": {**unit(E([0.01]), E([0.5])), "repair_cost": 1.0}, + "valve": unit(E([0.005]), E([0.5])), + }, + ) + with pytest.raises(Refused, match="No component's model is a") as error: + rbd.mean_availability_uncertainty(n_draws=3, seed=0) + for name in ( + "mean_availability_uncertainty", + "point_availability_uncertainty", + "mission_availability_uncertainty", + "expected_cost_rate_uncertainty", + ): + route = rbd.analysis_routes()[name] + assert route.route == "simulated" + assert str(error.value) in route.reason + assert "nested" not in str(error.value) + inner = RepairableRBD( + SERIES, + { + "pump": unit(fitted(10.0, 20), E([0.5])), + "valve": unit(E([0.01]), E([0.5])), + }, + ) + outer = RepairableRBD( + [("s", "sub"), ("sub", "c"), ("c", "t")], + {"sub": inner, "c": unit(E([0.02]), E([0.5]))}, + ) + with pytest.raises(Refused, match="inside nested RBD 'sub' are not"): + outer.mean_availability_uncertainty(n_draws=3, seed=0) + assert "nested RBD 'sub'" in ( + outer.analysis_routes()["mean_availability_uncertainty"].reason + ) + with pytest.raises(ValueError, match="not drawn here"): + outer.mean_availability_uncertainty({"sub": "fit"}, n_draws=3, seed=0) + # The nested diagram's own methods take them. + assert inner.mean_availability_uncertainty(n_draws=3, seed=0).std > 0 diff --git a/repyability/tests/test_result_objects.py b/repyability/tests/test_result_objects.py index dae900ca..5c462310 100644 --- a/repyability/tests/test_result_objects.py +++ b/repyability/tests/test_result_objects.py @@ -90,6 +90,33 @@ def test_the_results_of_the_exact_methods_go_through_json(repairable): assert as_json(results[4])["members"] == ["a", "b"] +def test_the_single_unit_policies_go_through_json(): + from repyability import ( + FailureLimitPolicy, + MaintenancePolicy, + NonRepairable, + ) + + policy = NonRepairable( + W([1000, 2.5]), planned_cost=1.0, unplanned_cost=5.0 + ).optimal_replacement_policy() + assert isinstance(policy, MaintenancePolicy) + limit = FailureLimitPolicy(7, 0.031) + for result in (policy, limit): + assert isinstance(result, _ResultMapping) + out = as_json(result) + assert set(out) == fields_of(result) + assert out["cost_rate"] == result.cost_rate + assert result["cost_rate"] == result.cost_rate + assert as_json(policy)["interval"] == policy.interval + assert as_json(limit)["failure_count"] == 7 + assert ( + repr(limit) == "FailureLimitPolicy(failure_count=7, cost_rate=0.031)" + ) + with pytest.raises(dataclasses.FrozenInstanceError): + policy.interval = 1.0 + + def test_a_simulated_cost_goes_through_json(repairable): cost = repairable.cost(1000.0, mc_samples=200, seed=1) out = as_json(cost) @@ -207,8 +234,7 @@ def test_the_capacity_mean_is_a_property(): warnings.simplefilter("error") assert capacity.mean == pytest.approx(expected) assert float(capacity.mean) == pytest.approx(expected) - with pytest.warns(FutureWarning, match="0.14"): - assert capacity.mean() == pytest.approx(expected) + assert type(capacity.mean) is float def test_spares_mean_is_still_a_property(): diff --git a/repyability/tests/test_shards.py b/repyability/tests/test_shards.py index cab3f60c..f06b6076 100644 --- a/repyability/tests/test_shards.py +++ b/repyability/tests/test_shards.py @@ -20,13 +20,9 @@ import surpyval as surv from repyability import NodeState, RepairableRBD, SimulationChunk, run_shard -from repyability.rbd import _streams -from repyability.rbd.repairable_rbd import ( - _WORKER, - _simulate_block, - _start_worker, - _Tally, -) +from repyability.rbd import _event_loop, _streams +from repyability.rbd._event_loop import _WORKER, _simulate_block, _start_worker +from repyability.rbd._tally import _Tally from repyability.rbd.shards import main from repyability.tests.catalogue import systems_of_every_kind from repyability.tests.test_performance_equivalence import binomial_first @@ -324,7 +320,8 @@ def test_workers_send_back_their_blocks_totals(): assert block._rows == [] and block.changes == [] assert len(block.uptimes) == 50 # And the same totals as the block's simulations added one by one. - serial = rbd._run( + serial = _event_loop._run( + rbd, 100.0, set(), set(), @@ -343,9 +340,11 @@ def test_workers_send_back_their_blocks_totals(): def test_parallel_workers_keep_each_simulations_replacements(): rbd = plant() run = dict(entropy=_streams.entropy_of(2), replacements=True) - serial = rbd._run(100.0, set(), set(), "p", 600, False, None, **run) - parallel = rbd._run( - 100.0, set(), set(), "p", 600, False, None, jobs=2, **run + serial = _event_loop._run( + rbd, 100.0, set(), set(), "p", 600, False, None, **run + ) + parallel = _event_loop._run( + rbd, 100.0, set(), set(), "p", 600, False, None, jobs=2, **run ) assert parallel.replacements == serial.replacements assert len(parallel.replacements) == 600 diff --git a/repyability/tests/test_simulation_chunks.py b/repyability/tests/test_simulation_chunks.py index 718a74cf..c4b3d017 100644 --- a/repyability/tests/test_simulation_chunks.py +++ b/repyability/tests/test_simulation_chunks.py @@ -14,7 +14,8 @@ from repyability import NonRepairableRBD, RepairableRBD, SimulationChunk from repyability.rbd import _montecarlo as montecarlo from repyability.rbd._exact import ExactSum, expansion -from repyability.rbd.repairable_rbd import _group_totals, _Tally +from repyability.rbd._tally import _Tally +from repyability.rbd._time_order import _group_totals E, W = surv.Exponential.from_params, surv.Weibull.from_params diff --git a/repyability/tests/test_simulation_engines.py b/repyability/tests/test_simulation_engines.py index 0fba4b48..813a5eee 100644 --- a/repyability/tests/test_simulation_engines.py +++ b/repyability/tests/test_simulation_engines.py @@ -26,8 +26,15 @@ from repyability import NodeState, PerfectReliability, RepairableRBD from repyability.non_repairable import NonRepairable -from repyability.rbd import _compiled, _streams, repairable_rbd -from repyability.rbd.repairable_rbd import Event +from repyability.rbd import ( + _compiled, + _crews, + _event_loop, + _streams, + _time_order, +) +from repyability.rbd._events import Event +from repyability.tests import timeline_reference from repyability.tests.catalogue import systems_of_every_kind from repyability.tests.keyed_draws import KeyedDraws, reference_draw from repyability.tests.test_performance_equivalence import ( @@ -234,7 +241,7 @@ def test_each_simulation_draws_from_the_start_of_its_columns(): # draws the other simulations take, and a block extended by more rows # keeps its rows. rbd = plain_rbds()["bridge"] - plan, complete = rbd._stream_plan(400.0, 5, False) + plan, complete = _event_loop._stream_plan(rbd, 400.0, 5, False) assert complete spec = plan.specs[(("a",), _streams.FAILURE)] block = plan.block(spec, 1) @@ -305,7 +312,7 @@ def reference(rbd, t_end, n, seed, working=(), broken=(), antithetic=False): system.append((t_end, 0)) starts = [1 if node not in broken else 0 for node in nodes] overlaps = overlaps_one_at_a_time(starts, events, system, t_end) - uptime = repairable_rbd.time_at_status(system, 1) + uptime = timeline_reference.time_at_status(system, 1) for c, node in enumerate(nodes): rate = rbd.costs.get(node, {}).get("downtime_cost") if rate is not None: @@ -455,7 +462,7 @@ def test_a_nested_rbd_can_be_held_working_or_broken(name, forced): def test_an_unstreamable_maintenance_time_still_takes_time(): rbd = systems_of_every_kind()["unstreamable maintenance"] - assert not rbd._stream_specs(200.0)[1] + assert not _event_loop._stream_specs(rbd, 200.0)[1] result = rbd.availability(200.0, mc_samples=30, seed=31) assert result.system_planned_outages > 0 assert result.node_uptime["a"] < 200.0 * 30 @@ -466,7 +473,7 @@ def test_an_unstreamable_component_leaves_the_others_alone(): # draws from, so its up time is the same. systems = systems_of_every_kind() mixed = systems["unstreamable"] - assert not mixed._stream_specs(100.0)[1] + assert not _event_loop._stream_specs(mixed, 100.0)[1] plain = RepairableRBD( [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], { @@ -525,7 +532,7 @@ def test_an_unknown_engine_is_refused(): ) def test_the_compiled_engine_says_what_it_cannot_run(name, reason): rbd = repairable_rbds()[name] - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert reason in _compiled.unsupported(rbd, plan, None) # "auto" runs it in Python. identical( @@ -536,17 +543,17 @@ def test_the_compiled_engine_says_what_it_cannot_run(name, reason): def test_what_the_compiled_engine_can_run(): for rbd in plain_rbds().values(): - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert _compiled.unsupported(rbd, plan, None) is None mixed = systems_of_every_kind()["unstreamable"] - plan, _ = mixed._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(mixed, 100.0, 1, False) assert "cannot be streamed" in _compiled.unsupported(mixed, plan, None) capacity = RepairableRBD( [("s", "a"), ("a", "t")], {"a": {"reliability": E([0.1]), "repairability": E([1.0])}}, capacity={"a": 10.0}, ) - plan, _ = capacity._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(capacity, 100.0, 1, False) assert _compiled.unsupported(capacity, plan, object()) == "capacities" # numba's own loop follows them (#155). assert _compiled.unsupported(capacity, plan, object(), numba=True) is None @@ -565,7 +572,7 @@ class LoggedUnit(NonRepairable): "sub": LoggedUnit(W([40, 2]), E([0.5])), }, ) - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert ( _compiled.unsupported(rbd, plan, None) == "the LoggedUnit of node 'sub'" @@ -815,7 +822,7 @@ def engines_agree(rbd, options): if "broken_nodes" in options and "b" not in rbd.components: nodes = list(rbd.components) options = {**options, "broken_nodes": [nodes[min(1, len(nodes) - 1)]]} - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert _compiled.unsupported(rbd, plan, None) is not None assert _compiled.unsupported(rbd, plan, None, numba=True) is None with warnings.catch_warnings(): @@ -857,7 +864,7 @@ def test_a_maintained_run_from_a_state_stays_in_python(upkeep, monkeypatch): rbd = maintained_rbds()["block, priced"] else: rbd = inspected_rbds()["staggered, priced"] - state = {"a": NodeState(age=0.0, phase=10.0)} + state = {"a": NodeState(age=0.0, since_scheduled=10.0)} with pytest.raises(NotImplementedError, match="started from a state"): rbd.availability( 100.0, mc_samples=5, seed=2, state=state, engine="numba" @@ -899,9 +906,9 @@ def test_what_numbas_loop_runs_besides_plain_components(): *standby_rbds().values(), *nested_rbds().values(), ]: - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) # Not given to an engine of the interface's version. - if rbd._crews_limited(): + if _crews._crews_limited(rbd): reason = "repair crews" elif rbd._standby: reason = "standby groups" @@ -931,7 +938,7 @@ def test_what_numbas_loop_runs_besides_plain_components(): # Its test time could be streamed, its life cannot. (unstreamed, "the models of node 'a'"), ]: - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert reason in _compiled.unsupported(rbd, plan, None, numba=True) @@ -1555,13 +1562,13 @@ def outer(inner): (on_condition(), "replacement on condition"), ]: rbd = outer(inner) - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) assert reason in _compiled.unsupported(rbd, plan, None, numba=True) # An engine of the interface's version is given no nested RBD. assert _compiled.unsupported(rbd, plan, None) == "nested RBDs" rbd = nested_rbds()["maintained inside"] - plan, _ = rbd._stream_plan(100.0, 1, False) - state = {"c": NodeState(age=0.0, phase=1.0)} + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) + state = {"c": NodeState(age=0.0, since_scheduled=1.0)} assert "started from a state" in _compiled.unsupported( rbd, plan, None, numba=True, states=state ) @@ -1666,7 +1673,7 @@ def test_the_engines_agree_on_capacities(name, options): rbd, extra = capacity_rbds()[name] if "broken_nodes" in options and "b" not in rbd.components: options = {**options, "broken_nodes": [list(rbd.components)[0]]} - plan, _ = rbd._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(rbd, 100.0, 1, False) # An engine of the interface's version is given no capacities. assert _compiled.unsupported(rbd, plan, object()) is not None assert _compiled.unsupported(rbd, plan, object(), numba=True) is None @@ -1680,7 +1687,7 @@ def test_the_engines_agree_on_capacities(name, options): def test_capacities_numba_does_not_follow(): wide = pairs_in_series(32) - plan, _ = wide._stream_plan(100.0, 1, False) + plan, _ = _event_loop._stream_plan(wide, 100.0, 1, False) assert "more than 63" in _compiled.unsupported( wide, plan, object(), numba=True ) @@ -1899,10 +1906,8 @@ def test_the_curve_is_the_same_from_changes_in_order(changes): deltas = rng.choice([-1, 1], times.size).astype(np.int64) order = np.argsort(times, kind="stable") for start in (0, 40): - shuffled = repairable_rbd._working_over_time( - times, deltas, 100.0, start - ) - ordered = repairable_rbd._working_over_time( + shuffled = _time_order._working_over_time(times, deltas, 100.0, start) + ordered = _time_order._working_over_time( times[order], deltas[order], 100.0, start ) for a, b in zip(shuffled, ordered): diff --git a/repyability/tests/test_small_probabilities.py b/repyability/tests/test_small_probabilities.py index 14290c42..3c2af829 100644 --- a/repyability/tests/test_small_probabilities.py +++ b/repyability/tests/test_small_probabilities.py @@ -21,6 +21,7 @@ RepairableRBD, RepeatedNode, ) +from repyability.rbd import _long_run #: Relative error allowed against an exact value: a few roundings. RTOL = 1e-14 @@ -548,6 +549,6 @@ def test_planned_outages_at_block_times(mttf): [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")], {"a": a, "b": b} ) U = 1 / (Fraction(1 / (1 / mttf)) + 1) - up_before = Fraction(rbd._block_cycle("a").before) - planned = rbd._outage_frequencies()[1] + up_before = Fraction(_long_run._block_cycle(rbd, "a").before) + planned = _long_run._outage_frequencies(rbd)[1] assert planned == approx(float(up_before * U / 100), 1e-12) diff --git a/repyability/tests/test_spares.py b/repyability/tests/test_spares.py index 2e62f674..f50813b5 100644 --- a/repyability/tests/test_spares.py +++ b/repyability/tests/test_spares.py @@ -93,7 +93,7 @@ def renewal_demands(rng, life, repair, maintenance, age, until): def test_age_replacement_counts_match_a_direct_simulation(): np.random.seed(3) - model = single(WORN)._replacements("c", True) + model = _spares._replacements(single(WORN), "c", True) args = (W([100.0, 2.5]), L([1.0, 0.6]), E([1 / 1.5]), 80.0) rng = np.random.default_rng(1) # From new over 500 hours. @@ -221,7 +221,13 @@ def test_what_is_refused_and_checked(): assert crewed.analysis_routes()["spares_demand"].route == r.REFUSED crewed.spares_demand(100.0, method="simulate", mc_samples=500, seed=1) nested = RepairableRBD([("s", "x"), ("x", "t")], {"x": single(PUMP)}) - assert nested.spares_demand(100.0) == {} + # A nested RBD's components' spares are not counted here: by default + # it refuses (routes.Refused), naming the components counted. + with pytest.raises(ValueError, match="nested RBD") as refused: + nested.spares_demand(100.0) + assert nested.analysis_routes()["spares_demand"].reason == str( + refused.value + ) with pytest.raises(ValueError, match="nested RBD"): nested.spares_demand(100.0, nodes=["x"]) never = single( diff --git a/repyability/tests/test_spares_block_and_tested.py b/repyability/tests/test_spares_block_and_tested.py index 529e3440..88acba89 100644 --- a/repyability/tests/test_spares_block_and_tested.py +++ b/repyability/tests/test_spares_block_and_tested.py @@ -9,7 +9,7 @@ import numpy as np import pytest import surpyval as surv -from scipy.stats import binom, poisson +from scipy.stats import binom, nbinom, poisson from repyability import RepairableRBD from repyability.rbd import _spares @@ -315,9 +315,15 @@ def test_a_lead_time_on_block_times_is_read_at_its_jump(): # intervals on less its phase, so their distribution jumps at each # interval: read on its own, each rounding takes it exactly, where # their mean blurred it over a step (an error of the step's order). - model = single( - block(W([100.0, 2.5]), 60.0, repair=W([4.0, 1.5]), duration=E([0.5])) - )._replacements("c", True) + model = _spares._replacements( + single( + block( + W([100.0, 2.5]), 60.0, repair=W([4.0, 1.5]), duration=E([0.5]) + ) + ), + "c", + True, + ) coarse, fine = ( np.array(_spares._palm(model, 120.0, steps)[1]) for steps in (256, 512) ) @@ -335,7 +341,7 @@ def test_the_long_run_is_an_interval_settled_from_new(): rbd.spares_demand(21 * 60.0)["c"].mean - rbd.spares_demand(20 * 60.0)["c"].mean ) - model = rbd._replacements("c", True) + model = _spares._replacements(rbd, "c", True) assert _spares.rate(model) * 60.0 == pytest.approx(settled, rel=2e-6) # From a random time, the mean on order is the rate times the lead time. stock = rbd.spares_stock(45.0, fill_rate=0.9)["c"] @@ -347,7 +353,9 @@ def test_the_long_run_is_an_interval_settled_from_new(): def test_renewals_in_no_time_through_a_typical_replacement(kind): # The typical replacement's counts take renewals in no time too, where # each interval starts new: they are the averages over the phase. - model = single(block(W([100.0, 2.0]), 60.0))._replacements("c", True) + model = _spares._replacements( + single(block(W([100.0, 2.0]), 60.0)), "c", True + ) assert not _spares.carries_over(model) tails = np.minimum.accumulate( np.clip(_spares._palm_tails(model, 45.0, kind), 0.0, 1.0) @@ -371,7 +379,7 @@ def test_a_part_with_a_member_whose_work_takes_time(): other = poisson.pmf(np.arange(40), 0.3) got, want = padded(stock.on_order, np.convolve(alone.on_order, other)) np.testing.assert_allclose(got, want, atol=2e-6) - rate = _spares.rate(rbd._replacements("a", True)) + rate = _spares.rate(_spares._replacements(rbd, "a", True)) share = rate / (rate + 0.01) at_a, at_b = padded( np.convolve(alone.on_order_at_demand, other), @@ -383,42 +391,153 @@ def test_a_part_with_a_member_whose_work_takes_time(): np.testing.assert_allclose(got, want, atol=2e-6) +def test_dead_on_arrival_with_instant_renewals_compounds_the_replacements(): + # #322: an exponential life ending at 0 with probability q, repaired + # and block replaced in no time. The lasting units' failures are + # Poisson, so a lead time tau < T from a random time holds + # Poisson(rate tau) of them and a block replacement with probability + # tau / T; before a lasting unit's replacement (a failure, at a + # uniform phase, as often as rate T to 1), the block replacement is in + # the lead time with probability tau / T too. Each of those brings a + # geometric number of units dead on arrival at its instant (a + # negative binomial for b of them), and a replacement has a geometric + # number of its instant's before it. + rate, q, T, tau = 0.01, 0.15, 60.0, 45.0 + rbd = single(block(E([rate], f0=q), T)) + stock = rbd.spares_stock(tau, fill_rate=0.9)["c"] + + def compounded(lasting): + out = np.zeros(60) + for b, p in enumerate(lasting): + if b == 0: + out[0] += p + else: + out[b:] += p * nbinom.pmf(np.arange(60 - b), b, 1.0 - q) + return out + + def lasting(p_block): + failures = poisson.pmf(np.arange(30), rate * tau) + return np.convolve(failures, [1.0 - p_block, p_block]) + + random = compounded(lasting(tau / T)) + before = compounded( + np.concatenate([[0.0], lasting(rate * tau / (rate * T + 1.0))]) + )[1:] + for got, want in ( + (stock.on_order, random), + (stock.on_order_at_demand, before), + ): + np.testing.assert_allclose(*padded(got, want), atol=1e-6) + model = _spares._replacements(rbd, "c", True) + assert _spares.rate(model) == pytest.approx( + (rate * T + 1.0) / T / (1.0 - q), rel=1e-6 + ) + report = rbd.analysis_routes() + assert report["spares_stock"].route == r.NUMERICAL + assert "where the lead time falls" in report["spares_stock"].reason + # A vanishing share dead on arrival gives the values without one. + plain, tiny = ( + single(block(E([rate], f0=f0), T)).spares_stock(tau, fill_rate=0.9)[ + "c" + ] + for f0 in (0.0, 1e-9) + ) + for name in ("on_order", "on_order_at_demand"): + np.testing.assert_allclose( + *padded(getattr(tiny, name), getattr(plain, name)), atol=1e-7 + ) + + @pytest.mark.parametrize( - "spec, reason", + "repair, duration", [ - ( - block(W([100.0, 2.0], f0=0.1), 60.0, duration=E([0.5])), - "while its repairs may take no time", - ), - ( - block(W([100.0, 2.0], f0=0.1), 60.0, repair=E([0.5])), - "while its block replacements may take no time", - ), + ("instant", None), + (E([0.5]), None), + ("instant", E([0.5])), + ], + ids=[ + "repairs and block replacements in no time", + "block replacements in no time", + "repairs in no time", ], - ids=["repairs in no time", "block replacements in no time"], ) -def test_dead_on_arrival_with_renewals_in_no_time_is_refused(spec, reason): - # Replacements could come several at one instant. +def test_a_unit_dead_on_arrival_against_long_histories(repair, duration): + # #322: a unit dead on arrival fails as it is put in, and its + # replacement, in no time, may be too: replacements come several at + # one instant, each a demand. Before a replacement, those at its + # instant before it are in its lead time. + life, interval, lead = W([100.0, 2.0], f0=0.1), 60.0, 45.0 + spec = block(life, interval, repair=repair) + if duration is not None: + spec["preventive"]["duration"] = duration rbd = single(spec) - with pytest.raises(NotImplementedError, match="one instant") as error: - rbd.spares_stock(30.0, fill_rate=0.9) - assert reason in str(error.value) - report = rbd.analysis_routes() - assert report["spares_stock"].route == r.REFUSED - assert report["spares_stock"].reason == str(error.value) - assert report["spares_demand"].route == r.NUMERICAL - # Taking time, a unit dead on arrival is counted. - timed = single( - block( - W([100.0, 2.0], f0=0.1), 60.0, repair=E([0.5]), duration=E([0.5]) + stock = rbd.spares_stock(lead, fill_rate=0.9)["c"] + assert rbd.analysis_routes()["spares_stock"].route == r.NUMERICAL + instant = surv.ExactEventTime.from_params([0.0]) + rng = np.random.default_rng(5) + times = block_history_with_work( + life, + instant if repair == "instant" else repair, + instant if duration is None else duration, + interval, + 400, + rng, + ) + random, arrival = [], [] + for row in times: + lo, hi = 20 * interval, row[-1] - lead - 20 * interval + starts = rng.uniform(lo, hi, 600) + random.append( + np.searchsorted(row, starts + lead) - np.searchsorted(row, starts) + ) + inner = np.flatnonzero((row > lo + lead) & (row < hi)) + arrival.append( + inner - np.searchsorted(row, row[inner] - lead, side="right") ) + for got, sample in ( + (stock.on_order, np.concatenate(random)), + (stock.on_order_at_demand, np.concatenate(arrival)), + ): + got, want = padded(got, np.bincount(sample) / len(sample)) + np.testing.assert_allclose(got, want, atol=0.006) + # From a random time, the lead time holds the long-run rate's worth; + # the count from new runs at that rate once past its start as new + # (and no block replacement at 0), and the simulation counts as it + # does, within its standard error. + rate = _spares.rate(_spares._replacements(rbd, "c", True)) + assert np.arange(len(stock.on_order)) @ stock.on_order == pytest.approx( + rate * lead, rel=1e-5 + ) + first, second = ( + rbd.spares_demand(k * interval)["c"].mean for k in (5, 10) + ) + assert second - first == pytest.approx(rate * 5 * interval, rel=1e-4) + demand = rbd.spares_demand( + 10 * interval, method="simulate", mc_samples=2000, seed=7 + )["c"] + assert demand.mean == pytest.approx( + second, abs=4.0 * demand.std / np.sqrt(2000) + ) + + +def test_a_unit_dead_on_arrival_at_a_block_time_is_not_replaced_there_twice(): + # #322: the unit put in at a block time by its replacement in no time + # may be dead on arrival, and the one put in at its failure, in no + # time, is at the block time too, after its replacement: it is not + # replaced there again. Rounded up, the count from new took it as put + # in just before the block time and replaced there, 0.035 out over + # two block times. + rbd = single(block(W([100.0, 2.0], f0=0.1), 60.0)) + horizon, mc_samples = 130.0, 20_000 + demand = rbd.spares_demand(horizon)["c"] + simulated = rbd.spares_demand( + horizon, method="simulate", mc_samples=mc_samples, seed=3 + )["c"] + got, want = padded(demand.probabilities, simulated.probabilities) + assert got[:2] == pytest.approx(0.0, abs=1e-9) # two block times + np.testing.assert_allclose( + got, want, atol=4.0 * np.sqrt(0.25 / mc_samples) ) - routes = timed.analysis_routes() - assert routes["spares_stock"].route == r.NUMERICAL - assert "a typical replacement" in routes["spares_stock"].reason - routes = single(block(W([100.0, 2.0]), 60.0)).analysis_routes() - assert routes["spares_stock"].route == r.NUMERICAL - assert "where the lead time falls" in routes["spares_stock"].reason # -- hidden failures found by tests ------------------------------------------- diff --git a/repyability/tests/test_staggered_and_partial_tests.py b/repyability/tests/test_staggered_and_partial_tests.py index c255b53d..af3c2fa0 100644 --- a/repyability/tests/test_staggered_and_partial_tests.py +++ b/repyability/tests/test_staggered_and_partial_tests.py @@ -12,6 +12,7 @@ from surpyval import Exponential from repyability import NodeState, RepairableRBD +from repyability.rbd import _intervals E = Exponential.from_params PAIR = [("s", "a"), ("s", "b"), ("a", "t"), ("b", "t")] @@ -255,7 +256,7 @@ def test_a_tested_unit_whose_tests_miss_failures_starts_new(): SINGLE, {"a": hidden(1e-3, 100.0, coverage=0.5, full_test=300.0)}, ) - for state in (NodeState(age=20.0), NodeState(phase=10.0)): + for state in (NodeState(age=20.0), NodeState(since_scheduled=10.0)): with pytest.raises(NotImplementedError, match="leave it out"): rbd.point_availability([50.0], state={"a": state}) with pytest.raises(NotImplementedError, match="leave it out"): @@ -269,7 +270,7 @@ def test_a_state_places_a_staggered_calendar(): SINGLE, {"a": hidden(rate, interval, offset=60.0)} ) plain = RepairableRBD(SINGLE, {"a": hidden(rate, interval)}) - state = {"a": NodeState(phase=10.0)} + state = {"a": NodeState(since_scheduled=10.0)} x = np.array([5.0, 50.0, 95.0, 250.0]) np.testing.assert_allclose( staggered.point_availability(x, state=state), @@ -286,7 +287,7 @@ def test_a_state_places_a_staggered_calendar(): ] assert simulated[0] == simulated[1] # From a test at 0 too. - state = {"a": NodeState(phase=0.0)} + state = {"a": NodeState(since_scheduled=0.0)} np.testing.assert_allclose( staggered.point_availability(x, state=state), plain.point_availability(x, state=state), @@ -306,7 +307,9 @@ def test_choosing_intervals_keeps_an_offsets_share(): "b": hidden(rate, 1000.0, offset=500.0, cost=10.0), }, ) - plan = rbd._with_intervals(inspection={"a": 2000.0, "b": 2000.0}) + plan = _intervals._with_intervals( + rbd, inspection={"a": 2000.0, "b": 2000.0} + ) assert plan._inspection["b"].offset == 1000.0 chosen = rbd.optimal_inspection_intervals( allowed={"a": [500.0, 1000.0, 2000.0], "b": [500.0, 1000.0, 2000.0]}, diff --git a/repyability/tests/test_start_from_state.py b/repyability/tests/test_start_from_state.py index 3f2b772d..b938a0ef 100644 --- a/repyability/tests/test_start_from_state.py +++ b/repyability/tests/test_start_from_state.py @@ -259,7 +259,7 @@ def test_a_stationary_calendar_starts_at_its_phase(): } rbd = RepairableRBD(ONE, {"a": unit}) phase = 30.0 - state = {"a": NodeState(stationary=True, phase=phase)} + state = {"a": NodeState(stationary=True, since_scheduled=phase)} x = np.array([0.0, 10.0, 69.0, 71.0, 150.0, 290.0]) # Long in service: as far from new as it has settled, at the phase. far = rbd.point_availability(5000.0 + phase + x) @@ -282,14 +282,14 @@ def test_a_tested_unit_was_last_known_up_at_its_last_test(): rbd = RepairableRBD(ONE, {"a": tested}) x = np.array([0.0, 30.0, 69.0, 71.0, 150.0]) since_test = np.where(x < 70.0, 30.0 + x, x - 70.0) - state = {"a": NodeState(age=1000.0, phase=30.0)} + state = {"a": NodeState(age=1000.0, since_scheduled=30.0)} np.testing.assert_allclose( rbd.point_availability(x, state=state), np.exp(-lam * since_test), atol=1e-12, ) # Put into service 10 ago, after its last test, 30 ago. - renewed = {"a": NodeState(age=10.0, phase=30.0)} + renewed = {"a": NodeState(age=10.0, since_scheduled=30.0)} since = np.where(x < 70.0, 10.0 + x, x - 70.0) np.testing.assert_allclose( rbd.point_availability(x, state=renewed), @@ -380,10 +380,12 @@ def test_maintained_pumps_are_what_the_simulation_estimates(state): @pytest.mark.parametrize( "start", [ - NodeState(age=40.0, phase=40.0), - NodeState(age=0.0, phase=30.0), - NodeState(alive=False, down_for=4.0, phase=95.0), - NodeState(alive=False, down_for=2.0, maintenance=True, phase=2.0), + NodeState(age=40.0, since_scheduled=40.0), + NodeState(age=0.0, since_scheduled=30.0), + NodeState(alive=False, down_for=4.0, since_scheduled=95.0), + NodeState( + alive=False, down_for=2.0, maintenance=True, since_scheduled=2.0 + ), ], ) def test_block_replacement_from_a_state_is_what_the_simulation_estimates( @@ -544,7 +546,9 @@ def test_a_run_from_a_state_is_simulated_in_python(): def test_what_a_state_cannot_be(): rbd = maintained_pair() with pytest.raises(ValueError, match="not on a calendar"): - rbd.point_availability(1.0, state={"a": NodeState(phase=1.0)}) + rbd.point_availability( + 1.0, state={"a": NodeState(since_scheduled=1.0)} + ) with pytest.raises(ValueError, match="not a component"): rbd.point_availability(1.0, state={"z": NodeState()}) with pytest.raises(ValueError, match="held working or broken"): @@ -561,7 +565,7 @@ def test_what_a_state_cannot_be(): preventive={"policy": "block", "interval": 100.0}, ) with pytest.raises(ValueError, match="less than its interval"): - block.availability(1.0, state={"a": NodeState(phase=100.0)}) + block.availability(1.0, state={"a": NodeState(since_scheduled=100.0)}) with pytest.raises(ValueError, match="no preventive maintenance"): block.availability( 1.0, state={"a": NodeState(alive=False, maintenance=True)} @@ -584,8 +588,13 @@ def test_what_a_state_is_not_taken_for(): repairability=E([1.0]), repair={"model": "kijima1", "q": 0.5}, ) - with pytest.raises(NotImplementedError, match="virtual age"): - imperfect.availability(10.0, state={"a": NodeState(age=5.0)}) + # The simulations take its age and virtual age (#269); the exact + # methods do not. + imperfect.availability( + 10.0, mc_samples=5, seed=0, state={"a": NodeState(age=5.0)} + ) + with pytest.raises(NotImplementedError, match="taken by the simulations"): + imperfect.point_availability(10.0, state={"a": NodeState(age=5.0)}) standby = single( reliability=E([0.01]), repairability=E([0.5]), @@ -613,6 +622,6 @@ class Own(NonRepairable): repairability=E([0.5]), inspection={"interval": 100.0}, ) - down = {"a": NodeState(alive=False, down_for=1.0, phase=10.0)} + down = {"a": NodeState(alive=False, down_for=1.0, since_scheduled=10.0)} run = tested.availability(50.0, mc_samples=50, seed=0, state=down) assert run.availability[0] == 0.0 diff --git a/repyability/tests/test_streams.py b/repyability/tests/test_streams.py index 66ff7fbd..a00b044e 100644 --- a/repyability/tests/test_streams.py +++ b/repyability/tests/test_streams.py @@ -9,7 +9,7 @@ import numpy as np import pytest -from repyability.rbd import _compiled, _philox, _streams +from repyability.rbd import _compiled, _event_loop, _philox, _streams from repyability.tests.keyed_draws import reference_uniform from repyability.tests.test_simulation_engines import identical, plain_rbds @@ -43,7 +43,7 @@ def test_the_uniforms_are_numpys_philox(name, uniforms): def test_a_block_holds_the_defined_uniforms_whatever_its_width(): rbd = plain_rbds()["bridge"] - plan, _ = rbd._stream_plan(400.0, 5, False) + plan, _ = _event_loop._stream_plan(rbd, 400.0, 5, False) spec = plan.specs[(("a",), _streams.FAILURE)] uniform = dataclasses.replace(spec, sampler=lambda u: u) for width in (1, 3, 8): diff --git a/repyability/tests/test_time_order.py b/repyability/tests/test_time_order.py index 1af4cbce..ad9628dc 100644 --- a/repyability/tests/test_time_order.py +++ b/repyability/tests/test_time_order.py @@ -1,5 +1,5 @@ """A run's changes put in time order (#201, #208): each simulation's changes -come as a sorted run, one run after another, and ``repairable_rbd`` nets +come as a sorted run, one run after another, and ``_time_order`` nets the +1 and -1 changes of state by time, groups the capacity's changes by time and merges its totals. Each is checked here against its definition, written out plainly: a dict of the changes at each time, their exact sums @@ -11,7 +11,7 @@ import numpy as np import pytest -from repyability.rbd import repairable_rbd +from repyability.rbd import _time_order from repyability.rbd._exact import ExactSum @@ -73,9 +73,7 @@ def test_the_working_count_after_each_time(seed, start): times, _, deltas = runs(rng) expected = by_time(np.r_[times, 0.0, 100.0], np.r_[deltas, 0, 0]) count = start + np.cumsum([sum(d) for d in expected.values()]) - time, working = repairable_rbd._working_over_time( - times, deltas, 100.0, start - ) + time, working = _time_order._working_over_time(times, deltas, 100.0, start) same(time, np.array(list(expected), dtype=float)) same(working, count.astype(float)) @@ -85,7 +83,7 @@ def test_the_working_count_at_the_edges(case): times = CASES[case] deltas = np.resize(np.array([-1, 1], dtype=np.int64), times.size) expected = by_time(np.r_[times, 0.0, 100.0], np.r_[deltas, 0, 0]) - time, working = repairable_rbd._working_over_time(times, deltas, 100.0, 7) + time, working = _time_order._working_over_time(times, deltas, 100.0, 7) same(time, np.array(list(expected), dtype=float)) same(working, 7.0 + np.cumsum([sum(d) for d in expected.values()])) @@ -101,12 +99,12 @@ def test_the_capacity_changes_grouped_by_time(seed): rng = np.random.default_rng(seed) times, steps, _ = runs(rng) expected = by_time(times, steps) - at, values, starts = repairable_rbd._by_time(times, steps) + at, values, starts = _time_order._by_time(times, steps) same(at, np.array(list(expected), dtype=float)) stops = np.append(starts[1:], values.size) for a, b, want in zip(starts, stops, expected.values()): assert sorted(values[a:b].tolist()) == sorted(want) - totals = repairable_rbd._group_totals(values, starts) + totals = _time_order._group_totals(values, starts) same(totals, np.array([exact_total(v) for v in expected.values()])) assert all( math.isclose(t, math.fsum(v), abs_tol=0.0) @@ -119,7 +117,7 @@ def test_the_net_changes_by_time(seed): rng = np.random.default_rng(seed) times, _, deltas = runs(rng) expected = by_time(times, deltas) - at, net = repairable_rbd._net_by_time(times, deltas) + at, net = _time_order._net_by_time(times, deltas) same(at, np.array(list(expected), dtype=float)) same(net, np.array([sum(d) for d in expected.values()], dtype=np.int64)) @@ -170,7 +168,7 @@ def test_the_capacity_totals_merged(case): step.update(zip(at.tolist(), change.tolist())) freed = dict.fromkeys(every, 0) freed.update(zip(free_at.tolist(), counts.tolist())) - times, totals, unlimited = repairable_rbd._capacity_totals( + times, totals, unlimited = _time_order._capacity_totals( at, change, free_at, counts, 10.0 ) same(times, np.array(every, dtype=float)) diff --git a/repyability/tests/test_timelines.py b/repyability/tests/test_timelines.py index df94b75d..3622d950 100644 --- a/repyability/tests/test_timelines.py +++ b/repyability/tests/test_timelines.py @@ -26,7 +26,14 @@ TimelineSimulation, ) from repyability import timelines as tl -from repyability.rbd import _compiled, _streams, _timeline_runs, modular +from repyability.rbd import ( + _ccf_groups, + _compiled, + _event_loop, + _streams, + _timeline_runs, + modular, +) from repyability.timelines import k_out_of_n, parallel, series W = surv.Weibull.from_params @@ -681,7 +688,8 @@ def recorded(rbd, engine: str, t: float, n: int, seed: int, **options): (whatever the components), as ``simulate_timelines`` gives them.""" working = set(options.get("working_nodes", ())) broken = set(options.get("broken_nodes", ())) - tally = rbd._run( + tally = _event_loop._run( + rbd, t, working, broken, @@ -774,7 +782,7 @@ def every_architecture(): return { name: rbd for name, rbd in repairable_kinds().items() - if not rbd._has_ccf() + if not _ccf_groups._has_ccf(rbd) } @@ -794,7 +802,10 @@ def test_every_engine_records_the_same_histories(name): if ( _compiled.available() and _compiled.unsupported( - rbd, rbd._stream_plan(300.0, 0, False)[0], None, numba=True + rbd, + _event_loop._stream_plan(rbd, 300.0, 0, False)[0], + None, + numba=True, ) is None ): diff --git a/repyability/tests/test_total_cost.py b/repyability/tests/test_total_cost.py index e6219f16..2051acf6 100644 --- a/repyability/tests/test_total_cost.py +++ b/repyability/tests/test_total_cost.py @@ -18,7 +18,7 @@ import surpyval as surv from repyability import CostResult, RepairableRBD, TotalCostAllocation -from repyability.rbd import redundancy_allocation +from repyability.rbd import _costs, redundancy_allocation from repyability.rbd.redundancy_allocation import lowest_total_cost E = surv.Exponential.from_params @@ -107,9 +107,12 @@ def test_acquisition_cost_alone_prices_no_running_cost(): assert result.by_category["repair"] == 0.0 assert rbd.expected_cost(10.0).total == 7.0 assert rbd.total_cost(1e6) == 7.0 - # With nothing priced at all, there is no cost to give. + # With nothing priced at all, there is no cost to give: it refuses. unpriced = RepairableRBD(SERIES, {"pump": spec(1e-3, 0.1)}) - assert unpriced.cost(t_simulation=10.0, mc_samples=5, seed=0) is None + with pytest.raises(ValueError, match="Nothing is priced"): + unpriced.cost(t_simulation=10.0, mc_samples=5, seed=0) + with pytest.raises(ValueError, match="Nothing is priced"): + unpriced.total_cost(10.0) # A cost of 0 is left out. free = RepairableRBD(SERIES, {"pump": spec(1e-3, 0.1, acquisition_cost=0)}) assert free.acquisition_costs == {} @@ -807,7 +810,8 @@ def by_hand(nx, ny): nx * (900.0 + H * 40.0 * failures(0.01, 0.2)) + ny * (400.0 + H * 10.0 * failures(0.03, 0.5)) + H * 500.0 * (1 - availability) - + H * rbd._node_cost_rate("a", 1.0) # the bridge costs nothing + + H + * _costs._node_cost_rate(rbd, "a", 1.0) # the bridge costs nothing ) return total, availability @@ -853,7 +857,7 @@ def test_nodes_in_series_with_inspected_nodes_elsewhere(): def test_the_series_program_is_used_exactly_when_it_applies(monkeypatch): - from repyability.rbd import repairable_rbd + from repyability.rbd import _repairable_allocation calls = [] @@ -861,7 +865,7 @@ def recording(*args): calls.append(args[7]) return lowest_total_cost(*args) - monkeypatch.setattr(repairable_rbd, "lowest_total_cost", recording) + monkeypatch.setattr(_repairable_allocation, "lowest_total_cost", recording) inspected = { "reliability": E([0.01]), "repairability": "instant", diff --git a/repyability/tests/test_uncertainty.py b/repyability/tests/test_uncertainty.py index 04101b10..1892081d 100644 --- a/repyability/tests/test_uncertainty.py +++ b/repyability/tests/test_uncertainty.py @@ -50,7 +50,7 @@ def test_fit_draws_follow_the_fits_normal_approximation(): times = np.linspace(20, 400, 40) model = surv.Exponential.fit(times) rate = float(np.ravel(model.params)[0]) - s = math.sqrt(float(np.ravel(model.hess_inv)[0])) / rate + s = math.sqrt(float(np.ravel(model.covariance())[0])) / rate result = one_node(model).sf_uncertainty( 100.0, {"c": "fit"}, n_draws=40_000, seed=0 ) @@ -75,16 +75,81 @@ def test_fit_draws_agree_with_surpyvals_confidence_bounds(): def test_fit_draws_keep_an_offset(): model = surv.Weibull.fit(np.linspace(200, 1800, 50) + 100, offset=True) rbd = one_node(model) - # Before the offset nothing can fail, in every draw. - with np.errstate(invalid="ignore"): + # Before the offset nothing can fail, in every draw. surpyval's + # covariance leaves the offset out (SurPyval#830), so it is held, and + # the draws say so. + with ( + np.errstate(invalid="ignore"), + pytest.warns(UserWarning, match="offset.*SurPyval#830"), + ): early = rbd.sf_uncertainty( 0.5 * model.gamma, {"c": "fit"}, n_draws=200 ) np.testing.assert_array_equal(early.samples, 1.0) - later = rbd.sf_uncertainty(1000.0, {"c": "fit"}, n_draws=200, seed=2) + with pytest.warns(UserWarning, match="offset"): + later = rbd.sf_uncertainty(1000.0, {"c": "fit"}, n_draws=200, seed=2) assert later.std > 0 +def shares_fit(lfp=True, zi=False): + """A Weibull fitted to 60 failures, with 40 units that never failed + (``lfp``) and 10 dead on arrival (``zi``).""" + x = [np.random.default_rng(5).weibull(2, 60) * 100] + c = [np.zeros(60)] + if lfp: + x, c = x + [np.full(40, 400.0)], c + [np.ones(40)] + if zi: + x, c = x + [np.zeros(10)], c + [np.zeros(10)] + x, c = np.concatenate(x), np.concatenate(c).astype(int) + return surv.Weibull.fit(x, c=c, lfp=lfp, zi=zi) + + +@pytest.mark.parametrize("share", ["lfp_p", "f0"]) +def test_fit_draws_vary_the_shares_that_never_fail_and_are_dead(share): + # Long after every unit that can fail has failed, the reliability is + # the share that never fails, 1 - lfp_p; just after 0 it is the share + # not dead on arrival, 1 - f0 (#267). Each share is drawn on the logit + # scale, its variance from surpyval's covariance() (the delta method), + # so the reliability there has known percentiles; before #267 it was + # held at its fitted value. + model = shares_fit(lfp=share == "lfp_p", zi=share == "f0") + p = float(getattr(model, share)) + sd = math.sqrt(float(model.covariance()[-1, -1])) / (p * (1 - p)) + at = 1e4 if share == "lfp_p" else 1e-9 + result = one_node(model).sf_uncertainty( + at, {"c": "fit"}, n_draws=40_000, seed=4 + ) + assert result.nominal == pytest.approx(1 - p) + logit = math.log(p / (1 - p)) + for q in (0.05, 0.25, 0.5, 0.75, 0.95): + # A high share is a low reliability. + drawn = 1 / (1 + math.exp(-(logit + st.norm.ppf(1 - q) * sd))) + assert_percentile(result.samples, 1 - drawn, q) + + +def test_fit_draws_every_parameter_with_the_fits_covariance(): + # A fit of both shares: the draws of all four parameters have the + # fit's covariance, to the skew of the log and logit scales over the + # draws: each variance within 15%, each correlation within 0.05. + from repyability.rbd._model_utils import parametric_spec + from repyability.rbd.uncertainty import draw_models + + model = shares_fit(lfp=True, zi=True) + drawn = draw_models(model, "fit", 20_000, np.random.default_rng(6), "c") + params = np.array([parametric_spec(m).params for m in drawn]) + assert parametric_spec(model).names == ["alpha", "beta", "lfp_p", "f0"] + drawn_cov, fitted = np.cov(params.T), model.covariance() + np.testing.assert_allclose(np.diag(drawn_cov), np.diag(fitted), rtol=0.15) + sd = np.sqrt(np.diag(fitted)) + np.testing.assert_allclose( + np.corrcoef(params.T), fitted / np.outer(sd, sd), atol=0.05 + ) + # Centred on the fit on the log and logit scales: the median is it. + np.testing.assert_allclose( + np.median(params, axis=0), parametric_spec(model).params, rtol=0.01 + ) + + def test_parameter_distributions(): # The rate is uniform between 1 and 3 per 1000 h. result = one_node(E([0.002])).sf_uncertainty( @@ -301,7 +366,7 @@ def test_a_model_on_the_edge_of_its_range_has_no_normal_approximation(): with pytest.raises(ValueError, match="edge of its range"): one_node(edge).sf_uncertainty(10.0, {"c": "fit"}, n_draws=5) broken = surv.Exponential.fit(np.linspace(20, 400, 40)) - broken.hess_inv = np.array([[np.nan]]) + broken.covariance = lambda: np.array([[np.nan]]) with pytest.raises(ValueError, match="finite"): one_node(broken).sf_uncertainty(10.0, {"c": "fit"}, n_draws=5) assert ( diff --git a/repyability/tests/test_uncertainty_importance.py b/repyability/tests/test_uncertainty_importance.py index a17f5790..e1eda3e8 100644 --- a/repyability/tests/test_uncertainty_importance.py +++ b/repyability/tests/test_uncertainty_importance.py @@ -80,7 +80,7 @@ def test_a_fit_s_covariance_and_derivatives(): parts = rbd.uncertainty_importance(x) assert parts.first_order == {"pump": 1.0} assert parts.variance == pytest.approx( - gradient @ np.asarray(pump.hess_inv) @ gradient, rel=1e-6 + gradient @ np.asarray(pump.covariance()) @ gradient, rel=1e-6 ) diff --git a/repyability/tests/test_units.py b/repyability/tests/test_units.py new file mode 100644 index 00000000..913efd69 --- /dev/null +++ b/repyability/tests/test_units.py @@ -0,0 +1,236 @@ +"""Units of measure: results do not depend on the unit a diagram is in, +and a duty cycle puts a life in operating time on the calendar.""" + +import math + +import numpy as np +import pytest +import surpyval as surv +from surpyval import AcceleratedLife, life_models + +from repyability import ( + NonRepairableRBD, + RegressionNode, + RepairableRBD, + StandbyModel, +) +from repyability.rbd._duty import _SCALED, on_calendar +from repyability.rbd.serialisation import rbd_from_json, rbd_to_json + +W = surv.Weibull.from_params +E = surv.Exponential.from_params +LN = surv.LogNormal.from_params +SERIES = [("s", "a"), ("a", "b"), ("b", "t")] +ONE = [("s", "a"), ("a", "t")] + + +# Results do not depend on the unit ----------------------------------------- + + +def diagrams(c): + """The same diagrams with every time multiplied by ``c``.""" + return { + "series": NonRepairableRBD( + SERIES, {"a": W([100 * c, 2]), "b": LN([math.log(300 * c), 0.8])} + ), + "cold standby": NonRepairableRBD( + ONE, {"a": StandbyModel([W([100 * c, 2])] * 2)} + ), + "cold standby, different units": NonRepairableRBD( + ONE, {"a": StandbyModel([W([100 * c, 2]), W([80 * c, 1.5])])} + ), + "warm standby": NonRepairableRBD( + ONE, + {"a": StandbyModel([W([100 * c, 2])] * 2, dormancy_factor=0.5)}, + ), + } + + +@pytest.mark.parametrize("c", [1e-6, 1e-3, 1e3, 1e6]) +def test_results_scale_with_the_unit(c): + # Tiny or huge times give the same answers as times near 1: no step + # or grid is sized in absolute time. + t = np.array([0.0, 50.0, 100.0, 200.0]) + for name, (base, scaled) in zip( + diagrams(1.0), zip(diagrams(1.0).values(), diagrams(c).values()) + ): + pairs = { + "sf": (base.sf(t), scaled.sf(t * c)), + "ff": (base.ff(t), scaled.ff(t * c)), + "df": (base.df(t), np.asarray(scaled.df(t * c)) * c), + "hf": (base.hf(t[1:]), np.asarray(scaled.hf(t[1:] * c)) * c), + "mean": (base.mean(), scaled.mean() / c), + } + for what, (x, y) in pairs.items(): + # Rounding aside: 1e-16 of a probability at 0, say. + scale = np.max(np.abs(x)) + np.testing.assert_allclose( + y, x, rtol=1e-6, atol=1e-12 * scale, err_msg=f"{name}: {what}" + ) + + +def test_differential_importance_does_not_depend_on_the_unit(): + # A LogNormal's mu is the log of a time: in years it is ln(8760) less + # than in hours, so moving it by a share of itself would weigh it by + # the unit. A proportional change of it is one of the time instead. + def shares(c): + rbd = NonRepairableRBD( + SERIES, + {"a": LN([math.log(3000 * c), 0.8]), "b": W([4000 * c, 1.5])}, + ) + return rbd.differential_importance( + 2000 * c, over="parameters", change="proportional" + ) + + hours, years = shares(1.0), shares(1 / 8760) + for key in hours: + assert years[key] == pytest.approx(hours[key], rel=1e-4, abs=1e-8) + + +def test_a_covariate_has_no_proportional_change(): + rng = np.random.default_rng(0) + load = rng.uniform(0.5, 2.0, 200) + x = rng.weibull(2.0, 200) * 80.0 / np.exp(0.4 * (load - 1)) + model = surv.WeibullAFT.fit(x, Z=load.reshape(-1, 1)) + rbd = NonRepairableRBD(ONE, {"a": RegressionNode(model, covariates=[1.0])}) + with pytest.raises(ValueError, match="covariates, whose zero"): + rbd.differential_importance( + 50.0, over="parameters", change="proportional" + ) + rbd.differential_importance(50.0, over="parameters", change="uniform") + + +def test_an_accelerated_life_model_is_a_regression_node(): + # An Arrhenius life of one stress (a temperature, in kelvin): fitted + # with one stress, though its life model has two parameters. + rng = np.random.default_rng(0) + kelvin = np.repeat([323.15, 348.15, 373.15], 30) + x = surv.Weibull.random(90, 100, 2, random_state=rng) * np.exp( + 3000 / kelvin - 3000 / 373.15 + ) + model = AcceleratedLife(surv.Weibull, life_models.Exponential).fit( + x, kelvin + ) + node = RegressionNode(model, covariates=[340.0]) + rbd = NonRepairableRBD(ONE, {"a": node}) + assert rbd.sf(100.0) == pytest.approx( + float(np.ravel(model.sf(100.0, np.array([[340.0]])))[0]) + ) + assert rbd_from_json(rbd_to_json(rbd)).mean() == pytest.approx(rbd.mean()) + assert [lever.name for lever in rbd.levers()] == ["covariate.0"] + with pytest.raises(ValueError, match="fitted with 1 covariate"): + RegressionNode(model, covariates=[340.0, 1.0]) + + +def test_a_probability_is_no_life_in_the_long_run(): + # A per-demand probability has no mean time: read as one, the long run + # took p = 0.1 for the mean life (availability 0.09, against the + # simulations' 0.999). + rbd = RepairableRBD( + ONE, + { + "a": { + "reliability": surv.FixedEventProbability.from_params(0.1), + "repairability": E([1.0]), + "repair_cost": 5.0, + } + }, + ) + routes = rbd.analysis_routes() + for name, call in [ + ("mean_availability", rbd.mean_availability), + ("system_failure_frequency", rbd.system_failure_frequency), + ("expected_cost_rate", rbd.expected_cost_rate), + ("birnbaum_importance", rbd.birnbaum_importance), + ]: + assert routes[name].route == "refused" + with pytest.raises(NotImplementedError, match="no mean time"): + call() + assert routes["availability"].route == "simulated" + + +# Duty cycles --------------------------------------------------------------- + +PARAMETERS = { + "Exponential": [0.01], + "Weibull": [100, 2], + "ExpoWeibull": [100, 2, 1.5], + "LogLogistic": [100, 3], + "Rayleigh": [100], + "LogNormal": [4, 0.5], + "Galton": [4, 0.5], + "Gamma": [2, 0.05], + "Normal": [100, 20], + "Gauss": [100, 20], + "Gumbel": [100, 20], + "GumbelLEV": [100, 20], + "Logistic": [100, 20], + "Uniform": [10, 200], +} + + +def test_every_scaled_distribution_is_checked(): + assert set(PARAMETERS) == set(_SCALED) + + +@pytest.mark.parametrize("name", sorted(PARAMETERS)) +def test_a_life_on_the_calendar(name): + # Operating a fraction d of the time, R_c(t) = R(d t). + model = getattr(surv, name).from_params(PARAMETERS[name]) + d, t = 0.3, np.array([5.0, 40.0, 90.0, 150.0]) + moved = on_calendar("a", model, d) + np.testing.assert_allclose( + np.ravel(moved.sf(t / d)), np.ravel(model.sf(t)), atol=1e-12 + ) + offset = W([100, 2], gamma=20.0) + np.testing.assert_allclose( + np.ravel(on_calendar("a", offset, d).sf(t / d)), + np.ravel(offset.sf(t)), + atol=1e-12, + ) + + +def duty_rbd(life, duty, **more): + spec = {"reliability": life, "repairability": E([0.5]), **more} + if duty is not None: + spec["duty"] = duty + return RepairableRBD(ONE, {"a": spec}) + + +def test_a_duty_is_the_life_on_the_calendar(): + # A quarter of the time: a Weibull life of scale 100 in operating + # time lasts 400 on the calendar; repairs and maintenance stay on it. + part = duty_rbd(W([100, 2]), 0.25, preventive={"interval": 300.0}) + calendar = duty_rbd(W([400, 2]), None, preventive={"interval": 300.0}) + assert part.mean_availability() == pytest.approx( + calendar.mean_availability(), rel=1e-12 + ) + first = part.availability(t_simulation=2000.0, mc_samples=50, seed=3) + second = calendar.availability(t_simulation=2000.0, mc_samples=50, seed=3) + assert first.system_failures == second.system_failures + # The levers are the life's in operating time, as given. + levers = {lever.name: lever.value for lever in part.levers()} + assert levers["reliability.alpha"] == 100.0 + again = rbd_from_json(rbd_to_json(part)) + assert again.mean_availability() == part.mean_availability() + assert again._init_args["components"]["a"]["duty"] == 0.25 + + +def test_a_duty_of_one_changes_nothing(): + assert duty_rbd(W([100, 2]), 1.0).mean_availability() == ( + duty_rbd(W([100, 2]), None).mean_availability() + ) + + +@pytest.mark.parametrize( + "life, duty, match", + [ + (W([100, 2]), 0.0, r"in \(0, 1\]"), + (W([100, 2]), 1.5, r"in \(0, 1\]"), + (W([100, 2]), "half", r"in \(0, 1\]"), + (surv.FixedEventProbability.from_params(0.1), 0.5, "on the calendar"), + ], +) +def test_a_duty_is_checked(life, duty, match): + with pytest.raises(ValueError, match=match): + duty_rbd(life, duty) diff --git a/repyability/tests/timeline_reference.py b/repyability/tests/timeline_reference.py new file mode 100644 index 00000000..ce781be9 --- /dev/null +++ b/repyability/tests/timeline_reference.py @@ -0,0 +1,215 @@ +"""The up/down timeline arithmetic, written from its definition: a +reference for the overlaps the event loops add up as they go (the time a +component and the system are both up, both down, either up, either down). +""" + +from collections import defaultdict +from typing import List, Tuple + +import numpy as np + + +def combined_timeline( + timeline_1: List[Tuple[float, int]], timeline_2: List[Tuple[float, int]] +): + """Merge two up/down timelines and count how many are up over time. + + Each timeline is a list of ``(time, change)`` pairs, as the + availability simulation records them for a node and for the system: a + first entry ``(0.0, 1)`` if it starts up or ``(0.0, 0)`` if it starts + down, then ``(t, 1)`` for each restoration and ``(t, -1)`` for each + failure, and a last entry ``(t_end, 0)`` closing the window. The + changes of the two timelines are summed at equal times and accumulated + in time order. + + Parameters + ---------- + timeline_1 : list[tuple[float, int]] + The first timeline, e.g. a node's. + timeline_2 : list[tuple[float, int]] + The second timeline, e.g. the system's. + + Returns + ------- + tuple[numpy.ndarray, numpy.ndarray] + The distinct times in increasing order, and at each time the number + of the two (0, 1 or 2) that are up from that time until the next. + + Examples + -------- + A node down on ``[2, 3)`` and a system down on ``[2, 5)``, over a + window of length 10: + + >>> from repyability.tests.timeline_reference import combined_timeline + >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] + >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] + >>> times, n_up = combined_timeline(node, system) + >>> times.tolist(), n_up.tolist() + ([0.0, 2.0, 3.0, 5.0, 10.0], [2, 0, 1, 2, 2]) + """ + joint_timeline: defaultdict = defaultdict(lambda: 0) + for t, e in timeline_1 + timeline_2: + joint_timeline[t] += e + events: np.ndarray = np.fromiter(joint_timeline.values(), dtype=np.int8) + timeline: np.ndarray = np.fromiter(joint_timeline.keys(), dtype=np.float64) + idx = np.argsort(timeline) + timeline = timeline[idx] + events = events[idx] + events = events.cumsum() + return timeline, events + + +def intersection(timeline, event_cumsum): + """Total time during which both of two timelines are up. + + Sums the lengths of the intervals ``[timeline[j], timeline[j + 1])`` + on which ``event_cumsum[j] == 2``. Pass ``2 - event_cumsum`` instead to + get the total time during which both are down. + + Parameters + ---------- + timeline : numpy.ndarray + Increasing times, as returned by ``combined_timeline``. + event_cumsum : numpy.ndarray + How many of the two are up from each time, as returned by + ``combined_timeline``. + + Returns + ------- + float + The total length of those intervals. + + Examples + -------- + >>> from repyability.tests.timeline_reference import ( + ... combined_timeline, + ... intersection, + ... ) + >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] + >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] + >>> times, n_up = combined_timeline(node, system) + >>> float(intersection(times, n_up)) # both up + 7.0 + >>> float(intersection(times, 2 - n_up)) # both down + 1.0 + """ + from_idx = np.where(event_cumsum[:-1] == 2)[0] + to_idx = from_idx + 1 + intersection = timeline[to_idx] - timeline[from_idx] + return intersection.sum() + + +def union(timeline, event_cumsum): + """Total time during which at least one of two timelines is up. + + Sums the lengths of the intervals ``[timeline[j], timeline[j + 1])`` + on which ``event_cumsum[j] > 0``. Pass ``2 - event_cumsum`` instead to + get the total time during which at least one is down. + + Parameters + ---------- + timeline : numpy.ndarray + Increasing times, as returned by ``combined_timeline``. + event_cumsum : numpy.ndarray + How many of the two are up from each time, as returned by + ``combined_timeline``. + + Returns + ------- + float + The total length of those intervals. + + Examples + -------- + >>> from repyability.tests.timeline_reference import ( + ... combined_timeline, + ... union, + ... ) + >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] + >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] + >>> times, n_up = combined_timeline(node, system) + >>> float(union(times, n_up)) # either up + 9.0 + >>> float(union(times, 2 - n_up)) # either down + 3.0 + """ + from_idx = np.where(event_cumsum[:-1] > 0)[0] + to_idx = from_idx + 1 + union = timeline[to_idx] - timeline[from_idx] + return union.sum() + + +def intersection_over_union( + node_timeline: List[Tuple[float, int]], + system_timeline: List[Tuple[float, int]], +): + """Intersection over union of a node's and the system's up time. + + The time both are up divided by the time at least one is up, for one + pair of timelines. (``RepairableRBD.availability`` computes its ``iou`` + measure from totals over all its simulations instead.) If neither is + ever up the result is nan, with a numpy warning. + + Parameters + ---------- + node_timeline : list[tuple[float, int]] + The node's timeline, in the format described in + ``combined_timeline``. + system_timeline : list[tuple[float, int]] + The system's timeline, in the same format. + + Returns + ------- + float + The ratio, in ``[0, 1]``: 1 when the two are up at exactly the same + times. + + Examples + -------- + >>> from repyability.tests import timeline_reference as tr + >>> intersection_over_union = tr.intersection_over_union + >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] + >>> system = [(0.0, 1), (2.0, -1), (5.0, 1), (10.0, 0)] + >>> round(float(intersection_over_union(node, system)), 4) # 7 / 9 + 0.7778 + """ + timeline, event_cumsum = combined_timeline(node_timeline, system_timeline) + return intersection(timeline, event_cumsum) / union(timeline, event_cumsum) + + +def time_at_status(timeline, status): + """Sum the gaps that follow a timeline's entries of a given value. + + Sums ``t[j + 1] - t[j]`` over every entry ``(t[j], value)`` of + ``timeline`` whose value equals ``status``. For a single node's or the + system's timeline, in the format described in ``combined_timeline``, + ``status=1`` gives its total up time: every up period starts at the + initial ``(0.0, 1)`` entry or at a restoration. (``status=-1`` would + miss a down period at the start, whose entry is ``(0.0, 0)``; the + simulation only uses ``status=1``.) + + Parameters + ---------- + timeline : list[tuple[float, int]] + One node's or the system's timeline. + status : int + The entry value whose following gaps are summed. + + Returns + ------- + float + The total time. + + Examples + -------- + >>> from repyability.tests.timeline_reference import time_at_status + >>> node = [(0.0, 1), (2.0, -1), (3.0, 1), (10.0, 0)] + >>> float(time_at_status(node, 1)) + 9.0 + """ + t = np.array([a for a, _ in timeline]) + events = np.array([b for _, b in timeline]) + from_idx = np.where(events[:-1] == status)[0] + to_idx = from_idx + 1 + union = t[to_idx] - t[from_idx] + return union.sum() diff --git a/repyability/timelines.py b/repyability/timelines.py index 020ff20f..80e0c80f 100644 --- a/repyability/timelines.py +++ b/repyability/timelines.py @@ -463,7 +463,7 @@ def _intervals(data: _Data, up: bool) -> np.ndarray: def _curve(data: _Data) -> Tuple[np.ndarray, np.ndarray]: """The fraction of histories up after each time at which one changed (and at 0 and the end): the times, in order, and the fractions.""" - from repyability.rbd.repairable_rbd import _working_over_time + from repyability.rbd._time_order import _working_over_time history, j = data.positions delta = 2 * _after(data, history, j) - 1 @@ -819,7 +819,7 @@ def from_outages( failed again at that instant, so they count as two failures. Real logs often hold one outage as several records, overlapping or touching (two work orders on one outage): ``merge=True`` makes each - run of them one outage (#179). + run of them one outage. Parameters ---------- @@ -1110,7 +1110,7 @@ def __eq__(self, other) -> bool: return _same(self, other) def to_dict(self) -> dict: - """The history as plain data, ready for ``json.dumps`` (#235). + """The history as plain data, ready for ``json.dumps``. Returns ------- @@ -1376,7 +1376,7 @@ def __eq__(self, other) -> bool: return _same(self, other) def to_dict(self) -> dict: - """The histories as plain data, ready for ``json.dumps`` (#235). + """The histories as plain data, ready for ``json.dumps``. Returns ------- diff --git a/repyability/utils/checks.py b/repyability/utils/checks.py index d402285f..929d89a9 100644 --- a/repyability/utils/checks.py +++ b/repyability/utils/checks.py @@ -37,7 +37,7 @@ def whole_number(value, name: str, minimum: int = 1) -> int: def structure_method(method) -> str: """``"p"`` or ``"c"``, from a structure ``method``: ``"p"`` or ``"paths"`` for the minimal path sets, ``"c"`` or ``"cuts"`` for the - minimal cut sets (#179); else a ``ValueError``.""" + minimal cut sets; else a ``ValueError``.""" try: return _STRUCTURE_METHODS[method] except (KeyError, TypeError): @@ -50,23 +50,60 @@ def structure_method(method) -> str: def simulation_options(method: str, given: dict) -> None: """Refuse the simulation options in ``given`` (those passed: not None or False) to ``method``, whose answer is exact unless - ``method="simulate"``: they would do nothing.""" + ``method="simulate"``: they would do nothing. A ``ValueError``, as + for any argument whose value does not fit the call, from every class + that simulates on request.""" passed = [ name for name, value in given.items() if value is not None and value is not False ] if passed: - raise TypeError( + raise ValueError( f"{method} takes {', '.join(passed)} only with " "method='simulate': its answer is otherwise exact." ) +def simulation_window(t_simulation) -> float: + """A simulation's window, ``t_simulation``, as a float: a positive and + finite time.""" + if ( + isinstance(t_simulation, bool) + or not isinstance(t_simulation, (int, float, np.integer, np.floating)) + or not (math.isfinite(t_simulation) and t_simulation > 0.0) + ): + raise ValueError( + f"t_simulation must be a positive, finite time, got " + f"{t_simulation!r}." + ) + return float(t_simulation) + + +def nonnegative_times(x) -> np.ndarray: + """``x`` as a 1-d float array of times, all finite and non-negative: + numbers, not text that reads as them (#233).""" + times = np.atleast_1d(real_array(x, "The times")) + if not np.all(np.isfinite(times)) or np.any(times < 0.0): + raise ValueError(f"Times must be finite and non-negative, got {x!r}.") + return times + + +def one_of(name: str, value, choices: tuple) -> None: + """Refuse ``value`` for the option ``name`` unless it is one of + ``choices``: "name must be 'a' or 'b', got 'c'.".""" + if value not in choices: + named = [repr(c) for c in choices] + listed = " or ".join( + [", ".join(named[:-1]), named[-1]] if len(named) > 1 else named + ) + raise ValueError(f"{name} must be {listed}, got {value!r}.") + + def is_number(value) -> bool: """Whether ``value`` is a real number: an int or a float (numpy's too, or any ``numbers.Real``), but not a bool, nor text that reads as - a number (#233).""" + a number.""" if isinstance(value, (bool, np.bool_)): return False if isinstance(value, Real): @@ -82,7 +119,7 @@ def number_or_nan(value) -> float: """``value`` as a float if it is a real number (see ``is_number``), else NaN: for a check that then refuses it in its own words. Text such as ``"8760"``, which ``float`` would read, and booleans, which it would - read as 1 and 0, are not numbers here (#233).""" + read as 1 and 0, are not numbers here.""" return float(value) if is_number(value) else math.nan @@ -90,7 +127,7 @@ def real_array(value, name: str) -> np.ndarray: """``value`` (a number or an array of them) as a float array, or a TypeError naming ``name``: text, which numpy would read as a number (``"8760"``), and booleans, which it would read as 1 and 0, are not - numbers here (#233).""" + numbers here.""" array = np.asarray(value) kind = array.dtype.kind if kind == "O" and not any( @@ -112,7 +149,7 @@ def real_array(value, name: str) -> np.ndarray: def seed(value): """``value``, a seed, if it is one (see ``SEEDS``), else a TypeError - or ValueError saying what one is (#232). A numpy ``Generator`` is no + or ValueError saying what one is. A numpy ``Generator`` is no seed: the simulations draw from streams of their own, which one number seeds.""" if value is None: @@ -144,7 +181,7 @@ def seed(value): def unfitted_distribution(model) -> "str | None": """The name of the surpyval distribution ``model`` is, where it is the distribution itself (``surv.Weibull``) rather than a model of it - (fitted, or made with ``from_params``); None otherwise (#233).""" + (fitted, or made with ``from_params``); None otherwise.""" if not type(model).__module__.startswith("surpyval"): return None if not (hasattr(model, "fit") and hasattr(model, "from_params")): @@ -157,7 +194,7 @@ def unfitted_distribution(model) -> "str | None": def no_distribution(model, what: str) -> None: """Raise if ``model``, given as ``what``, is a surpyval distribution - itself rather than a model of it, saying how to make one (#233).""" + itself rather than a model of it, saying how to make one.""" name = unfitted_distribution(model) if name is not None: raise TypeError( diff --git a/repyability/utils/deprecation.py b/repyability/utils/deprecation.py index 6d5c744f..d975974b 100644 --- a/repyability/utils/deprecation.py +++ b/repyability/utils/deprecation.py @@ -1,23 +1,28 @@ """Deprecated names, arguments and calls. -A deprecation gives one minor release's notice: what 0.13 deprecates goes -in 0.14 (``NEXT_REMOVAL``). Each warns with a ``FutureWarning``, which -Python always shows, as the notice is short: - -- the search's ``offsets`` of ``optimal_inspection_intervals``, renamed - ``offset_shares`` (``renamed``, #222); -- calling ``CapacityDistribution.mean()``, now a property (``called``, - #235); -- the simulation options of a ``StandbyModel``'s, ``LoadSharingModel``'s - or ``DegradingNode``'s exact ``mean``, which it ignores (``ignored``, - #233). - -What 0.12 deprecated went in 0.13: calling ``SparesDemand.mean()`` and -``std()``, properties since 0.12 (#184), and ``StandbyModel``'s and -``LoadSharingModel``'s ``mc_samples``, ``lower`` and ``seed``, which set a -fit to simulated lifetimes that 0.12 removed (#149). What 0.11 deprecated -went in 0.12 (#149): its old names are refused with the names that took -their place (``refuses_removed_names``, #232). +A deprecation gives one minor release's notice: what 0.14 deprecates goes +in 0.15 (``NEXT_REMOVAL``). Each warns with a ``FutureWarning``, which +Python always shows, as the notice is short: an argument renamed +(``renamed``; an attribute, ``renamed_attribute``), a result's value +called that is now a property (``called``), or arguments a method ignores +(``ignored``). 0.14 deprecates: + +- ``NodeState``'s ``phase``, renamed ``since_scheduled`` (``renamed`` for + the argument, ``renamed_attribute`` for reading it, #350). + +What 0.13 deprecated went in 0.14: ``optimal_inspection_intervals``' +``offsets``, renamed ``offset_shares`` (#222); calling +``CapacityDistribution.mean()``, a property since 0.13 (#235); the +simulation options given to the exact ``mean`` of a ``StandbyModel``, +``LoadSharingModel`` or ``DegradingNode``, which it would ignore (#233); +and ``rbd._sensitivity``'s ``levers`` and ``Lever``, the names callers +used before the levers were public (#244). What 0.12 deprecated went in +0.13: calling ``SparesDemand.mean()`` and ``std()``, properties since 0.12 +(#184), and ``StandbyModel``'s and ``LoadSharingModel``'s ``mc_samples``, +``lower`` and ``seed``, which set a fit to simulated lifetimes that 0.12 +removed (#149). What 0.11 deprecated went in 0.12 (#149): its old names +are refused with the names that took their place +(``refuses_removed_names``, #232). """ import functools @@ -29,8 +34,8 @@ from repyability.utils.wrappers import outside_level -#: The release that removes what 0.13 deprecates. -NEXT_REMOVAL = "0.14" +#: The release that removes what 0.14 deprecates. +NEXT_REMOVAL = "0.15" def _called_warning(name: str, removal: str) -> None: @@ -121,12 +126,29 @@ def ignored( ) -def renamed(method: str, old: str, new: str, why: str) -> None: +def renamed( + method: str, old: str, new: str, why: str, removal: str = NEXT_REMOVAL +) -> None: """Warn that ``method``'s argument ``old`` is renamed ``new``, for the - reason ``why``: deprecated in 0.13, it is refused in ``NEXT_REMOVAL``.""" + reason ``why``: deprecated, it is refused in ``removal`` + (``NEXT_REMOVAL`` by default).""" warnings.warn( f"{method}'s {old}= is renamed {new}=, as {why}. Passing {old}= is " - f"deprecated, and {NEXT_REMOVAL} will refuse it.", + f"deprecated, and {removal} will refuse it.", + FutureWarning, + stacklevel=outside_level(), + ) + + +def renamed_attribute( + owner: str, old: str, new: str, why: str, removal: str = NEXT_REMOVAL +) -> None: + """Warn that ``owner``'s attribute ``old`` (read as ``owner.old``) is + renamed ``new``, for the reason ``why``: deprecated, reading it is + refused in ``removal`` (``NEXT_REMOVAL`` by default).""" + warnings.warn( + f"{owner}.{old} is renamed {new}, as {why}. Reading {old} is " + f"deprecated, and {removal} will refuse it.", FutureWarning, stacklevel=outside_level(), ) @@ -148,7 +170,7 @@ def renamed(method: str, old: str, new: str, why: str) -> None: def refuses_removed_names(function): """``function``, refusing the simulation-count names 0.12 removed with the names that took their place, where Python would say only that the - keyword was unexpected (#232); ``function`` itself where it takes + keyword was unexpected; ``function`` itself where it takes neither ``mc_samples`` nor ``max_samples``.""" if getattr(function, "refuses_removed_names", False): return function diff --git a/repyability/utils/wrappers.py b/repyability/utils/wrappers.py index a2dafa17..2521605b 100644 --- a/repyability/utils/wrappers.py +++ b/repyability/utils/wrappers.py @@ -74,7 +74,7 @@ def numpy_seed(seed): def node_names(func): """``func`` (a method) with a bare string given to one of its - ``NODE_ARGUMENTS`` taken as one node's name (#225), not as its + ``NODE_ARGUMENTS`` taken as one node's name, not as its characters: ``working_nodes="belt"`` is ``["belt"]``, where iterating it gave ``{"b", "e", "l", "t"}``, and an error that changed from run to run with the strings' hashes. ``func`` itself where it has none of @@ -130,7 +130,12 @@ def conditional_survival(model, x, X, *args, **kwargs): already survived to ``X``: .. math:: - R(x \\mid X) = \\frac{R(X + x)}{R(X)} + R(x \\mid X) = \\frac{R(X + x)}{R(X)} = e^{-(H(X + x) - H(X))} + + It is worked out from the cumulative hazard ``Hf`` where the model has + one, which keeps its precision where ``R(X + x)`` and ``R(X)`` are + both too small for a float (an age far past the model's lives), + and as the ratio of ``sf`` otherwise. Parameters ---------- @@ -145,17 +150,27 @@ def conditional_survival(model, x, X, *args, **kwargs): Returns ------- float or numpy.ndarray - The conditional survival probability, clipped to ``[0, 1]``; where the - item has all but surely failed by ``X`` (``R(X) ≈ 0``) it is ``0``. + The conditional survival probability, clipped to ``[0, 1]``; where + ``X`` cannot be reached (``R(X)`` is 0, or rounds to it without a + cumulative hazard) it is ``0``. A float if both ``x`` and ``X`` are scalars, otherwise an array. """ scalar_in = np.ndim(x) == 0 and np.ndim(X) == 0 x = np.atleast_1d(np.asarray(x, dtype=float)) X = np.atleast_1d(np.asarray(X, dtype=float)) - denom = np.asarray(model.sf(X, *args, **kwargs), dtype=float) - numer = np.asarray(model.sf(x + X, *args, **kwargs), dtype=float) + hazard = getattr(model, "Hf", None) with np.errstate(divide="ignore", invalid="ignore"): - out = numer / denom + if hazard is None: + out = np.asarray( + model.sf(x + X, *args, **kwargs), dtype=float + ) / np.asarray(model.sf(X, *args, **kwargs), dtype=float) + else: + out = np.exp( + -( + np.asarray(hazard(x + X, *args, **kwargs), dtype=float) + - np.asarray(hazard(X, *args, **kwargs), dtype=float) + ) + ) out = np.where(np.isfinite(out), out, 0.0) out = np.clip(out, 0.0, 1.0) return out.item() if scalar_in else out diff --git a/scripts/calendar_policy_chart.py b/scripts/calendar_policy_chart.py new file mode 100644 index 00000000..0102c26a --- /dev/null +++ b/scripts/calendar_policy_chart.py @@ -0,0 +1,103 @@ +"""Draw the user guide's chart of a calendar policy's thresholds +(docs/guide/img/calendar-thresholds.svg): the threshold age at each weekly, +quarterly and annual shut over a year of the plant in +docs/guide/maintenance.md. + +The chart is drawn here, outside the package (which has no plotting +dependency). Run it again after a change to the calendar policy: + + python scripts/calendar_policy_chart.py +""" + +from pathlib import Path + +import matplotlib + +matplotlib.use("svg") +import matplotlib.pyplot as plt # noqa: E402 +import surpyval as surv # noqa: E402 + +from repyability import Calendar, NonRepairable, Opportunity # noqa: E402 + +OUT = ( + Path(__file__).resolve().parent.parent + / "docs" + / "guide" + / "img" + / "calendar-thresholds.svg" +) + +SERIES = ( + ("weekly", "#2a78d6", "o"), + ("quarterly", "#eb6834", "s"), + ("annual", "#1baf7a", "D"), +) +TEXT, MUTED, SURFACE = "#262624", "#6b6a64", "#fcfcfb" + + +def main() -> None: + bearing = NonRepairable(surv.Weibull.from_params([1000.0, 2.5])) + plant = Calendar( + [ + Opportunity("weekly", 3.0, period=7.0), + Opportunity("quarterly", 2.0, period=91.0), + Opportunity("annual", 1.0, period=364.0), + ], + failure=20.0, + anytime=4.0, + ) + policy = bearing.optimal_calendar_policy(plant) + figure, axes = plt.subplots(figsize=(8, 4.2), dpi=100) + figure.patch.set_facecolor(SURFACE) + axes.set_facecolor(SURFACE) + for name, colour, marker in SERIES: + points = policy.schedule(kinds=(name,)) + days = [t for t, _, _, _ in points] + ages = [a for _, _, a, _ in points] + axes.plot( + days, + ages, + linestyle="none", + marker=marker, + markersize=6 if name == "weekly" else 9, + color=colour, + markeredgecolor=SURFACE, + markeredgewidth=1.5, + label=name, + zorder=3 if name != "weekly" else 2, + ) + if name != "weekly": + for day, age in zip(days, ages): + axes.annotate( + f"{age:.0f} d", + (day, age), + textcoords="offset points", + xytext=(8, -4), + color=TEXT, + fontsize=9, + ) + axes.set_xlabel("Day of the plant's year", color=MUTED) + axes.set_ylabel("Replace if at least this old (days)", color=MUTED) + axes.set_xlim(-10, 374) + axes.set_ylim(0, None) + axes.grid(axis="y", color="#e4e3dd", linewidth=0.8) + for side in ("top", "right"): + axes.spines[side].set_visible(False) + for side in ("left", "bottom"): + axes.spines[side].set_color("#c3c2b7") + axes.tick_params(colors=MUTED) + axes.legend(frameon=False, loc="upper left", labelcolor=TEXT) + axes.set_title( + "The threshold at each shut rises as the cheap annual shut nears", + color=TEXT, + loc="left", + fontsize=11, + ) + figure.tight_layout() + OUT.parent.mkdir(parents=True, exist_ok=True) + figure.savefig(OUT, format="svg", metadata={"Date": None}) + print(f"Wrote {OUT}") + + +if __name__ == "__main__": + main() diff --git a/scripts/check.sh b/scripts/check.sh new file mode 100755 index 00000000..53574e88 --- /dev/null +++ b/scripts/check.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# The checks CI's lint job runs, in the same order, before work goes into +# dev (the one automated run is on the pull request from dev into master). +# Give test files or directories to run them too, on every core: +# +# scripts/check.sh # lint only +# scripts/check.sh repyability/tests/test_units.py # lint, then those tests +set -euo pipefail +cd "$(dirname "$0")/.." + +black --check repyability +isort --check-only repyability +python -m flake8 repyability +mypy repyability + +if [ "$#" -gt 0 ]; then + python -m pytest -n auto -q "$@" +fi diff --git a/stories/README.md b/stories/README.md new file mode 100644 index 00000000..8b9b0e8c --- /dev/null +++ b/stories/README.md @@ -0,0 +1,41 @@ +# User stories + +The questions people (and the agents working for them) bring to RePyability +that it should be able to answer. These files record *what* is wanted and +*how we will know it is answered*. Units of work go in issues, the larger +ideas and their designs in [ROADMAP.md](../ROADMAP.md), and the outcomes +they serve here. + +| File | Area | Stories | +|---|---|---| +| [maintenance-economics.md](maintenance-economics.md) | The net present value of maintenance plans, designs and crews, kept current as the plant changes | MX-01 to MX-25 | + +## Format + +Each story has + +- an **ID** (`MX-03`), stable once written, so code, tests and issues can + cite it; +- a **persona**: who asks, and in what job; +- the **story**: *as a ..., I want ..., so that ...*; +- the **question in their words**, as they would type it to RePyability + or to an agent; +- the **setting**: a concrete system, taken where possible from the docs, + the tutorial or a study already in the repository, so the story can be + checked against numbers we know; +- **acceptance criteria**: what an answer must contain, and the + independent checks (an oracle) it must pass, so the answer is known to + be right and not just present; +- what it **needs**: the building blocks (listed at the top of each file) + it depends on; +- **references**, where a standard text sets out the question; +- a **status**: *open* (not answerable without the analyst writing it by + hand), *partial* (part of it can be done today, said how) or *met* (with + the test that shows it). + +## Adding a story + +Start from a question someone has actually asked, or a study in the docs +or an issue. Keep the setting concrete and small enough to check by hand +or by an independent calculation. Number new stories after the last one in +the file; do not reuse an ID. diff --git a/stories/maintenance-economics.md b/stories/maintenance-economics.md new file mode 100644 index 00000000..b6f12d35 --- /dev/null +++ b/stories/maintenance-economics.md @@ -0,0 +1,719 @@ +# Maintenance economics: the net present value of a plan + +Stories MX-01 to MX-25: valuing a plan (MX-01 to MX-17) and keeping its +value current as the plant changes (MX-18 to MX-25). Statuses as of 0.13 +and what is merged on `dev` for 0.14 (16 October 2026). The plan of work +is in [ROADMAP.md](../ROADMAP.md). + +Every story is answered in model time: the models' own unit, from 0 or +from a state. A question asked in dates (a month, a financial year, a +budget that resets on 1 July) is turned into model time by the +application that asks it, and the answer back into dates; RePyability +keeps no calendar (see the roadmap's "Scope"). + +## Why + +The people who decide on maintenance plans, spares, crews and redundancy +decide on money, at a cost of capital, over a horizon, and over 20 years +discounting changes which option wins. The end-to-end pump-station study +in #184 (item 3) found exactly that and had to discount by hand. + +0.12 took the first step: `total_cost` and `allocate_redundancy` take a +`discount_rate` (a continuous rate per unit time of the models), buying +the components at the start and discounting the **long-run** cost rate +over the horizon, `(1 - exp(-r H)) / r`. Everything else (but for one +part's skipped replacement, `ReplacementPlan.skip_next`, on `dev` for +0.14) is still undiscounted: the costs from new (`expected_cost`, where the early years +weigh most), the interval optimisers, the simulations and their +distributions. There is no value side (what the plant earns while it +runs), no plan of actions at given times, no cash-flow table, and no way +to keep a plan's value current as the plant changes. + +The question they ask is not "what is the cheapest plan per hour in the +long run" but "**which plan creates the most value, at the least cost, +over the life of the asset, and how sure are we?**" + +## Background + +### The expected NPV is a linear functional of what is already exact + +By linearity of expectation, the expected present value of a cost charged +at events is the cost times the discounted expected number of events: + +```text +E[PV] = c * integral_0^H exp(-delta t) dM(t), delta = ln(1 + r) +``` + +with `M(t)` the expected number of those events by `t`, which +`expected_events(t)` already gives exactly on a grid of times. Costs and +value that accrue per hour discount against the point availability: + +```text +PV(downtime cost) = integral exp(-delta t) * c_d * (1 - A(t)) dt +PV(value) = integral exp(-delta t) * v(t) * A(t) dt +``` + +So expected NPV needs only `A(t)` and `M(t)`, which RePyability already has +for independent components, maintenance, inspections and (for exponential +components) repair crews. Two consequences: + +- **The long-run rate is not enough.** Discounting weighs the early years + most, and from new a wear-out system fails less than its long-run rate. + A prototype on top of `expected_cost` (two Weibull pumps in parallel + feeding a compressor, 10 years at 8%, 900 per hour of margin) put the + undiscounted cheapest pump PM interval at 10,000 h and the highest-NPV + one at 14,000 h: deferring spend is worth something. +- **The like-for-like objective is the equivalent annual cost**, + `EAC = delta * PV` (or the annuity form over a finite horizon): the + discounted counterpart of `expected_cost_rate`, comparable across plans + of different lengths. As `delta -> 0` it must reduce to the existing cost + rate, which gives every story a free regression check. + +Distributional questions (percentiles, the chance of missing a hurdle rate, +contract penalties on annual availability) are not linear in the events +and need the simulated distribution, each event discounted at its own time. + +### Building blocks + +The stories need these. They are capabilities, not a plan of work; one +change may deliver several. + +| ID | Building block | +|---|---| +| **B1** | Expected NPV and EAC on the exact curves: a constant value rate, event and rate costs, acquisition cost; a cash-flow table by year, category and component. Fixed-time events (block maintenance, or actions at given times) discounted at their exact times, not at grid midpoints. | +| **B2** | Value that varies in time: a price curve, capacity times price, any `v(t)`, as functions of model time; escalation per cost stream; real or nominal rates. | +| **B3** | Planned maintenance on the timeline: outages at given model times, with a duration and a scope (renew some components, restore others to a virtual age, take the system down); recurring campaigns with opportunistic work; mid-life upgrades that swap a component's life model at a capital cost; retirement with a salvage value from each component's age. | +| **B4** | The simulated NPV distribution: each simulated event discounted at its time; percentiles, the chance of missing a hurdle, conditional value at risk; non-linear payoffs such as availability-contract penalties; plans compared with common random numbers. | +| **B5** | Crews as cash flows and as options: crew salaries as rate costs; a contract crew called in when the repair queue reaches a threshold, at a call-out fee and a premium rate. | +| **B6** | `objective="npv"` / `"eac"` in the optimisers (`optimal_replacement_intervals`, `optimal_inspection_intervals`, `allocate_redundancy`), with constraints kept (availability, PFDavg); the discount-rate sensitivity (rho). | +| **B7** | Parameter uncertainty carried into NPV: the fitted models' uncertainty propagated to the NPV and to the decision (see the uncertainty-importance issue, #196). | +| **B8** | A live plant: the diagram, the plan, the valuation and each component's current state held together, updated by events (a sensor reading, a replacement, a repair, a failure, an inspection, a refit of a life model, a new price curve), each re-valuing the plan from the state now, recomputing only what the event touched, and recording what changed the value and by how much. | +| **B9** | Re-optimising a plan from the state now (receding horizon): the actions to bring forward, defer, add or drop, with what each is worth, only proposed when it beats the current plan by a margin, within crews, outage windows, budgets and safety limits. | +| **B10** | An agent-facing contract: every input and result as plain JSON with its units, a stable schema, `analysis_routes` coverage of every NPV analysis, refusals that say what to do, and a valuation reproducible from its saved inputs (state, plan, models, prices, seed). | + +Valuing from the components' current states uses the existing `state=` +(`NodeState`, which has a virtual age since #269), which the NPV must +accept wherever the curves do. + +### Shape of an answer + +Not an API decision, but what every story's answer should carry: + +- the **NPV** (value less costs, or costs alone when no value is given), + the **EAC**, and the undiscounted total for comparison; +- a **cash-flow table** by year, category (repair, replace, preventive, + inspection, downtime, setup, crews, capital, salvage, value) and + component, so users can apply their own tax, depreciation or WACC + conventions outside the package; +- how it was computed (exact, numerical or simulated, as + `analysis_routes()` reports for every other analysis), and a clear + refusal, with the reason, where it cannot be. + +--- + +## Maintenance engineer + +### MX-01: Does my PM interval still hold at our cost of capital? + +- **Persona:** a maintenance engineer setting a component's age-replacement + interval, asked by finance to justify it at the company's WACC. +- **Story:** As a maintenance engineer, I want the replacement interval + that minimises the discounted cost, so that the interval I set is the + one the business case assumes. +- **In their words:** "What's the best replacement age for this bearing at + an 8% discount rate, and how much does it differ from the undiscounted + one?" +- **Setting:** the bearing of Lesson 8 and the maintenance guide: + `NonRepairable(surv.Weibull.from_params([1000, 2.5]))` with + `cp = 1`, `cu = 5`, whose undiscounted optimum is 493.0 h at a cost rate + of 0.003462 per hour. +- **Acceptance criteria:** + - The optimal interval and its EAC at 0%, 8% and 15% a year (time unit + stated), and the EAC curve over a range of intervals. + - At 0% the answer equals `optimal_replacement_policy()` (493.0 h, + 0.003462) to the optimiser's tolerance. + - It matches Fox's (1966) closed form for discounted age replacement + over an infinite horizon, integrated independently: + `PV(T) = [c_u * int_0^T e^(-delta t) f(t) dt + c_p * e^(-delta T) R(T)] / (1 - L(delta))`, + `L(delta) = int_0^T e^(-delta t) f(t) dt + e^(-delta T) R(T)`. + - The same question at system level (`RepairableRBD` with + `"preventive"`) agrees with the component answer for a single node. +- **Needs:** B1, B6. +- **References:** Fox (1966), *Age replacement with discounting*; + Jardine & Tsang, *Maintenance, Replacement and Reliability*, ch. 2–3. +- **Status:** open. The undiscounted optimum is + `optimal_replacement_policy()`; on `dev` for 0.14, + `ReplacementPlan.skip_next(discount_rate)` (#319) discounts one + decision about the plan, not the interval. + +### MX-02: Overhaul the gearbox now, or defer it to next financial year? + +- **Persona:** a maintenance planner with a repairable gearbox under + minimal repair and a budget that resets on 1 July. +- **Story:** As a maintenance planner, I want the value of each overhaul + date, so that I can defend deferring (or not deferring) an overhaul + across a budget boundary. +- **In their words:** "If I push the gearbox overhaul from May to August, + what does it cost us in expected failures, and what do we gain from + spending later?" +- **Setting:** the gearbox of the maintenance guide (`Repairable`, minimal + repair, `optimal_overhaul_policy().cost_rate` 0.5848), with the + overhaul as an action at a given time on the model's timeline (May and + August turned into times by the application). +- **Acceptance criteria:** + - The NPV of the plan for each candidate overhaul time, the difference + between them, and the expected number of failures before each. + - With no discounting and the overhaul on its optimal cycle, the cost + rate equals `optimal_overhaul_policy().cost_rate`. + - An independent check: the expected failures before the overhaul from + the power-law cumulative intensity, discounted by hand. +- **Needs:** B1, B3 (an overhaul at a given time). +- **Status:** open. + +### MX-03: Bundle the compressor overhaul into the pump shutdown? + +- **Persona:** a maintenance planner at a plant where every stop has a + fixed set-up cost (isolation, permits, scaffolding, lost production). +- **Story:** As a maintenance planner, I want to compare separate stops + with one campaign that does both jobs, so that I can see whether sharing + the set-up cost is worth doing some work early or late. +- **In their words:** "Should we do the compressor overhaul in the same + shutdown as the pump replacements every four years, instead of on its + own schedule?" +- **Setting:** a pump pair and a compressor in series (as in the + prototype), with a maintenance group's set-up cost charged once per stop + (the existing `"group"` and `"opportunity"` keys), compared with a + four-yearly campaign. +- **Acceptance criteria:** + - The NPV and EAC of each plan, with the set-up cost and downtime shown + separately in the cash-flow table. + - The campaign interval (and which jobs join it) that maximises NPV. + - Undiscounted, the separate-stops plan's cost rate equals today's + `expected_cost_rate` for the same group. +- **Needs:** B1, B3 (campaigns), B6. +- **Status:** partial: groups and opportunistic maintenance exist + undiscounted, and on `dev` for 0.14 `replacement_policies(shut_every=)` + (#319) finds a part's best policy against periodic shuts, undiscounted; + campaigns at given times do not exist. + +### MX-04: Proof-test intervals under a SIL target, at least cost + +- **Persona:** a functional safety engineer who must meet a PFDavg target + and would like to spend as little as possible doing it. +- **Story:** As a functional safety engineer, I want the test plan with + the lowest present cost that still meets the PFDavg target, so that + compliance is met without over-testing. +- **In their words:** "What's the cheapest proof-test plan for the two + pressure switches that keeps PFDavg under 1e-4, and should the tests be + staggered?" +- **Setting:** the 1oo2 trip of #184 (item 4) with a beta factor of 10%, + where staggering two annual tests by six months takes PFDavg from 9.6e-4 + to 4.9e-4. +- **Acceptance criteria:** + - The plan (intervals and offsets) minimising PV of tests plus downtime + subject to PFDavg <= target, with its PFDavg and PV. + - The offset (at least the half-interval stagger) is searched, not fixed + at 0. + - PFDavg checked against the closed form for a 1oo2 group with beta + (IEC 61508-6, annex B), computed independently. +- **Needs:** B1, B6 (constrained, with offsets). +- **References:** IEC 61508-6; #184 item 4. +- **Status:** partial: since 0.12 `optimal_inspection_intervals` searches + the offsets too (`offset_shares=`, so named since 0.13) and meets the + target, undiscounted. + +--- + +## Asset manager and finance + +### MX-05: Is the standby pump worth buying? + +- **Persona:** an asset manager writing the business case for a capital + purchase, doing life-cycle costing. +- **Story:** As an asset manager, I want the NPV, payback and IRR of adding + a standby unit, so that the capital request states its return the way + finance expects. +- **In their words:** "We're thinking of putting in a standby pump. + Over 15 years at 8%, does it pay for itself, and when?" +- **Setting:** `alone` and `with_standby` from the costs guide (Weibull + 1000/2.5 pumps, 5000 per replacement, 1000 per preventive replacement, + 500 per hour of lost production; best cost rates 13.14 alone and 6.985 + with the standby), with an acquisition cost for the second pump. +- **Acceptance criteria:** + - The NPV of each design, the incremental NPV of the standby, its + discounted payback year and its IRR (the rate at which the incremental + NPV is zero). + - Each design at its own best PM interval (the interval changes with the + design, as the guide shows). + - The yearly cash-flow table reconciles to the NPV. + - An independent check: discounting `expected_cost(t)` on a fine grid by + hand gives the same PV within the grid's error. +- **Needs:** B1, B6. +- **References:** IEC 60300-3-3 (life cycle costing); ISO 15663. +- **Status:** partial: `total_cost(horizon, discount_rate=)` (0.12) gives + each design's present value at its long-run cost rate; the costs from + new, payback, IRR and the cash-flow table do not exist. + +### MX-06: Should we add a fourth pump train? + +- **Persona:** an asset manager deciding on redundancy for a cooling-water + pump station over a 20-year life. +- **Story:** As an asset manager, I want redundancy allocation to optimise + NPV, so that the design it recommends is the one with the best return, + not the lowest undiscounted cost. +- **In their words:** "Two-of-three or two-of-four pump trains? Over 20 + years at 8%, which is worth more?" +- **Setting:** the 2-of-3 station of #184, with lives fitted in SurPyval + from censored field data, one repair crew, PM and proof tests; the + alternative is a fourth train (seal, bearing and motor in series). +- **Acceptance criteria:** + - The NPV of each design and the winner at 0% and at 8%; the story + exists because the winner can flip. + - `allocate_redundancy(..., objective="npv")` (or equivalent) returns the + same design as comparing the NPVs by hand. + - A whole train (a set of nodes) can be the unit copied (#184 item 2; + `trains=` since 0.12). +- **Needs:** B1, B6. +- **Status:** partial: 0.12's `allocate_redundancy(trains=..., + discount_rate=...)` answers it with the long-run cost rate (a fourth + train that pays undiscounted no longer does at 15% a year). Left: the + costs from new, the value side, and a crew-limited plant. + +### MX-07: What's the most we should pay for Vendor B's valve? + +- **Persona:** a procurement or reliability engineer comparing quotes for + a more reliable part. +- **Story:** As an engineer comparing vendors, I want the break-even price + of a more reliable component, so that I know the most a reliability + improvement is worth before negotiating. +- **In their words:** "Vendor B's valve has a characteristic life 40% + longer. It costs more. What's the most we should pay for it?" +- **Setting:** the plant of Lesson 7 (two pumps and a valve; + `expected_cost_rate()` 121.23, and 44.63 with the valve perfect), with + Vendor B's valve as a different Weibull. +- **Acceptance criteria:** + - The PV of running cost with each valve, and the break-even price + difference (the PV saved). + - An upper bound: the PV saved by a perfect valve + (`working_nodes=["valve"]`), which no real valve can exceed. + - The inverse question: the life Vendor B's valve must have to break + even at a quoted price. +- **Needs:** B1. +- **Status:** partial: the undiscounted saving per hour is available today. + +### MX-08: Refurbish the compressor in year 8, or replace it with the new model? + +- **Persona:** an asset manager planning mid-life works. +- **Story:** As an asset manager, I want to value a mid-life upgrade that + changes a component's life model, with salvage at the end, so that + refurbish, replace and run-on can be compared on one basis. +- **In their words:** "In year 8 we can refurbish the compressor or put in + the new model. Which is worth more over the remaining 12 years?" +- **Setting:** the compressor of the prototype (Weibull 30,000 h, 2.2), + with a refurbishment that restores it to a virtual age (Kijima) and a + replacement with a different Weibull at a capital cost. +- **Acceptance criteria:** + - The NPV of run-on, refurbish and replace, with salvage value at the + horizon from each component's age. + - Retiring early is not free: the salvage value enters the cash-flow + table. +- **Needs:** B1, B3 (upgrades, salvage). +- **Status:** open. + +### MX-09: What's our P90, and how likely are we to breach the availability contract? + +- **Persona:** a commercial manager whose contract pays a penalty when a + year's availability falls below 95%. +- **Story:** As a commercial manager, I want the distribution of NPV and + the expected penalty, so that I can price the contract's risk and report + a P90 to the board. +- **In their words:** "What's the P10/P50/P90 NPV of this plant over the + contract, and what's the chance we pay the availability penalty in any + given year?" +- **Setting:** any of the plants above, with a penalty per percentage point + of annual availability below 95%. +- **Acceptance criteria:** + - The NPV percentiles, the chance of missing a hurdle NPV, and the + present value of expected penalties. + - The penalty is computed from each simulated year's availability, not + from the mean availability (the story exists because those differ). + - The simulated mean NPV agrees with the exact expected NPV (B1) within + its confidence interval. + - A hand-written Monte Carlo of the yearly availabilities agrees within + its confidence interval. +- **Needs:** B1, B4. +- **Status:** open. + +--- + +## Operations and trading + +### MX-10: Which month should the outage go in? + +- **Persona:** a generator's asset manager working with the trading desk. +- **Story:** As an asset manager, I want to value a planned outage at + each candidate time against the forward price curve, so that the outage + goes where it costs the least margin, net of the extra failure risk of + waiting. +- **In their words:** "We need a 21-day outage next year. Which month + costs us least, given the forward curve and the chance the unit fails + before we get to it?" +- **Setting:** a generating unit with a wear-out (Weibull) life, a 21-day + planned outage that renews it, and a seasonal forward price curve, as a + function of model time (the application turns the months into times). +- **Acceptance criteria:** + - The NPV for each candidate start time (each month's), the best, and + its value over the worst. + - The probability of a forced outage before the planned one, for each + start time. + - An independent check: enumerating the twelve start times by hand, + with the lost margin of each outage window from the curve. +- **Needs:** B1, B2 (price curve), B3 (an outage at a given time). +- **Status:** open. + +### MX-11: Is it worth running hot? + +- **Persona:** an operations manager who can raise output at the cost of + faster wear. +- **Story:** As an operations manager, I want to value running at a higher + load against the life it consumes, so that I can decide when the extra + margin is worth it. +- **In their words:** "If we run at 70 °C instead of 30 °C we make more, + but the bearings wear out faster. Does it pay?" +- **Setting:** the load schedule of the condition-based guide (30 °C for + 2000 h, then 70 °C), with value proportional to load. +- **Acceptance criteria:** + - The NPV of each operating schedule, with value and maintenance cost + separated. + - The switch-over time that maximises NPV. +- **Needs:** B1, B2; covariate-dependent lives in a `RepairableRBD`, which + today only the non-repairable side supports. +- **Status:** open. + +### MX-12: Pump 1 is at 18,000 hours. Replace it at the March outage, or run it? + +- **Persona:** a reliability engineer making a live decision about one + unit, from its history. +- **Story:** As a reliability engineer, I want to value the options for a + unit from its current condition, so that the decision uses what we know + about this unit, not a new one. +- **In their words:** "Pump 1 has run 18,000 hours since its last repair. + Do we replace it at the March outage, refurbish it, or run it to + failure?" +- **Setting:** a repairable pump whose repairs are imperfect, with the + truth of SurPyval's repairable-fleet scenario card (Kijima type I, + `q = 0.4`, lives Weibull 30 months, 2.5), and an outage at a given time + (March's, in model time). +- **Acceptance criteria:** + - The NPV of each option from the unit's current virtual age + (`state=`), with salvage at the horizon. + - An independent check: simulating forward from the virtual age by hand. + - The fit side (the virtual age from the unit's own history) comes from + SurPyval (`GeneralizedRenewal.unit_states()`, since 0.23); + RePyability takes the state. +- **Needs:** B1, B3. +- **Status:** partial, on `dev` for 0.14: the simulations start a Kijima + unit from its virtual age (`NodeState(age=, virtual_age=)`, #269), + which a fitted `GeneralizedRenewal`'s `unit_states()` gives, though the + exact methods refuse such a state; and for a part replaced rather than + repaired, `NonRepairable.optimal_next_replacement(age, + scheduled_outages=)` (#319) compares replacing now, at each scheduled + outage, at its best age and running to failure, undiscounted. The NPV, + refurbishment and salvage do not exist. + +--- + +## Crews, spares and data + +### MX-13: One crew, two crews, or a call-out contract? + +- **Persona:** a maintenance manager at a plant run with one crew. +- **Story:** As a maintenance manager, I want the value of a second crew + and of a call-out contract, so that I can choose between hiring, keeping + a contractor on retainer, or neither. +- **In their words:** "Is a second crew worth its salary? What if we only + call a contractor when two pumps are down at once?" +- **Setting:** a 2-of-3 pump station of exponential units with one repair + crew (as in #184 item 5); the contract crew is called when the repair + queue reaches 1 or 2, at a call-out fee and a premium hourly rate. +- **Acceptance criteria:** + - The NPV with 1 and 2 crews, net of crew cost; the NPV with the call-out + at each threshold, and the best threshold. + - The option's value: NPV with the call-out minus without, compared with + the cost of a permanent second crew. + - An exact independent check: the continuous-time Markov chain built in + the test with scipy, its discounted reward + `PV = e_0^T (delta I - Q)^(-1) r` from the all-up state, sharing no + code with `_crew_chain.py`. +- **Needs:** B1, B5. +- **Status:** partial: `expected_cost(t)` already runs over time with + `repair_crews` for exponential components, so the static crew comparison + can be discounted by hand, and since 0.12 the intervals can be chosen + with `assume_unlimited_crews=True` and checked by simulation with + `with_intervals(plan)`; the call-out option does not exist. + +### MX-14: How many seals should we hold, with a 12-week lead time? + +- **Persona:** a stores or reliability engineer setting stock levels. +- **Story:** As a stores engineer, I want the stock level that maximises + NPV, so that the capital tied up in spares is weighed against the + downtime a stock-out causes. +- **In their words:** "How many seals should we keep for the 20 stations, + given they take 12 weeks to arrive?" +- **Setting:** the spares guide's fleet of 20 with + `spares_stock(2016.0, fill_rate=0.95, fleet=20)`, with a holding cost + per unit per year and the downtime cost of waiting for a part. +- **Acceptance criteria:** + - The NPV for each stock level and the best one, alongside the stock a + fill-rate target gives. + - Pooled stock across interchangeable parts (#183) valued the same way. +- **Needs:** B1, B2 (holding cost as a rate cost); #183. +- **Status:** partial: stock for a fill rate or stock-out target exists, + and since 0.12 for pooled parts (`parts=`, #183), without costs. + +### MX-15: What is better data worth? + +- **Persona:** a reliability engineer deciding whether to gather more + failure data before committing to a plan. +- **Story:** As a reliability engineer, I want the expected value of + better information about a component's life, so that I can justify (or + not) tearing down more units or fitting a sensor. +- **In their words:** "Is it worth tearing down five more pumps to firm up + the Weibull before we set the PM interval?" +- **Setting:** the tutorial's 40 pumps, 19 still running, fitted in + SurPyval; the decision is the PM interval of MX-01's kind. +- **Acceptance criteria:** + - The expected NPV lost to parameter uncertainty (the expected value of + perfect information) for the PM-interval decision. + - The expected value of the sample information from n more teardowns. + - An independent check: sampling the fitted parameters from their + covariance by hand, re-optimising the interval for each draw, and + averaging. +- **Needs:** B1, B6, B7. +- **References:** #196 (uncertainty importance). +- **Status:** open. + +--- + +## Agents + +### MX-16: One call that answers "what is this plan worth?" + +- **Persona:** an agent (working through an application's tools or the + Python API) answering an engineer's question in a chat. +- **Story:** As an agent, I want one call that returns the NPV, the EAC, + the yearly cash-flow table and how it was computed, and a clear refusal + when it cannot, so that I do not hand-roll discounting and get it wrong. +- **In their words:** (the engineer) "What's this maintenance plan worth + over 15 years at 7%?" +- **Acceptance criteria:** + - The answer's shape above: NPV, EAC, undiscounted total, cash-flow + table, method. + - `analysis_routes()` reports the NPV analyses like every other: exact, + numerical, simulated or refused, with the method's own reason. + - Units stated in the result (time unit of the models, the rate's + period), so an agent cannot mix hours and years silently. +- **Needs:** B1; `analysis_routes` coverage. +- **Status:** open. + +### MX-17: The plan said one thing; the outage log says another + +- **Persona:** an agent or engineer reviewing a plan against what + happened. +- **Story:** As an engineer, I want to compare a plan's expected cash flows + with those implied by the real outage history, so that I can see where + the assumptions were wrong and re-value the plan from today. +- **In their words:** "We've had two years on this plan. How does what + actually happened compare with what we expected, and what's the plan + worth now?" +- **Setting:** a diagram with an outage log (`timelines`), the plan + valued at the start, and the components' current states. +- **Acceptance criteria:** + - Expected and actual cash flows per year so far, by category and + component. + - The plan re-valued from the current states (`state=`) for the rest of + the horizon. +- **Needs:** B1, B3; the timelines' measures. +- **Status:** open. + +--- + +## Live valuation: keeping the plan's value current + +A plan's value changes every time the plant does: a vibration reading +rises, a pump is replaced, a seal fails, a month of new failure data +narrows a fit, the forward curve moves. Today each of those means rebuilding +the analysis by hand. The aim is that RePyability holds the plant as it is +now and re-values the plan as each event arrives, so that an application +(or an agent) can keep a maintenance plan optimal by NPV through all the +churn of real maintenance work. + +The building blocks are B8 to B10, on top of B1 to B7. RePyability already +starts every exact analysis of a `RepairableRBD` from the components' current +states (`state=`, `NodeState`: age, up or down, how long down, the phase +of its schedule), and its simulations from an imperfectly repaired unit's +virtual age (#269); what is missing is the state a sensor gives (a +degradation level), a refitted model, the events that move the state, and +the valuation that follows it. + +### MX-18: A sensor reading moves the plan's value + +- **Persona:** a condition-monitoring engineer with monthly vibration + readings on the main bearings of a pump station. +- **Story:** As a condition-monitoring engineer, I want each new reading + to update the value of the current maintenance plan, so that a + deteriorating bearing shows up as money, not just as a trend line. +- **In their words:** "Bearing 2 just read 5.5 mm/s. What does that do to + the plan, and should we change it?" +- **Setting:** the gamma-process bearing of SurPyval's condition-monitoring + scenario card (healthy 1.0 mm/s, alarm 7 mm/s, increments shape 4.0 per + year and scale 0.5), as one component of the pump station of MX-06, with + a plan that replaces bearings every three years. +- **Acceptance criteria:** + - The plan's NPV before and after the reading, and the change attributed + to the bearing: its remaining-life distribution from 5.5 mm/s (from + SurPyval's degradation model) replaces its age-based life. + - The live update equals a full re-valuation from scratch with the same + state, to the exact methods' precision (incremental is not + approximate). + - The bearing's conditional life checked independently: the gamma + process's first passage from 5.5 to 7 mm/s by simulation. + - Fast enough to run on every reading: about a second for a plant of 50 + components with exact curves. +- **Needs:** B1, B8; SurPyval: a degradation model's remaining life from a + current level, as a life model RePyability can condition on. +- **Status:** partial, on `dev` for 0.14: a fitted gamma or Wiener process + is a component's life, and replacement on condition can go by its + measured level (#271); a reading as a component's state, and the + valuation, do not exist. + +### MX-19: A component is replaced, or repaired imperfectly + +- **Persona:** a maintenance planner closing work orders. +- **Story:** As a planner, I want a completed work order to reset the + component in the plant model, so that the plan's value and its next + actions follow what was actually done. +- **In their words:** "We replaced the seal on pump 3 yesterday, and + overhauled the gearbox (not as good as new). Update the plan." +- **Acceptance criteria:** + - A replacement sets the component new (age 0) at the work order's time; + an imperfect repair sets its virtual age (Kijima, with the restoration + factor fitted in SurPyval); the NPV and the component's next scheduled + action move accordingly. + - Events can arrive late and out of order (a work order closed a week + after the job) and give the same state as in order. + - Checked against re-valuing from scratch with the resulting states. +- **Needs:** B3, B8. +- **Status:** partial: the state after a replacement (`NodeState(age=0)`) + or, on `dev` for 0.14, after an imperfect repair + (`NodeState(virtual_age=)`, #269, simulations only) can be given; the + events, their order and the valuation do not exist. + +### MX-20: A failure happens + +- **Persona:** an operations manager on call. +- **Story:** As an operations manager, I want a failure to update the + plant's state, its expected downtime and the plan's value at once, so + that I see what the failure costs and what the plan now recommends. +- **In their words:** "Pump 2 tripped. What does it cost us, and does the + plan change?" +- **Acceptance criteria:** + - The failed component is down (`NodeState(alive=False)`) and queued for + a crew if crews are limited; spares are drawn from stock; the NPV + change is attributed to the failure. + - The plan's other actions are re-valued from the new state (the + standby now carries the duty, so its replacement may come forward). +- **Needs:** B5, B8, B9. +- **Status:** partial: the exact analyses already start from a component + that is down (`NodeState(alive=False, down_for=...)`); the valuation and + the spares draw-down do not exist. + +### MX-21: New failure data updates the fleet's life models + +- **Persona:** a reliability engineer who refits life models monthly. +- **Story:** As a reliability engineer, I want a refit (or a Bayesian + update) of a component type's life to update every plant that uses it, + with its uncertainty, so that the plan's value reflects what we now know. +- **In their words:** "We've had four more seal failures this quarter. Does + that change the seal replacement interval anywhere?" +- **Acceptance criteria:** + - Replacing a component type's model (with its covariance) re-values + every component of that type, keeping each one's own age or condition. + - The NPV distribution from parameter uncertainty (B7) narrows or moves + as the data say, checked against re-sampling the fit by hand. + - The value of the next piece of information is reported (MX-15). +- **Needs:** B7, B8; SurPyval: refitting with new data cheaply, or a + sequential update. +- **Status:** open. + +### MX-22: Re-optimise the plan after an event + +- **Persona:** a maintenance planner and the agent that assists them. +- **Story:** As a planner, I want the NPV-optimal plan re-computed from the + plant as it is now, so that I'm told when the best plan has changed, by + how much, and what to do differently. +- **In their words:** "Given everything that's happened this month, is + the plan still the best one?" +- **Acceptance criteria:** + - From the current state, the plan that maximises NPV over the horizon + within the constraints (crews, outage windows, a budget, availability + or PFDavg limits), warm-started from the current plan. + - Changes proposed only when they beat the current plan by a stated + margin (so the plan does not churn on noise), each with its value: + "bring pump 2's replacement forward to the March outage: +$45k". + - An independent check on a small plant: enumerate the candidate plans + and value each from scratch. +- **Needs:** B6, B8, B9. +- **Status:** open. + +### MX-23: The forward curve moves + +- **Persona:** a generator's asset manager working with the trading desk. +- **Story:** As an asset manager, I want a new forward price curve to + re-value the plan and re-time its outages, so that planned work follows + the market. +- **In their words:** "Winter prices just jumped. Should the overhaul move + to autumn?" +- **Acceptance criteria:** + - The NPV under the new curve, and the outage timing re-optimised + (MX-10's question from the state now). + - Re-valuing under a new curve reuses the plant's availability curves + (prices change the value, not the reliability), so it is cheap. +- **Needs:** B2, B8, B9. +- **Status:** open. + +### MX-24: What is the plan worth now, and what changed? + +- **Persona:** an agent, working through an application's tools, asked by + an engineer for a status report. +- **Story:** As an agent, I want the plant's current valuation and the + history of what moved it, in one call with a stable schema, so that I can + answer "what changed since last week?" correctly and explain it. +- **In their words:** (the engineer) "What's the plan worth now, why did + it drop, and what should we do next?" +- **Acceptance criteria:** + - The current NPV, EAC and cash-flow table; each event since a given + time with its effect on the NPV, the effects adding up to the total + change; the recommended next actions with their value (MX-22). + - The plant (structure, plan, models, states, prices, events) saves to + and loads from JSON, and a saved valuation reproduces exactly. + - Units in every result (the models' time unit, currency, the rate's + period). +- **Needs:** B8, B10. +- **Status:** open. + +### MX-25: Tell me when the best action changes + +- **Persona:** a reliability lead who doesn't want to watch a dashboard. +- **Story:** As a reliability lead, I want to be told when the plan's value + drops by more than a threshold, or when the recommended next action + changes, so that attention goes where the money is. +- **In their words:** "Email me if the best thing to do on any of my + stations changes." +- **Acceptance criteria:** + - RePyability reports, for each update, whether the NPV moved beyond a + threshold and whether the recommended actions changed (with the + reason); the application turns that into alerts. +- **Needs:** B8, B9, B10. +- **Status:** open (RePyability's part: the change report). +