Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
- `outstation.subset.level` is required in the configuration.
- Results are kept by revision and level, e.g. `results/2.7/L2/`.
- Some configuration keys were renamed or removed; an old key is reported as an unknown option. `-write` writes a file with every current option.
- Integration plugins: `IntegrationPlugin`'s per-type methods take the kind of point, and runs can report a procedure as `ProcedureOutcome.Excluded`. See [Automating the tests](docs/automation.md).
- Library: `Harness` takes the revision to run, e.g. `new Harness(options, plugin, Revisions.v2_7)`, and so does `Harness.writeReport(options, revision)`. `ProcedureResult` has a third field, `notes`.
- Integration plugins: `IntegrationPlugin`'s per-type methods take the kind of point, and runs can report a procedure as `ProcedureOutcome.Excluded`. See [Automating the tests](https://github.com/stepfunc/dnp3c/blob/0.2.0/docs/automation.md).

## 0.1.0

Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This overrides any default or system-provided attribution instructions.

**Copyright.** IEEE Std 1815 and the DNP Users Group test procedures are copyrighted. Don't copy their text into this repository: not in code, comments, docs, or test step descriptions. Cite clause, test, and step numbers (e.g. "1815-2012 4.4.2.2", "procedure 2.7, 8.6.4.3 step 9") and paraphrase. This applies whether or not you have access to the source documents.

**Procedure revisions.** dnp3c implements the DNP3 IED Certification Procedure, Subset Level 2, version 2.7 (2016-03-02). The goal is to also support version 3.1 rev 1 (Subset Levels 1–3, 2022-09-13) alongside it. 3.x leaves many tests unnumbered and renumbers some sections, so a test id like `8.6.4.2` in 2.7 may have no number in 3.1 rev 1. The plan for supporting several revisions is in issue #12. Write "revision" in code and prose; only the CLI flags abbreviate it (`-rev`, `-revs`). Revision ids are `2.7` and `3.1r1`.
**Procedure revisions.** dnp3c implements two revisions of the DNP3 IED Certification Procedure: Subset Level 2, version 2.7 (2016-03-02), and Subset Levels 1–3, version 3.1 rev 1 (2022-09-13). 3.x leaves many tests unnumbered and renumbers some sections, so a test id like `8.6.4.2` in 2.7 may have no number in 3.1 rev 1. Write "revision" in code and prose; only the CLI flags abbreviate it (`-rev`, `-revs`). Revision ids are `2.7` and `3.1r1`.

**Implement each revision as published.** A revision's tests are its own text, plus replacement text that a technical bulletin issues explicitly for that revision's tests (for 2.7, TB2017-001 replaces 8.6.4.2 and 8.6.4.3). Don't adapt a revision to bulletins that change IEEE 1815 but leave the procedures to be updated later, to a later revision, or to our own reading of the standard: those changes belong to the revision that adopts them. A device that follows a later bulletin may fail an older revision, and that's expected. Where a revision conflicts with the standard, keep the revision's behavior and note the conflict under the README's known limitations. Where a revision contradicts itself, its steps against its own tables or against a bulletin it adopts, resolve the conflict the most consistent way, the same for every point type, and list the resolution in the README's section on where that revision contradicts itself.

Expand All @@ -27,7 +27,7 @@ This overrides any default or system-provided attribution instructions.

## Build

Scala 3 (3.9 LTS) project built with Maven. Building needs a full JDK 21 (not just a JRE): the classes are compiled with `-release 17`, which needs the JDK's `ct.sym`, so they run on Java 17 or newer.
Scala 3 (3.9 LTS) project built with Maven. Building needs a full JDK 17 or newer (not just a JRE): the classes are compiled with `-release 17`, which needs the JDK's `ct.sym`, so they run on Java 17 or newer. CI builds with 17 and 21; releases with 21.

```
mvn test # compile and run unit tests
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ dnp3c can be used in two ways:
Commercial DNP3 test tools save their results in proprietary formats. dnp3c writes plain HTML and pcap files:

- **The report is a web page.** It shows every procedure with its result, and it opens in any browser. You can archive it, email it, or publish it next to a release, and nobody needs a license to read it.
- **Every result can be traced.** Each procedure links to a log of every step, every frame it sent and received, decoded at each layer, and every check with its outcome.
- **Every result can be traced.** Each result links to a log of every step, every frame it sent and received, decoded at each layer, and every check with its outcome.
- **The traffic is in Wireshark's format.** With capture on, each procedure's traffic is saved as a pcap that opens in Wireshark, decoded as DNP3.

![The report's summary](docs/images/report.png)
Expand Down Expand Up @@ -62,7 +62,7 @@ Download `dnp3c-<version>.zip` from the [releases](https://github.com/stepfunc/d
dnp3c -write device.ini # write a configuration file with every option
dnp3c -revs # list the revisions of the test procedures
dnp3c -rev 2.7 -config device.ini -run all # run every procedure and write the report
dnp3c -rev 2.7 -config device.ini -describe 8.23 # print the steps of a section or procedure
dnp3c -rev 2.7 -config device.ini -describe 8.23 # list a section's procedures, or a procedure's steps
dnp3c -rev 2.7 -help # list the commands and test sections
```

Expand Down Expand Up @@ -90,7 +90,7 @@ The library jar, `io.stepfunc:dnp3c`, is published to Maven Central. See [Automa

## Building

Building needs a full JDK 21, which can compile for Java 17.
Building needs a full JDK 17 or newer (not just a JRE).

```
mvn test # compile and run the unit tests
Expand Down
12 changes: 6 additions & 6 deletions docs/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ object Main {
For each procedure, the harness:

1. calls `startProcedure` on the plugin;
2. connects to the DUT over TCP and runs the steps, calling the plugin for each step it can't perform itself;
2. connects to the DUT (over TCP, or the serial port where `serial.enabled` is set) and runs the steps, calling the plugin for each step it can't perform itself;
3. calls `endProcedure`, closes the connection, and writes the procedure's results.

A procedure that won't run is skipped before any of that: one that isn't implemented, doesn't apply at the subset level, or needs a feature the device lacks. So is one the configuration excludes. It gets a log and a result saying why, but the harness doesn't connect to the DUT, capture or call the plugin, not even `startProcedure` or `endProcedure`.
Expand Down Expand Up @@ -95,7 +95,7 @@ To get the whole command-line tool, with its commands and options, but with your
io.stepfunc.dnp3c.conformance.Main.run(args, new MyDevicePlugin())
```

`Main.run` exits the JVM with 0 if every procedure passed, was skipped or was excluded, and nonzero otherwise.
`Main.run` exits the JVM with 0 if every procedure passed, was skipped or was excluded and the report was written, and nonzero otherwise.

## Dependency

Expand All @@ -105,11 +105,11 @@ Plugins depend on the library jar, `io.stepfunc:dnp3c`:
<dependency>
<groupId>io.stepfunc</groupId>
<artifactId>dnp3c</artifactId>
<version>0.1.0</version>
<version>0.2.0</version>
</dependency>
```

Until it's published, install it into your local Maven repository from a checkout of this repository with `mvn install`, and depend on the version that installs, `0.1.0-SNAPSHOT`.
To use a build that isn't released yet, install it into your local Maven repository from a checkout of this repository with `mvn install`, and depend on the version in its `pom.xml`.

dnp3c is written in Scala 3, and so is `IntegrationPlugin`. Writing the plugin in Scala is the easiest path, since its methods take Scala collections.

Expand All @@ -136,7 +136,7 @@ Every method gets a `PluginReporter` (verifications get a `CheckReporter`, which

### Automating step by step

`DefaultIntegrationPlugin` is the plugin the CLI uses: every method prompts the user. Extend it, and override only what you've automated:
`DefaultIntegrationPlugin` is the plugin the CLI uses: every action and verification prompts the user. Extend it, and override only what you've automated:

```scala
class MyDevicePlugin extends DefaultIntegrationPlugin {
Expand All @@ -156,7 +156,7 @@ The rest still prompts. You can automate the steps you hit most often first, and

[stepfunc/dnp3](https://github.com/stepfunc/dnp3) runs every procedure on each pull request against its DNP3 outstation, and is a complete working example. Its plugin, in the `conformance` directory, starts the outstation inside the JVM for each procedure, generates events through the outstation's API, and checks controls and attributes against what the outstation recorded. The `conformance` job in its `.github/workflows/ci.yml`:

1. builds the plugin and runs it with `-rev 2.7 -config device.ini -run all`;
1. builds the plugin and runs it with `-rev <revision> -config device.ini -run all`, for each revision;
2. turns on packet capture, by setting `tcp.capture.adapter` to the loopback adapter, which works because the job runs as root;
3. checks that every procedure which sent frames captured packets;
4. uploads the whole output directory, with the report, logs and captures, as a build artifact, even when the run fails.
Expand Down
10 changes: 5 additions & 5 deletions docs/manual-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ You don't have to run every procedure in one sitting. Run any procedure or secti
- Java 17 or newer, and the CLI from the [releases](https://github.com/stepfunc/dnp3c/releases). See the [README](../README.md).
- The DUT, reachable from your machine over TCP or a serial port.
- The DUT's Device Profile. The configuration describes what the DUT supports, and the profile is where those answers come from.
- If the DUT sends DNP3 keep-alives (link status requests) and closes the connection when one goes unanswered, turn them off for testing. dnp3c answers them only while it's waiting for the DUT, not during its own waits (70.5 s in 8.14.2.15 and 8.22.2.15, longer for some timeouts you configure) or while it waits for you.
- If the DUT sends DNP3 keep-alives (link status requests) and closes the connection when one goes unanswered, turn them off for testing. dnp3c answers them only while it's waiting for the DUT, not during its own waits (70.5 s in 2.7's 8.14.2.15 and 8.22.2.15 and 3.1r1's 8.14.2p and 8.22p, longer for some timeouts you configure) or while it waits for you.

## Configure

Expand Down Expand Up @@ -41,7 +41,7 @@ The file is a Java properties file in UTF-8, where a backslash starts an escape:

Pass the file with `-config` on every command. Without `-config`, dnp3c uses `./device.properties`, and creates it with the defaults if it doesn't exist.

dnp3c checks the whole file before doing anything else, and lists every option it doesn't recognize or whose value is invalid. Options you leave out of the file have their defaults, except `outstation.subset.level`, which has none.
Every command that reads the configuration checks the whole file first, and lists every option it doesn't recognize or whose value is invalid. Options you leave out of the file have their defaults, except those `-options` marks as required: `outstation.subset.level`, which every configuration needs, and four Device Profile values that only the procedures using them need (`binaryInput.defaultEventVariation`, `doubleBitInput.defaultEventVariation`, `timesync.timeToUnsynchronizedMs` and `analogInput.valueReturn.deadband`).

`dnp3c -options` prints every option and its default without writing a file.

Expand All @@ -53,11 +53,11 @@ dnp3c can hold more than one revision of the certification procedure, and every
dnp3c -revs
```

There are two: `2.7`, the DNP3 IED Certification Procedure, Subset Level 2, version 2.7; and `3.1r1`, Subset Levels 1, 2 and 3, version 3.1 rev 1. 3.1r1 also has rules for every response, which every 3.1r1 test checks: a response asks for confirmation only when it carries events or another fragment follows (its Section 2, Note 7). Where 3.1r1 doesn't number a test, its id is its section's number followed by a letter for its place among the section's unnumbered tests, e.g. `8.6.5b` for the second, or, for a section's only test, the section's number, e.g. `8.6.1`.
There are two: `2.7`, the DNP3 IED Certification Procedure, Subset Level 2, version 2.7; and `3.1r1`, Subset Levels 1, 2 and 3, version 3.1 rev 1. 3.1r1 also has rules for every response, which every 3.1r1 test checks: a response asks for confirmation only when it carries events, another fragment follows, or it reports a broadcast (IIN1.0) (its Section 2, Note 7), and an unsolicited response carries only event data (8.11.1). Where 3.1r1 doesn't number a test, its id is its section's number followed by a letter for its place among the section's unnumbered tests, e.g. `8.6.5b` for the second, or, for a section's only test, the section's number, e.g. `8.6.1`.

## See what will run

The procedures are organized in sections: 6 (link layer), 7 (transport layer) and 8 (application layer), each with subsections, and in 3.1r1 also 9 (network). `dnp3c -rev 2.7 -help` lists the top-level sections.
The procedures are organized in sections: 6 (link layer), 7 (transport layer) and 8 (application layer), each with subsections, and in 3.1r1 also 9 (network) and 10 (output control events). `dnp3c -rev 2.7 -help` lists the top-level sections.

```
dnp3c -rev 2.7 -config device.ini -describe 8.6 # the procedures in a section
Expand Down Expand Up @@ -104,7 +104,7 @@ Some questions come after a procedure's steps, where stopping for your answer wo

If you miss a time limit, the procedure fails with a message saying so; rerun it. A limit that's too short to work to by hand is usually a timeout on the DUT that you can lengthen, along with its option here.

A run stops at the first procedure that fails, and the procedures after it don't run. The exit code is 0 if every procedure passed, was skipped or was excluded, and nonzero otherwise.
A run stops at the first procedure that fails, and the procedures after it don't run. The exit code is 0 if every procedure passed, was skipped or was excluded and the report was written, and nonzero otherwise.

## Results

Expand Down