Add detailed traces to the output of train!

[ablaom]

This PR is a major breaking update implementing the proposal at #64.

It additionally closes: #47, #62 and replaces #66, #68, #70, #71.

Detailed document strings have been added to all public methods and New Documenter.jl documentation replaces the README.md docs. The demos in /examples are now integrated into the new documentation.

Draft release notes below tag breaking changes. The main change is to the output of train! which will be a vector of new ControlTrace objects, which, by default, record a complete history of the internal state of each control with intuitive API. Here's a demonstration:

using IterationControl
import IterationControl.SquareRooter  # demonstration "model" type
using DataFrames

model = SquareRooter(9)

julia> traces = IterationControl.train!(
    model,
    Step(),
    Threshold(0.01),
    WithLossDo(x->x<0.01),
)
[ Info: final loss: 0.0005493331704400362
[ Info: final training loss: 0.02343785763332562
[ Info: Stop triggered by Threshold(0.01) stopping criterion.
3-element Vector{Any}:
 ControlTrace{<:Step}, ending (new_iterations = 4, new_steps = 4)
 ControlTrace{<:Threshold}, ending (loss = 0.0005493331704400362, excess = -0.009450666829559964)
 ControlTrace{<:WithLossDo}, ending (loss = 0.0005493331704400362, called_back = true)

julia> trace = traces[end]
ControlTrace
  control: WithLossDo{var"#2#3"}(var"#2#3"(), false, nothing)
  loss: [ …, 2.5599999999999987, 0.1417301038062284, 0.0005493331704400362]
  called_back: Bool[ …, 0, 0, 1]
  table: (loss = …, called_back = …)
  triggered_stop: false

julia> trace.loss
4-element Vector{Float64}:
 16.0
  2.5599999999999987
  0.1417301038062284
  0.0005493331704400362

julia> DataFrame(trace)
4×2 DataFrame
 Row │ loss          called_back
     │ Float64       Bool
─────┼───────────────────────────
   1 │ 16.0                false
   2 │  2.56               false
   3 │  0.14173            false
   4 │  0.000549333         true

Requirements

• Merge of Add user interface point for accessing the internal state of a stopping criterion EarlyStopping.jl#37 and new EarlyStopping.jl 1.0 release

Draft release notes

• (enhancement - breaking) The form of the return value of
  IterationControl.train!(model, ...) has changed. Now a vector of ControlTrace
  objects is returned. By default, each trace records a complete history of (an public
  form) of the control's internal state. For example, the trace of any callback control
  records a history of the values returned by the calling function.

• (new default - breaking) Step() is now the same as Step(1)

• (new default - breaking) Callback() is now the same as Callback(identity)

• (new default - breaking) WithNumberDo() is now the same as WithNumberDo(identity)

• (new default - breaking) WithLossDo() is now the same as WithLossDo(identity)

• (new default - breaking) WithTraininLossesDo() is now the same as WithTrainingLossesDo(identity)

• (API change - breaking for custom controls) (for new implementations)
  takedown(control, verbosity, state) should now only return a short (possible empty)
  message string, which it should log to :info if it's non-empty and verbosity > 0. Ideally the message should fit on one line. (Detailed information can be accessed
  with EarlyStopping.public_state(control, state).) A new control trait
  needs_ingest(model) should be implemented for custom controls that require
  IterationControl.ingest!(model, datum) to be implemented to support model.

• (logging changes) What train! logs to Info (apart from control-specific logging)
  has changed; logging is less verbose by default.

• Support for julia < 1.10 is dropped.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB