Plot functions should optionally return the data they are based on

[hmgaudecker]

Summary

Plot functions should be able to return the data they are based on, not only the
go.Figure. correlation_heatmap.py already works this way; the other visualisation
modules do not, and there is no public way to get at the numbers behind a figure.

Motivation

Two independent reasons, one general and one specific.

General. A figure is a presentation choice frozen at render time. When the data
behind it is available, the figure can be re-rendered later with a different layout,
template, aspect ratio or language, and the same numbers can go into a table. When it is
not, the only way to change anything is to re-run the estimation.

Specific. Projects running inside a secure environment often cannot export figures
at all, or can export them only under conditions that make it not worth it. Statistics
Netherlands, for example, accepts tables in xlsx/csv/xml and requires every data
point in a figure to rest on at least ten underlying observations, with the per-bin
counts supplied separately. Exporting the underlying table and re-rendering outside is
both simpler and more useful. That is only possible if the library will hand over the
data.

Precedent, already in the library

skillmodels/common/correlation_heatmap.py separates the two concerns cleanly:

def get_measurements_corr(...) -> pd.DataFrame: ...
def get_quasi_scores_corr(...) -> pd.DataFrame: ...
def get_scores_corr(...) -> pd.DataFrame: ...
def plot_correlation_heatmap(...): ...

The data accessors are public and documented, and the plot function sits on top of them.
This issue asks for the same shape elsewhere.

Some other public functions already return data and need nothing:

• variance_decomposition.decompose_measurement_variance -> pd.DataFrame
• variance_decomposition.summarize_measurement_reliability -> pd.DataFrame
• simulate_data.simulate_policy_effect -> pd.DataFrame

Gap

These return figures only:

Function	Current return
visualize_transition_equations.get_transition_plots	go.Figure
visualize_transition_equations.combine_transition_plots	go.Figure
visualize_factor_distributions.univariate_densities	dict[str, go.Figure]
visualize_factor_distributions.bivariate_density_contours	dict[tuple[str, str], go.Figure]
visualize_factor_distributions.bivariate_density_surfaces	dict[tuple[str, str], go.Figure]
visualize_factor_distributions.combine_distribution_plots	go.Figure
diagnostic_plots.plot_residual_boxplots	go.Figure | dict[int, go.Figure]
diagnostic_plots.plot_likelihood_contributions	go.Figure | dict[int, go.Figure]

In every case the data preparation already exists as private helpers, so this is mostly
a matter of promoting and documenting what is there rather than writing new logic:

• visualize_transition_equations: _prepare_plot_data_for_factor_pair,
  _prepare_single_period_plot_data, _get_state_ranges,
  _prepare_data_for_one_plot_fixed_quantile_2d,
  _prepare_data_for_one_plot_average_2d
• visualize_factor_distributions: _process_data, _get_one_state_per_period,
  _calculate_kde_for_3d

Proposal

Either of these would solve it; the first matches the existing precedent more closely.

1. Public data accessors. Add get_transition_plot_data(...) -> pd.DataFrame,
   get_factor_distribution_data(...) -> pd.DataFrame, and so on, and have the plot
   functions call them. Mirrors correlation_heatmap.py exactly.
2. A return_data flag on the plot functions, returning (figure, data) when set.
   Fewer new names, but it makes the return type conditional on an argument, which is
   harder to type and to discover.

Tidy long-form DataFrames would be the most useful shape — one row per plotted point,
with the facet/period/factor identifiers as columns, so several figures' data can be
concatenated or written to one workbook.

Note on residuals

plot_residual_boxplots and plot_likelihood_contributions are listed above for
completeness, but they are a special case: residuals are individual-level data and
several secure environments will not release them in any form. Data accessors for those
two are useful for in-environment work, not for export.

Workaround in the meantime

The arrays can be recovered from the figure object (fig.data[i].x, .y, .z), which
works but depends on trace layout, so a change in how a figure is assembled silently
changes the recovered table. That is the reason for opening this rather than keeping the
workaround.