From 19bc092aaeff0e960bbbf2782c6116109928829d Mon Sep 17 00:00:00 2001 From: WilliamK112 <164879897+WilliamK112@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:34:50 -0500 Subject: [PATCH 1/2] docs: clarify strict fallback errors and native recovery Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com> --- docs/fallback-matrix.md | 65 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/docs/fallback-matrix.md b/docs/fallback-matrix.md index ccd475a..077e330 100644 --- a/docs/fallback-matrix.md +++ b/docs/fallback-matrix.md @@ -72,6 +72,71 @@ fd.clean(df, engine="polars", fallback_policy="warn") # FallbackWarning - CLI: `freshdata clean data.parquet --engine duckdb --fallback-policy error`. - Preview without running anything: `fd.plan(df, engine="duckdb").fallback_reason`. +### Catching a strict fallback + +The following example assumes the Polars extra is installed +(`pip install "freshdata-cleaner[polars]"`). The default balanced strategy needs +the pandas decision engine, so strict policy raises before that pipeline runs: + +```python +import polars as pl +import freshdata as fd +from freshdata.execution import PlanGenerator + +frame = pl.DataFrame({"value": [1, 2]}) +backend = "polars" + +try: + fd.clean(frame, engine=backend, fallback_policy="error", verbose=False) +except fd.FallbackError as err: + diagnostic = { + "error_type": type(err).__name__, + "requested_backend": backend, + "message": str(err), + "args": err.args, + } + print(diagnostic["message"]) +``` + +The exact message for this configuration is: + +```text +freshdata polars backend fell back to pandas at step 'pipeline': strategy='balanced' runs the accuracy-first decision engine, which is evaluated by the pandas backend. strategy="conservative" with default options is the fully native path; see docs/fallback-matrix.md for what each backend runs natively. +``` + +`FallbackError` is a `RuntimeError` with one diagnostic message in `err.args`; +`err.args[0]` gives the same text as `str(err)`. It currently has no dedicated +`backend`, `trigger`, or error-code attributes. Catch the exception type to +classify a refused fallback, and retain your requested backend/configuration +alongside the message. Treat the text as a diagnostic, not a stable error code. + +For this compatible input, select the native representation-repair path and +disable heuristic dtype inference. A lazy output also avoids materializing the +cleaned result: + +```python +lazy, report = fd.clean( + frame, + engine=backend, + fallback_policy="error", + strategy="conservative", + fix_dtypes=False, + output_format="polars-lazy", + return_report=True, + verbose=False, +) +print(report.backend, report.materialized) # polars False +``` + +Other options or input shapes can still require fallback; use the matrix above +to address the stated reason. +`PlanGenerator(fd.CleanConfig(), backend=backend).fallback_reason()` previews +configuration-driven reasons without accessing the input; input-driven checks +can still reject a run during ingestion. Keep the exception handler for those +cases. If the pandas decision engine is what you need, explicitly materialize +this native source with `fd.clean(frame.to_pandas(), engine="pandas")` and omit +the native-only `fallback_policy` argument. + ## What the report tells you afterwards Every engine run stamps: From 46e122139121dd3c396958f9ba6bf0c8860430e9 Mon Sep 17 00:00:00 2001 From: WilliamK112 <164879897+WilliamK112@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:07:45 -0500 Subject: [PATCH 2/2] docs: show the complete strict fallback diagnostic Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com> --- docs/fallback-matrix.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/fallback-matrix.md b/docs/fallback-matrix.md index 077e330..1030127 100644 --- a/docs/fallback-matrix.md +++ b/docs/fallback-matrix.md @@ -79,6 +79,8 @@ The following example assumes the Polars extra is installed the pandas decision engine, so strict policy raises before that pipeline runs: ```python +from pprint import pprint + import polars as pl import freshdata as fd from freshdata.execution import PlanGenerator @@ -95,7 +97,7 @@ except fd.FallbackError as err: "message": str(err), "args": err.args, } - print(diagnostic["message"]) + pprint(diagnostic, sort_dicts=False) ``` The exact message for this configuration is: