fix: make the sphinx-gallery scraper capture every shown figure

[larsoner]

Link to issue

Closes #4722
Closes #4959
Closes sphinx-gallery/sphinx-gallery#1238
Closes sphinx-gallery/sphinx-gallery#1632

Description of change

Fix sphinx-gallery thumbnail generation!

Demo

Testing strategy

Added some unit tests to make sure it works. It uses SG (added to dev_optional so CIs install it) to make sure the contract holds.

Additional information (optional)

I am a SG maintainer. I used Claude Opus 5 to draft the changes here but I iterated with it a lot and have reviewed the final diff.

Guidelines

• I have reviewed the pull request guidelines and the Code of Conduct and confirm that this PR follows them.
• I have added an entry to the changelog if needed (not required for documentation PRs).

[sr-murthy]

Tried it (replaced plotly dep in project TOML with a ref to your fork branch of Plotly), cleared out the docs build folder, and ran Sphinx build again (make -C docs html), but still get the same placeholder thumbs (in the docs auto_examples/images/thumb folder - the full size figures are correctly drawn in the HTMLs.

I can see that the Plotly version in my env. is your branch because the UV log has the following line:

...
DEBUG Requirement already installed: plotly==6.9.0 (from git+https://github.com/larsoner/plotly.py@5ef36f39d12315835d211c39ae5ff2357c06a70b)
...

Conf changes for Plotly PNG rendering are as described here. Folder structure for the examples folder containing the Python files, is also as recommended, with auto_examples adjacent to examples. Here is the gallery conf. in docs/conf.py:

# Sphinx gallery conf.
sphinx_gallery_conf = {
    "examples_dirs": "sources/plot-gallery/examples",  # `docs`-relative path to Python scripts
    "gallery_dirs": "sources/plot-gallery/auto_examples",  # `docs`-relative path to gallery outputs
    "image_scrapers": ("matplotlib", "plotly.io._sg_scraper.plotly_sg_scraper",),
}

Wouldn't it be simpler for Sphinx gallery to support a post-build hook (a custom function maybe) that created the thumbnails by resizing and exporting the full sized figures into PNGs in the appropriate location?

[larsoner]

Wouldn't it be simpler for Sphinx gallery to support a post-build hook (a custom function maybe) that created the thumbnails by resizing the full sized PNGs into the appropriate location?

I don't think we should rework / discuss the SG API contracts here -- it'll be a much bigger discussion if needed, and require an understanding of design decisions and code evolution over the years where we settled on the existing design.

Instead I'd like to see if we can get things working properly for you using this branch. Can you make clean and try again with the latest version of the PR? I pushed some fixe -- the previous code required some SG changes and fig.show() to be used etc. but this version should work, even with the simpler:

    "image_scrapers": ("matplotlib", "plotly"),

I confirmed this works in sphinx-gallery/sphinx-gallery#1635 (see the updated example render) so hopefully it works for you now!

[sr-murthy]

Thx, I'll try it out, and let you know.

[sr-murthy]

@larsoner The changes seem to work (Plotly pointing to rev. e3b95636) for my plot gallery, so thank you. But please note that thumbnail generation will only work if the build environment has both kaleido and a Chromium-based browser installed, and this needs to be made clear in the documentation (more details in point 4 below).

I've done several local and remote (RTD) builds to check consistency and reproducibility. There are a few points you may wish to note:

• The build does seem to have been slowed down significantly - I think the initial build (after clearing the build folder) took almost a minute, but subsequent builds now are averaging about 30 seconds. Usually these builds, at least for my project, take less than 10 seconds.
• My guess is that the build times would go up in line with the number of plots in the gallery [?]
• I see that Chrome is used - I see two separate instantiations of it - to enable the kaleido exports for generating thumbnails. This would obviously fail if either the user or build environment did not have Chrome (or a Chromium-based browser) or did not have kaleido installed: in this case you get the following warning:

WARNING: plotly static image export is unavailable, so example thumbnails will fall back to a placeholder image. Static export requires Kaleido and a Chromium-based browser; see https://plotly.com/python/static-image-export/ for installation instructions. The failure was: ValueError: 
Image export using the "kaleido" engine requires the Kaleido package,
which can be installed using pip:

and the result is placeholder thumbnails, but the correct full sized figures are still drawn in the HTML pages. It's probably a good idea to note this somewhere in the Sphinx gallery docs. RTD builds had placeholder thumbnails because although kaleido is part of dependencies and is installed, the build environment initially did not have a Chromium-based browser installed - this required a pre-install step to be added to the RTD YML:

...
  jobs:
    pre_install:
      - uv run plotly_get_chrome -y

Once this is done, RTD builds are fine.

P. S. I had an earlier version of the gallery working, that used statically generated images, and with some scripting it would have come very close to what I have now. But Sphinx gallery is a good solution, so it's fine.

[sr-murthy]

BTW in the gallery conf. it is enough to have:

    {
        ...
        "image_scrapers": ("plotly",),
    }

You don't need matplotlib if you're only using Plotly.

[larsoner]

I see that Chrome is used

Yeah we need some way to get the thumbnail image from the rendered HTML page. Likely also the slowdown cause but I'll check.

It's probably a good idea to note this somewhere in the Sphinx gallery docs

Yes you're right!

[sr-murthy]

I've removed the point about opaque thumbnails on-hover - I'm guessing this could be fixed with the theme CSS [?], but anyway it's not an essential.

[larsoner]

Okay I pushed a change to reuse a single kaleido server instead of spinning one up for every figure. Can you see if that's faster?

[sr-murthy]

That appears to be quicker (no duplicate Chrome instances) - about 16 seconds total. That's fine.

[camdecoster]

Thanks for the PR! Before we review, could you please provide some testing steps to show that these changes fix the underlying issues?

[sr-murthy]

Thanks for the PR! Before we review, could you please provide some testing steps to show that these changes fix the underlying issues?

Is that me? My testing procedure was:

1. Re-install Plotly in the local env. pointing to this PR branch (rev. 6894c4b).
2. Clear out the Sphinx build folder(docs/_build), and build again (make -C docs html).
3. Check the plot gallery page (thumbnails, and figures in the figure pages).

I did about 10 builds locally, and several on RTD also. All fine.

[larsoner]

Quickest way to get something you can look at would maybe be to run sphinx gallery's own doc build on the branch from this PR : sphinx-gallery/sphinx-gallery#1635 . But that's also exactly what CircleCI does over there so you can view its result here. That plus @sr-murthy testing is maybe enough already?

@sr-murthy do you have a branch you could open in whatever repo you've been working on that could show it working as well?

If it helps for review, I could alternatively put together a tiny reproduction sphinx setup and upload that somewhere.

[sr-murthy]

@larsoner Yes, in fact it's this one.

ISARICResearch/IsaricAnalytics#18

At the bottom of that PR is a link to the complete RTD build site of that PR, and the plot gallery page on this site I'm posting below:

https://isaricanalytics--18.org.readthedocs.build/en/18/sources/visualisation/index.html

The problem prior to the fix was that the thumbnails were placeholder images as displayed here. The figures inside the figure pages were never an issue, so they render properly as before.