.. _advanced_usage:

==============
Advanced usage
==============

This page contains more advanced topics in case you want to understand how
to use Sphinx-Gallery more deeply.

.. contents:: **Contents**
    :local:
    :depth: 2

Extend your Makefile for Sphinx-Gallery
=======================================

This section describes some common extensions to the documentation Makefile
that are useful for Sphinx-Gallery.

Cleaning the gallery files
--------------------------

Once your gallery is working you might need completely remove all generated
files by Sphinx-Gallery to have a clean build. For this we recommend adding the
following to your Sphinx ``Makefile``:

.. code-block:: bash

    clean:
            rm -rf $(BUILDDIR)/*
            rm -rf auto_examples/

You need to adapt the second ``rm`` command if you have changed the
``gallery_dirs`` config variable.

Build the gallery without running any examples
----------------------------------------------

If you wish to build your gallery without running examples first (e.g., if an
example takes a long time to run), add the following to your ``Makefile``.

.. code-block:: bash

    html-noplot:
            $(SPHINXBUILD) -D plot_gallery=0 -b html $(ALLSPHINXOPTS) $(SOURCEDIR) $(BUILDDIR)/html
            @echo
            @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

Know your Gallery files
=======================

The Gallery has been built, now you and all of your project's users
can already start enjoying it. All the temporary files needed to
generate the gallery (rst files, images, cache objects, etc) are
stored where you configured in ``gallery_dirs``. The final files that go
into the HTML version of your documentation have a particular
namespace, to avoid collisions with your own files and images.

Our namespace convention is to prefix everything with ``sphx_glr`` and
change path separators with underscores. For example the first image
generated by the example 'plot_0_sin.py' has the name
``sphx_glr_plot_0_sin_001.png`` and its thumbnail is
``sphx_glr_plot_0_sin_thumb.png``

You can also include part of a gallery script elsewhere in your documentation
using the :rst:dir:`literalinclude` directive, in order to limit code
duplication:


.. code-block:: rst

   .. literalinclude:: ../examples/plot_0_sin.py
      :language: python
      :start-after: # License: BSD 3 clause
      :end-before: # To avoid matplotlib

The above directive inserts the following block:

.. literalinclude:: ../examples/plot_0_sin.py
    :language: python
    :start-after: # License: BSD 3 clause
    :end-before: # To avoid matplotlib

.. warning::

   Using literalinclude is fragile and can break easily when examples are
   changed (all the more when line numbers are used instead of ``start-after``
   and  ``end-before``). Use with caution: linking directly to examples is
   a more robust alternative.

Cross referencing
-----------------
You can also cross reference an example using similar naming convention. For
example if we want to reference the example
:ref:`sphx_glr_auto_examples_plot_0_sin.py`, we just call its reference
``:ref:`sphx_glr_auto_examples_plot_0_sin.py```.
Note that we have included the path to the example file (relative to
the ``conf.py`` file) after ``sphx_glr_``. Path separators are replaced with
underscores.

.. _warning_errors:

Understanding warning and error outputs
=======================================

Any warnings or errors that occur when executing code blocks in the gallery
Python files will be printed in pink during building of the documentation. The
``.py`` file path and the line number that the error occurred in will also be
printed. For example, the example
:ref:`sphx_glr_auto_examples_no_output_plot_raise.py` will raise the following
error::

    File "<full_path>/examples/no_output/plot_raise.py", line 27, in <module>
        iae
    NameError: name 'iae' is not defined

Problems in the text (reST) blocks of the gallery Python files will result
in warnings or errors when Sphinx is converting the generated ``.rst`` files
to HTML. These will be printed by Sphinx in pink, after code block errors,
during building of the documentation. In this case, the ``.rst`` file path and
``.rst`` file line number will be printed. To fix the problem, you will need
to amend the original ``.py`` file, **not** the generated ``.rst`` file.
To figure out where the problem is, you will need to match the content of the
``.rst`` file at the line number printed to the original ``.py`` file.

Example ``.rst`` warning::

    <full_path>/auto_examples/plot_example.rst:19: WARNING: Explicit markup ends without a blank line; unexpected unindent.

The warning above occurred due to line 19 in ``plot_example.rst``. The
original ``plot_example.py`` file will need to be amended to fix it.
Sphinx-Gallery only (re)builds new, modified or failed examples, so
re-running the documentation build should rebuild just the modified example,
allowing for quick iteration.

.. _custom_scraper:

Write a custom image scraper
============================

.. warning:: The API for custom scrapers is currently experimental.

By default, Sphinx-Gallery supports image scraping for Matplotlib
(:func:`~sphinx_gallery.scrapers.matplotlib_scraper`). If you wish to capture
output from other python packages, first determine if the object you wish to
capture has a ``_repr_html_`` method. If so, you can use the configuration
``capture_repr`` (:ref:`capture_repr`) to control the display of the object,
without the need to write a custom scraper. This configuration allows capture
of the raw html output, in a process similar to other html-based displays such
as `jupyter <https://jupyter.org/>`_. If the first option does not work,
this section describes how to write a custom scraper.

Image scrapers are functions (or callable class instances) that do the following
things:

1. Collect a list of images created in the latest execution of code.
2. Write these images to disk in PNG, JPEG, SVG, GIF, or WebP format (with .png,
   .jpg, .svg, .gif, or .webp extensions, respectively)
3. Return reST that embeds these figures in the built documentation.

The function should take the following inputs (in this order):

1. ``block`` - a Sphinx-Gallery ``.py`` file is separated into consecutive
   lines of 'code' and reST 'text', called 'blocks'. For each
   block, a `namedtuple` with ("type", "content", "lineno") items
   (e.g. ``('code', 'print("Hello world")', 5)``) is created.

   * 'label' is a string that can either be ``'text'`` or ``'code'``. In this
     context, it should only be ``'code'`` as this function is only called for
     code blocks.
   * 'content' is a string containing the actual content of the code block.
   * 'lineno' is an integer, indicating the line number that the block
     starts at.

2. ``block_vars`` - dictionary of configuration and runtime variables. Of
   interest for image scrapers is the element ``'image_path_iterator'`` which
   is an iterable that returns an absolute path to an image file name
   adhering to Sphinx-Gallery naming convention. The path directs to the
   ``gallery_dirs/images`` directory (:ref:`configure_and_use_sphinx_gallery`)
   and the image file name is ``'sphx_glr_'`` followed by the name of the
   source ``.py`` file then a number, which starts at 1 and increases by 1 at
   each iteration. The default file format is ``.'png'``. For example:
   ``'home/user/Documents/module/auto_examples/images/sphx_glr_plot_mymodule_001.png'``.
   If a different image extension is desired, the scraper is responsible for
   replacing the default `.png` extension. Only supported image extensions
   (see above) will enable the file to be picked up by Sphinx-Gallery.

3. ``gallery_conf`` - dictionary containing the configuration of Sphinx-Gallery,
   set under ``sphinx_gallery_conf`` in ``doc/conf.py`` (:ref:`configuration`).
   Of note, the :ref:`image_srcset <image_srcset>` configuration will provide
   user specified image resolutions (as floats) and can be used by your custom scraper
   to enable multi-resolution images.

It should return a string containing the reST for embedding this figure in the
documentation. See :func:`~sphinx_gallery.scrapers.matplotlib_scraper` for an
example of a scraper function (click on 'source' below the function name to see
the source code). The :func:`~sphinx_gallery.scrapers.matplotlib_scraper` uses
the helper function :func:`sphinx_gallery.scrapers.figure_rst` to help generate
reST (see below).

This function will be called once for each code block of your examples.
Sphinx-Gallery will take care of scaling images for the gallery
index page thumbnails. PNG, JPEG and WebP images are scaled using Pillow, and
SVG and GIF images are copied.

.. warning:: SVG images do not work with ``latex`` build modes, thus will not
             work while building a PDF version of your documentation. You may
             want to consider `sphinxcontrib-svg2pdfconverter
             <https://pypi.org/project/sphinxcontrib-svg2pdfconverter/>`_.

Example 1: a Matplotlib-style scraper
-------------------------------------

For example, we will show sample code for a scraper for a hypothetical package.
It uses an approach similar to what
:func:`sphinx_gallery.scrapers.matplotlib_scraper` does under the hood, which
use the helper function :func:`sphinx_gallery.scrapers.figure_rst` to
create the standardized reST. If your package will be used to write an image
file to disk (e.g., PNG or JPEG), we recommend you use a similar approach::

    def my_module_scraper(block, block_vars, gallery_conf):
        import mymodule
        # We use a list to collect references to image names
        image_names = list()
        # The `image_path_iterator` is created by Sphinx-Gallery, it will yield
        # a path to a file name that adheres to Sphinx-Gallery naming convention.
        image_path_iterator = block_vars['image_path_iterator']

        # Define a list of our already-created figure objects.
        list_of_my_figures = mymodule.get_figures()

        # Iterate through figure objects, save to disk, and keep track of paths.
        for fig, image_path in zip(list_of_my_figures, image_path_iterator):
            fig.save_png(image_path)
            image_names.append(image_path)

        # Close all references to figures so they aren't used later.
        mymodule.close('all')

        # Use the `figure_rst` helper function to generate the reST for this
        # code block's figures. Alternatively you can define your own reST.
        return figure_rst(image_names, gallery_conf['src_dir'])

This code could be defined either in your ``conf.py`` file, or as a module that
you import into your ``conf.py`` file (see :ref:`importing_callables`).
The configuration needed to use this scraper would look like::

    sphinx_gallery_conf = {
        ...
        'image_scrapers': ('matplotlib', "my_module._scraper.my_module_scraper"),
    }

Where Sphinx-Gallery will parse the string ``"my_module._scraper.my_module_scraper"``
to import the callable function.

Example 2: detecting image files on disk
----------------------------------------

Here's another example that assumes that images have *already been written to
disk*. In this case we won't *generate* any image files, we'll only generate
the reST needed to embed them in the documentation. Note that the example scripts
will still need to be executed to scrape the files, but the images
don't need to be produced during the execution.

We assume the function is defined within your
package in a module called ``_scraper``. Here is the scraper code::

    from glob import glob
    import shutil
    import os
    from sphinx_gallery.scrapers import figure_rst

    def png_scraper(block, block_vars, gallery_conf):
        # Find all PNG files in the directory of this example.
        path_current_example = os.path.dirname(block_vars['src_file'])
        pngs = sorted(glob(os.path.join(path_current_example, '*.png')))

        # Iterate through PNGs, copy them to the Sphinx-Gallery output directory
        image_names = list()
        image_path_iterator = block_vars['image_path_iterator']
        seen = set()
        for png in pngs:
            if png not in seen:
                seen |= set(png)
                this_image_path = image_path_iterator.next()
                image_names.append(this_image_path)
                shutil.move(png, this_image_path)
        # Use the `figure_rst` helper function to generate reST for image files
        return figure_rst(image_names, gallery_conf['src_dir'])

Then, in our ``conf.py`` file, we include the following code::

   sphinx_gallery_conf = {
       ...
       'image_scrapers': ('matplotlib', 'my_module._scraper.png_scraper'),
   }

Example 3: matplotlib with SVG format
-------------------------------------
The :func:`sphinx_gallery.scrapers.matplotlib_scraper` supports ``**kwargs``
to pass to :meth:`matplotlib.figure.Figure.savefig`, one of which is the
``format`` argument. See :ref:`custom_scraper` for supported formats.
To use SVG you can define the following function and ensure it is
:ref:`importable <importing_callables>`::

    def matplotlib_svg_scraper(*args, **kwargs):
        return matplotlib_scraper(*args, format='svg', **kwargs)

Then in your ``conf.py``::

    sphinx_gallery_conf = {
        ...
        'image_scrapers': ("sphinxext.matplotlib_svg_scraper",),
        ...
    }

You can also use different formats on a per-image basis, but this requires
writing a customized scraper class or function.

.. _mayavi_scraper:

Example 4: Mayavi scraper
-------------------------
Historically, Sphinx-Gallery supported scraping Mayavi figures as well as
matplotlib figures. However, due to the complexity of maintaining the scraper,
support was deprecated in version 0.12.0. To continue using a Mayavi scraping,
consider using something like the following::

    from sphinx_gallery.scrapers import figure_rst

    def mayavi_scraper(self, block, block_vars, gallery_conf):
        from mayavi import mlab
        image_path_iterator = block_vars['image_path_iterator']
        image_paths = list()
        e = mlab.get_engine()
        for scene, image_path in zip(e.scenes, image_path_iterator):
            try:
                mlab.savefig(image_path, figure=scene)
            except Exception:
                mlab.close(all=True)
                raise
            # make sure the image is not too large
            scale_image(image_path, image_path, 850, 999)
            if 'images' in gallery_conf['compress_images']:
                optipng(image_path, gallery_conf['compress_images_args'])
            image_paths.append(image_path)
        mlab.close(all=True)
        return figure_rst(image_paths, gallery_conf['src_dir'])

Integrate custom scrapers with Sphinx-Gallery
---------------------------------------------

Sphinx-Gallery plans to internally maintain only one scraper: matplotlib.
If you have extended or fixed bugs with this scraper, we welcome PRs
to improve it!

On the other hand, if you have developed a custom scraper for a different
plotting library that would be useful to the broader community, we encourage
you to get it working with Sphinx-Gallery and then maintain it externally
(probably in the package that it scrapes), and then integrate and advertise
it with Sphinx-Gallery. You can:

1. Contribute it to the list of externally supported scrapers located in
   :ref:`reset_modules`.
2. Optional: add a custom hook to your module root to simplify scraper use.
   Taking PyVista as an example, adding ``pyvista._get_sg_image_scraper()``
   that returns the ``callable`` scraper to be used by Sphinx-Gallery allows
   PyVista users to just use strings as they already can for
   ``'matplotlib'``::

       sphinx_gallery_conf = {
           ...
           'image_scrapers': ('pyvista',)
       }

   Sphinx-Gallery will import the named module (here, ``pyvista``) and use the
   ``_get_sg_image_scraper`` function defined there as a scraper.

.. _custom_reset:

Resetting before each example
=============================

Sphinx-Gallery supports 'resetting' function(s) that are run before and/or after
each example script is executed. This is used natively in Sphinx-Gallery to 'reset'
the behavior of the visualization packages ``matplotlib`` and ``seaborn``
(:ref:`reset_modules`). However, this functionality could be used to reset other
libraries, modify the resetting behavior for a natively-reset library or run
an arbitrary custom function at the start and/or end of each script.

This is done by adding a custom function to the resetting tuple defined in
``conf.py``. The function should take two variables: a dictionary called
``gallery_conf`` (which is
your Sphinx-Gallery configuration) and a string called ``fname`` (which is the
file name of the currently-executed Python script). These generally don't need
to be used in order to perform whatever resetting behavior you want, but must
be included in the function definition for compatibility reasons.

For example, to reset matplotlib to always use the ``ggplot`` style, you could do::

   def reset_mpl(gallery_conf, fname):
       from matplotlib import style
       style.use('ggplot')

Any custom functions can be defined (or imported) in ``conf.py`` and given to
the ``reset_modules`` configuration key. To add the function defined above (assuming
you've make it :ref:`importable <importing_callables>`)::

   sphinx_gallery_conf = {
       ...
       'reset_modules': ("sphinxext.reset_mpl", "seaborn"),
   }

In the config above ``"seaborn"`` refers to the native seaborn resetting
function (see :ref:`reset_modules`).

.. note:: Using resetters such as ``reset_mpl`` that deviate from the
          standard behavior that users will experience when manually running
          examples themselves is discouraged due to the inconsistency
          that results between the rendered examples and local outputs.

If the custom function needs to be aware of whether it is being run before or
after an example, a function signature with three parameters can be used, where
the third parameter is required to be named ``when``::

    def reset_mpl(gallery_conf, fname, when):
        import matplotlib as mpl
        mpl.rcParams['lines.linewidth'] = 2
        if when == 'after' and fname=='dashed_lines':
            mpl.rcParams['lines.linestyle'] = '-'

The value passed into ``when`` can be ``'before'`` or ``'after'``.
If ``reset_modules_order`` in the :ref:`configuration <reset_modules_order>`
is set to ``'before'`` or ``'after'``, ``when`` will always be the same value
as what ``reset_modules_order`` is set to.
This function signature is only useful when used in conjunction with
``reset_modules_order`` set to ``'both'``.

Altering Sphinx-Gallery CSS
===========================

The Sphinx-Gallery ``.css`` files that control the appearance of your example
gallery can be found `here
<https://github.com/sphinx-gallery/sphinx-gallery/tree/master/sphinx_gallery/_static>`_.
These default ``.css`` files are added to your build. Specifically, they are
copied into ``_build/html/_static/`` of your ``gallery_dir``.

You can add your own custom ``.css`` files by using the
:doc:`Sphinx configuration <sphinx:usage/configuration>` ``html_static_path``.
For example, list any path(s) that contain your custom static files, using the
``html_static_path`` configuration, in your ``conf.py`` file::

    html_static_path = ['_static']
    # Custom CSS paths should either relative to html_static_path
    # or fully qualified paths (eg. https://...)
    html_css_files = ['my_custom.css', 'other_change.css']

The default Sphinx-Gallery ``.css`` files are copied to your build **after**
files in your ``html_static_path`` config. This means that files in your
``html_static_path`` that are named the same as Sphinx-Gallery ``.css`` files
will be over-written. You can easily avoid this as all Sphinx-Gallery ``.css``
files are prepended with 'sg\_' (e.g., 'sg_gallery.css'). More details on
this can be found in `PR #845
<https://github.com/sphinx-gallery/sphinx-gallery/pull/845>`_.

Custom css can be used for example to alter the appearance of
:ref:`code links <stylizing_code_links>` and
:ref:`thumbnail size <setting_thumbnail_size>`.

Hide the download buttons in the example headers
------------------------------------------------

.. code-block:: css

    div.sphx-glr-download-link-note {
        display: none;
    }

Disable thumbnail text on hover
-------------------------------

.. code-block:: css

    .sphx-glr-thumbcontainer[tooltip]:hover::before,
    .sphx-glr-thumbcontainer[tooltip]:hover::after {
        display: none;
    }

Using (only) Sphinx-Gallery styles
==================================

If you just want to make use of sphinx-Gallery CSS files, instead of using
the ``sphinx_gallery.gen_gallery`` extension, you can use in ``conf.py``::

    extensions = ['sphinx_gallery.load_style']

This will only cause the ``gallery.css`` file to be added to your build.

Using Sphinx-Gallery sidebar components
=======================================

Sphinx-Gallery provides two built-in components (which are essentially `Sphinx templates
<https://www.sphinx-doc.org/en/master/development/templating.html>`__):
``sg_download_links`` and ``sg_launcher_links``. They are primarily designed for use in
`secondary sidebars <https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/layout.html#secondary-sidebar-right>`__
of ``pydata-sphinx-theme``, but also work with any other configuration that uses
templates, e.g., `html_sidebars
<https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_sidebars>`__.

- ``sg_download_links``: Displays the download links of source code, Jupyter notebook,
  and the zip of them for an example.
- ``sg_launcher_links``: Displays the launcher links for JupyterLite and Binder for an
  example.

Note that it is safe to use these components on any page, even if the page is not a
Sphinx-Gallery example page. The components work by grabbing specific Sphinx-Gallery
classes from the page, hence for non-Sphinx-Gallery pages, the components will simply
not render anything.

Using these components does not mean that the original download and launcher links will
be removed from the page. In fact, they cannot be removed as otherwise the components
cannot grab the necessary information. To visually remove the original links in favor
of the components, one can use ``display: none`` in CSS to hide them. The relevant
classes are:

.. literalinclude:: _static/hide_links.css
   :language: css

Example
-------

Here we will show an example of how to use these components in the secondary sidebar of
``pydata-sphinx-theme``. Navigate around :ref:`examples-index` for an example of what
the sidebar would look like.

Configure the components to be used in the secondary sidebar:

.. code-block:: python

    html_theme_options = {
        ...
        "secondary_sidebar_items": ["page-toc", "sg_download_links", "sg_launcher_links"],
    }

Add the CSS to hide the original download and launcher links (suppose that the CSS is
placed in ``_static/hide_links.css``):

.. code-block:: python

    html_static_path = ["_static"]

    def setup(app):
        # ...
        app.add_css_file("hide_links.css")

The above will include the CSS in all pages. If one wants to add the CSS only for
certain pages, one can alternatively do the following (only add to pages that start with
``auto_examples/``):

.. code-block:: python

    html_static_path = ["_static"]

    def hide_sg_links(app, pagename, templatename, context, doctree):
        if pagename.startswith("auto_examples/"):
            app.add_css_file("hide_links.css")

    def setup(app):
        # ...
        app.connect("html-page-context", hide_sg_links)
