Documentation on Stephen Finucane (Fin-oo-can)

A Lion, A Head, and A Dash of YAML

stephenfinucane@hotmail.com (Stephen Finucane) — Sat, 29 Feb 2020 00:00:00 +0000

This talk, delivered at PyCon Limerick 2020, was a repeat of a talk I’d previously given at FOSDEM 2018. Once again, this talk provided a quick dive into the somewhat complicated world of Sphinx extensions.

If you’ve read documentation for a Python library, you’ve almost certainly seen the end product of a docutils or Sphinx-powered documentation toolchain. Sphinx builds upon docutils and together, they provide the tool of choice for a majority of projects across the Python ecosystem.

One of the great advantages of both tools is their inherent customisability and extensibility: not only can you customise the output of the tools, be that HTML, LaTeX or something else, but you can also write extensions that allow you to do things up to and including automatically generating documentation for you.

In this presentation, which is part talk, part workshop, we’ll look at the basics for building and using your own extension for these tools, with the goal of solving the kinds of issues that have been seen in large Python-first communities like OpenStack. Some prior Python experience will be necessary, but no Sphinx or docutils experience is assumed.

More information is available on the PyCon Limerick 2020 website.

Working With Documentation, The Openstack Way

stephenfinucane@hotmail.com (Stephen Finucane) — Tue, 30 Apr 2019 00:00:00 +0000

This talk was co-presented with Alex Settle at the Denver OpenStack Summit in April 2019. It serves as a brief history of documentation within OpenStack along with a how-to on contributing to documentation today.

Back in the old days, OpenStack’s documentation was managed by a central team and written in the unspeakable horror that is DocBook. Things have changed a lot since then, with documentation now spread out across multiple projects and Sphinx the documentation tool du jour. This presents a lot of opportunities for people, be they long-term contributors or novices, to make a real impact to OpenStack’s documentation. In this talk, we detail how one can get started contributing in this area, be it through writing your own guides, using Sphinx to automatically generate docs or simply fixing a typo. Viva la documentation!

More information is available on the OpenStack Summit website.

Who Needs pandoc When You Have Sphinx?

stephenfinucane@hotmail.com (Stephen Finucane) — Sat, 02 Feb 2019 00:00:00 +0000

This was a talk I gave at FOSDEM 2019. In this talk, subtitled “An exploration of the parsers and builders of the Sphinx documentation tool”, I explored some of the less well known aspects of both Sphinx and docutils.

Using Sphinx doesn’t necessarily mean using reStructuredText for input and HTML for output. We explore Sphinx’s newfound support for Markdown as well as it’s broad range output formats available, before moving onto an overview of how you can develop your parser and builder extensions.

Sphinx is the documentation tool of choice for an increasing amount of projects, both inside and outside Python. Thanks to the success of platforms like Read the Docs, building documentation with this toolkit has never been easier. Most people associate Sphinx with the reStructuredText syntax provided by docutils, and typically output documents to HTML and, occasionally, PDF. However, Sphinx is capable of so much more. Since Sphinx 1.8, it is possible to use source documents written in the CommonMark syntax, while the amount of builders provided both in-tree and out-of-tree continue to grow.

Through this talk, we explore how one can build their documentation using Sphinx and Markdown source documents. We detail the variety of builders available as part of the standard Sphinx installation and as third-party extensions, including some basic configuration tips for the more commonly useful ones like man pages and LaTeX/PDF. Finally, we provide a high-level overview on how you can go about writing your own extensions to parse other plain-text documentation formats and output in additional documentation format. The latter of these will touch on docutils, a foundational component of Sphinx, so one can understand the intermediate documentation model it provides.

Full information is available on the FOSDEM website.

A Lion, A Head, and A Dash of YAML

stephenfinucane@hotmail.com (Stephen Finucane) — Sun, 04 Feb 2018 00:00:00 +0000

This talk, delivered at FOSDEM 2018, provided a quick dive into the somewhat complicated world of Sphinx extensions. Sphinx is an incredibly powerful tool but, surprisingly for a documentation, suffers from poor quality documentation. This talk aimed to fill in some of the blanks for users that aren’t so familiar with the tool and its inner workings.

The Sphinx documentation tool provides tremendous extensibility and theming options; options which are massively underutilized. We demonstrate how you can go about writing your own extensions and themes for Sphinx and ultimately how you treat your docs like code.

If you work in open source, then it’s increasingly likely that you’ve come across the Sphinx documentation tool. Originally written by and for the Python community, the past few years have seen an increasingly rapid adoption by non-Python projects. Recent adopters of Sphinx include the Linux kernel, Open vSwitch and the Dataplane Development Kit (DPDK), while other projects such as QEMU are preparing plans to migrate. Meanwhile, the Python ecosystem itself has only grown in strength and communities such as OpenStack have adopted the tool and have made extensive use of it.

Unfortunately, despite the widespread adoption and existence of multiple extensions, Sphinx is not always a very well understood tool. Extensions provide a pathway to greater functionality, for example, automatically generating documentation or generating docs in different output formats. Themes, meanwhile, provide a way to give a consistent, project-focused feel to your docs. The OpenStack community makes significant use of these extensibility features, which allow us to solve a number of problems in an automated manner.

How can I avoid documenting a command line application twice: once in code and once in my docs?

How can I take a large dataset and automatically convert this into multiple human-readable documents?

How can I move documents about without breaking links?

How can I automatically spell-check my documents?

How can I style my docs to match my project website?

Through this talk we will demonstrate how each of the above problems, and how many more, can be solved in an easy and automated manner through the power of Sphinx extensions and themes. We also demonstrate how you can share your extensions and themes with the world, if you’re so inclined

More information is available on the FOSDEM website.

Building PDFs for OpenStack documentation

stephenfinucane@hotmail.com (Stephen Finucane) — Wed, 06 Dec 2017 00:00:00 +0000

I’ve only ever really worked with HTML and man page builds for the documentation various of various OpenStack projects. However, OpenStack uses Sphinx across the board and Sphinx, being the awesome tool that it is, supports many other output formats. In this instance, I was interested in PDF.

Sphinx doesn’t actually provide a native PDF builder (although other packages do). Instead, you have to generate LaTeX sources and then generate a PDF for this. This means you need to have a pretty well fleshed out TeX installation. As such, the first step is to install the dependencies.

Install dependencies

We need a number of packages to build the LaTeX sources and convert them into a PDF. I’m using Fedora 27 which means package names may need to be adjusted accordingly. Do note, however, that these names should work equally well on RHEL and RHEL derivatives like CentOS).

texlive (TeX Live)

Per the description of this package in the Fedora repos.

The TeX Live software distribution offers a complete TeX system for a variety of Unix, Macintosh, Windows and other platforms. It encompasses programs for editing, typesetting, previewing and printing of TeX documents in many different languages, and a large collection of TeX macros and font libraries.

The distribution includes extensive general documentation about TeX, as well as the documentation for the included software packages.

This is the basic thing we need to actually build LaTeX documents and convert them to PDF. The package itself includes multiple other dependencies, either because they are necessary for a minimal install or they form a standard library of sorts. I’m not sure which.
latexmk

Per the description of this package in the Fedora repos.

latexmk is a perl script for running LaTeX the correct number of times to resolve cross references, etc.; it also runs auxiliary programs (bibtex, makeindex if necessary, and dvips and/or a previewer as requested). It has a number of other useful capabilities, for example to start a previewer and then run latex whenever the source files are updated, so that the previewer gives an up-to-date view of the document. The script runs on both UNIX and MS-WINDOWS (95, ME, XP, etc). This script is a corrected and improved version of the original version of latexmk.

This is a tool that Sphinx uses, via some makefiles it includes with the built sources to ease converting the built LaTeX sources into PDFs.

You also need a variety of other packages which aren’t included as part of the texlive installation. The base Sphinx template has the following requirements.

texlive-fncychap
texlive-titlesec
texlive-tabulary
texlive-framed
texlive-wrapfig
texlive-upquote
texlive-capt-of
texlive-needspace

In addition, openstackdocstheme, the theme used for all official OpenStack projects, provides a LaTeX template with the following dependencies.

texlive-polyglossia

Installation of all the above.

$ sudo dnf install texlive latexmk texlive-fncychap texlive-titlesec \
    texlive-tabulary texlive-framed texlive-wrapfig texlive-upquote \
    texlive-capt-of texlive-needspace texlive-polyglossia

Get some documentation

I didn’t have specific project in mind, so I’ve decided to build the project documentation for the openstackdocstheme project. As mentioned previously, this project provides the theme used for all Official OpenStack projects. Given that this is the base for most projects’ documentation, it figures that it contains all the necessary configuration to build PDFs. If, however, you are planning to build docs for some other project, you should ensure that this is project is configured correctly for PDF builds.

To begin, clone the repo locally.

$ git clone https://github.com/openstack/openstackdocstheme

Once cloned, create a virtual environment and install the dependencies.

$ virtualenv .venv
$ source .venv
$ pip install -e .

In the case of openstackdocstheme, Sphinx isn’t installed by default. Let’s install that too.

$ pip install Sphinx

Build the docs

Now that we have both our dependencies and source documentation in place, it’s time to actually build some PDFs. Sphinx actually provides a number of helpful targets in the Makefile generated by sphinx-quickstart. However, OpenStack projects don’t tend to use these so we’re going to take the (still exceedingly simple) two-step process. Firstly, build the LaTeX sources. Seeing as this is a Python project, we’re going to use the setuptools integration.

$ python setup.py build_sphinx -b latex

All OpenStack projects place their documentation source and builds in doc/source and doc/build respectively. The above command should generate a number of files in doc/build/latex including a very helpful makefile. We’re going to use that now to complete the process of generating a PDF.

$ make -C doc/build/latex

This should emit a lot of text, followed by something that looks like this.

Output written on os-doc-demo.pdf (23 pages).
Transcript written on os-doc-demo.log.
=== TeX engine is 'XeTeX'
Latexmk: Index file 'os-doc-demo.idx' was written
Latexmk: Log file says output to 'os-doc-demo.pdf'
Latexmk: All targets (os-doc-demo.pdf) are up-to-date

Once complete, you can find your compiled PDF in the same build directory (in this case, doc/build/latex/os-doc-demo.pdf)