For the purpose of this demo, I have created a Python module. demo.py which contains a class and three basic functions (all annotated with docstrings with the exception of one function). It is this module that I will create documentation for in this article. The content of this demo.py module is below:
1. Configuration
The first thing is to configure everything. You will need to install VS Code and set up a new project along with Sphinx. There are a few options here. You can a) set up a new project using Cookiecutter (where the relevant Sphinx configuration will be generated along with standardized directories) or b) create your own project structure and install sphinx separately.
option A: install Cookiecutter
In the terminal, pip installs Cookiecutter and then creates a new project:
pip install cookiecutter
cookiecutter https://github.com/drivendata/cookiecutter-data-science
Next, answer the questions that appear in the terminal window and your new project will be created. The Sphinx framework will be stored in the /docs directory of your project.
Option B: Sphinx Quick Start
If you don't like the Cookiecutter template, you can create your own project structure from scratch and install Sphinx. It's a good idea to create a documentation directory and install sphinx there. In the terminal:
mkdir docs
cd docspip install sphinx
sphinx-quickstart
2. Understand the Sphinx folder structure
After you have installed Sphinx using one of the options above, some files will appear in your project's documentation directory. He conf.py The file is the key configuration file that you will edit to customize your documentation; You will find more details about this in the next section. He index.rst The file acts as content for your documentation. You can find more information about the index.rst archive here. He getting-started.rst and commands.rst The files are suggested templates for your documentation. You can delete them if necessary. The creation files (make.bat and Makefile) are used to carry out documentation. You don't need to edit them, but you will call them in the terminal window when you are ready to do the documentation.
3. conf.py file
The configuration file is where the magic happens. This file is used during the build process, so it is essential that you configure it correctly. Below are some steps to modify the conf.py archive:
Uncomment the sys.path line (line 20 in my configuration):
# sys.path.insert(0, os.path.abspath('.'))
Change the path of os.path.abspath to the relative location of the code you want to document (relative to the conf.py archive). For example, the Python modules I want to document are located inside the src/ directory of my project. Therefore, I will change os.path.abspath to the /src directory which is located in the main folder of the conf.py archive. You can specify the relative location using the . and / syntax:
sys.path.insert(0, os.path.abspath('../src'))"""
# you can use the following syntax to specify relative locations:
'.' # current path of conf.py
'..' # parent path of conf.py
'../..' # parent of the parent path of conf.py
"""
Add relevant extensions. You will need to add some extensions to the conf.py file for additional functionality when creating your documentation. All of them are optional and you can have fun exploring the different extensions available. here. Here are the 5 extensions I recommend as a minimum:
- sphinx.ext.autodoc – use docstring documentation
- autodocsum — generate a tabular summary of all docstrings at the top of the html page listing only the docstring summaries. Useful when you have many docstrings. Note. You will need to install autodocsumm in the terminal.
- sphinx.ext.napoleon — allows Sphinx to parse Google doc strings
- sphinx.ext.viewcode — adds a link to an html page containing the source code for each module
- sphinx.ext.coverage – Provides a summary of how many classes/functions etc. They have document chains. Good coverage means that a code base is well explained.
Here's how to include these extensions in the conf.py file (line 29 in my configuration):
# add in the extension names to the empty list variable 'extensions'
extensions = (
'sphinx.ext.autodoc',
'sphinx.ext.napoleon',
'autodocsumm',
'sphinx.ext.coverage'
)# add in this line for the autosummary functionality
auto_doc_default_options = {'autosummary': True}
change the topic. The default theme in the documentation is pretty clean, although you may prefer to play with different options by changing the 'html_theme' variable (line 94 in my configuration) from 'default' to one of the standard ones. Theme Options or some third party options. In this demo, I will show the default themes and Read Documents.
html_theme = 'sphinx_rtd_theme' # read the docs theme. This variable is 'default' by default.
Note. you will need to install any non-standard (third party) themes.
4. Create the html pages.
Now that you conf.py The file is set up and you have glorious docstrings in your code, we're ready to do some scraping and create some html pages.
Generate .rst files from your Python packages
These files are the precursors to html pages and are the native format of Sphinx. These must be generated before creating the html files. You will use the sphinx.apidoc command, which uses the autodoc extension to locate all Python modules (for example, any .py file) within the sys.path location that you specified in the conf.py archive. There are some optional parameters to include when using the apidoc command which you can find in the documentationbut I used the following template:
Note. In the terminal, change the directory to the root of the project to run the following code.
sphinx-apidoc -f -o output_dir module_dir/
-F (force overwriting of any existing generated files).
-o output_addr (directory to place the output files. If it does not exist, it is created). Note. replace 'output_dir' with a directory name of your choice. I set mine up in the /docs directory.
module_addr (location of Python packages to document)
After running this command, there should be newly generated .rst files in the documents folder.
Notice that two new .rst files have been generated: data.rst and modules.rst. In addition to modules.rst, a .rst file will be generated for each directory that contains at least one Python module. In my example, data.rst is generated since I saved my demo.py file in src/data directory. If you have multiple Python module directories within the location you specified in sys.path in the conf.py file, multiple .rst files will be generated. Note. These files do not contain the removed documentation yet, they only contain the information necessary for autodoc to create the html files in the next step.
Edit index.rst file
Remember, index.rst acts as a content page, so we need to edit this file to include all the Python modules we want to document. Luckily, the modules.rst refers to the source location of all Python modules identified in sys.path, so you can simply add this file to index.rst.
To do this, open the index.rst file and add 'modules' below the toctree (table of contents tree) section. Make sure there is a line between the :max Depth: parameter and the .rst file names.
Note. 'Getting-started' and 'Commands' will already be in the index.rst file. You can remove them from this file if you don't want to generate html pages (although an “intro” page is probably a good idea).)
make html files
Now we can use the make files in your documentation directory to create the html files. These files will appear in the _build/html/ directory inside your documentation folder. You can preview these in VS code if you download the 'HTML Preview' extension.
Change the directory where the make.bat file is located and run the following command in the cmd terminal:
make html
Note. If you are using the Windows PowerShell terminal (instead of cmd), use the following syntax:
.\make.bat html
Better advice. If you get a warning when using the make html command saying “autodoc: could not import module”, this is most likely because autodoc could not find your modules since sys.path was not set correctly in conf .py. Make sure this points to the directory where your Python modules are located.
Edit html files
If you want to edit your docstrings and update your html files with the changes, you can do so using the following command:
make clean html
Let's take a look at our documentation!
As I mentioned above, I have created documentation of my Python module. demo.py in two different themes seen in the images below; 'default' (left image) and 'Read the documents' (right image). The content is identical but the appearance is different. Let's take note of the main features:
- Navigation bar on the left side
- A summary of all classes or functions belonging to the module in tables at the top of the page (thanks to the 'autodocsumm' extension)
- Detailed list of docstring components for all functions and classes below the summary
