Tutorial

Package Helper 2 helps you create, develop and maintain a package. If you use all the tools presented in this tutorial, things will work this way:

• You create the file structure of your package in less than a minute with Cookiecutter.

• You use the IDE PyCharm. It is configured to:

  • Generate the documentation of your package locally,

  • Run the unit tests.

  • Generate a local html page displaying what parts of the package are covered by the unit tests.

• Your project is on GitHub. When you push modifications:

  • GitHub automatically generates the documentation in order to check if it works correctly,

  • If the branch is your default branch (“main” or “master”), Github automatically publishes the documentation online,

  • GitHub automatically runs the unit tests on several versions of Python.

  • Codecov displays what parts of the package are covered by the unit tests.

• When you make a release on GitHub, GitHub automatically publishes your package on PyPI. This way, any Python user can install it with pip install.

Tick the tools that you want to use:

• Virtualenv: virtual environment. Recommended especially if your use a third-party package that is still in a 0.x.x release (which means that its API is not considered stable yet).
• Github: collaborative version control. In that case, you can use also:
  • Codecov: check if the unit tests cover the whole package.
  • PyPI: make the package installable with pip install.

Preliminaries: Once and for All

Create Accounts on the Websites

Ensure that you have accounts (preferably with the same login) on:

• GitHub,

• Codecov,

• PyPI.

Install Cookiecutter

In a terminal (e.g. Anaconda Prompt):

pip install cookiecutter

Install Git

Git will be used to download the package template that will be used by Cookiecutter.

Install git: https://git-scm.com/downloads. You may need to restart your computer.

Change the Documentation Style in PyCharm

Perform this step only if you want to use the Numpy documentation style (which we recommend).

In the “Welcome to PyCharm” window (before you open a project): Customize → All settings → Tools → Python Integrated Tools → Docstrings → Docstring format → NumPy.

Register Your GitHub Account in PyCharm

In the “Welcome to PyCharm” window (before you open a project): Customize → All settings → Version Control → GitHub → Add account → Log In via GitHub → log in and accept to grant access.

Create Your package

This section is adapted from: https://cookiecutter-pypackage.readthedocs.io/en/latest/tutorial.html. Creating your package should take less than 5 minutes with the minimal options (cookiecutter + PyCharm), and less than 30 minutes if you use all the goodies (Virtual environment, GitHub, etc).

Generate Your Package with Cookiecutter

1. Your project will need a project name (e.g. My Toy Package) and a project slug (typically my_toy_package). Before starting, check that your project slug is not used in PyPI, even if you do not plan to use PyPI for the moment.

2. In a terminal (e.g. Anaconda Prompt):

   1. Go to the parent directory of where you want to put the directory of your package, e.g. D:\GitHub\.

   2. cookiecutter https://github.com/francois-durand/package_helper_2.git

   3. Answer the questions. Here is an example (some explanations follow):

      full_name [François Durand]: François Durand
      email [fradurand@gmail.com]: fradurand@gmail.com
      github_username [francois-durand]: francois-durand
      project_name [Python Boilerplate]: My Toy Package
      project_slug [my_toy_package]:
      project_short_description [Python Boilerplate contains all the boilerplate
      you need to create a Python package.]: My Toy Package is a beautiful package.
      version [0.1.0]:
      Select main_git_branch_name:
      1 - main
      2 - master
      Choose from 1, 2 (1, 2) [1]:
      use_pytest [y]:
      use_codecov [y]:
      Select command_line_interface:
      1 - No command-line interface
      2 - Click
      3 - Argparse
      Choose from 1, 2, 3 (1, 2, 3) [1]:
      create_author_file [y]:
      Select open_source_license:
      1 - GNU General Public License v3
      2 - MIT license
      3 - BSD license
      4 - ISC license
      5 - Apache Software License 2.0
      6 - Not open source
      Choose from 1, 2, 3, 4, 5, 6 (1, 2, 3, 4, 5, 6) [1]:

3. Optionally, in your file manager, open the directory of your package: the whole file structure is now in there!

Some explanations now:

• main_git_branch_name: your choice must be consistent with the settings of your git application. It is the name of your default branch in your git projects. If you do not know, it is probably "master", but there is a current tendency to call it "main" instead and this will probably become standard in a near future.

• use_pytest: there are essentially three ways to do unit tests in Python: unittest (the standard solution), pytest (the recommended test package) and doctest (where tests are integrated in the docstrings). If you are new to testing, I suggest using doctest. But even so, pytest is useful to configure your tests (as we will do a bit later). For this reason, in all cases, I suggest to answer yes.

• use_codecov: you use Codecov to assess the coverage of your code by your tests. I suggest answering yes anyway, even if you do not plan to use Codecov for the moment.

• Click and Argparse allow you to easily call your program with unix-style command, e.g. python my_program.py --help. You can choose either of them without harm, even if you do not use it for the moment. But personally, I answer no.

• create_author_file: I suggest to answer yes.

Open Your Project and Create a Virtual Environment (or not)

A virtual environment is essentially a Python installation dedicated to your project, with its own versions of the third-party packages. It ensures that if you reuse this project several months later, it will still work… However, it can be difficult to use in conjunction with Jupyter Notebook. If using Jupyter is important, I suggest that you do not use virtual environments.

In PyCharm:

1. Choose "Open" and select the directory of your package.

2. A prompt window named "Creating Virtual Environment" opens. If you want to create a virtual environment, validate with the default options. Otherwise, cancel.

3. In the bottom right part of the window, you will probably see some background tasks running. Wait until they are finished.

4. In the bottom right part of the window, the interpreter should be indicated: something like "Python X.X (your_package)". If "No Interpreter" appears instead, click on it and select the desired interpreter (virtual environment or not).

Install Dev Requirements

The file requirements_dev.txt contains the list of the third-party packages that you need for development, such as sphinx or pytest, but that the end-user does not need when using your package.

In PyCharm's terminal:

1. Ensure you are in the directory of your package (e.g. D:\GitHub\my_toy_package).

2. Ensure that your virtual environment is activated: there should be (venv) at the beginning of the line. If not:

   Windows: venv\Scripts\activate
   Linux:   source venv/bin/activate

3. pip install -r requirements_dev.txt

Install Your Package in “Development Mode”

This way, your package behaves as if it were installed, but any change you make will have effect immediately.

In PyCharm's terminal, you should still be in the directory of your package. Do:

python setup.py develop

Check That Everything Is Working Locally

1. In PyCharm: menu Run → Run → All Tests. It runs all the tests of the project. If you are new to testing, you can take example from the classes provided by the template, such as my_class_1.py.

2. Open the file cov/index.html (in your web browser). It displays what parts of your code are covered by the tests.

3. In PyCharm: menu Run → Run → Generate docs. It generates the html documentation of your project. The template provides examples of classes, such as the file my_class_1.py. You can use them as models if you are new to Sphinx documentation.

4. Open the file build/index.html (in your web browser). It displays the html documentation of your project.

Create the GitHub Repo

In PyCharm:

1. From the file README.rst of your package, copy the short description sentence, e.g.: "My Toy Package is a beautiful package." (you will paste it in the form below).

2. Menu VCS → Share project on GitHub.

3. Fill in the form and validate, e.g.:

   New repository name: my_toy_package
   Remote name: origin
   Description: My Toy Package is a beautiful package.

4. Accept to add all the files as proposed by PyCharm.

In a browser, you can go to your GitHub account to check that everything is there.

Check your GitHub Actions

In GitHub:

1. GitHub page of your package → Actions.

2. Check that the actions are successes (it may take several minutes). The "build" action runs the tests of the package. The "docs" actions builds the documentation, and publishes it online if the branch is your default git branch ("main" or "master").

GitHub Settings

On the GitHub page of your package:

1. Tell GitHub Pages that the documentation files are in the "gh-pages" branch of your project:

   1. Settings → Pages → Source.

   2. Select "gh-pages".

   3. Select "/ (root)".

   4. Save.

2. Configure Codecov:

   1. On Codecov's website, log in with your GitHub account. On your main page: Not yet setup → select your project (you may have to wait for syncing before it appears). Then copy the Codecov token.

   2. Back to GitHub: Settings → Secrets → New repository secret. Name: CODECOV_TOKEN. Value: paste the codecov token.

3. Register your PyPI credentials as "secrets", so that GitHub can automatically publish your package on PyPI for you:

   1. Settings → Secrets.

   2. Select "New repository secret". Name: PYPI_USERNAME. Value: your PyPI username.

   3. Select "New repository secret". Name: PYPI_PASSWORD. Value: your PyPI password.

Your First Commit

In PyCharm, make a slight modification, e.g. in the file README.rst. Then commit/push:

1. Menu Git → Commit.

2. Enter a commit message.

3. Commit and push.

4. Push.

Check That Everything Is Working Online

1. GitHub page of your package → Actions. Wait until your actions are finished.

2. Check the documentation:

   1. GitHub page of your package (main page) → near the bottom of the page, follow the link to your documentation. Check that the documentation is there.

   2. In the table of contents, click on the first page (e.g. My Toy Package). Depending on your initial choice of options, you should have up to four badges:

      1. PyPI: package or version not found (there will be the version number after your first release).

      2. Build: passing.

      3. Docs: passing.

      4. Codecov: with a percentage.

   3. In the table of contents: Reference → MyClass1. You should see the documentation of the first example of class provided by the template.

3. On Check your test coverage::

   1. GitHub page of your package (main page) → near the bottom of the page, click on the Codecov badge.

   2. You can navigate in your project to see what parts of the code are covered by the tests.

If you wish, you are now ready to release your first version (cf. below).

Develop and Maintain Your Package

Release a Version

Test your package:

1. Run the tests (in PyCharm and/or with GitHub actions).

2. Check your coverage (locally in cov/index.html and/or on Codecov) and add tests if necessary.

In PyCharm:

1. Check that all new classes are in the file __init__.py and in the Reference section of the documentation.

2. Generate the documentation (in PyCharm and/or with GitHub actions) in order to check that it is working.

3. If you were working on a secondary branch (which is desirable), do what you have to do: pull request to "main" or "master", etc.

4. Ensure that you are now in your default git branch ("main" or "master").

5. Update the file HISTORY.rst. Stick to pure .rst syntax: never use Sphinx' specific directives such as :class:`MyClass`.

6. In PyCharm's terminal, do one of the following (except for the first release):

   • bumpversion patch (version x.y.z → x.y.(z+1)) when you made a backwards-compatible modification (such as a bug fix).

   • bumpversion minor (version x.y.z → x.(y+1).0) when you added a functionality.

   • bumpversion major (version x.y.z → (x+1).0.0) when you changed the API. Note: in versions 0.y.z, the API is not expected to be stable anyway.

7. Commit/push.

On GitHub's website:

1. Go to “releases”.

2. Select “Draft a new release” or “Create a new release”.

3. Add a tag name as in HISTORY.rst, e.g. 0.1.0.

4. Add a release title as in HISTORY.rst, e.g. First release.

5. Add release notes as in HISTORY.rst, e.g. * First release on PyPI..

6. Select “Publish release”.

After a few minutes, GitHub has finished the built and it is deployed on PyPI. If you want to check, search for your package name on PyPI and check that the version number is correct. Note that the PyPI badge may take several more minutes before being updated.

If the Deployment on PyPI Fails...

Check that the readme will be correctly rendered on PyPI.

In a terminal:

python setup.py bdist
twine check dist/the_name_of_the_file.zip

where the_name_of_the_file must be replaced by the relevant file name.

Add a Module (= a File)

Typically, this is a file sub_package\my_class, containing the class MyClass.

1. In the file __init__.py: add the shortcut.

2. In the directory reference of the documentation: add the auto-documentation.

Use a Third-Party Package

For example, you want to use Numpy in your module.

1. Open the file setup.py.

2. In the list requirements, add the name of the package (e.g. 'numpy').