The PyPI Playbook: Publishing and Maintaining Python Packages#

Table of Contents#

• Introduction
• Understanding PyPI and Package Distribution
• Setting Up a Project Directory
• Creating a Minimal setup.py or pyproject.toml
• Building and Testing Locally
• Publishing to TestPyPI
• Proper Versioning
• Advanced Topics: pyproject.toml, Wheels, and Distribution Formats
• Maintaining Your Package
• Professional-Level Expansions
• Conclusion

Introduction#

If you’ve ever written Python code that could benefit others—or if you’ve ever wanted to reuse your own code in multiple contexts without duplicating files—you may have wondered how to turn a Python project into a publishable package. The Python Package Index (PyPI) is a central repository where countless Python projects live, waiting to be installed via the familiar pip install command.

This blog post introduces you to the entire lifecycle of creating, publishing, and maintaining a Python package. From choosing how to structure your code to advanced best practices, this resource is designed to get you started quickly and guide you toward professional-level package management.

Whether you’re an enthusiast building your first library or a seasoned developer looking to refine your packaging workflow, you’ll find everything you need in this guide. Let’s dive in and explore the PyPI playbook.

Understanding PyPI and Package Distribution#

PyPI is the primary repository of software for the Python ecosystem. It hosts thousands of open-source libraries in virtually every domain—web development, data science, scientific computing, and more. When developers run pip install <package_name>, the client (pip) retrieves the package metadata and distribution files from PyPI, then installs them into the user’s environment.

PyPI has the following core elements:

1. Packages: Distributable modules or libraries. Each package can have one or more distributions—often labeled as sdist (source distribution) or a wheel (a pre-built binary distribution).
2. Metadata: Each package has metadata that includes dependencies, version, license, maintainers, etc.
3. Index/Repository: The central server (PyPI / TestPyPI) hosting distribution files.
4. Users: The developers who publish or download packages.

Why publish to PyPI at all?

• Visibility: You gain a presence in the Python community.
• Convenience: Anyone can install your code with a single command.
• Collaboration: Open-source projects can quickly gather feedback and contributions.
• Versioning: You can release updates in an organized manner.

Understanding how PyPI operates is crucial before you package anything. Let’s move on to the practical side: creating a project directory and preparing it for publication.

Setting Up a Project Directory#

A clear and consistent directory structure paves the way for a smooth packaging experience. Although Python packages can technically take many shapes, following community standards helps users and other developers understand and contribute to your project more easily.

Here’s a recommended structure for a simple Python package:

1
my_project/
2
    my_project/
3
        __init__.py
4
        core.py
5
        helpers.py
6
    tests/
7
        test_core.py
8
        test_helpers.py
9
    requirements.txt
10
    setup.py
11
    README.md
12
    LICENSE
13
    .gitignore

Breaking it down:

• my_project/: The top-level directory for your repository.
• my_project/ (nested): The actual Python package. The __init__.py file marks this directory as a Python package.
• tests/: Place your test files here. Name them systematically, for instance test_<module>.py, or use a test framework structure.
• setup.py: A traditional file to define metadata, dependencies, and packaging configurations (though newer pyproject.toml approaches are increasingly popular).
• requirements.txt: Lists required packages for development or testing (optional, but often helpful).
• README.md: Provides users with an overview of your project, usage instructions, and any other relevant details.
• LICENSE: Indicates how your package can be used or distributed.
• .gitignore: Lists files and directories that should be excluded from version control (e.g., __pycache__, virtual environments, etc.).

Even though setup.py is a popular approach, many projects are moving to pyproject.toml for packaging. We’ll explore both approaches in the sections to come.

Creating a Minimal setup.py or pyproject.toml#

The core of publishing your package is providing metadata and instructions about how to install and manage your library. The classic approach is via a setup.py file. The newer approach involves putting all or most of this configuration into pyproject.toml.

1
from setuptools import setup, find_packages
2

3
setup(
4
    name='my_project',
5
    version='0.1.0',
6
    packages=find_packages(),
7
    install_requires=[],
8
    python_requires='>=3.7',
9
    author='Your Name',
10
    author_email='youremail@example.com',
11
    description='A sample Python project',
12
    long_description=open('README.md', 'r', encoding='utf-8').read(),
13
    long_description_content_type='text/markdown',
14
    url='https://github.com/yourusername/my_project',
15
    classifiers=[
16
        'Programming Language :: Python :: 3.7',
17
        'License :: OSI Approved :: MIT License',
18
    ],
19
    license='MIT',
20
    include_package_data=True,
21
)

1
[build-system]
2
requires = ["setuptools", "wheel"]
3
build-backend = "setuptools.build_meta"
4

5
[project]
6
name = "my_project"
7
version = "0.1.0"
8
description = "A sample Python project"
9
readme = "README.md"
10
requires-python = ">=3.7"
11
license = { file = "LICENSE" }
12
authors = [
13
  { name = "Your Name", email = "youremail@example.com" }
14
]
15
classifiers = [
16
  "Programming Language :: Python :: 3.7",
17
  "License :: OSI Approved :: MIT License"
18
]
19
dependencies = []

Building and Testing Locally#

Before you publish any package, testing it locally is essential. This involves creating a source distribution (sdist) and/or a wheel distribution, then installing it in a controlled environment.

1
python -m venv venv
2
source venv/bin/activate   # On macOS/Linux
3
venv\Scripts\activate      # On Windows

1
# If using setup.py
2
python setup.py sdist bdist_wheel
3

4
# Or using pyproject.toml with setuptools
5
python -m build

1
pip install dist/my_project-0.1.0-py3-none-any.whl

1
>>> import my_project
2
>>> my_project.some_function()

Publishing to TestPyPI#

Once your package installs locally without problems, you’re ready to push it to an online testing environment. TestPyPI (https://test.pypi.org/) is a safe place to practice the publishing process without messing up the main PyPI index.

1
pip install twine

1
# Using setup.py
2
python setup.py sdist bdist_wheel
3

4
# Or with pyproject.toml
5
python -m build

1
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

1
pip install --index-url https://test.pypi.org/simple/ my_project

Proper Versioning#

Versioning communicates changes, improvements, and backward incompatibilities to your users. Python’s packaging ecosystem often uses Semantic Versioning (SemVer) to indicate the scope of changes. A typical SemVer string looks like MAJOR.MINOR.PATCH:

• MAJOR: Increment this for backward-incompatible changes.
• MINOR: Increment this for added functionality in a backward-compatible manner.
• PATCH: Increment this for backward-compatible bug fixes or minor changes.

Examples:

• 0.1.0: An initial development release.
• 1.0.0: A stable release indicating a relatively polished API.
• 1.2.3: Incrementing the patch version for bug fixes, the minor version for new features, or the major version for breaking changes.

In practice, the version string you set in setup.py or pyproject.toml should align with the version of your actual code. Many developers use version automation tools (like bumpversion or robust continuous integration scripts) that handle the version increments and tag creation.

Advanced Topics: pyproject.toml, Wheels, and Distribution Formats#

Python packaging is evolving, and developers now have more flexible and feature-rich options. Understanding the following advanced concepts can improve the reliability and compatibility of your packages.

1
[project.optional-dependencies]
2
dev = [
3
  "pytest >=6.0",
4
  "mypy",
5
  "flake8"
6
]
7
docs = [
8
  "sphinx",
9
  "sphinx-rtd-theme"
10
]
11

12
[tool.coverage.run]
13
branch = true
14
source = ["my_project"]
15

16
[tool.isort]
17
profile = "black"

1
pip install "my_project[dev]"
2
pip install "my_project[docs]"

Maintaining Your Package#

Publishing your package often marks the beginning of a project’s real lifecycle. Active maintenance involves tracking issues, adding enhancements, and ensuring compatibility with the latest Python releases.

Professional-Level Expansions#

Once you have mastered the basic workflow of packaging and distributing code—to the point of easily pushing new releases—there are numerous advanced or professional steps you can implement to further streamline your process and elevate your Python project.

Conclusion#

Creating and maintaining a package on PyPI may seem daunting at first, but once you break down the steps—structuring your project, writing a setup.py or using pyproject.toml, building distributions, testing locally, and finally uploading to (Test)PyPI—the process becomes more approachable. By adopting best practices like semantic versioning, thorough testing, and continuous integration, you can ensure your Python package remains reliable and useful for a long time.

As your confidence and ambitions grow, you can explore more sophisticated workflows: wheels, platform-specific builds, advanced dependency management, code coverage, and continuous deployment. Ultimately, the Python packaging ecosystem offers a wealth of tools and standards that streamline the entire lifecycle of your project. Embrace them, and you’ll find that publishing on PyPI is not just about sharing code, but about contributing to the broader Python community with professionalism and polish.