# 41. Packages

# 41. Packages

A package is a module that can contain other modules. In CPython, a package is not a separate object category. It is still a module object, but it has import metadata that tells the import system where to look for submodules.

At the Python level, this directory can be a package:

```text
app/
    __init__.py
    config.py
    server.py
```

You can import it as:

```python
import app
import app.config
from app.server import run
```

The important rule is simple:

```text
A package is a module with submodule search locations.
```

For a normal file module, the import system loads one file. For a package, the import system also records where child modules may be found.

## 41.1 Packages Are Modules

A package object has type `module`.

```python
import email

print(type(email))
print(email.__name__)
```

Output:

```text
<class 'module'>
email
```

A package has a module dictionary just like any other module.

```python
import email

print(email.__dict__)
```

The package namespace stores normal names:

```text
__name__
__doc__
__package__
__loader__
__spec__
__path__
__file__
__cached__
```

It can also store functions, classes, constants, imported submodules, and re-exported public API names.

A package is therefore both:

```text
a namespace object
a container for submodule lookup
```

## 41.2 The Role of `__init__.py`

A regular package usually has an `__init__.py` file.

```text
project/
    app/
        __init__.py
        config.py
        routes.py
```

When CPython imports `app`, it executes:

```text
app/__init__.py
```

The code in `__init__.py` initializes the package namespace.

Example:

```python
# app/__init__.py

VERSION = "1.0.0"

def create_app():
    return "app"
```

Then:

```python
import app

print(app.VERSION)
print(app.create_app())
```

The file `__init__.py` is not just a marker. It is executable module code.

## 41.3 Minimal Package Import

For this layout:

```text
demo/
    __init__.py
    util.py
```

and this code:

```python
import demo
```

CPython roughly performs:

```text
find package named "demo"
create module object for "demo"
set package metadata
insert "demo" into sys.modules
execute demo/__init__.py
bind name "demo" in caller namespace
```

After import:

```python
import sys
import demo

print(sys.modules["demo"] is demo)
```

Output:

```text
True
```

The package is cached in `sys.modules` under its fully qualified name.

## 41.4 Importing Submodules

For:

```python
import demo.util
```

CPython imports the parent package first.

Conceptually:

```text
import demo
then search demo.__path__ for util
then import demo.util
then set demo.util attribute
```

After a successful import:

```python
import demo.util

print(demo.util)
print(demo.util.__name__)
```

Output shape:

```text
<module 'demo.util' from '.../demo/util.py'>
demo.util
```

The submodule is cached separately:

```python
import sys
import demo
import demo.util

print(sys.modules["demo"])
print(sys.modules["demo.util"])
print(demo.util is sys.modules["demo.util"])
```

A parent package and a submodule are different module objects.

## 41.5 Fully Qualified Module Names

Packages create hierarchical module names.

```text
app
app.config
app.server
app.server.http
```

Each imported module has a fully qualified name.

```python
import app.server.http

print(app.__name__)
print(app.server.__name__)
print(app.server.http.__name__)
```

Output:

```text
app
app.server
app.server.http
```

The fully qualified name is the key used in `sys.modules`.

```python
import sys

print(sys.modules["app"])
print(sys.modules["app.server"])
print(sys.modules["app.server.http"])
```

This name-based identity is important. Importing the same file under two different names can create two independent module objects.

## 41.6 Package Search Locations

A package has `__path__`.

```python
import app

print(app.__path__)
```

For a regular package, `__path__` usually contains the package directory.

```text
['/path/to/project/app']
```

When CPython imports:

```python
import app.config
```

it searches `app.__path__`, not the full top-level `sys.path`.

This distinction is central:

| Import | Search path |
|---|---|
| `import app` | `sys.path` |
| `import app.config` | `app.__path__` |
| `import app.server.http` | `app.server.__path__` |

A package controls where its children are found.

## 41.7 `__spec__` and Packages

Every modern imported module has a `__spec__`.

For packages, the module spec includes submodule search locations.

```python
import app

print(app.__spec__)
print(app.__spec__.name)
print(app.__spec__.origin)
print(app.__spec__.submodule_search_locations)
```

For a package, this is usually non-null:

```python
app.__spec__.submodule_search_locations
```

For an ordinary module, it is usually `None`.

The import system uses this to distinguish modules from packages.

## 41.8 Package Metadata

A regular package commonly has these attributes:

| Attribute | Meaning |
|---|---|
| `__name__` | Fully qualified package name |
| `__package__` | Package context used for relative imports |
| `__path__` | Locations searched for submodules |
| `__spec__` | Import specification |
| `__loader__` | Loader that initialized the package |
| `__file__` | Path to `__init__.py`, if file-backed |
| `__cached__` | Path to cached bytecode, if available |

Example:

```python
import json

print(json.__name__)
print(json.__package__)
print(json.__path__)
print(json.__file__)
print(json.__cached__)
```

A package is therefore observable as a normal object.

## 41.9 `__package__`

The `__package__` attribute controls relative import resolution.

Inside `app/server.py`:

```python
from .config import Settings
```

The leading dot means:

```text
resolve "config" relative to the current package
```

For the module `app.server`, the package context is usually:

```text
app
```

So:

```python
from .config import Settings
```

resolves to:

```python
from app.config import Settings
```

For a package module such as `app`, `__package__` is usually `"app"`.

For a submodule such as `app.server`, `__package__` is usually `"app"`.

For a nested submodule such as `app.http.server`, `__package__` is usually `"app.http"`.

## 41.10 Package Execution Order

Given this layout:

```text
app/
    __init__.py
    config.py
    server.py
```

and this import:

```python
import app.server
```

CPython executes in this order:

```text
1. app/__init__.py
2. app/server.py
```

If `server.py` imports `config.py`:

```python
# app/server.py
from . import config
```

then execution becomes:

```text
1. app/__init__.py
2. app/server.py starts
3. app/config.py executes
4. app/server.py continues
```

Parent packages are loaded before child modules.

## 41.11 Package Attributes for Submodules

After:

```python
import app.server
```

the parent package usually receives an attribute:

```python
app.server
```

This attribute points to the submodule object.

Equivalent view:

```python
import sys
import app.server

assert app.server is sys.modules["app.server"]
```

This binding matters because user code often navigates through package attributes:

```python
import app.server

app.server.run()
```

The import system maintains the connection between the parent package and the child module.

## 41.12 `import package.module` vs `from package import module`

These two forms are similar but not identical in name binding.

```python
import app.config
```

binds `app` in the caller namespace.

```python
from app import config
```

binds `config` in the caller namespace.

Both normally load `app.config`.

Example:

```python
import app.config

print(app.config)
```

```python
from app import config

print(config)
```

The loaded module object is usually the same:

```python
import app.config
from app import config

print(app.config is config)
```

Output:

```text
True
```

The difference is the name placed in the importing module’s namespace.

## 41.13 Public Package API

A package can expose a clean public API through `__init__.py`.

Example layout:

```text
httpkit/
    __init__.py
    client.py
    response.py
    errors.py
```

Internal files:

```python
# httpkit/client.py

class Client:
    ...
```

```python
# httpkit/errors.py

class HTTPKitError(Exception):
    ...
```

Facade:

```python
# httpkit/__init__.py

from .client import Client
from .errors import HTTPKitError

__all__ = ["Client", "HTTPKitError"]
```

Users can write:

```python
from httpkit import Client, HTTPKitError
```

This lets the package author change internal file layout while preserving public imports.

## 41.14 `__all__`

The name `__all__` defines the public names used by star import.

```python
__all__ = ["Client", "HTTPKitError"]
```

For:

```python
from httpkit import *
```

Python imports names listed in `httpkit.__all__`.

Without `__all__`, star import exports names that do not begin with `_`.

`__all__` is also useful as documentation. It tells readers which names are intended as public package API.

## 41.15 Package Facades and Import Cost

A package facade improves ergonomics but can increase import time.

This is convenient:

```python
# package/__init__.py

from .database import Database
from .server import Server
from .client import Client
from .analytics import Tracker
```

But now:

```python
import package
```

loads all those modules.

That may be expensive if those modules import large dependencies, initialize native libraries, read files, or perform configuration.

A lighter package facade may expose only cheap names:

```python
# package/__init__.py

__version__ = "1.0.0"
```

Then users import heavy components directly:

```python
from package.client import Client
```

Good package design balances convenience, startup time, and dependency clarity.

## 41.16 Lazy Package Attributes

A package can lazily expose attributes using module-level `__getattr__`.

```python
# package/__init__.py

def __getattr__(name):
    if name == "Client":
        from .client import Client
        return Client
    raise AttributeError(name)
```

Then:

```python
import package

Client = package.Client
```

imports `.client` only when `Client` is requested.

This can reduce startup cost while preserving a nice public API.

The tradeoff is complexity. Lazy package exports make import behavior less obvious, and errors may appear later.

## 41.17 Regular Packages

A regular package has an `__init__.py`.

```text
pkg/
    __init__.py
    mod.py
```

Properties:

```text
executes __init__.py when imported
has __file__ pointing to __init__.py
has __path__ pointing to package directory
can define package-level API
can contain submodules and subpackages
```

Most application packages and libraries use regular packages.

## 41.18 Namespace Packages

A namespace package has no single `__init__.py`.

It can be assembled from multiple directories.

Example:

```text
dir1/
    plugins/
        alpha.py

dir2/
    plugins/
        beta.py
```

If both `dir1` and `dir2` are on `sys.path`, then `plugins` can be a namespace package.

```python
import plugins.alpha
import plugins.beta
```

The package `plugins` may have a `__path__` containing both locations.

Namespace packages are useful when multiple distributions contribute to one package namespace.

## 41.19 Regular Package vs Namespace Package

| Feature | Regular package | Namespace package |
|---|---|---|
| Has `__init__.py` | Yes | No |
| Executes package initialization code | Yes | No single initializer |
| Can define package-level names directly | Yes | Limited |
| Can span multiple directories | Usually no | Yes |
| Common use | Normal libraries and apps | Plugin namespaces, split distributions |

A regular package gives explicit initialization. A namespace package gives flexible composition.

## 41.20 Subpackages

A subpackage is a package inside another package.

```text
app/
    __init__.py
    api/
        __init__.py
        users.py
        posts.py
```

You can import:

```python
import app.api
import app.api.users
```

The import system resolves each level.

```text
app
app.api
app.api.users
```

Each level has its own module object and its own `sys.modules` entry.

```python
import sys
import app.api.users

print(sys.modules["app"])
print(sys.modules["app.api"])
print(sys.modules["app.api.users"])
```

## 41.21 Relative Imports in Packages

Relative imports are common inside packages.

```python
from .config import Settings
from .storage import Store
from ..core import errors
```

Dots mean levels:

| Syntax | Meaning |
|---|---|
| `from . import x` | Import sibling from current package |
| `from .x import y` | Import from child module in current package |
| `from .. import x` | Import from parent package |
| `from ..x import y` | Import from sibling under parent package |

Relative imports make internal dependencies independent of the top-level package name.

They also require correct package context. Running a package file directly can break them.

## 41.22 Direct Script Execution Problem

Given:

```text
app/
    __init__.py
    main.py
    config.py
```

Inside `main.py`:

```python
from .config import Settings
```

This works:

```bash
python -m app.main
```

This may fail:

```bash
python app/main.py
```

When executed by file path, `main.py` becomes `__main__`, not `app.main`. The module may lose the package context needed for relative imports.

Use module execution for package code:

```bash
python -m app.main
```

## 41.23 `__main__.py`

A package can define `__main__.py`.

```text
tool/
    __init__.py
    __main__.py
    cli.py
```

Then:

```bash
python -m tool
```

executes:

```text
tool/__main__.py
```

Example:

```python
# tool/__main__.py

from .cli import main

main()
```

This is the standard way to make a package executable.

## 41.24 Package Initialization Side Effects

Because `__init__.py` executes during import, package initialization can have side effects.

```python
# package/__init__.py

print("loading package")
connect_to_service()
```

Then:

```python
import package
```

performs that work immediately.

This can be problematic for:

```text
startup time
tests
CLI responsiveness
server cold starts
configuration ordering
optional dependencies
import cycles
```

Keep package initialization small when possible.

Good `__init__.py` files usually contain:

```text
version constants
cheap re-exports
small compatibility shims
public API declarations
```

Avoid heavy work unless import-time initialization is part of the explicit package contract.

## 41.25 Package Dependency Graphs

Package structure should reflect dependency direction.

A clean application might look like:

```text
app/
    __init__.py
    main.py
    config.py
    domain/
        __init__.py
        users.py
        posts.py
    storage/
        __init__.py
        db.py
    web/
        __init__.py
        routes.py
```

Good dependency direction:

```text
main imports web
web imports domain
web imports storage
storage imports domain
domain imports no app-specific infrastructure
```

Poor dependency direction:

```text
domain imports web
storage imports main
config imports route handlers
__init__.py imports everything
```

Bad package dependency graphs often produce circular imports.

## 41.26 Circular Imports in Packages

Circular imports are common in packages because modules often import siblings.

Example:

```python
# app/users.py
from .posts import Post

class User:
    ...
```

```python
# app/posts.py
from .users import User

class Post:
    ...
```

This may fail because `app.users` and `app.posts` need each other during top-level execution.

Possible fixes:

Move shared definitions:

```text
app/
    models.py
    users.py
    posts.py
```

Use local imports:

```python
def create_post():
    from .posts import Post
    return Post()
```

Use type-checking-only imports:

```python
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from .posts import Post
```

The structural fix is usually best. Circular imports often indicate that module boundaries need adjustment.

## 41.27 Package-Level Re-Exports and Circular Imports

Re-exporting too much in `__init__.py` can create import cycles.

Example:

```python
# app/__init__.py
from .server import Server
from .config import Config
```

Then inside `server.py`:

```python
from app import Config
```

This forces `app.__init__` to finish while it is still importing `server`.

A safer internal import is:

```python
from .config import Config
```

Inside a package, prefer importing from the defining module rather than from the package facade.

The package facade is mostly for external users.

## 41.28 Private Modules

Python uses naming conventions for private modules.

```text
package/
    __init__.py
    public.py
    _internal.py
    _compat.py
```

A leading underscore means the module is internal by convention.

```python
from package._internal import helper
```

This is allowed, but package authors may change `_internal` without preserving compatibility.

Public API should be documented and re-exported deliberately.

## 41.29 `src` Layout

Many Python projects use a `src` layout.

```text
project/
    pyproject.toml
    src/
        package/
            __init__.py
            core.py
    tests/
        test_core.py
```

This layout helps prevent accidental imports from the repository root.

Without `src`, tests may import local files by accident even when the installed package is broken.

With `src`, the package must be installed or the path must be configured correctly, which better matches user behavior.

## 41.30 Package Data

Packages can contain data files.

```text
package/
    __init__.py
    templates/
        page.html
    data/
        defaults.json
```

Do not assume package data lives in an ordinary file-system directory. Packages may be imported from zip files or other loaders.

Prefer `importlib.resources`:

```python
from importlib.resources import files

data = files("package.data").joinpath("defaults.json").read_text()
```

This asks the import system for the resource instead of manually building paths from `__file__`.

## 41.31 `__file__` Limitations

Many packages have `__file__`.

```python
import package

print(package.__file__)
```

But robust code should not assume all modules and packages have normal file paths.

Some modules may be:

```text
built in
frozen
loaded from zip files
loaded by custom importers
namespace packages
```

For package resources, use import-system APIs rather than direct path arithmetic when possible.

## 41.32 Package Version Values

Packages often define `__version__`.

```python
# package/__init__.py

__version__ = "1.2.3"
```

This is simple and common.

A package may also use installed package metadata:

```python
from importlib.metadata import version

__version__ = version("package-name")
```

The second approach avoids duplicating version strings, but it can fail if the package is not installed as metadata.

For libraries, keep version handling simple and predictable.

## 41.33 Public API Stability

Package layout and public API are different things.

Internal layout:

```text
library/
    _client.py
    _transport.py
    _errors.py
```

Public API:

```python
from library import Client, LibraryError
```

The package can preserve the public API while changing internals.

```python
# library/__init__.py

from ._client import Client
from ._errors import LibraryError

__all__ = ["Client", "LibraryError"]
```

This separation gives maintainers freedom to refactor.

## 41.34 Package Import Time

Package import time can be measured.

```bash
python -X importtime -c "import package"
```

Slow package imports often come from:

```text
large transitive imports
heavy package __init__.py files
runtime configuration loading
native library initialization
network or file-system work
plugin auto-discovery
large type-hint imports at runtime
```

Improvement usually starts with making `__init__.py` smaller.

## 41.35 Package Initialization Pattern

A practical `__init__.py` often looks like:

```python
"""
Public API for examplekit.
"""

from .client import Client
from .errors import ExampleKitError

__all__ = [
    "Client",
    "ExampleKitError",
]

__version__ = "0.1.0"
```

This is acceptable when `client` and `errors` are cheap to import.

For heavier modules:

```python
__all__ = ["Client", "ExampleKitError", "__version__"]

__version__ = "0.1.0"

def __getattr__(name):
    if name == "Client":
        from .client import Client
        return Client
    if name == "ExampleKitError":
        from .errors import ExampleKitError
        return ExampleKitError
    raise AttributeError(name)
```

Use the lazy form only when startup cost justifies the complexity.

## 41.36 Packages and CPython Internals

At the CPython level, package import is still module import.

The difference appears in import metadata:

```text
regular module:
    __spec__.submodule_search_locations = None

package:
    __spec__.submodule_search_locations = [...]
    __path__ = [...]
```

When importing a child module, the import system uses the parent package path.

Simplified:

```python
def import_child(parent, child_name):
    fullname = parent.__name__ + "." + child_name
    path = parent.__path__
    spec = find_spec(fullname, path)
    return load(spec)
```

The real import system handles locking, errors, namespace packages, caches, and loader protocols.

## 41.37 Package Objects and Attribute Lookup

A package object uses normal module attribute lookup.

```python
import package

package.name
```

This looks in the package dictionary.

If a package defines module-level `__getattr__`, missing attributes can be computed dynamically.

```python
def __getattr__(name):
    ...
```

But imported submodules are usually stored as attributes on the parent package.

```python
import package.submodule

print(package.submodule)
```

This is why package namespaces can grow as imports happen.

## 41.38 Failed Package Imports

If package import fails during `__init__.py`, the import system removes the failing module from `sys.modules` in many cases.

Example:

```python
# broken/__init__.py

raise RuntimeError("failed")
```

Then:

```python
import broken
```

raises an exception.

The import system must avoid leaving a broken module cached as if it were successfully initialized.

Submodule failures can be subtler. A parent package may import successfully while a child module fails.

## 41.39 Duplicate Package Identity

Duplicate package identity happens when the same package is importable under different names.

Example path problem:

```text
project/
    app/
        __init__.py
        state.py
```

One import path:

```python
import app.state
```

Another accidental path:

```python
import state
```

Now the same file may be loaded twice:

```text
sys.modules["app.state"]
sys.modules["state"]
```

Consequences:

```text
two module global dictionaries
two singleton instances
two registry objects
two class identities
two caches
```

This is a common source of strange bugs.

Avoid it by using consistent absolute package imports and avoiding unsafe `sys.path` edits.

## 41.40 Packages and Type Identity

Classes are objects stored in modules.

If the same module is imported twice under different names, its classes are created twice.

```python
# app/models.py

class User:
    pass
```

If loaded as both:

```python
import app.models
import models
```

then:

```python
app.models.User is models.User
```

may be:

```text
False
```

An object created from one class may fail `isinstance` checks against the other.

This is why module identity matters for packages.

## 41.41 Packages and Entry Points

Installed packages can expose console scripts through packaging metadata.

A command-line entry point may call:

```text
package.module:function
```

Conceptually, running the command imports the module and calls the function.

Example target:

```python
# tool/cli.py

def main():
    ...
```

Entry point:

```text
tool = tool.cli:main
```

This means CLI startup cost includes importing `tool.cli` and its dependencies.

Keep CLI entry modules small when startup matters.

## 41.42 Plugin Packages

Packages often support plugins.

A plugin architecture may use:

```text
namespace packages
entry points
importlib
explicit plugin lists
dynamic discovery
```

A simple explicit plugin loader:

```python
import importlib

def load_plugin(name):
    return importlib.import_module(f"app_plugins.{name}")
```

A package-based plugin system should avoid importing every plugin eagerly unless needed.

Plugin imports often have side effects, such as registration.

```python
# plugin_alpha.py

from registry import register

register("alpha", handler)
```

This is useful, but import order becomes part of program behavior.

## 41.43 Packages and Distribution Names

Import package names and distribution package names can differ.

Example:

```text
distribution name: beautifulsoup4
import name: bs4
```

The import system knows import names. Packaging tools know distribution names.

This distinction matters when reading dependency lists or package metadata.

```python
import bs4
```

does not say the installed distribution was named `bs4`.

## 41.44 Packages and `pyproject.toml`

Modern Python packages commonly use `pyproject.toml`.

A project may declare:

```toml
[project]
name = "examplekit"
version = "0.1.0"
```

The distribution name is `examplekit`.

The import package may be:

```text
src/examplekit/
    __init__.py
```

For CPython’s import system, only the importable package layout and `sys.path` matter at runtime. Build metadata matters before runtime, during installation and packaging.

## 41.45 Editable Installs

In development, packages are often installed in editable mode.

```bash
python -m pip install -e .
```

This makes the package importable from the working tree.

From CPython’s perspective, the result is still path-based import. The environment has been configured so that the package source location appears in import resolution.

Editable installs help tests and tools import the package as users would, while still using live source files.

## 41.46 Package Debugging Checklist

When a package import behaves strangely, inspect:

```python
import sys
import package

print(package)
print(package.__name__)
print(getattr(package, "__file__", None))
print(getattr(package, "__path__", None))
print(package.__spec__)
print(sys.modules.get("package"))
```

For a submodule:

```python
import package.submodule

print(package.submodule)
print(package.submodule.__name__)
print(package.submodule.__file__)
print(sys.modules.get("package.submodule"))
```

For path issues:

```python
import sys

for p in sys.path:
    print(p)
```

For resolution without importing:

```python
import importlib.util

print(importlib.util.find_spec("package"))
print(importlib.util.find_spec("package.submodule"))
```

## 41.47 Minimal Package Import Model

A simplified model for importing a submodule:

```python
def import_package_child(parent_name, child_name):
    parent = import_module(parent_name)

    fullname = parent_name + "." + child_name

    if fullname in sys.modules:
        return sys.modules[fullname]

    spec = find_spec(fullname, parent.__path__)
    if spec is None:
        raise ModuleNotFoundError(fullname)

    module = module_from_spec(spec)
    sys.modules[fullname] = module

    try:
        spec.loader.exec_module(module)
    except Exception:
        del sys.modules[fullname]
        raise

    setattr(parent, child_name, module)
    return module
```

This model captures the package-specific parts:

```text
load parent first
search parent.__path__
cache child by fully qualified name
bind child as parent attribute
```

## 41.48 Good Package Design Rules

Keep package initialization small.

Use `__init__.py` to expose stable public API, not to run the application.

Prefer direct internal imports from defining modules.

Avoid having internal modules import from the package facade.

Avoid direct script execution for package modules. Use `python -m package.module`.

Use `importlib.resources` for package data.

Use `__all__` to document public names.

Use namespace packages only when split package composition is needed.

Avoid modifying `sys.path` inside package code.

Keep dependency direction clear.

## 41.49 Common Package Failures

| Symptom | Likely cause |
|---|---|
| Relative import failed | Module run as a script instead of with `-m` |
| Partially initialized package | Circular import |
| Missing package attribute | Submodule has not been imported or facade does not export it |
| Two singleton instances | Same module imported under two names |
| Slow `import package` | Heavy `__init__.py` or transitive imports |
| Works in repo but fails after install | Bad package layout or missing package data |
| Standard module shadowed | Local file or package name collision |

## 41.50 Key Points

A package is a module with submodule search locations.

A regular package executes `__init__.py` during import.

Submodules are cached in `sys.modules` under fully qualified names.

A package’s `__path__` controls where child modules are searched.

Relative imports depend on package context.

Namespace packages allow one package namespace to span multiple locations.

Package facades improve API ergonomics but can increase import cost and create cycles.

Most package bugs come from circular imports, path mistakes, duplicate module identity, heavy initialization, or running package files directly.