40. Modules and Imports

A module is Python’s basic unit of code loading, namespace isolation, and reuse. In CPython, a module is both a language-level object and a runtime record in the import system.

At the Python level, a module is what you get after executing:

import math
import os
import json

Each imported name is bound to a module object, package object, function, class, or other exported object. At the CPython level, import is a coordinated process involving bytecode instructions, import hooks, module specifications, loaders, finders, sys.modules, package paths, file-system lookup, bytecode caches, import locks, and module execution.

The import system is not a simple file include mechanism. It is a runtime protocol.

40.1 What a Module Is

A module is an object of type module.

import sys

print(type(sys))
print(sys.__name__)

Output:

<class 'module'>
sys

A module object owns a dictionary. That dictionary is the module’s global namespace.

import math

print(math.__dict__["pi"])
print(math.__dict__["sqrt"])

When CPython executes a module file, top-level assignments write into this dictionary.

For a file named config.py:

debug = True
port = 8080

def connect():
    return port

CPython creates a module object, prepares its namespace, executes the compiled code object inside that namespace, and leaves the resulting bindings in config.__dict__.

Conceptually:

module object
    __dict__
        "__name__"      -> "config"
        "__file__"      -> ".../config.py"
        "__spec__"      -> ModuleSpec(...)
        "debug"         -> True
        "port"          -> 8080
        "connect"       -> function object

A module is therefore a mutable namespace object.

40.2 Module Objects in CPython

In CPython, module objects are implemented by the PyModuleObject type.

A simplified mental model is:

typedef struct {
    PyObject_HEAD
    PyObject *md_dict;
    PyObject *md_name;
    PyObject *md_doc;
    PyObject *md_state;
    PyObject *md_weaklist;
    PyModuleDef *md_def;
} PyModuleObject;

The exact fields can change, but the important point is stable: a module has an associated dictionary and optional C-level module definition/state.

The dictionary stores ordinary Python names. When Python code evaluates a global name inside a module, CPython usually looks in that module dictionary first.

For example:

x = 10

def f():
    return x

The function f does not copy x. It stores a reference to the module globals dictionary through its function object. When f executes return x, CPython resolves x using global lookup against that dictionary.

40.3 Import Is Execution

Importing a Python module executes its top-level code.

For example.py:

print("loading example")

value = 42

def get_value():
    return value

The first import executes the file:

import example

Output:

loading example

A second import usually does not execute the file again:

import example

No output appears because CPython finds the existing module in sys.modules.

This behavior is fundamental. Import has side effects because module top-level code runs.

Good module top-level code usually contains definitions and cheap initialization:

CONSTANT = 100

def parse(text):
    ...

Risky module top-level code performs expensive or externally visible work:

connect_to_database()
delete_old_files()
start_threads()
make_network_request()

Such code runs during import, sometimes before the application is fully initialized.

40.4 The Import Statement

The statement:

import package.module

does not directly mean “open this file.”

It means:

resolve a module name
find a module specification
create or reuse a module object
initialize import-related attributes
execute the module if needed
bind a name in the caller's namespace

The statement:

import os.path

usually binds os, not os.path, in the local namespace:

import os.path

print(os)
print(os.path)

The statement:

from os import path

binds path directly:

from os import path

print(path)

The statement:

from math import sqrt

imports the module if needed, then retrieves sqrt from that module and binds it in the caller’s namespace.

40.5 Bytecode for Import

CPython compiles import statements into bytecode.

Example:

import math

The compiled code uses import-related bytecode instructions. The exact instruction sequence varies by Python version, but conceptually it does this:

load import machinery
import module named "math"
bind result to name "math"

For:

from math import sqrt

the bytecode conceptually does this:

import module named "math"
load attribute "sqrt"
bind local/global name "sqrt"

You can inspect this with dis:

import dis

def f():
    import math
    return math.sqrt(9)

dis.dis(f)

The import statement is part of normal bytecode execution. There is no separate preprocessor step.

40.6 __import__

At the language level, import statements eventually route through import machinery exposed through builtins.__import__.

import builtins

print(builtins.__import__)

You can call it directly:

math_module = __import__("math")
print(math_module.sqrt(9))

But most code should not call __import__ directly. Use importlib.import_module for dynamic imports:

import importlib

mod = importlib.import_module("math")
print(mod.sqrt(9))

The function __import__ exists because import is dynamic. Python code can import modules by string name at runtime.

40.7 sys.modules

sys.modules is the central module cache.

It is a dictionary mapping fully qualified module names to module objects.

import sys
import math

print(sys.modules["math"] is math)

Output:

True

Before loading a module, the import system checks sys.modules.

Conceptually:

if fullname in sys.modules:
    return sys.modules[fullname]
else:
    module = load_module(fullname)
    sys.modules[fullname] = module
    return module

The real process is more careful because it must handle packages, circular imports, failed imports, locks, and loader protocols.

The key property remains: imports are cached by module name.

40.8 Why Modules Are Inserted Before Execution

CPython usually inserts a module into sys.modules before executing its code.

This is necessary for circular imports.

Suppose a.py contains:

import b

x = 1

and b.py contains:

import a

y = 2

When a imports b, and b imports a, the import system must avoid infinite recursion. It does this by placing the partially initialized module object into sys.modules.

The tradeoff is that circular imports can observe incomplete modules.

Example:

# a.py
import b

x = 1

# b.py
import a

print(a.x)

This may fail because a.x has not been assigned yet when b reads it.

Circular imports are not forbidden, but they require care. The usual fix is to move imports inside functions, move shared definitions into a third module, or avoid top-level cross-dependencies.

40.9 Module Initialization Sequence

A simplified import sequence for a Python source module looks like this:

1. Receive module name, such as "pkg.mod".
2. Check sys.modules.
3. Search sys.meta_path for a finder.
4. Finder returns a ModuleSpec.
5. Import machinery creates a module object.
6. Module is inserted into sys.modules.
7. Loader executes module code.
8. Import machinery returns the module object.
9. Import statement binds names in caller namespace.

For a source file, execution means:

read source
decode source
compile source to code object
execute code object in module namespace

For an extension module, execution means calling native initialization code.

For a built-in module, execution uses built-in initialization logic compiled into CPython.

40.10 ModuleSpec

Modern Python import uses ModuleSpec objects to describe how a module should be loaded.

A module spec contains information such as:

module name
loader
origin
package search locations
cached bytecode path
whether the module is a package

You can inspect a module’s spec:

import json

print(json.__spec__)
print(json.__spec__.name)
print(json.__spec__.origin)
print(json.__spec__.loader)
print(json.__spec__.submodule_search_locations)

For a normal module, submodule_search_locations is usually None.

For a package, it contains paths where submodules may be found.

40.11 Finders and Loaders

The import system separates finding from loading.

A finder answers:

Can this module name be found?
If yes, what spec describes it?

A loader answers:

How should this module be created and executed?

This separation allows Python to import from many places:

source files
bytecode files
built-in modules
extension modules
zip archives
namespace packages
custom import hooks
memory-backed module stores
remote systems, if a custom importer implements it

The standard import system is extensible because it is protocol-based.

40.12 sys.meta_path

sys.meta_path is the first major hook point in the import system.

It is a list of finder objects. Each finder can decide whether it knows how to handle a module name.

import sys

for finder in sys.meta_path:
    print(finder)

A simplified import search does this:

for finder in sys.meta_path:
    spec = finder.find_spec(fullname, path, target)
    if spec is not None:
        return spec

Typical entries handle:

built-in modules
frozen modules
path-based modules

The path-based finder is responsible for searching directories and other path entries.

40.13 sys.path

sys.path is the list of import search locations for top-level modules.

import sys

for entry in sys.path:
    print(entry)

When you write:

import mymodule

and mymodule is not built in or frozen, the path-based import system searches entries in sys.path.

Entries are usually:

directory of the running script
current working directory in interactive mode
standard library directories
site-packages directories
paths from PYTHONPATH
virtual environment paths
zip archives

This is why changing sys.path changes import behavior.

import sys

sys.path.insert(0, "/custom/modules")

import mymodule

This can be useful in controlled tools, but it can also create fragile import behavior.

40.14 Packages

A package is a module that can contain submodules.

Historically, a directory became a package by containing an __init__.py file:

pkg/
    __init__.py
    parser.py
    lexer.py

Then:

import pkg.parser

loads pkg first, then pkg.parser.

The file pkg/__init__.py executes when the package is imported.

For example:

# pkg/__init__.py
print("loading package")
version = "1.0"

import pkg
print(pkg.version)

A package is still a module object. The difference is that it has package search locations.

40.15 Package Attributes

Packages usually define import-related attributes:

__name__
__package__
__path__
__spec__
__file__
__cached__

The important package-specific attribute is __path__.

import package

print(package.__path__)

__path__ tells the import system where to search for submodules inside that package.

For:

import package.submodule

the import system searches package.__path__, not the top-level sys.path.

40.16 Namespace Packages

Python supports namespace packages. These are packages without a single __init__.py file.

A namespace package can be spread across multiple directories.

Example:

dir1/
    plugins/
        alpha.py

dir2/
    plugins/
        beta.py

If both dir1 and dir2 are on sys.path, Python can treat plugins as a namespace package.

Then both may work:

import plugins.alpha
import plugins.beta

Namespace packages are useful for plugin systems and separately distributed package fragments.

They also make import resolution more complex because a package may have multiple search locations.

40.17 Absolute Imports

An absolute import starts from the top-level import namespace.

import package.module
from package import module

Inside a package, this still searches for a top-level package named package.

Absolute imports are preferred for clarity when referring to external or top-level modules.

Example:

from project.config import Settings

This tells the reader where the imported name comes from.

40.18 Relative Imports

A relative import is resolved against the current package.

from . import parser
from .lexer import tokenize
from ..config import Settings

Relative imports depend on __package__.

They only work when the module is executed as part of a package. This is why running a package module directly as a script can break relative imports.

For example:

project/
    app/
        __init__.py
        main.py
        config.py

Inside main.py:

from .config import Settings

This works when executed with:

python -m app.main

It may fail when executed as:

python app/main.py

because direct script execution changes how the module is named and packaged.

40.19 __main__

The module executed as the program entry point is named __main__.

print(__name__)

When run as a script:

python script.py

Output:

__main__

When imported:

import script

the module name is:

script

This is why Python programs commonly use:

def main():
    ...

if __name__ == "__main__":
    main()

The guard prevents program entry code from running during import.

40.20 Running Modules With -m

The command:

python -m package.module

runs a module by import name rather than by file path.

This matters because the module gets correct package context.

For package code, prefer:

python -m package.module

over:

python package/module.py

The -m form allows relative imports to work because CPython knows the module’s package.

40.21 Bytecode Cache Files

CPython may cache compiled bytecode in __pycache__.

Example:

package/
    module.py
    __pycache__/
        module.cpython-312.pyc

The bytecode cache avoids recompiling source on every import when the cache is valid.

A .pyc file contains:

magic number
cache metadata
marshaled code object

The magic number identifies the bytecode format version. It changes when CPython changes bytecode incompatibly.

CPython checks cache validity using timestamp-based or hash-based invalidation, depending on how the file was produced.

40.22 Source Modules and Code Objects

For a normal .py module, the loader compiles source into a code object.

Then it executes the code object in the module dictionary.

Conceptually:

module = types.ModuleType("example")
code = compile(source_text, filename, "exec")
exec(code, module.__dict__)

This is close to the real model, although the actual import system handles many more details.

The important point is that module execution is ordinary code execution with a module dictionary as the global namespace.

40.23 Built-in Modules

Built-in modules are compiled into the CPython executable or linked runtime.

Examples often include:

import sys
import builtins
import time

Built-in module availability depends on platform and build configuration.

A built-in module does not require locating a .py file. Its loader initializes it from C-level definitions.

You can inspect built-in module names:

import sys

print(sys.builtin_module_names)

Built-in modules provide core runtime services needed before the full file-based import system is available.

40.24 Frozen Modules

Frozen modules are Python modules embedded into the CPython binary as frozen bytecode or equivalent static data.

They help bootstrap import machinery before the file system import system is fully operational.

This creates a bootstrapping problem:

importlib implements imports
but importlib itself must be imported
so parts of importlib are frozen

Frozen modules solve this cycle by making selected Python code available without ordinary source-file import.

40.25 Extension Modules

Extension modules are native shared libraries loaded into CPython.

On Unix-like systems, these are commonly .so files. On Windows, they are commonly .pyd files.

Example imports:

import _sqlite3
import _ssl
import _hashlib

An extension module provides an initialization function called by CPython. Modern extension modules use multi-phase initialization when possible.

Extension modules must follow CPython’s C API rules:

create module object
define methods
manage reference ownership
set exceptions on failure
return initialized module

Because extension modules run native code inside the Python process, bugs can crash the interpreter.

40.26 Single-Phase and Multi-Phase Initialization

Older extension modules often use single-phase initialization. The initialization function creates and returns a module object in one step.

Modern extension modules can use multi-phase initialization. In this model, module creation and module execution are separated.

This better matches Python-level import semantics and supports per-module state more cleanly.

Multi-phase initialization is important for:

subinterpreter compatibility
module reloading behavior
cleaner module state
avoiding process-global mutable state
future isolation improvements

A C extension that stores all state in global C variables may work in simple cases, but it can behave badly with subinterpreters or repeated initialization.

40.27 Import Locks

Imports need locking.

Without locking, two threads could try to import and initialize the same module at the same time.

The import system uses locks to ensure that a module is not executed concurrently by multiple threads in unsafe ways.

This matters especially when module initialization has side effects:

# database.py
connection_pool = create_pool()

If two threads imported this module concurrently without locking, they might create duplicate global state or observe partial initialization.

The import lock prevents many such races.

40.28 Reloading Modules

Python can reload a module using importlib.reload.

import importlib
import config

importlib.reload(config)

Reloading re-executes the module code using the existing module object.

This has subtle consequences.

Suppose config.py originally contains:

value = 1

After editing it to:

value = 2

a reload updates config.value.

But existing references elsewhere may still point to old objects.

from config import value

import config
import importlib

importlib.reload(config)

print(value)        # old binding
print(config.value) # new module attribute

Reload is useful for development tools, notebooks, and plugin systems, but it is not a full process reset.

40.29 Import Side Effects

Import side effects are often the source of confusing behavior.

This module has a visible side effect:

# noisy.py
print("imported noisy")

This module has a hidden side effect:

# registry.py
handlers = {}

def register(name, fn):
    handlers[name] = fn

# plugin.py
from registry import register

def handle(x):
    return x

register("plugin", handle)

Importing plugin mutates registry.handlers.

This pattern is common in plugin systems, ORMs, test frameworks, and web frameworks. It can be useful, but it means import order becomes part of program behavior.

40.30 Lazy Imports

A lazy import delays importing a module until it is needed.

Example:

def parse_json(text):
    import json
    return json.loads(text)

This can reduce startup time or avoid optional dependencies during unused code paths.

But lazy imports have tradeoffs:

errors appear later
first call may become slower
dependency structure becomes less visible
circular imports may be hidden rather than fixed

Lazy imports are useful when used deliberately. They should not become a default workaround for poor module structure.

40.31 Optional Imports

Optional imports are common for feature detection.

try:
    import uvloop
except ImportError:
    uvloop = None

Be careful with broad exception handling. This is often wrong:

try:
    import plugin
except Exception:
    plugin = None

It hides real bugs inside plugin.

Prefer catching ImportError or ModuleNotFoundError narrowly, and when needed, verify which module failed.

try:
    import optional_backend
except ModuleNotFoundError as exc:
    if exc.name != "optional_backend":
        raise
    optional_backend = None

This avoids hiding missing transitive dependencies.

40.32 Import Name Binding

Different import forms bind different names.

The import system loads modules. The import statement then binds names in the current namespace.

These are related but separate operations.

40.33 Star Imports

A star import copies exported names into the current namespace.

from module import *

If the module defines __all__, Python imports those names.

__all__ = ["connect", "close"]

Without __all__, Python imports names that do not start with an underscore.

Star imports are usually discouraged outside interactive sessions and package facade modules because they obscure where names come from.

A controlled package facade may use them carefully:

# package/__init__.py
from .client import Client
from .errors import PackageError

__all__ = ["Client", "PackageError"]

40.34 Package Facades

A package can re-export names from submodules.

# library/__init__.py
from .client import Client
from .config import Config

__all__ = ["Client", "Config"]

Then users can write:

from library import Client

instead of:

from library.client import Client

This improves API ergonomics but can increase import cost. If library.__init__ imports many heavy submodules, then import library becomes expensive.

Good package facades balance convenience and startup cost.

40.35 Import Performance

Import time matters for command-line tools, server cold starts, tests, and short-running scripts.

Import cost comes from:

file-system searches
source decoding
bytecode validation
compilation if cache is missing
module execution
transitive imports
native extension loading
top-level initialization

You can inspect import timing with:

python -X importtime -c "import your_package"

This prints a tree of import timings.

Common ways to improve import performance:

avoid heavy top-level work
delay optional imports
reduce large dependency chains
avoid importing test-only modules at runtime
keep package __init__.py small
avoid broad convenience imports in hot paths

40.36 Import Errors

The common import-related exceptions are:

Example:

from package import missing_name

If package exists but missing_name does not, Python may raise ImportError.

Example:

import missing_package

Usually raises:

ModuleNotFoundError

Import failure diagnosis should distinguish:

the target module is missing
a transitive dependency is missing
the module exists but raised during execution
the requested exported name is missing
a native extension failed to load

40.37 Circular Imports

Circular imports happen when modules depend on each other during top-level execution.

Example:

# users.py
from posts import Post

class User:
    ...

# posts.py
from users import User

class Post:
    ...

This can fail because each module needs the other before either has finished initializing.

Common fixes:

1. Move shared types into a third module.

models/
    base.py
    users.py
    posts.py

2. Use local imports for runtime-only dependencies.

def create_post():
    from posts import Post
    return Post()

3. Use postponed type annotations.

from __future__ import annotations

class User:
    posts: list[Post]

4. Depend on interfaces rather than concrete modules.

The best fix is usually structural. Circular imports often reveal that module boundaries are poorly chosen.

40.38 Import and Type Checking

Type hints can create import cycles if annotations import runtime objects.

A common pattern is:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from posts import Post

class User:
    def add_post(self, post: "Post") -> None:
        ...

TYPE_CHECKING is false at runtime, so the import is visible to type checkers but skipped during execution.

This reduces runtime import cycles while keeping type information.

Modern Python also supports postponed annotation evaluation in several contexts, which further reduces runtime imports for typing.

40.39 Import Hooks

Import hooks let programs customize import behavior.

A custom finder can be placed on sys.meta_path.

A custom loader can create and execute modules from nonstandard sources.

Use cases include:

zip import
plugin systems
test isolation
sandboxed module loading
import tracing
encrypted module stores
remote module stores
generated modules

A minimal finder skeleton looks like:

class Finder:
    def find_spec(self, fullname, path=None, target=None):
        if fullname == "virtual_module":
            ...
        return None

A loader must implement the appropriate loader protocol, usually including module creation or execution.

Import hooks are powerful. They affect global program behavior, so they should be narrow and predictable.

40.40 importlib

importlib is the standard library interface to the import system.

Common operations:

import importlib

mod = importlib.import_module("json")
mod = importlib.reload(mod)

Useful lower-level pieces include:

importlib.util.find_spec
importlib.util.module_from_spec
spec.loader.exec_module
importlib.machinery.PathFinder
importlib.machinery.SourceFileLoader

Manual loading can look like this:

import importlib.util

spec = importlib.util.spec_from_file_location("custom_name", "/path/to/file.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

This creates and executes a module from a specific file.

For normal application code, prefer ordinary imports. Use manual importlib loading only when building tools, plugin systems, loaders, or runtime module systems.

40.41 Module Identity

Module identity is based on the key in sys.modules.

If the same file is imported under two different names, CPython can create two distinct module objects.

Example problem:

project/
    package/
        __init__.py
        settings.py

If one part of the program imports:

import package.settings

and another path manipulation causes:

import settings

then the same file may be loaded twice under different names.

That can duplicate module globals:

two registries
two singleton objects
two class identities
two caches

This is one reason direct sys.path manipulation can be dangerous.

40.42 Module Globals Are Shared State

A module-level variable is shared by all code that imports the module.

# state.py
count = 0

def increment():
    global count
    count += 1
    return count

Every importer observes the same module object:

import state

state.increment()
state.increment()

This is useful for constants, registries, caches, and singletons. It can also make testing harder because state persists across imports.

A test may need to reset module state explicitly:

import state

def test_increment():
    state.count = 0
    assert state.increment() == 1

Or isolate behavior by avoiding mutable module globals.

40.43 Import-Time Configuration

Modules often read configuration at import time:

import os

DEBUG = os.environ.get("DEBUG") == "1"

This makes configuration fixed at import time. If the environment changes later, DEBUG does not update automatically.

A more flexible design reads configuration when needed:

import os

def debug_enabled():
    return os.environ.get("DEBUG") == "1"

or centralizes configuration loading:

class Settings:
    def __init__(self):
        self.debug = os.environ.get("DEBUG") == "1"

settings = Settings()

Import-time configuration is simple, but it can surprise tests and long-running programs.

40.44 Module-Level __getattr__

Modules can define __getattr__ to customize attribute access for missing names.

# package/__init__.py

def __getattr__(name):
    if name == "heavy":
        from . import heavy
        return heavy
    raise AttributeError(name)

This can implement lazy exports.

Then:

import package

package.heavy

imports the heavy submodule only when requested.

Module-level __getattr__ is useful for compatibility shims, deprecations, and lazy loading. It should remain simple because it changes normal attribute access behavior.

40.45 Module-Level __dir__

A module can also define __dir__.

def __dir__():
    return ["Client", "Config", "connect"]

This controls what appears in:

dir(module)

It is mainly useful for modules with dynamic attributes.

40.46 The Import System During Startup

During CPython startup, the import system itself must be initialized carefully.

The runtime needs enough import machinery to load the standard library, but much of the import system is written in Python.

The bootstrap sequence uses built-in and frozen modules to bring up importlib.

Conceptually:

initialize runtime
initialize builtins and sys
initialize frozen importlib bootstrap code
configure import machinery
initialize sys.path
load site if enabled
start executing user code

This is why startup internals are more constrained than ordinary runtime imports.

40.47 site and Environment Setup

After core import machinery is initialized, CPython commonly imports the site module unless disabled with -S.

The site module configures additional import paths, including site-packages directories.

It may also process:

.pth files
user site-packages
virtual environment path adjustments
sitecustomize
usercustomize

This means the import environment at application start depends on interpreter flags, virtual environments, installation layout, and environment variables.

40.48 Virtual Environments and Imports

A virtual environment changes where Python looks for installed packages.

It usually changes:

sys.prefix
sys.exec_prefix
site-packages paths
script entry points

The interpreter binary may be shared or copied, but the import environment points to the virtual environment’s package directories.

This is why:

python -m pip install requests

inside a virtual environment makes requests importable only inside that environment.

The import system itself is the same. The search paths differ.

40.49 Importing From Zip Files

Python can import modules from zip archives if the archive is on sys.path.

Example:

python app.zip

or:

import sys

sys.path.insert(0, "modules.zip")
import mymodule

Zip import uses an importer that knows how to find module files inside the archive.

This demonstrates why import is path-entry based rather than only directory based. A sys.path entry can be handled by a custom path hook.

40.50 Path Hooks and Path Importers

For path-based imports, CPython uses path hooks to turn sys.path entries into importer objects.

Conceptually:

sys.path entry
    ↓
sys.path_hooks
    ↓
path importer
    ↓
find module spec

The cache sys.path_importer_cache stores importer objects for path entries.

import sys

print(sys.path_hooks)
print(sys.path_importer_cache)

This avoids rebuilding importer objects repeatedly.

40.51 Import Security Concerns

Import searches paths. That makes path order security-sensitive.

If the current directory appears before the standard library, a local file can shadow a standard module.

Example:

project/
    json.py

Then:

import json

may import the local json.py instead of the standard library json.

This can cause bugs or security problems.

Defensive practices:

avoid naming files after standard library modules
avoid unsafe sys.path insertion
run applications from expected working directories
use virtual environments
inspect module.__file__ when debugging
prefer python -m package.module for package code

To inspect what was imported:

import json

print(json.__file__)

40.52 Import and Testing

Tests often stress import behavior.

Common test issues include:

tests depend on working directory
local files shadow installed packages
package imported twice under different names
module global state leaks between tests
environment variables read at import time
plugins register themselves during import

A robust test setup imports the package the same way users do.

Prefer testing installed package behavior:

python -m pytest

from a clean environment, rather than relying on incidental path layout.

40.53 Import and Application Design

Good Python application structure reduces import complexity.

A common layout:

project/
    pyproject.toml
    src/
        app/
            __init__.py
            main.py
            config.py
            service.py
            storage.py
    tests/
        test_service.py

The src layout helps catch accidental imports from the repository root.

A clean module dependency graph points inward:

main
    depends on service
service
    depends on storage and config
storage
    depends on database driver
config
    depends on environment parsing

Avoid designs where low-level modules import high-level application entry points.

40.54 Import and Public API Design

A package’s import surface is part of its public API.

For example:

from library import Client

is a public contract if documented.

Changing the internal module location should not break users if the package facade preserves the public import:

# library/__init__.py
from ._client import Client

__all__ = ["Client"]

Private modules often use a leading underscore:

library/
    __init__.py
    _client.py
    _protocol.py
    public.py

This is a convention, not an access restriction.

40.55 Import Debugging Checklist

When import behavior is confusing, inspect these values:

import sys
import module

print(module)
print(module.__name__)
print(getattr(module, "__file__", None))
print(getattr(module, "__spec__", None))
print(getattr(module, "__package__", None))
print(sys.path)

For package issues:

import package

print(package.__path__)
print(package.__spec__.submodule_search_locations)

For cache issues:

import sys

print(sys.modules.get("module_name"))

For timing:

python -X importtime -c "import module_name"

For direct resolution:

import importlib.util

print(importlib.util.find_spec("module_name"))

40.56 Minimal Import Algorithm

A simplified import function can be written as:

def import_module(fullname):
    if fullname in sys.modules:
        return sys.modules[fullname]

    spec = find_spec(fullname)
    if spec is None:
        raise ModuleNotFoundError(fullname)

    module = module_from_spec(spec)
    sys.modules[fullname] = module

    try:
        spec.loader.exec_module(module)
    except Exception:
        del sys.modules[fullname]
        raise

    return module

The real CPython import system is more complex, but this skeleton captures the central flow:

cache lookup
spec discovery
module creation
cache insertion
module execution
error cleanup
return module

40.57 Common Failure: Partially Initialized Module

A common error looks like:

AttributeError: partially initialized module 'x' has no attribute 'y'

This often means a circular import or module shadowing problem.

Example circular import:

# a.py
import b

class A:
    ...

# b.py
import a

class B(a.A):
    ...

When b reads a.A, module a exists in sys.modules, but its class A has not yet been defined.

The fix is usually to restructure the modules so that class definitions do not require importing each other during top-level execution.

40.58 Common Failure: Shadowing

If a file is named after a standard library module, imports may resolve to the wrong file.

Example:

random.py

Inside it:

import random

This may import itself instead of the standard library random.

Symptoms include:

partially initialized module
missing expected attributes
recursive import behavior
strange module.__file__

Check:

import random
print(random.__file__)

Rename the local file and remove stale cache files if needed.

40.59 Common Failure: Running a Package File Directly

Given:

app/
    __init__.py
    main.py
    config.py

Inside main.py:

from .config import Settings

This may fail:

python app/main.py

because main.py is executed as __main__, not as app.main.

Use:

python -m app.main

from the directory containing app.

This preserves package context and makes relative imports work.

40.60 Key Points

A module is a runtime object with a namespace dictionary.

Importing a module executes its top-level code once per module name in sys.modules.

The import system is built around finders, loaders, module specs, path hooks, and caches.

Packages are modules with submodule search locations.

Circular imports expose partially initialized modules because CPython inserts modules into sys.modules before execution.

The import system is programmable through importlib, sys.meta_path, path hooks, and loaders.

Most import problems come from circular dependencies, path shadowing, direct execution of package files, import-time side effects, or duplicated module identity.