# 42. The Import Lock

# 42. The Import Lock

The import lock is the synchronization machinery that prevents unsafe concurrent imports. In CPython, imports are not only name lookups. They may create module objects, mutate `sys.modules`, execute arbitrary Python code, initialize extension modules, update package attributes, compile source files, read bytecode caches, and run package initialization code.

Without locking, two threads could import the same module at the same time and observe inconsistent module state.

The import lock exists because import is execution, and execution mutates shared runtime state.

## 42.1 Why Imports Need Locking

Consider this module:

```python id="ez8v8a"
# cache.py

print("initializing cache")

items = {}

def get(key):
    return items[key]
```

Now consider two threads doing this at the same time:

```python id="2pklaf"
import cache
```

Without synchronization, both threads might:

```text id="oy5qmp"
create a module object
insert or overwrite sys.modules["cache"]
execute cache.py
initialize items twice
observe a partially initialized module
bind different module objects
```

The import system must ensure that only one thread initializes a given module at a time. Other threads should either wait or receive the already initialized module.

## 42.2 Import Mutates Global Runtime State

Import touches shared state.

Important shared structures include:

```text id="yxk2bl"
sys.modules
sys.meta_path
sys.path
sys.path_hooks
sys.path_importer_cache
parent package attributes
module dictionaries
bytecode cache files
extension module state
```

The most important one is `sys.modules`.

```python id="ul20gj"
import sys

print(type(sys.modules))
```

`sys.modules` is a normal dictionary, but it is central to module identity. If concurrent imports corrupt module insertion or replacement, the runtime can observe duplicated modules or incomplete modules.

## 42.3 The Simple Mental Model

A simplified import with locking looks like this:

```python id="d8ycx5"
def import_module(name):
    lock = get_import_lock_for(name)

    with lock:
        if name in sys.modules:
            return sys.modules[name]

        spec = find_spec(name)
        module = module_from_spec(spec)
        sys.modules[name] = module

        try:
            spec.loader.exec_module(module)
        except Exception:
            del sys.modules[name]
            raise

        return module
```

The real implementation has more cases, but the central idea is:

```text id="dsb489"
for a given module name, only one thread should execute that module's initialization at a time
```

## 42.4 Per-Module Locks

Modern CPython uses module-specific import locks in the import machinery. The goal is to avoid serializing all imports unnecessarily while still preventing two threads from initializing the same module concurrently.

Conceptually:

```text id="r8nrgm"
import a        locks module name "a"
import b        locks module name "b"
import a again  waits for "a" if another thread is initializing it
```

This gives more concurrency than a single global import lock, while preserving safety for each module identity.

The lock key is the fully qualified module name:

```text id="ev5p7h"
json
json.decoder
email.message
package.submodule
```

Each name has its own import synchronization boundary.

## 42.5 Import Lock vs the GIL

The Global Interpreter Lock and the import lock solve different problems.

| Mechanism | Purpose |
|---|---|
| GIL | Protects interpreter execution and many internal object operations |
| Import lock | Prevents unsafe concurrent module initialization |

The GIL does not remove the need for import locking.

A module import can perform file I/O, execute arbitrary Python code, release the GIL in native code, or wait on other operations. Another thread can attempt the same import while the first import is still in progress.

The import lock protects the higher-level invariant:

```text id="u3ib58"
one module name should not be initialized concurrently by multiple threads
```

## 42.6 Import Is Reentrant

Imports can trigger more imports.

Example:

```python id="4hqnm0"
# app.py
import config
import server
```

```python id="d6chge"
# server.py
import logging
import socket
```

Importing `app` imports `server`, and importing `server` imports other modules.

The import lock machinery must support nested imports.

Conceptually:

```text id="k2va9e"
import app
    lock app
    execute app.py
        import server
            lock server
            execute server.py
                import socket
                    lock socket
                    execute socket.py
```

This is normal.

A thread that is importing one module must be allowed to import another module while the first module is still initializing.

## 42.7 Recursive Imports of the Same Module

A recursive import of the same module can happen through circular imports.

```python id="z153co"
# a.py
import b
x = 1
```

```python id="zkwdfu"
# b.py
import a
y = 2
```

When `a` imports `b`, and `b` imports `a`, the second import of `a` should not deadlock waiting for itself.

This is one reason import locking must track ownership and recursion carefully.

The import system handles this by returning the partially initialized module from `sys.modules` when appropriate.

That prevents infinite recursion and self-deadlock, but it exposes partially initialized state.

## 42.8 Partially Initialized Modules

During import, CPython places the module in `sys.modules` before executing its code.

This supports circular imports.

For:

```python id="4jdauu"
# a.py
import b
value = 1
```

and:

```python id="ayh1bo"
# b.py
import a
print(a.value)
```

The module `a` exists in `sys.modules` before `value` is assigned. So `b` can import `a`, but `a.value` may not exist yet.

The import lock prevents two concurrent initializations. It does not make incomplete modules complete.

Different problems:

| Problem | Import lock helps? |
|---|---|
| Two threads initializing same module | Yes |
| Circular import observing incomplete module | No |
| Module shadowing | No |
| Bad `sys.path` order | No |
| Top-level side effects | Only prevents concurrent duplication |

The lock provides thread safety. It does not fix dependency structure.

## 42.9 Import Lock Acquisition Order

Imports form dependency chains. A module may hold its own import lock while importing another module.

Example:

```text id="m3czjg"
Thread 1:
    lock a
    import b
    wait for lock b

Thread 2:
    lock b
    import a
    wait for lock a
```

This is a classic deadlock shape.

CPython’s import machinery includes deadlock detection around module locks. When it detects a circular wait, it can avoid hanging forever and instead handle the partially initialized module path.

The important design point is that import locks must handle reentrant and cyclic dependency graphs.

## 42.10 Threaded Import Example

This example starts several threads that import the same module.

```python id="2bqtk9"
# slowmod.py
import time

print("slowmod start")
time.sleep(2)
value = 42
print("slowmod end")
```

```python id="kqyd0z"
# main.py
import threading

def worker():
    import slowmod
    print(slowmod.value)

threads = [threading.Thread(target=worker) for _ in range(5)]

for t in threads:
    t.start()

for t in threads:
    t.join()
```

Expected behavior:

```text id="mq93uj"
slowmod start
slowmod end
42
42
42
42
42
```

The module body executes once. Other threads wait for the import to complete, then use the cached module.

## 42.11 Importing Different Modules Concurrently

Per-module locks allow different modules to be imported concurrently when dependencies permit.

Thread 1:

```python id="jg44xg"
import alpha
```

Thread 2:

```python id="vyvgaf"
import beta
```

If `alpha` and `beta` are independent, their imports do not need to block on the same module lock.

However, they may still contend on:

```text id="4h21u8"
the GIL
file-system operations
path importer caches
extension module initialization
shared package parents
custom finder state
```

Import concurrency is possible, but import remains a complex global operation.

## 42.12 Parent Packages and Submodule Locks

For:

```python id="bbvi06"
import package.submodule
```

the import system must load `package` first.

Conceptually:

```text id="9mmvrw"
lock package
initialize package
lock package.submodule
initialize package.submodule
bind package.submodule attribute
```

If several threads import different children of the same package:

```python id="rp9s87"
import package.alpha
import package.beta
```

they may share the parent package initialization step.

Once the parent is initialized, children can be handled by their own module locks, subject to package path and finder behavior.

## 42.13 The Global Import Lock in `_imp`

CPython exposes low-level import lock functions through the `_imp` module.

```python id="8j3c5t"
import _imp

print(_imp.lock_held())
```

The `_imp` module includes functions such as:

```text id="iqvyee"
acquire_lock
release_lock
lock_held
```

These are low-level implementation interfaces. Normal Python code should not use them for application synchronization.

They exist for import machinery and compatibility.

Using them incorrectly can deadlock the process or interfere with the import system.

## 42.14 Import Locks and Custom Importers

Custom finders and loaders must assume they may be called under import synchronization.

A finder should avoid slow, blocking, or reentrant behavior when possible.

A loader’s `exec_module` method executes module code. It may import other modules.

Example loader shape:

```python id="viqfze"
class Loader:
    def create_module(self, spec):
        return None

    def exec_module(self, module):
        module.value = 42
```

If `exec_module` imports other modules, it participates in the same locking graph.

Custom importers should avoid acquiring unrelated locks in inconsistent orders. Otherwise they can create deadlocks outside CPython’s own import-lock logic.

## 42.15 Import Lock and `sys.modules`

The lock protects the critical section around module initialization.

Important operations include:

```text id="hu7f15"
checking sys.modules
creating the module
inserting the module
executing module code
removing module on failure
returning the initialized module
```

The most sensitive moment is insertion before execution.

```text id="fzh60s"
sys.modules[name] = module
exec_module(module)
```

This order supports circular imports, but it means other code can observe the module before execution finishes.

The lock ensures other threads importing the same name wait for completion instead of executing it again.

## 42.16 Import Failure and Lock Release

If import fails, locks must be released.

Example:

```python id="3sf7se"
# broken.py
raise RuntimeError("boom")
```

```python id="bj6o9l"
try:
    import broken
except RuntimeError:
    pass
```

After failure, another thread should not be blocked forever.

The import system must:

```text id="rhkz4v"
release the module lock
clean up sys.modules when appropriate
propagate the exception
allow future import attempts
```

A later import can try again:

```python id="36ur7b"
import broken
```

and will execute the module again unless a failed module object was intentionally left by special machinery.

## 42.17 Native Extension Modules

Extension modules complicate import locking.

An extension module may:

```text id="bc8bib"
run native initialization code
allocate process-global state
register types
import other modules
release the GIL
use static C variables
interact with subinterpreters
```

The import lock prevents concurrent initialization of the same extension module name, but extension authors still need to write initialization code carefully.

Modern multi-phase initialization helps extension modules store state per module object rather than only in C globals.

## 42.18 Subinterpreters

Subinterpreters add another dimension.

Each interpreter has its own module dictionary state for many modules, but native extension modules may still have process-global C state unless designed otherwise.

Import locking must be considered relative to interpreter state and extension state.

For extension authors, this means:

```text id="l90pr8"
avoid mutable process-global module state
prefer per-module state
support multi-phase initialization
consider subinterpreter isolation
```

The import lock prevents concurrent import races, but it does not automatically make extension module state isolated across interpreters.

## 42.19 Import Lock and Reload

`importlib.reload(module)` re-executes module code.

```python id="q3ox9f"
import importlib
import config

importlib.reload(config)
```

Reload must coordinate with import state because it mutates an existing module dictionary.

During reload, other code may still hold references to:

```text id="8a8ikh"
the module object
old functions
old classes
old constants
old imported names
```

The import lock can prevent simultaneous conflicting reload/import operations for the module name, but reload remains semantically tricky.

Reload does not update all external references.

## 42.20 Import Lock and Module State Visibility

The import lock controls import operations. It does not make module globals transactional.

If module code mutates global state during import:

```python id="uwbmge"
registry = []

registry.append("phase 1")
registry.append("phase 2")
ready = True
```

Code involved in circular imports may observe:

```text id="spya66"
registry exists
registry contains only phase 1
ready does not exist yet
```

The import lock prevents concurrent duplicate execution, but partially initialized modules are still visible through circular imports.

## 42.21 Avoiding Import-Time Races

Application code should minimize import-time mutation.

Prefer this:

```python id="1kciks"
# service.py

class Service:
    ...

def create_service(config):
    return Service(config)
```

Over this:

```python id="5wtu0a"
# service.py

config = load_config()
service = Service(config)
service.start()
```

The second form performs work at import time. It is harder to test, harder to reload, and more sensitive to import order.

Import-time work should usually be limited to definitions and cheap constants.

## 42.22 Safe Top-Level Code

Safe top-level module code usually includes:

```text id="xbk1bu"
imports
constants
class definitions
function definitions
small table definitions
cheap feature detection
type aliases
```

Riskier top-level code includes:

```text id="ippw6v"
network calls
database connections
thread startup
event loop startup
large file reads
global registration with side effects
process-wide configuration
monkey patching
```

The import lock serializes initialization, but expensive or side-effect-heavy imports still hurt startup and concurrency.

## 42.23 Import Lock and Dead Imports

A “dead import” is an import that waits for a module that cannot finish importing because of a dependency cycle or external lock.

Example:

```python id="jt6y2f"
# a.py
import threading
import b

lock = threading.Lock()
```

```python id="niivsa"
# b.py
import a
```

Simple circular imports are usually handled by partial module visibility. But if module code acquires external locks, starts threads, or waits for events during import, deadlocks become much easier to create.

Avoid waiting on threads or external locks at import time.

## 42.24 Custom Locks During Import

This is risky:

```python id="626g6o"
# registry.py
import threading

lock = threading.Lock()

with lock:
    import plugin
```

If `plugin` imports `registry` and tries to acquire the same lock, the program can deadlock.

Better design:

```python id="9ez7pd"
# registry.py
import threading

lock = threading.Lock()
handlers = {}

def load_plugin(name):
    import importlib
    module = importlib.import_module(name)
    return module

def register(name, handler):
    with lock:
        handlers[name] = handler
```

Keep application locks outside import dependency cycles where possible.

## 42.25 Import Lock and Plugin Systems

Plugin systems often import modules dynamically.

```python id="iixpre"
import importlib

def load_plugins(names):
    for name in names:
        importlib.import_module(name)
```

If plugins register themselves at import time, loading is serialized per module name but still mutates shared registries.

Safer plugin design separates import from registration:

```python id="d13tr6"
def load_plugin(name):
    module = importlib.import_module(name)
    return module.setup
```

Then call setup in a controlled phase:

```python id="p8oelv"
for setup in setups:
    setup(registry)
```

This makes initialization order explicit.

## 42.26 Import Lock and Startup Performance

The import lock can affect startup performance in threaded programs.

If many threads start and import dependencies lazily, several may block on the same imports.

A common improvement is to import major dependencies during single-threaded startup:

```python id="e6dmwr"
def main():
    import logging
    import database
    import server

    server.run()
```

This front-loads initialization before worker threads begin.

For server programs, the usual pattern is:

```text id="iq8ig9"
configure process
import dependencies
initialize application
start worker threads or event loop
```

Avoid making worker threads discover and import large dependency graphs independently.

## 42.27 Diagnosing Import Lock Problems

Symptoms of import-lock or import-cycle problems include:

```text id="m5z0iy"
program hangs during import
thread dump shows imports in multiple threads
partially initialized module errors
module attributes missing only during startup
plugin loading deadlocks
reload behaves inconsistently
```

Useful debugging tools:

```python id="1mccj7"
import sys

print(sys.modules.get("module_name"))
```

```python id="5cy0rx"
import _imp

print(_imp.lock_held())
```

For hangs, use a thread dump.

```python id="csvfsw"
import faulthandler

faulthandler.dump_traceback()
```

Or enable fault handler from the command line:

```bash id="7wjpm2"
python -X faulthandler app.py
```

## 42.28 Import Timing and Lock Contention

Import timing can be inspected with:

```bash id="gr9wta"
python -X importtime -c "import your_package"
```

This reports where time is spent during import.

It does not directly show lock contention, but it helps identify slow imports that may become contention points.

Slow imports usually deserve attention when they happen:

```text id="7c79oz"
inside worker startup
inside request paths
inside plugin discovery
inside command-line entry points
inside test collection
```

## 42.29 Import Lock and Async Code

Async code does not make imports asynchronous.

An import statement inside an `async def` still runs synchronously when that part of the coroutine executes.

```python id="fg7lak"
async def handler():
    import heavy_module
    return heavy_module.run()
```

The first call to `handler` that reaches the import may block the event loop while the module loads.

For async servers, import heavy dependencies before starting the event loop, or move expensive initialization into explicit async startup hooks.

## 42.30 Import Lock and Multiprocessing

Each process has its own interpreter state and its own import state.

With multiprocessing, child processes import modules separately.

This matters more on platforms or start methods that spawn fresh interpreters.

For example, with spawn-style process creation, the child process imports the main module.

This is why multiprocessing code needs:

```python id="w0j45j"
if __name__ == "__main__":
    main()
```

Without the guard, child processes may re-run top-level code during import.

The import lock only protects imports inside one process. It does not synchronize imports across processes.

## 42.31 Import Lock and Bytecode Cache Writes

When CPython imports source modules, it may read or write `.pyc` files.

Concurrent imports could otherwise race around bytecode cache generation.

The import machinery handles this carefully. Still, bytecode caches are an optimization, not the source of module identity.

The source of identity is:

```text id="ka9zk4"
module name in sys.modules
```

The bytecode cache only stores compiled code objects for faster later imports.

## 42.32 Import Lock and Filesystem Changes

The import lock does not make filesystem state stable.

If files are created, deleted, or replaced while imports are happening, behavior can still be confusing.

Examples:

```text id="50enmw"
deployment replacing package files during process startup
tests generating modules dynamically
plugin files being written while discovery runs
zip archive changed while importing
```

Use stable deployment methods. Avoid mutating importable code while a running process is importing it.

## 42.33 The Import Lock Is Not an Application Lock

Do not use import as a synchronization mechanism.

This is a poor pattern:

```python id="xdy9zr"
def initialize_once():
    import initialize_side_effects
```

It relies on import caching to run initialization once.

Prefer explicit one-time initialization:

```python id="q7vhsy"
_lock = threading.Lock()
_initialized = False

def initialize_once():
    global _initialized

    if _initialized:
        return

    with _lock:
        if _initialized:
            return
        do_initialization()
        _initialized = True
```

Import caching is for modules. Application lifecycle should be explicit.

## 42.34 Design Rule: Import Should Be Boring

The safest imports are boring.

A boring import:

```text id="tjyc8p"
defines names
sets constants
imports dependencies
does no external work
finishes quickly
```

A surprising import:

```text id="8vcz4t"
starts threads
opens sockets
loads large models
registers global plugins
patches builtins
changes logging globally
reads mutable external config
```

The import lock can make surprising imports less racy. It cannot make them easy to reason about.

## 42.35 Minimal Import-Lock Model

A compact model:

```text id="iwck6e"
Thread imports module M.
Import system acquires lock for M.
If M is already initialized, return it.
If M is initializing in another thread, wait.
If this thread is already involved in the cycle, use partial module state.
Create and cache M before execution.
Execute M.
On success, mark M initialized and release lock.
On failure, clean up and release lock.
```

This model explains why imports are thread-safe enough for normal use, why circular imports can still expose incomplete modules, and why import-time side effects remain dangerous.

## 42.36 Key Points

The import lock prevents concurrent initialization of the same module name.

The lock protects module loading, not arbitrary module semantics.

The GIL and import lock solve different problems.

Imports are reentrant because modules can import other modules during execution.

Circular imports are handled by inserting modules into `sys.modules` before execution, which can expose partially initialized modules.

Custom importers, plugins, extension modules, subinterpreters, reloads, and threaded startup all make import locking more important.

Good application design keeps imports fast, deterministic, and mostly free of external side effects.