34. Exception Handling

Exception handling is the control-flow system CPython uses when an operation cannot complete normally. It covers explicit raise statements, failed operations, failed imports, failed calls, generator termination, context manager cleanup, traceback construction, and propagation through frames.

At source level, exceptions look like this:

try:
    value = risky()
except ValueError:
    value = 0

At runtime, CPython must:

execute the protected bytecode range
detect failure
record the active exception
find a matching handler
restore the frame stack to a valid state
jump to handler bytecode
run cleanup code
propagate if no handler matches

Exceptions are not bolted onto the interpreter. They are integrated into the evaluation loop, frame state, thread state, bytecode metadata, and object model.

34.1 What an Exception Is

An exception is an object that represents abnormal control flow.

Most exceptions are instances of classes derived from BaseException.

raise ValueError("bad input")

This creates or uses an exception object and transfers execution out of the current normal path.

The exception hierarchy begins with:

BaseException
    SystemExit
    KeyboardInterrupt
    GeneratorExit
    Exception
        ValueError
        TypeError
        RuntimeError
        OSError
        ...

Most application-level exceptions derive from Exception, not directly from BaseException.

This matters because:

except Exception:
    ...

does not normally catch KeyboardInterrupt, SystemExit, or GeneratorExit.

34.2 Normal Return vs Exception Return

A Python function can exit in two main ways:

normal return
exception propagation

Normal return:

def f():
    return 42

Conceptually:

LOAD_CONST 42
RETURN_VALUE

Exception exit:

def f():
    raise ValueError("bad")

Conceptually:

create ValueError object
set exception state
unwind frame

A caller receives either:

a returned PyObject pointer
or an error indication with exception state set

At the C API level, many functions follow this shape:

PyObject *result = some_operation();
if (result == NULL) {
    /* exception is set */
    return NULL;
}

The NULL return signals failure. The actual exception is stored in interpreter thread state.

34.3 Exceptions as Control Flow

Exceptions are used for errors, but also for structured control flow.

Examples:

StopIteration ends iteration
StopAsyncIteration ends async iteration
GeneratorExit closes generators
KeyboardInterrupt interrupts execution
SystemExit requests interpreter exit
ImportError reports import failure
AttributeError reports missing attributes

The for loop depends on StopIteration internally:

for x in xs:
    body(x)

Conceptually:

iterator = iter(xs)

while true:
    try:
        x = next(iterator)
    except StopIteration:
        break
    body(x)

So exceptions are part of normal interpreter protocols.

34.4 Raising an Exception

A raise statement transfers control to the exception machinery.

raise ValueError("bad")

The bytecode roughly performs:

load ValueError
load "bad"
call ValueError("bad")
raise resulting exception

A raise can use:

raise SomeError
raise SomeError("message")
raise existing_exception
raise

A bare raise is valid only while handling an active exception:

try:
    risky()
except ValueError:
    raise

It reraises the currently handled exception.

34.5 Exception Classes and Instances

Python lets you raise an exception class or an exception instance.

raise ValueError

CPython normalizes this to an instance when needed.

raise ValueError("bad")

already provides an instance.

Internally, exception handling often needs a normalized triple or equivalent state:

exception type
exception value
traceback

Modern CPython represents and manages this state through internal exception structures, but the conceptual model remains useful.

34.6 Tracebacks

A traceback records where an exception traveled.

Example:

def a():
    b()

def b():
    c()

def c():
    1 / 0

a()

The traceback contains entries for active frames:

a
b
c
ZeroDivisionError

Each traceback entry refers to:

frame
code object
instruction position
source line information
next traceback entry

A traceback is structured runtime data, not just formatted text.

This is why exceptions can retain local variables indirectly:

exception
    traceback
        frame
            locals

34.7 Frame Unwinding

When an exception is not handled in the current frame, CPython unwinds the frame and propagates the exception to the caller.

Example:

def f():
    raise ValueError

def g():
    f()

def h():
    g()

If no handler exists:

frame f raises
frame f unwinds
frame g receives exception
frame g unwinds
frame h receives exception
frame h unwinds
top level prints traceback

If a handler exists in g, propagation stops there:

def g():
    try:
        f()
    except ValueError:
        return 0

The exception handler becomes the new control-flow target.

34.8 Exception Tables

Modern CPython uses exception tables associated with code objects to describe protected regions and handlers.

A try statement:

try:
    risky()
except ValueError:
    recover()

compiles into:

bytecode for risky()
bytecode for handler
exception table mapping protected range to handler

The exception table records information such as:

protected bytecode start
protected bytecode end
handler target
stack depth to restore
handler kind

When an exception occurs, the interpreter uses the current instruction position to search the table for a handler.

This avoids maintaining some older block-stack machinery for normal execution and lets zero-cost exception regions avoid overhead when no exception occurs.

34.9 Stack Restoration

An exception may occur while temporary values are on the frame stack.

Example:

x = f(g(), h())

If h() raises, the stack may contain:

f
result_of_g

The call to f never happens. CPython must release temporary references and restore the stack to the depth expected by the exception handler.

Exception table metadata tells the interpreter what stack depth to restore before entering a handler.

This is essential for correctness. A handler must start with a known stack shape.

34.10 Matching an Exception

An except clause checks whether the active exception matches a type or tuple of types.

try:
    risky()
except ValueError:
    handle()

The handler matches if the exception is an instance of ValueError or a subclass.

Tuple matching:

except (ValueError, TypeError):
    handle()

matches either type.

The matching operation uses exception class relationships. It is not a string comparison.

34.11 Handler Order

Handlers are tested in source order.

try:
    risky()
except Exception:
    handle_general()
except ValueError:
    handle_value()

The ValueError handler is unreachable because ValueError is a subclass of Exception.

Correct ordering puts specific handlers first:

try:
    risky()
except ValueError:
    handle_value()
except Exception:
    handle_general()

The compiler does not generally reject unreachable exception handlers. The runtime follows the order given.

34.12 Binding the Exception Variable

An except ... as name clause binds the exception object.

try:
    risky()
except ValueError as exc:
    print(exc)

Inside the handler:

exc -> exception instance

After the handler, Python clears this binding to reduce reference cycles involving tracebacks.

Conceptually:

except ValueError as exc:
    ...
finally:
    del exc

This prevents a common retention cycle:

exception
    traceback
        frame
            locals
                exception

34.13 Bare except

A bare handler catches almost everything:

try:
    risky()
except:
    handle()

It catches exceptions derived from BaseException, including KeyboardInterrupt and SystemExit.

This is usually too broad.

Prefer:

except Exception:
    handle()

when handling ordinary application errors.

A bare except may still be appropriate when code must perform cleanup and then reraise:

try:
    risky()
except:
    cleanup()
    raise

34.14 else Blocks

A try statement can have an else block.

try:
    value = parse()
except ValueError:
    value = default
else:
    use(value)

The else block runs only if the try block completed without an exception.

It does not run if:

the try block raises
the try block returns
the try block breaks
the try block continues

At bytecode level, this is normal branch control flow around the handler block.

34.15 finally Blocks

A finally block runs when control leaves the try block.

try:
    risky()
finally:
    cleanup()

The cleanup runs for:

normal fallthrough
return
exception
break
continue

Example:

def f():
    try:
        return 1
    finally:
        print("cleanup")

The return is pending while the finally body runs.

If the finally body raises, it replaces the pending return:

def f():
    try:
        return 1
    finally:
        raise RuntimeError("cleanup failed")

This function raises RuntimeError instead of returning 1.

34.16 return Inside finally

A return inside finally overrides earlier exceptions or returns.

def f():
    try:
        raise ValueError("bad")
    finally:
        return 10

This returns 10. The ValueError is suppressed.

This behavior is legal but dangerous. It can hide errors.

At runtime, the finally block controls the final exit path. If it returns, that return becomes the function’s outcome.

34.17 Context Managers and Exceptions

A with statement is built on exception handling.

with manager as value:
    body(value)

Conceptually:

mgr = manager
exit = mgr.__exit__
value = mgr.__enter__()
try:
    body(value)
except BaseException as exc:
    suppress = exit(type(exc), exc, exc.__traceback__)
    if not suppress:
        raise
else:
    exit(None, None, None)

If __exit__ returns a true value, the exception is suppressed.

This is how context managers can implement transaction rollback, file closing, locks, temporary state, and resource cleanup.

34.18 with Bytecode

A with statement compiles into bytecode that:

loads the context manager
calls __enter__
stores the as-target
executes the body
calls __exit__ on normal exit
calls __exit__ on exceptional exit
suppresses or reraises based on return value

The interpreter must keep enough state to call __exit__ even if the body raises.

This makes with a structured form of try/finally plus exception suppression.

34.19 Chained Exceptions

Python records exception context.

Example:

try:
    int("x")
except ValueError:
    raise RuntimeError("parse failed")

The RuntimeError has a context pointing to the original ValueError.

Traceback output says something like:

During handling of the above exception, another exception occurred

Explicit chaining uses from:

raise RuntimeError("parse failed") from exc

Suppressing context uses:

raise RuntimeError("parse failed") from None

Exception objects have fields such as:

__context__
__cause__
__suppress_context__
__traceback__

34.20 Exception State in Thread State

The active exception is stored in interpreter thread state.

This is why C functions can signal failure by returning NULL while leaving the exception elsewhere.

Conceptually:

thread state
    current exception
    handled exception stack

When a C helper fails:

PyErr_SetString(PyExc_ValueError, "bad");
return NULL;

The caller checks the return value and knows an exception is set.

The evaluation loop then propagates or handles it.

34.21 C API Error Convention

CPython C API functions usually follow one of several error conventions.

Pointer-returning functions:

PyObject *obj = PyLong_FromString(text, NULL, 10);
if (obj == NULL) {
    return NULL;
}

Integer-returning functions:

int rc = PyObject_SetAttrString(obj, "x", value);
if (rc < 0) {
    return NULL;
}

Boolean-like checks may return 1, 0, or -1:

int ok = PyObject_IsTrue(obj);
if (ok < 0) {
    return NULL;
}

The error result and exception state must agree. Returning an error code without setting an exception is usually a bug.

34.22 Raising From C

C code raises Python exceptions by setting error state.

Example:

PyErr_SetString(PyExc_TypeError, "expected integer");
return NULL;

For formatted messages:

PyErr_Format(PyExc_ValueError, "bad value: %d", value);
return NULL;

For propagating an existing exception, C code returns the error sentinel without overwriting the exception.

This pattern lets errors move through many C functions until the evaluation loop finds a Python handler or exits to top level.

34.23 Clearing Exceptions

Sometimes C code intentionally handles an exception and clears it.

Conceptually:

PyObject *value = PyObject_GetAttrString(obj, "optional");
if (value == NULL) {
    if (PyErr_ExceptionMatches(PyExc_AttributeError)) {
        PyErr_Clear();
        value = default_value;
    }
    else {
        return NULL;
    }
}

Clearing the wrong exception can hide real bugs.

Python code has a similar pattern:

try:
    value = obj.optional
except AttributeError:
    value = default

The key rule is to catch only the exception you intend to handle.

34.24 Exceptions During Exception Handling

A handler can raise another exception.

try:
    risky()
except ValueError:
    recover_badly()

If recover_badly() raises, the new exception replaces the handler’s normal outcome while retaining context about the original exception.

A finally block can also raise during cleanup.

CPython must preserve enough state to build useful traceback chains.

34.25 Exceptions and Generators

Generators use exceptions for control.

A generator ends by raising StopIteration internally to the caller.

def gen():
    yield 1

g = gen()
next(g)
next(g)

The second next(g) raises StopIteration.

A return value inside a generator becomes the value attached to StopIteration.

def gen():
    return 42
    yield

g = gen()
try:
    next(g)
except StopIteration as exc:
    print(exc.value)

The output is:

42

Generator finalization uses GeneratorExit.

34.26 PEP 479 and Generators

A StopIteration accidentally raised inside a generator body is transformed into a RuntimeError.

Example:

def gen():
    raise StopIteration
    yield

This avoids subtle bugs where an accidental StopIteration silently terminates the generator.

The generator protocol still uses StopIteration at the boundary. The transformation applies inside generator execution.

34.27 Exceptions and Coroutines

Coroutines also use exception paths for cancellation and failure.

An awaited coroutine can complete normally:

return value

or fail:

raise exception

Cancellation is typically represented by an exception injected into coroutine execution.

The coroutine frame resumes with an exception rather than a normal value.

This means async frameworks rely deeply on CPython’s frame suspension, resumption, and exception propagation model.

34.28 Exception Groups

Modern Python includes exception groups for representing multiple exceptions together.

raise ExceptionGroup("many", [ValueError("a"), TypeError("b")])

They are handled with except*:

try:
    raise ExceptionGroup("many", [ValueError("a"), TypeError("b")])
except* ValueError as group:
    handle_values(group)
except* TypeError as group:
    handle_types(group)

This matters for concurrent programs where multiple tasks may fail at once.

The interpreter must split and match exception groups across except* handlers.

34.29 except*

except* differs from ordinary except.

Ordinary except chooses one handler for one active exception.

except* can split an exception group and route different sub-exceptions to different handlers.

Conceptually:

ExceptionGroup(ValueError, TypeError)
    except* ValueError receives ValueError subgroup
    except* TypeError receives TypeError subgroup

Any unmatched subgroup continues propagating.

This adds structured multi-error handling without discarding individual exception identities.

34.30 Traceback Retention

Tracebacks keep frames alive.

Example:

saved = None

def f():
    big = bytearray(100_000_000)
    raise RuntimeError

try:
    f()
except RuntimeError as exc:
    saved = exc

The saved exception may retain its traceback. The traceback retains the frame. The frame retains local variable big.

Retention chain:

saved exception
    traceback
        frame
            locals
                big bytearray

This is why long-lived exceptions can retain substantial memory.

34.31 Cleaning Tracebacks

You can avoid retention by not storing exceptions longer than needed, or by clearing traceback references.

try:
    f()
except RuntimeError as exc:
    handle(exc)
    exc = None

For explicit cleanup:

exc.__traceback__ = None

Use this carefully because tracebacks are useful for debugging.

The general rule is: storing exceptions stores context.

34.32 finally and Reference Cleanup

finally is often used to release resources.

resource = acquire()
try:
    use(resource)
finally:
    resource.close()

The preferred form for resource management is usually a context manager:

with acquire() as resource:
    use(resource)

Both forms depend on exception handling machinery to ensure cleanup runs when control leaves the protected block.

34.33 Exceptions and __del__

Exceptions raised in object finalizers are handled specially.

class C:
    def __del__(self):
        raise RuntimeError("bad finalizer")

An exception from __del__ cannot propagate normally to user code at the point where garbage collection occurs. CPython reports it through unraisable exception machinery.

This is why finalizers should avoid raising.

34.34 Unraisable Exceptions

Some exceptions occur where normal propagation is impossible.

Examples:

__del__ finalizers
weakref callbacks
some cleanup hooks
background finalization contexts

CPython reports these through unraisable exception handling, available via sys.unraisablehook.

This preserves diagnostic information without breaking impossible control-flow contexts.

34.35 Exceptions and Imports

Import failure uses exceptions.

import missing_module

raises ModuleNotFoundError.

An import can fail at several stages:

module search
loader creation
source reading
bytecode loading
module execution
submodule import
package initialization

If module execution raises, the import fails with that exception.

Because importing a module runs its top-level code, arbitrary exceptions can arise during import.

34.36 Exceptions and Attribute Lookup

Missing attribute lookup raises AttributeError.

obj.missing

But custom lookup can raise anything:

class C:
    @property
    def x(self):
        raise RuntimeError("failed")

This matters for getattr, hasattr, and dynamic frameworks.

hasattr catches AttributeError, not arbitrary exceptions.

34.37 Exceptions and Iteration

Iteration uses StopIteration.

Manual equivalent:

it = iter(xs)

while True:
    try:
        x = next(it)
    except StopIteration:
        break
    body(x)

At the bytecode level, loop instructions recognize iterator exhaustion and branch out of the loop.

This is a normal, expected exception path.

34.38 Exceptions and Pattern Matching

Pattern matching can use failure internally without exposing exceptions for ordinary non-matches.

match value:
    case {"x": x}:
        ...
    case _:
        ...

The matching engine must distinguish:

pattern does not match
operation raises a real exception

A failed pattern should move to the next case. A real exception should propagate.

34.39 Exceptions and Bytecode Instructions

Many bytecode instructions can raise.

Examples:

The evaluation loop must assume most instructions can fail.

34.40 Exception Safety in the Evaluation Loop

Every instruction implementation needs an error path.

Simplified:

PyObject *result = operation();
if (result == NULL) {
    goto error;
}
push(result);

The error path must:

preserve the active exception
release temporary references
restore stack state
find a handler or unwind
update traceback information
avoid clobbering unrelated exception state

This is one of the hardest parts of interpreter implementation.

34.41 Exceptions and Reference Counting

Exceptions are Python objects, so they are reference-counted.

Tracebacks, frames, and exception objects can form cycles:

exception
    traceback
        frame
            locals
                exception

CPython has special cleanup behavior around exception variables to reduce these cycles. The cyclic garbage collector can also collect unreachable cycles.

Still, storing exceptions can extend object lifetimes.

34.42 Exception Normalization

Internally, CPython often needs to normalize an exception so it has a concrete exception instance.

Input forms:

raise ValueError
raise ValueError("bad")

Both become an exception type and instance.

Normalization ensures later code can inspect:

exception class
exception instance
traceback
cause
context
notes

Exception normalization can itself fail if constructing the exception instance fails.

34.43 Exception Notes

Python exceptions can carry notes.

try:
    raise ValueError("bad")
except ValueError as exc:
    exc.add_note("while parsing config")
    raise

Notes provide additional diagnostic text in traceback output.

They are stored on the exception object and do not change matching behavior.

34.44 Syntax Errors

Syntax errors are exceptions too, but they arise before normal bytecode execution.

eval("if")

raises SyntaxError.

Compilation-phase exceptions include:

SyntaxError
IndentationError
TabError

These occur during parsing or compilation, before the evaluation loop executes the resulting code.

34.45 Memory Errors

Allocation failure raises MemoryError when CPython can report it.

Example operations that can allocate:

creating objects
building lists
creating strings
expanding dictionaries
constructing tracebacks
formatting exception messages

Exception handling itself may allocate, so the runtime must be careful when reporting low-memory conditions.

34.46 Keyboard Interrupts and Signals

KeyboardInterrupt is usually raised when the interpreter processes an interrupt signal such as Ctrl-C.

The signal is not handled at arbitrary machine instructions. CPython records pending signal state and checks at safe points in the evaluation loop.

When processed, the interpreter raises KeyboardInterrupt in the executing thread.

This is another example of exception machinery serving external control flow.

34.47 System Exit

sys.exit() raises SystemExit.

import sys
sys.exit(1)

If uncaught, the interpreter exits with the given status.

Because it is an exception, it can be caught:

try:
    sys.exit(1)
except SystemExit:
    print("caught")

This is why SystemExit derives directly from BaseException, so broad except Exception handlers do not usually suppress process exit.

34.48 Common Misunderstandings

34.49 Reading Strategy

To study exception handling, disassemble small examples.

Start with:

def f(x):
    try:
        return 10 / x
    except ZeroDivisionError:
        return 0

Then inspect:

import dis
dis.dis(f)

Also inspect exception table output when available:

dis.show_code(f)

Then test:

try/finally
try/except/else
with statements
raise from
bare raise
generators with return
ExceptionGroup and except*

For each case, track:

where the protected range begins
where the handler begins
what stack cleanup is required
which exception is active
whether the frame returns or unwinds
what traceback is retained

34.50 Chapter Summary

Exception handling in CPython is a structured control-flow system. It uses exception objects, thread-state exception storage, frame unwinding, traceback construction, exception tables, stack restoration, handler matching, and cleanup blocks.

The core model is:

operation fails
    ↓
exception state is set
    ↓
current frame looks for a handler
    ↓
handler found: restore stack and jump
    ↓
no handler: unwind frame and propagate
    ↓
top level: print traceback or terminate

Exceptions connect many parts of the runtime: bytecode execution, function calls, context managers, generators, coroutines, imports, attribute lookup, C API error conventions, signal handling, and memory management.

Understanding exceptions means understanding both error handling and a major part of Python’s control-flow machinery.