# 39. Closures and Cells

# 39. Closures and Cells

Closures let a nested function use variables from an enclosing function after that enclosing function has returned.

```python
def make_adder(n):
    def add(x):
        return x + n
    return add

add10 = make_adder(10)
print(add10(5))
```

Output:

```text
15
```

The variable `n` belongs to `make_adder`, but `add` still uses it later. CPython supports this by moving captured variables into cell objects. The inner function keeps references to those cells.

## 39.1 Nested Functions

A nested function is a function defined inside another function.

```python
def outer():
    def inner():
        return 1

    return inner
```

The `def inner` statement executes when `outer` runs. It creates a function object and binds it to the local name `inner`.

Calling `outer()` returns that function object:

```python
fn = outer()
print(fn())
```

The function object for `inner` contains:

```text
code object
globals dictionary
defaults
keyword defaults
annotations
closure cells, if needed
```

If `inner` does not refer to outer local variables, it does not need a closure.

## 39.2 Free Variables

A free variable is a variable used by a code object but defined in an enclosing scope.

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

Inside `inner`, `x` is a free variable.

You can inspect this:

```python
def outer():
    x = 10

    def inner():
        return x

    return inner

fn = outer()

print(fn.__code__.co_freevars)
```

Output:

```text
('x',)
```

The inner code object records that it needs `x` from outside.

## 39.3 Cell Variables

A cell variable is a local variable that must be captured by an inner function.

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

For `outer`, `x` is a cell variable because an inner function uses it.

Inspect:

```python
print(outer.__code__.co_cellvars)
```

Output:

```text
('x',)
```

The same variable is seen from two directions:

```text
outer:
    x is a cell variable

inner:
    x is a free variable
```

## 39.4 Why Cells Exist

A normal local variable lives in a frame slot.

```python
def f():
    x = 10
    return x
```

After `f` returns, its frame can be destroyed. Its local slots disappear.

But this cannot work for closures:

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

After `outer` returns, `inner` still needs `x`.

CPython solves this by storing `x` in a cell object.

Conceptually:

```text
outer frame
    x slot points to cell
        cell contains 10

inner function
    closure tuple points to same cell
```

When `outer` returns, the frame can disappear, but the cell remains alive because `inner` references it.

## 39.5 Cell Objects

A cell object is a small container that holds a reference to another Python object.

Conceptually:

```text
cell
    contents -> object
```

For:

```python
def outer():
    x = 10
    def inner():
        return x
    return inner
```

the closure looks like:

```text
inner function
    __closure__:
        cell for x
            contents: 10
```

Inspect it:

```python
fn = outer()

print(fn.__closure__)
print(fn.__closure__[0].cell_contents)
```

Output shape:

```text
(<cell at 0x...: int object at 0x...>,)
10
```

The cell, not the whole outer frame, is what survives.

## 39.6 Function Closure Tuple

A function object has a `__closure__` attribute.

```python
def outer():
    x = 10

    def inner():
        return x

    return inner

fn = outer()

print(fn.__closure__)
```

`__closure__` is either `None` or a tuple of cell objects.

For this example:

```text
fn.__closure__ -> (cell_for_x,)
```

The order of cells corresponds to `fn.__code__.co_freevars`.

```python
print(fn.__code__.co_freevars)
for cell in fn.__closure__:
    print(cell.cell_contents)
```

Output:

```text
('x',)
10
```

## 39.7 Reading Closure Metadata

Use this example:

```python
def outer(a):
    b = 2

    def inner(c):
        return a + b + c

    return inner

fn = outer(10)

print(outer.__code__.co_cellvars)
print(fn.__code__.co_freevars)
print(fn.__closure__)
```

Output shape:

```text
('a', 'b')
('a', 'b')
(<cell ...>, <cell ...>)
```

The outer function has cell variables `a` and `b`.

The inner function has free variables `a` and `b`.

The returned function carries cells for both.

## 39.8 `LOAD_DEREF`

Closure variables are accessed with dereference bytecode.

For:

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

`inner` does not use `LOAD_FAST` for `x`. It uses closure access.

Conceptually:

```text
LOAD_DEREF x
RETURN_VALUE
```

`LOAD_DEREF` reads the contents of a cell.

Similarly, storing into a closure variable uses a dereference-oriented store instruction.

## 39.9 Disassembling a Closure

Use `dis`:

```python
import dis

def outer():
    x = 10

    def inner():
        return x

    return inner

dis.dis(outer)

fn = outer()
dis.dis(fn)
```

Look for:

```text
MAKE_CELL
LOAD_CLOSURE
MAKE_FUNCTION
LOAD_DEREF
STORE_DEREF
```

Exact instructions vary by CPython version, but the conceptual operations are stable:

```text
create cell for captured local
create inner function with closure
load from closure cell inside inner
```

## 39.10 Closure Creation

When CPython creates a function object for a nested function, it attaches closure cells if the nested function needs free variables.

Conceptually:

```text
outer executes
    create cell for x
    load inner code object
    load closure cell for x
    make function object with closure
    return function object
```

For:

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

the returned function has:

```text
inner.__code__
inner.__globals__
inner.__closure__ = (cell_for_x,)
```

That closure tuple is how `inner` sees `x`.

## 39.11 Closures Capture Variables, Not Values

Closures capture variables through cells, not snapshots of values.

```python
def outer():
    x = 1

    def inner():
        return x

    x = 2
    return inner

fn = outer()
print(fn())
```

Output:

```text
2
```

The inner function sees the current contents of the cell. The cell was updated before `outer` returned.

Conceptually:

```text
cell x initially contains 1
cell x later contains 2
inner reads cell x
```

## 39.12 Shared Cells

Multiple inner functions can share the same cell.

```python
def outer():
    x = 0

    def get():
        return x

    def set_value(v):
        nonlocal x
        x = v

    return get, set_value

get, set_value = outer()

print(get())
set_value(10)
print(get())
```

Output:

```text
0
10
```

Both functions reference the same cell for `x`.

```text
get.__closure__[0]       -> cell x
set_value.__closure__[0] -> same cell x
```

The cell provides shared mutable binding.

## 39.13 `nonlocal`

`nonlocal` tells the compiler that assignment should target an enclosing function variable, not create a new local.

```python
def outer():
    x = 0

    def inc():
        nonlocal x
        x += 1
        return x

    return inc
```

Without `nonlocal`, assignment makes `x` local to `inc`:

```python
def outer():
    x = 0

    def bad():
        x += 1
        return x

    return bad
```

Calling `bad()` raises `UnboundLocalError` because `x += 1` tries to read local `x` before it has a value.

With `nonlocal`, CPython emits dereference operations against the outer cell.

## 39.14 `global` vs `nonlocal`

`global` targets the module namespace.

```python
x = 0

def f():
    global x
    x = 10
```

`nonlocal` targets an enclosing function scope.

```python
def outer():
    x = 0

    def inner():
        nonlocal x
        x = 10
```

Comparison:

| Declaration | Target |
|---|---|
| `global x` | Module global dictionary |
| `nonlocal x` | Nearest enclosing function scope with `x` |
| no declaration with assignment | Current local scope |

`nonlocal` cannot target a module global. It requires an enclosing function binding.

## 39.15 Closure and Scope Analysis

The compiler determines closure layout during symbol table analysis.

For each code block, it classifies names as:

```text
local
global explicit
global implicit
free
cell
```

Example:

```python
def outer():
    x = 1

    def inner():
        return x
```

Classification:

```text
outer:
    x = local, promoted to cell
    inner = local

inner:
    x = free
```

This classification decides which bytecode instructions are emitted.

```text
local variable    -> LOAD_FAST / STORE_FAST
global name       -> LOAD_GLOBAL / STORE_GLOBAL
closure variable  -> LOAD_DEREF / STORE_DEREF
```

## 39.16 Closure Lifetime

A cell lives as long as something references it.

Common owners:

```text
function closure tuple
active frame
generator frame
coroutine frame
another cell or object graph
```

Example:

```python
def outer():
    x = [1, 2, 3]

    def inner():
        return x

    return inner

fn = outer()
```

The list remains alive:

```text
fn
    __closure__
        cell
            list [1, 2, 3]
```

When `fn` becomes unreachable, the closure tuple and cell can be released, and the list can be released if no other references exist.

## 39.17 Closures Do Not Usually Keep Whole Frames Alive

A common misunderstanding is that a closure keeps the entire outer frame alive.

Usually it does not.

```python
def outer():
    a = "captured"
    b = "not captured"

    def inner():
        return a

    return inner
```

The returned function needs `a`, but not `b`.

Conceptually:

```text
inner keeps cell for a
inner does not keep b
outer frame can be destroyed
```

This is more efficient than preserving the whole frame.

However, if a frame object itself is captured through introspection, then it can keep all locals alive.

## 39.18 Late Binding in Loops

Closures capture variables, not per-iteration values.

```python
funcs = []

for i in range(3):
    funcs.append(lambda: i)

print([f() for f in funcs])
```

Output:

```text
[2, 2, 2]
```

All lambdas close over the same variable `i`. After the loop ends, `i` is `2`.

This is late binding: the value is looked up when the function runs, not when it is created.

## 39.19 Capturing Current Values With Defaults

Use a default argument to capture the current value.

```python
funcs = []

for i in range(3):
    funcs.append(lambda i=i: i)

print([f() for f in funcs])
```

Output:

```text
[0, 1, 2]
```

Here, each lambda has its own default argument value.

This uses function defaults, not closure cells.

Conceptually:

```text
lambda i=0: i
lambda i=1: i
lambda i=2: i
```

## 39.20 Closures in Comprehensions

Comprehensions have their own scope, but closures inside them can still show late binding.

```python
funcs = [lambda: x for x in range(3)]
print([f() for f in funcs])
```

Output:

```text
[2, 2, 2]
```

The loop variable `x` belongs to the comprehension scope, and all lambdas close over that same cell.

Use defaults to capture per-iteration values:

```python
funcs = [lambda x=x: x for x in range(3)]
print([f() for f in funcs])
```

Output:

```text
[0, 1, 2]
```

## 39.21 Closures and Mutability

A closure can reference a mutable object.

```python
def outer():
    xs = []

    def add(x):
        xs.append(x)
        return xs

    return add

add = outer()

print(add(1))
print(add(2))
```

Output:

```text
[1]
[1, 2]
```

No `nonlocal` is needed because the function mutates the list object. It does not rebind the name `xs`.

This works:

```python
xs.append(x)
```

This needs `nonlocal`:

```python
xs = xs + [x]
```

The second form reassigns the name.

## 39.22 Rebinding vs Mutating

Compare:

```python
def outer():
    xs = []

    def add(x):
        xs.append(x)

    return add
```

and:

```python
def outer():
    xs = []

    def add(x):
        xs = xs + [x]

    return add
```

The first mutates the object referenced by `xs`.

The second assigns to `xs`, so the compiler treats `xs` as local to `add` unless declared `nonlocal`.

Correct rebinding:

```python
def outer():
    xs = []

    def add(x):
        nonlocal xs
        xs = xs + [x]
        return xs

    return add
```

## 39.23 Closures and Function Factories

Closures are often used for function factories.

```python
def power(exp):
    def apply(x):
        return x ** exp
    return apply

square = power(2)
cube = power(3)

print(square(5))
print(cube(5))
```

Output:

```text
25
125
```

Each call to `power` creates a new cell for `exp`.

```text
square closure:
    exp = 2

cube closure:
    exp = 3
```

The code object for `apply` may be shared, but the closure cells differ.

## 39.24 Closures and Decorators

Decorators commonly use closures.

```python
def log_calls(fn):
    def wrapper(*args, **kwargs):
        print("calling", fn.__name__)
        return fn(*args, **kwargs)

    return wrapper
```

Use:

```python
@log_calls
def add(a, b):
    return a + b
```

Conceptually:

```python
def add(a, b):
    return a + b

add = log_calls(add)
```

The returned `wrapper` closes over the original `fn`.

```text
wrapper.__closure__
    cell for fn -> original add function
```

## 39.25 Closures and State

Closures can store private state.

```python
def counter():
    n = 0

    def inc():
        nonlocal n
        n += 1
        return n

    return inc

c = counter()

print(c())
print(c())
```

Output:

```text
1
2
```

The state `n` is not stored on an instance. It is stored in a closure cell.

This is similar to a small object with one method and private state.

## 39.26 Closure State vs Object State

Closure version:

```python
def counter():
    n = 0

    def inc():
        nonlocal n
        n += 1
        return n

    return inc
```

Object version:

```python
class Counter:
    def __init__(self):
        self.n = 0

    def inc(self):
        self.n += 1
        return self.n
```

Comparison:

| Feature | Closure | Object |
|---|---|---|
| State storage | Cell variables | Instance attributes |
| Multiple operations | Less convenient | Natural |
| Introspection | Less explicit | More explicit |
| Small callback state | Good fit | Good fit |
| Rich behavior | Awkward | Better |

Closures are best for small captured state. Classes are better for larger protocols.

## 39.27 Closures and Reference Cycles

Closures can participate in cycles.

```python
def outer():
    funcs = []

    def inner():
        return funcs

    funcs.append(inner)
    return inner

fn = outer()
```

Reference graph:

```text
inner function
    closure cell
        funcs list
            inner function
```

This cycle can be collected by CPython’s cyclic garbage collector when unreachable.

Cycles involving finalizers or external resources need more care.

## 39.28 Closures and Memory Retention

Closures can retain large objects.

```python
def make_reader():
    data = bytearray(100_000_000)

    def read():
        return data[0]

    return read

reader = make_reader()
```

The large `data` object remains alive as long as `reader` does.

Retention chain:

```text
reader function
    closure tuple
        cell
            large bytearray
```

If the closure no longer needs the large object, clear or avoid capturing it.

## 39.29 Avoiding Accidental Capture

This captures `self`:

```python
class C:
    def make_callback(self):
        def callback():
            return self.value
        return callback
```

The returned callback keeps the instance alive.

Sometimes this is intended. Sometimes it creates unexpected retention.

You can capture only the needed value:

```python
class C:
    def make_callback(self):
        value = self.value

        def callback():
            return value

        return callback
```

Now the callback keeps `value`, not necessarily the whole instance.

## 39.30 Inspecting Closure Contents

Use `__closure__` carefully:

```python
def outer():
    x = {"a": 1}

    def inner():
        return x

    return inner

fn = outer()

for name, cell in zip(fn.__code__.co_freevars, fn.__closure__):
    print(name, cell.cell_contents)
```

Output:

```text
x {'a': 1}
```

This is useful for learning and debugging, but production code should rarely depend on closure internals.

## 39.31 Empty Cells

A closure cell can be empty in some edge cases.

Example patterns involving deletion can produce empty cells:

```python
def outer():
    x = 1

    def inner():
        return x

    del x
    return inner
```

Calling:

```python
fn = outer()
fn()
```

raises an error because the cell no longer has contents.

Accessing `cell.cell_contents` for an empty cell raises `ValueError`.

## 39.32 Closures and `del`

Deleting a nonlocal variable clears the cell.

```python
def outer():
    x = 1

    def delete():
        nonlocal x
        del x

    def read:
        return x

    return read, delete
```

Corrected version:

```python
def outer():
    x = 1

    def delete():
        nonlocal x
        del x

    def read():
        return x

    return read, delete
```

Use:

```python
read, delete = outer()
print(read())
delete()
print(read())
```

The final `read()` raises because the cell is empty.

## 39.33 Closures and Defaults Are Different

Defaults are stored on the function object.

```python
def f(x=[]):
    return x
```

Closure cells are stored in `__closure__`.

```python
def outer():
    x = []

    def inner():
        return x

    return inner
```

Inspect:

```python
print(f.__defaults__)
print(outer().__closure__)
```

They solve different problems.

| Mechanism | Stores |
|---|---|
| Default argument | Value used when argument is omitted |
| Closure cell | Variable from enclosing scope |

The default-argument trick for late binding works because defaults are evaluated at function creation time.

## 39.34 Closures and Lambdas

`lambda` functions follow the same closure rules as `def`.

```python
def outer():
    x = 10
    return lambda y: x + y

fn = outer()
print(fn(5))
```

Output:

```text
15
```

The lambda has a free variable `x`.

```python
print(fn.__code__.co_freevars)
print(fn.__closure__[0].cell_contents)
```

A lambda is only a compact function expression. It does not have a special closure model.

## 39.35 Closures and Generators

A generator can close over variables.

```python
def outer(limit):
    def gen():
        for i in range(limit):
            yield i

    return gen

make = outer(3)
print(list(make()))
```

The generator function closes over `limit`.

When `make()` is called, it creates a generator object. That generator object can access the closure cell.

There are two layers:

```text
generator function
    closure cell for limit

generator object
    suspended frame when running
    references generator function/code/closure state
```

## 39.36 Closures and Coroutines

Async functions can close over variables too.

```python
def outer(delay):
    async def wait_then_return(value):
        await sleep(delay)
        return value

    return wait_then_return
```

The async function closes over `delay`.

Calling it creates a coroutine object. While suspended, the coroutine keeps its frame state, and the function carries closure cells.

This can retain captured objects across awaits.

## 39.37 Closure Performance

Accessing a closure variable is usually slower than accessing a fast local.

Fast local:

```text
LOAD_FAST
```

Closure variable:

```text
LOAD_DEREF
```

`LOAD_DEREF` must read through a cell.

In most code, this cost is minor. In very hot loops, local binding can matter.

Example:

```python
def outer(value):
    def inner(xs):
        v = value
        total = 0
        for x in xs:
            total += x + v
        return total
    return inner
```

Here, `v` is a fast local inside `inner`, while `value` is read from a cell once.

## 39.38 Closure Debugging

When debugging closure behavior, inspect:

```python
fn.__code__.co_freevars
fn.__closure__
cell.cell_contents
```

Example:

```python
def outer():
    x = 1
    y = 2

    def inner():
        return x + y

    return inner

fn = outer()

for name, cell in zip(fn.__code__.co_freevars, fn.__closure__):
    print(name, cell.cell_contents)
```

This shows which values the function can access through its closure.

## 39.39 Common Misunderstandings

| Misunderstanding | Correct model |
|---|---|
| Closures copy values | They capture variables through cells |
| A closure keeps the whole outer frame alive | Usually it keeps only needed cells |
| `nonlocal` means global | It targets an enclosing function scope |
| Mutating a captured list needs `nonlocal` | Only rebinding the name needs `nonlocal` |
| Lambdas have different closure rules | Lambdas and `def` use the same closure model |
| Loop closures capture each iteration value | They capture the loop variable |
| Defaults and closures are the same | Defaults store values; closures store cells |
| Closure variables are fast locals | They are accessed through cells |

## 39.40 Reading Strategy

Start with:

```python
def outer():
    x = 10

    def inner():
        return x

    return inner
```

Inspect:

```python
import dis

fn = outer()

print(outer.__code__.co_cellvars)
print(fn.__code__.co_freevars)
print(fn.__closure__)
print(fn.__closure__[0].cell_contents)

dis.dis(outer)
dis.dis(fn)
```

Then test:

```text
nonlocal rebinding
multiple inner functions sharing one variable
lambda inside loops
default-argument capture
closures holding large objects
closures inside generators
closures inside async functions
```

For each case, track:

```text
which scope binds the name
whether the name becomes a cell variable
which inner function sees it as a free variable
which function owns the closure tuple
how long the cell remains alive
```

## 39.41 Chapter Summary

Closures are CPython’s mechanism for letting nested functions access variables from enclosing function scopes. When an outer local variable is used by an inner function, CPython stores that variable in a cell. The inner function carries a closure tuple containing the needed cells.

The core model is:

```text
outer function local used by inner function
    ↓
local becomes a cell variable
    ↓
inner function records it as a free variable
    ↓
function object stores closure tuple
    ↓
cell remains alive after outer frame returns
    ↓
inner function reads or writes cell contents
```

Closures explain nested functions, decorators, function factories, callback state, `nonlocal`, late binding in loops, and memory retention through captured variables.