# 70. Calling Python From C

# 70. Calling Python From C

Calling Python from C means using the CPython C API to invoke Python objects from native code. The callable may be a function, method, class, instance with `__call__`, bound method, coroutine function, or any other object accepted by Python’s call protocol.

This operation appears in both extension and embedding code.

```text
extension module:
    Python calls C
        C calls Python callback

embedded application:
    C application starts Python
        C calls Python module function
```

At the C level, Python calls are ordinary object operations. A callable is a `PyObject *`. Arguments are Python objects. The result is a new `PyObject *`, or `NULL` if an exception occurred.

## 70.1 Basic Call Model

The simplest call shape is:

```c
PyObject *result =
    PyObject_CallObject(callable, args);
```

Where:

| Value | Meaning |
|---|---|
| `callable` | Python object to call |
| `args` | Tuple of positional arguments, or `NULL` |
| `result` | New reference on success |
| `NULL` | Failure with exception set |

Conceptually:

```text
C code
    owns callable reference
    builds argument tuple
    calls callable
    receives result object
    handles exception or decrefs result
```

This corresponds to Python code like:

```python
result = callable(*args)
```

## 70.2 Checking Callability

Before calling an object, native code can check:

```c
if (!PyCallable_Check(callable)) {
    PyErr_SetString(PyExc_TypeError, "object is not callable");
    return NULL;
}
```

This mirrors:

```python
callable(obj)
```

A callable may be:

```text
Python function
built-in function
bound method
class
callable instance
C extension function
functools.partial object
```

Do not assume the object is a function. Call the protocol.

## 70.3 Building Positional Arguments

`PyObject_CallObject` expects a tuple.

Example:

```c
PyObject *args = PyTuple_New(2);
if (args == NULL) {
    return NULL;
}

PyObject *a = PyLong_FromLong(10);
if (a == NULL) {
    Py_DECREF(args);
    return NULL;
}

PyObject *b = PyLong_FromLong(20);
if (b == NULL) {
    Py_DECREF(a);
    Py_DECREF(args);
    return NULL;
}

PyTuple_SET_ITEM(args, 0, a);  /* steals a */
PyTuple_SET_ITEM(args, 1, b);  /* steals b */

PyObject *result = PyObject_CallObject(callable, args);
Py_DECREF(args);

if (result == NULL) {
    return NULL;
}

return result;
```

The tuple owns the argument references after `PyTuple_SET_ITEM`.

## 70.4 Safer Argument Construction

`PyTuple_Pack` is simpler and safer for small argument lists.

```c
PyObject *a = PyLong_FromLong(10);
if (a == NULL) {
    return NULL;
}

PyObject *b = PyLong_FromLong(20);
if (b == NULL) {
    Py_DECREF(a);
    return NULL;
}

PyObject *args = PyTuple_Pack(2, a, b);

Py_DECREF(a);
Py_DECREF(b);

if (args == NULL) {
    return NULL;
}

PyObject *result = PyObject_CallObject(callable, args);
Py_DECREF(args);

return result;
```

`PyTuple_Pack` does not steal references. It increments references for inserted items. The caller must decref `a` and `b`.

## 70.5 Calling With No Arguments

For no arguments:

```c
PyObject *result = PyObject_CallNoArgs(callable);
```

This is clearer and avoids constructing an empty tuple manually.

Equivalent Python:

```python
result = callable()
```

Always check:

```c
if (result == NULL) {
    return NULL;
}
```

## 70.6 Calling With One Argument

For one positional argument:

```c
PyObject *arg = PyLong_FromLong(42);
if (arg == NULL) {
    return NULL;
}

PyObject *result = PyObject_CallOneArg(callable, arg);
Py_DECREF(arg);

if (result == NULL) {
    return NULL;
}

return result;
```

This avoids tuple construction in many paths.

Equivalent Python:

```python
result = callable(42)
```

## 70.7 Calling With Keyword Arguments

Use `PyObject_Call`.

```c
PyObject *args = NULL;
PyObject *kwargs = NULL;
PyObject *result = NULL;

args = PyTuple_Pack(1, path_obj);
if (args == NULL) {
    goto error;
}

kwargs = PyDict_New();
if (kwargs == NULL) {
    goto error;
}

if (PyDict_SetItemString(kwargs, "strict", Py_True) < 0) {
    goto error;
}

result = PyObject_Call(callable, args, kwargs);
if (result == NULL) {
    goto error;
}

Py_DECREF(args);
Py_DECREF(kwargs);
return result;

error:
Py_XDECREF(args);
Py_XDECREF(kwargs);
return NULL;
```

Equivalent Python:

```python
result = callable(path, strict=True)
```

`PyDict_SetItemString` does not steal references. `Py_True` is a borrowed singleton, and the dict takes its own reference internally.

## 70.8 `PyObject_CallFunction`

`PyObject_CallFunction` builds arguments from a format string.

```c
PyObject *result =
    PyObject_CallFunction(callable, "si", "name", 42);
```

Equivalent Python:

```python
result = callable("name", 42)
```

Common format units:

| Format | Meaning |
|---|---|
| `s` | C string to Python `str` |
| `i` | C int to Python `int` |
| `l` | C long to Python `int` |
| `d` | C double to Python `float` |
| `O` | Existing `PyObject *` |
| `N` | Existing `PyObject *`, steals reference |

Use this API carefully. It is compact, but ownership can become less obvious than explicit object construction.

## 70.9 Calling Methods

To call a named method:

```c
PyObject *result =
    PyObject_CallMethod(obj, "close", NULL);
```

Equivalent Python:

```python
result = obj.close()
```

With arguments:

```c
PyObject *result =
    PyObject_CallMethod(obj, "write", "s", "hello");
```

Equivalent Python:

```python
result = obj.write("hello")
```

For performance-sensitive code, avoid repeated string method lookup inside tight loops. Cache the method or use lower-level APIs carefully.

## 70.10 Attribute Lookup Before Calling

Calling a method manually has two steps:

```c
PyObject *method =
    PyObject_GetAttrString(obj, "process");
if (method == NULL) {
    return NULL;
}

PyObject *result =
    PyObject_CallOneArg(method, arg);

Py_DECREF(method);

if (result == NULL) {
    return NULL;
}

return result;
```

This mirrors:

```python
method = obj.process
result = method(arg)
```

The attribute lookup may run Python code through descriptors, properties, or custom `__getattribute__`.

## 70.11 Error Handling

If a call fails, the result is `NULL` and an exception is set.

```c
PyObject *result = PyObject_CallObject(callable, args);
if (result == NULL) {
    return NULL;
}
```

Propagating the error is often correct.

To print and clear it in embedding code:

```c
if (result == NULL) {
    PyErr_Print();
    return -1;
}
```

To fetch it:

```c
PyObject *type = NULL;
PyObject *value = NULL;
PyObject *tb = NULL;

PyErr_Fetch(&type, &value, &tb);
PyErr_NormalizeException(&type, &value, &tb);

/* inspect or translate */

Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(tb);
```

Extension functions normally propagate errors. Embedded applications often translate them into host diagnostics.

## 70.12 Result Ownership

A successful call returns a new reference.

```c
PyObject *result = PyObject_CallNoArgs(callable);
if (result == NULL) {
    return NULL;
}

/* use result */

Py_DECREF(result);
Py_RETURN_NONE;
```

If returning the result to Python, pass ownership upward:

```c
return result;
```

Do not decref before returning.

Incorrect:

```c
Py_DECREF(result);
return result;  /* use-after-free */
```

## 70.13 Callback Storage

Native objects often store Python callbacks.

Example structure:

```c
typedef struct {
    PyObject_HEAD
    PyObject *callback;
} HandlerObject;
```

Setter:

```c
static int
Handler_set_callback(HandlerObject *self, PyObject *value)
{
    if (value != NULL && !PyCallable_Check(value)) {
        PyErr_SetString(PyExc_TypeError, "callback must be callable");
        return -1;
    }

    Py_XINCREF(value);
    Py_XDECREF(self->callback);
    self->callback = value;
    return 0;
}
```

Deallocator:

```c
static void
Handler_dealloc(HandlerObject *self)
{
    Py_XDECREF(self->callback);
    Py_TYPE(self)->tp_free((PyObject *)self);
}
```

The object owns a strong reference to the callback.

## 70.14 Calling Stored Callbacks

```c
static PyObject *
Handler_fire(HandlerObject *self, PyObject *args)
{
    if (self->callback == NULL) {
        PyErr_SetString(PyExc_RuntimeError, "no callback set");
        return NULL;
    }

    return PyObject_CallObject(self->callback, args);
}
```

If the callback fails, the exception propagates naturally.

This gives Python code normal traceback behavior.

## 70.15 Borrowed References Around Calls

Calling Python can execute arbitrary code. That code can mutate containers, release references, run finalizers, import modules, trigger garbage collection, or call back into the extension.

This is unsafe:

```c
PyObject *item = PyList_GetItem(list, 0);  /* borrowed */

PyObject *result = PyObject_CallNoArgs(callback);
if (result == NULL) {
    return NULL;
}

/* item may no longer be valid */
```

Safer:

```c
PyObject *item = PyList_GetItem(list, 0);
if (item == NULL) {
    return NULL;
}

Py_INCREF(item);

PyObject *result = PyObject_CallNoArgs(callback);
if (result == NULL) {
    Py_DECREF(item);
    return NULL;
}

Py_DECREF(result);

/* item is still valid */

Py_DECREF(item);
Py_RETURN_NONE;
```

Before calling Python, own every object you need afterward.

## 70.16 Reentrant Calls

Calling Python from C may reenter your extension.

Example:

```text
C extension method starts
    calls Python callback
        callback calls same C extension object
            extension sees partially updated state
```

This can break invariants.

Guard state explicitly:

```c
if (self->running) {
    PyErr_SetString(PyExc_RuntimeError, "reentrant call");
    return NULL;
}

self->running = 1;

PyObject *result = PyObject_CallNoArgs(callback);

self->running = 0;
```

Use cleanup-safe patterns so flags are reset even on failure.

## 70.17 Cleanup-Safe Reentrancy Guard

```c
static PyObject *
run_callback(HandlerObject *self)
{
    PyObject *result = NULL;

    if (self->running) {
        PyErr_SetString(PyExc_RuntimeError, "reentrant call");
        return NULL;
    }

    self->running = 1;

    result = PyObject_CallNoArgs(self->callback);

    self->running = 0;

    if (result == NULL) {
        return NULL;
    }

    return result;
}
```

For more complex functions, use a single cleanup label:

```c
self->running = 1;

result = PyObject_CallNoArgs(self->callback);
if (result == NULL) {
    goto done;
}

/* more work */

done:
self->running = 0;
return result;
```

## 70.18 Calling Python From Native Threads

A native thread must hold the GIL before calling Python APIs.

```c
PyGILState_STATE state;

state = PyGILState_Ensure();

/* safe to call Python APIs */

PyGILState_Release(state);
```

Example:

```c
void
worker_thread_callback(PyObject *callback)
{
    PyGILState_STATE state;
    PyObject *result;

    state = PyGILState_Ensure();

    result = PyObject_CallNoArgs(callback);
    if (result == NULL) {
        PyErr_Print();
    } else {
        Py_DECREF(result);
    }

    PyGILState_Release(state);
}
```

The callback object must remain alive while the native thread may use it. Store an owned reference.

## 70.19 GIL and Long Native Work

If C code calls Python, then performs long native work, separate the phases.

```c
PyObject *result = PyObject_CallNoArgs(callback);
if (result == NULL) {
    return NULL;
}

/* convert result to native data while GIL is held */

Py_BEGIN_ALLOW_THREADS

long_native_compute();

Py_END_ALLOW_THREADS

Py_DECREF(result);
Py_RETURN_NONE;
```

Do not touch Python objects while the GIL is released.

## 70.20 Vectorcall

Modern CPython uses vectorcall for efficient calls.

The idea is to pass arguments as a C array rather than packing them into a tuple.

Conceptual shape:

```text
callable
args[0]
args[1]
...
nargs
kwargs names
```

This reduces temporary allocation.

Public APIs include helpers such as:

```c
PyObject_Vectorcall
```

where available for the target API level.

Vectorcall matters most in tight call-heavy paths. For ordinary extension code, `PyObject_CallNoArgs`, `PyObject_CallOneArg`, and `PyObject_Call` are often clearer.

## 70.21 Classes Are Callables

Calling a class constructs an instance.

```c
PyObject *obj =
    PyObject_CallObject((PyObject *)type, args);
```

Equivalent Python:

```python
obj = Type(*args)
```

This invokes the normal Python construction protocol:

```text
type.__call__
    tp_new / __new__
    tp_init / __init__
```

Do not bypass this unless you need low-level allocation semantics.

## 70.22 Coroutine Functions

Calling an async function does not run it to completion. It returns a coroutine object.

Python:

```python
coro = async_func()
```

C:

```c
PyObject *coro = PyObject_CallNoArgs(async_func);
if (coro == NULL) {
    return NULL;
}
```

The coroutine must then be awaited or scheduled by an event loop.

Embedding code that calls async Python must integrate with `asyncio` or another scheduler. Calling the function itself only creates the coroutine object.

## 70.23 Calling Python During Finalization

Avoid calling Python during interpreter shutdown unless you know the runtime is still valid.

During finalization:

```text
modules may be cleared
globals may be set to None
imports may fail
destructors may run in unusual order
threads may be stopping
```

Callbacks from C destructors are especially dangerous during shutdown.

A safe design avoids arbitrary Python calls from native finalizers.

## 70.24 Common Bugs

| Bug | Cause |
|---|---|
| Passing non-tuple to `PyObject_CallObject` | Incorrect argument construction |
| Forgetting to decref result | Leak |
| Decrefing result before return | Use-after-free |
| Holding borrowed reference across callback | Dangling pointer |
| Calling without GIL | Crash or corruption |
| Ignoring `NULL` result | Continuing after exception |
| Reentrant callback corrupts state | Missing guard |
| Calling async function expecting result | Got coroutine object |
| Using Python during shutdown | Finalization crash |

## 70.25 Practical Checklist

Before calling Python from C:

| Question | Required answer |
|---|---|
| Do I hold the GIL? | Yes |
| Is the object callable? | Check if uncertain |
| Are arguments valid Python objects? | Yes |
| Are borrowed references protected across the call? | Yes |
| Is `NULL` result handled? | Yes |
| Is returned reference decrefed or returned? | Yes |
| Can the callback reenter my code? | Guard if needed |
| Can the call happen during shutdown? | Avoid or handle explicitly |

## 70.26 Chapter Summary

Calling Python from C uses the same object protocol as Python code. A callable is a `PyObject *`, arguments are Python objects, and the result is a new reference or `NULL` with an exception set.

The main APIs are `PyObject_Call`, `PyObject_CallObject`, `PyObject_CallNoArgs`, `PyObject_CallOneArg`, `PyObject_CallFunction`, and `PyObject_CallMethod`.

The hard parts are not the call itself. The hard parts are ownership, error handling, borrowed-reference lifetime, reentrancy, GIL discipline, async behavior, and interpreter shutdown.