58. `ctypes`

The ctypes module is CPython’s built-in foreign function interface. It lets Python code load shared libraries, call C functions, define C-compatible data types, pass pointers, receive callbacks from native code, and inspect or mutate raw memory.

ctypes is part of the standard library, but it sits very close to the C ABI. It can cross the normal safety boundary of Python. Correct use requires understanding memory layout, calling conventions, object lifetime, pointer ownership, and platform-specific binary interfaces.

58.1 The Role of ctypes

ctypes answers one central question:

How can Python call functions from a native shared library without writing a C extension module?

Example:

import ctypes

libc = ctypes.CDLL(None)

On Unix-like systems, CDLL(None) loads the current process image, which usually exposes symbols from the C runtime and already-loaded shared libraries.

A more explicit example:

import ctypes

libm = ctypes.CDLL("libm.so.6")
libm.cos.restype = ctypes.c_double
libm.cos.argtypes = [ctypes.c_double]

print(libm.cos(0.0))

ctypes is useful for:

calling small C libraries
prototyping extension boundaries
binding system APIs
loading plugins
testing native ABI behavior
working with memory buffers
interacting with legacy shared libraries

It is less suitable for large, performance-critical bindings where a C extension, Cython, cffi, pybind11, or a generated binding layer may be clearer and safer.

58.2 Shared Libraries

A shared library is a binary file loaded into a process at runtime.

Common formats:

ctypes loads libraries through loader objects:

Example:

import ctypes

lib = ctypes.CDLL("/usr/lib/libexample.so")

After loading, exported symbols can be accessed as attributes:

func = lib.some_function

or by indexing:

func = lib["some_function"]

The result is a callable foreign function object.

58.3 Function Signatures

A C function has a binary calling contract. Python must know how to convert arguments and return values.

ctypes uses two attributes:

func.argtypes
func.restype

Example C function:

double hypot(double x, double y);

Python binding:

import ctypes

libm = ctypes.CDLL("libm.so.6")

libm.hypot.argtypes = [ctypes.c_double, ctypes.c_double]
libm.hypot.restype = ctypes.c_double

print(libm.hypot(3.0, 4.0))

Without argtypes, ctypes guesses conversions. That can work for simple integers, but it is unsafe for pointers, floating-point values, structs, and platform-sensitive types.

A disciplined binding sets both argtypes and restype for every function.

58.4 Scalar C Types

ctypes provides Python classes for C scalar types.

Example:

import ctypes

x = ctypes.c_int(42)

print(x)
print(x.value)

ctypes scalar objects are mutable boxes around C-compatible storage:

x.value = 100

This differs from normal Python integers, which are immutable Python objects.

58.5 Python Values to C Values

When calling a foreign function, ctypes converts Python objects into C-compatible values.

Example:

import ctypes

libc = ctypes.CDLL(None)

libc.abs.argtypes = [ctypes.c_int]
libc.abs.restype = ctypes.c_int

print(libc.abs(-10))

The Python integer -10 is converted to a C int.

If the Python value does not fit the target type, behavior can be surprising. Always choose the C type that exactly matches the library header.

For example, size_t should usually be represented as:

ctypes.c_size_t

not c_int.

Pointer-sized integer types should use:

ctypes.c_void_p
ctypes.c_size_t
ctypes.c_ssize_t

depending on meaning.

58.6 Return Types

restype controls how native return values are converted back to Python.

Example:

func.restype = ctypes.c_int

means the raw C return value is interpreted as int.

For a pointer-returning function:

func.restype = ctypes.c_void_p

For a string pointer:

func.restype = ctypes.c_char_p

Care is required with c_char_p. It converts a char * return value to Python bytes by reading until a null byte. That is correct for borrowed null-terminated strings, but wrong for arbitrary binary buffers.

For raw addresses, use c_void_p.

58.7 Pointers

ctypes supports pointer types.

import ctypes

IntPtr = ctypes.POINTER(ctypes.c_int)

x = ctypes.c_int(42)
p = ctypes.pointer(x)

print(p.contents.value)

pointer(x) creates a pointer object pointing to x.

byref(x) passes a lightweight reference suitable for function calls:

lib.some_func(ctypes.byref(x))

Use byref() for output parameters when the pointer does not need to live independently.

Conceptually:

ctypes.c_int object
    owns C-sized storage

ctypes.byref(object)
    temporary pointer for call

ctypes.pointer(object)
    real pointer object keeping referent alive

58.8 Output Parameters

Many C APIs return data through pointer arguments.

C shape:

int parse(const char *text, int *out_value);

Python binding:

import ctypes

# lib.parse.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_int)]
# lib.parse.restype = ctypes.c_int

out = ctypes.c_int()

# rc = lib.parse(b"123", ctypes.byref(out))
# if rc == 0:
#     print(out.value)

The Python object owns storage for out. The C function writes into that storage through the pointer.

This pattern is common in system APIs.

58.9 Arrays

ctypes can create fixed-size C arrays.

import ctypes

IntArray4 = ctypes.c_int * 4

arr = IntArray4(1, 2, 3, 4)

print(arr[0])
print(arr[3])

The array stores contiguous C values, not Python object references.

Memory shape:

int[4]
+------+------+------+------+
|  1   |  2   |  3   |  4   |
+------+------+------+------+

Arrays can be passed to C functions expecting pointers to their first element.

58.10 Structures

C structs are modeled with ctypes.Structure.

C shape:

struct Point {
    int x;
    int y;
};

Python shape:

import ctypes

class Point(ctypes.Structure):
    _fields_ = [
        ("x", ctypes.c_int),
        ("y", ctypes.c_int),
    ]

p = Point(3, 4)

print(p.x)
print(p.y)

The _fields_ list defines memory layout.

Ordering matters. Field types matter. Platform ABI rules matter.

You can inspect layout:

print(ctypes.sizeof(Point))
print(Point.x.offset)
print(Point.y.offset)

The offsets should match the C compiler’s struct layout.

58.11 Padding and Alignment

C structs may contain padding bytes for alignment.

Example:

import ctypes

class Example(ctypes.Structure):
    _fields_ = [
        ("a", ctypes.c_char),
        ("b", ctypes.c_int),
    ]

print(ctypes.sizeof(Example))
print(Example.a.offset)
print(Example.b.offset)

Even though char is 1 byte and int is often 4 bytes, the struct may be 8 bytes or more because b must be aligned.

Conceptual layout on a common ABI:

offset 0: char a
offset 1: padding
offset 2: padding
offset 3: padding
offset 4: int b

Never guess struct layout. Match the C header and verify with sizeof and offsets when binding native libraries.

58.12 Packed Structures

Some C APIs use packed structs.

ctypes supports _pack_.

import ctypes

class Packed(ctypes.Structure):
    _pack_ = 1
    _fields_ = [
        ("a", ctypes.c_char),
        ("b", ctypes.c_int),
    ]

Packing changes alignment and field offsets.

Use _pack_ only when the C header explicitly specifies packed layout, such as with compiler pragmas or attributes.

Incorrect packing is a common cause of memory corruption.

58.13 Unions

C unions are modeled with ctypes.Union.

import ctypes

class Number(ctypes.Union):
    _fields_ = [
        ("i", ctypes.c_int),
        ("d", ctypes.c_double),
    ]

n = Number()
n.i = 10
print(n.i)

All fields share the same memory.

Conceptually:

same bytes interpreted as int or double

Unions are common in system APIs, binary formats, and tagged data structures. They must be paired with correct tag logic at the Python layer.

58.14 String Pointers

c_char_p represents char * interpreted as a null-terminated byte string.

import ctypes

s = ctypes.c_char_p(b"hello")

print(s.value)

For a function expecting const char *, c_char_p is often correct.

For writable buffers, use create_string_buffer():

import ctypes

buf = ctypes.create_string_buffer(100)

print(ctypes.sizeof(buf))

This allocates mutable C memory.

Example:

buf.value = b"hello"
print(buf.raw)

Distinguish carefully:

58.15 Buffers

ctypes can work with raw memory buffers.

import ctypes

buf = ctypes.create_string_buffer(b"abc")

print(buf.value)
print(buf.raw)

.value reads until the first null byte.

.raw returns the entire buffer.

For binary data, prefer .raw because null bytes may appear inside the data.

buf = ctypes.create_string_buffer(b"a\x00b", 3)

print(buf.value)
print(buf.raw)

Output shape:

b'a'
b'a\x00b'

58.16 Memory Addresses

ctypes.addressof() returns the memory address of a ctypes object.

import ctypes

x = ctypes.c_int(42)

addr = ctypes.addressof(x)
print(hex(addr))

You can cast addresses to pointer types:

p = ctypes.cast(addr, ctypes.POINTER(ctypes.c_int))
print(p.contents.value)

This is powerful and unsafe.

The address is only valid while the underlying object remains alive and unmoved. CPython does not move objects, but the ctypes storage object must still be kept alive by Python references.

58.17 Casting

ctypes.cast() reinterprets a pointer as another pointer type.

import ctypes

x = ctypes.c_int(0x41424344)

p = ctypes.pointer(x)
q = ctypes.cast(p, ctypes.POINTER(ctypes.c_ubyte))

print(q[0])

This is equivalent to C-style pointer reinterpretation.

It does not convert data. It changes how the same bytes are interpreted.

Incorrect casts can violate alignment, size, lifetime, or aliasing expectations of the native code.

58.18 Callbacks

C libraries sometimes call function pointers supplied by the caller. ctypes can create C-callable callbacks from Python functions.

Example shape:

import ctypes

CALLBACK = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int)

@CALLBACK
def py_callback(x):
    return x + 1

The callback object must be kept alive while C may call it.

Bad:

lib.register_callback(CALLBACK(lambda x: x + 1))

If Python garbage collects the callback object, C may later call a dangling function pointer.

Good:

callback = CALLBACK(py_callback)
lib.register_callback(callback)

# Keep callback referenced for as long as native code may use it.

Callbacks cross from C into Python, so they must reacquire interpreter state and interact with the GIL.

58.19 Error Handling

C APIs often signal errors through return values and global error state.

Unix-style APIs may set errno.

Windows APIs may set GetLastError().

ctypes supports this with loader options:

lib = ctypes.CDLL("libc.so.6", use_errno=True)

Then:

err = ctypes.get_errno()

For Windows:

lib = ctypes.WinDLL("kernel32", use_last_error=True)
err = ctypes.get_last_error()

A robust binding wraps native calls and converts errors into Python exceptions.

Example pattern:

def check_zero(result, func, args):
    if result == 0:
        err = ctypes.get_errno()
        raise OSError(err)
    return result

# func.errcheck = check_zero

errcheck lets you attach result validation to a foreign function.

58.20 errcheck

errcheck is called after a foreign function returns.

def check_negative(result, func, args):
    if result < 0:
        raise OSError("native call failed")
    return result

# lib.some_call.errcheck = check_negative

It receives:

result
function object
arguments

It may return a transformed result or raise an exception.

This is useful for centralizing error conversion and output parameter handling.

58.21 Ownership

Ownership is the hardest part of ctypes.

A C function may return:

borrowed pointer
newly allocated pointer
pointer to static storage
pointer owned by caller-provided buffer
pointer valid only until next call

The Python binding must know which case applies.

Example C API:

char *make_message(void);
void free_message(char *);

Python binding must call the matching free function:

ptr = lib.make_message()
try:
    ...
finally:
    lib.free_message(ptr)

Do not free memory unless the library contract says you own it. Do not ignore ownership when the contract says you must free it.

Mismatched allocation and freeing across runtimes can crash the process.

58.22 Lifetime

Python references must keep ctypes objects alive while native code uses their memory.

Example:

import ctypes

def make_pointer():
    x = ctypes.c_int(42)
    return ctypes.pointer(x)

p = make_pointer()
print(p.contents.value)

This works because the pointer object keeps the referent alive.

But raw addresses do not:

def make_address():
    x = ctypes.c_int(42)
    return ctypes.addressof(x)

addr = make_address()

After the function returns, x may be gone. The address is invalid.

The same applies to buffers, arrays, structs, and callbacks. Keep Python owner objects alive for as long as C might access their storage.

58.23 The GIL

Foreign calls through ctypes.CDLL normally release the GIL during the call. This lets other Python threads run while native code executes.

ctypes.PyDLL does not release the GIL and checks for Python exceptions after the call. It is intended for calling Python C API functions.

Use PyDLL when calling functions that require the GIL, especially CPython API functions.

Example concept:

pythonapi = ctypes.PyDLL(None)

Many CPython C API functions are exposed through:

ctypes.pythonapi

Calling CPython C API functions through ctypes is possible, but dangerous. Incorrect signatures or reference ownership mistakes can crash the interpreter.

58.24 ctypes.pythonapi

ctypes.pythonapi exposes the Python C API from the running process.

Example:

import ctypes

ctypes.pythonapi.Py_GetVersion.restype = ctypes.c_char_p

print(ctypes.pythonapi.Py_GetVersion())

This calls a CPython C API function.

This is useful for experiments and diagnostics, but should not replace proper extension modules for serious C API integration.

Problems include:

reference ownership mistakes
wrong signatures
GIL assumptions
ABI changes
borrowed references
object lifetime bugs
process crashes

58.25 Calling Conventions

A calling convention defines how arguments are passed, who cleans the stack, and how return values are delivered.

On many Unix-like platforms, ordinary C calls use one dominant ABI per architecture.

On Windows, different APIs may use different conventions.

ctypes separates these with loaders:

ctypes.CDLL
ctypes.WinDLL
ctypes.OleDLL

Using the wrong calling convention can corrupt the stack or crash the process.

58.26 Platform Sensitivity

ctypes code is often platform-sensitive.

Differences include:

library names
symbol names
integer sizes
long size
pointer size
struct alignment
calling convention
errno behavior
wide character representation
dynamic loader rules

Example:

ctypes.c_long

has different size on common 64-bit Unix and 64-bit Windows ABIs.

Use fixed-width or semantic types where possible:

ctypes.c_int32
ctypes.c_uint64
ctypes.c_size_t
ctypes.c_void_p

58.27 ctypes and the Buffer Protocol

ctypes arrays and buffers can often interoperate with Python’s buffer protocol.

Example:

import ctypes

arr = (ctypes.c_ubyte * 4)(1, 2, 3, 4)

view = memoryview(arr)
print(view.tolist())

This is useful for binary data, I/O buffers, and integration with libraries that accept buffer-like objects.

The buffer protocol provides a safer alternative to raw pointer manipulation when possible.

58.28 Relationship to C Extensions

ctypes and C extensions solve related but different problems.

ctypes calls existing shared libraries. A C extension creates a Python module compiled specifically for CPython.

58.29 Relationship to struct

struct packs and unpacks bytes according to C-like layouts.

ctypes creates live C-compatible memory objects.

Use struct for binary serialization:

import struct

data = struct.pack("<I", 42)
value = struct.unpack("<I", data)[0]

Use ctypes for ABI interaction:

import ctypes

x = ctypes.c_uint32(42)

They overlap conceptually, but the intended uses differ.

58.30 Common Failure Modes

ctypes failures can be severe.

Unlike normal Python errors, many ctypes mistakes can terminate the process.

58.31 Defensive Binding Pattern

A safer ctypes binding follows a strict pattern.

import ctypes

class NativeError(RuntimeError):
    pass

lib = ctypes.CDLL("libexample.so", use_errno=True)

lib.example_open.argtypes = [ctypes.c_char_p]
lib.example_open.restype = ctypes.c_void_p

lib.example_close.argtypes = [ctypes.c_void_p]
lib.example_close.restype = None

def _check_ptr(ptr):
    if not ptr:
        err = ctypes.get_errno()
        raise OSError(err, "example_open failed")
    return ptr

class Handle:
    def __init__(self, path: str):
        raw = lib.example_open(path.encode("utf-8"))
        self._ptr = _check_ptr(raw)

    def close(self):
        if self._ptr:
            lib.example_close(self._ptr)
            self._ptr = None

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc, tb):
        self.close()

    def __del__(self):
        self.close()

The important ideas:

declare signatures once
wrap raw pointers
centralize error checking
model ownership explicitly
provide close()
support context managers
avoid exposing raw addresses

58.32 Why ctypes Matters for CPython Internals

ctypes matters because it exposes the boundary between Python objects and native memory.

It shows how CPython can:

load shared libraries
convert Python values to C ABI values
represent C structs and arrays
pass pointers
call native functions
receive native callbacks
interact with platform loaders
call parts of the Python C API

It is also a reminder that CPython’s safety guarantees end at the foreign function boundary. Once Python code passes raw addresses to native code, memory safety depends on the external ABI contract.

58.33 Chapter Summary

The ctypes module is CPython’s standard foreign function interface. It can load shared libraries, call C functions, define C-compatible types, pass pointers, create callbacks, and manipulate raw memory.

For CPython internals, ctypes is important because it exposes the relationship between Python objects, native ABI values, dynamic libraries, C memory layout, callbacks, the GIL, and the Python C API. It is powerful, but incorrect signatures, ownership mistakes, and lifetime bugs can crash the interpreter.