# Cross-Target Debugging ### Cross-Target Debugging Cross-target debugging means debugging a program built for a different machine, operating system, or CPU architecture than the one you are sitting at. For example, you may write Zig code on an x86_64 laptop, build it for `aarch64-linux`, copy it to a Raspberry Pi, and debug it from your main computer. The basic workflow is: ```text write code on host machine build for target machine run program on target machine connect debugger from host machine inspect program state ``` The host is your development machine. The target is the machine where the program runs. #### Why Cross-Target Debugging Exists Cross-compilation is useful because the target machine may be slow, small, remote, or inconvenient to use directly. This is common with: ```text Raspberry Pi boards ARM servers embedded Linux devices containers virtual machines remote production-like systems bare-metal boards WebAssembly runtimes ``` You may want to compile on a fast workstation but run on the real target. That creates a debugging problem: the executable is built on one machine but runs somewhere else. #### Build With Debug Information A debugger needs debug information. Without it, the debugger may only show raw addresses and assembly instructions. For learning, build in debug mode: ```bash zig build-exe main.zig -target aarch64-linux ``` Debug mode is the default when you do not pass a release optimization flag. Avoid this while debugging source-level behavior: ```bash zig build-exe main.zig -target aarch64-linux -O ReleaseFast ``` Optimized builds may reorder code, inline functions, remove variables, and make stepping confusing. Start with debug builds. Use release builds later when testing performance or final behavior. #### A Simple Remote Debugging Model A common model uses a debug server on the target and a debugger on the host. ```text host machine target machine ------------ -------------- lldb or gdb debug server + program ``` The target runs the program under a small debug server. The host debugger connects to it. For GDB-based workflows, the target may run: ```bash gdbserver :1234 ./main ``` Then the host connects: ```bash gdb ./main (gdb) target remote target-device:1234 ``` This lets the host debugger control a program running on the target. #### The Executable Must Match The debugger on the host should use the same executable that runs on the target. If the target runs one build but the debugger reads symbols from another build, line numbers and variables may be wrong. Good habit: ```text build once copy that exact binary to the target debug using that exact binary on the host ``` Do not rebuild between copying and debugging unless you copy the new binary too. #### Source Paths Matter Debug information usually records source file paths. If the debugger cannot find your source files, it may show addresses and function names but not source lines. This can happen when: ```text the target path differs from the host path the build happened in a container the source tree moved the debug info contains absolute paths from another machine ``` A debugger can often remap source paths, but the beginner rule is simpler: build from the same source directory you debug from when possible. #### Debugging an ARM Linux Program Suppose you have this program: ```zig const std = @import("std"); fn add(a: i32, b: i32) i32 { return a + b; } pub fn main() void { const result = add(20, 22); std.debug.print("result = {d}\n", .{result}); } ``` Build it for 64-bit ARM Linux: ```bash zig build-exe main.zig -target aarch64-linux ``` Copy it to the device: ```bash scp main user@device:/home/user/main ``` On the target device: ```bash chmod +x main gdbserver :1234 ./main ``` On the host machine: ```bash gdb ./main ``` Inside GDB: ```gdb target remote device:1234 break add continue ``` When the program reaches `add`, the debugger stops there. You can inspect variables: ```gdb print a print b next print result ``` This is the core debugging loop. #### LLDB LLDB is another debugger. It is commonly used on macOS and with LLVM-based toolchains. A typical LLDB workflow looks like: ```bash lldb ./main ``` Then inside LLDB: ```lldb breakpoint set --name main run ``` For remote targets, LLDB can connect to debug servers, but the setup depends on the target and platform. The important concept is the same: the debugger needs the binary, symbols, source paths, and a way to control the running process. #### Debugging Containers Containers are another cross-target case. You may build a Linux binary on your host, then run it inside a container. The operating system boundary is lighter than a separate machine, but debugging still has practical issues. Common problems include: ```text debugger not installed in the container container lacks permissions for ptrace binary path differs between host and container source paths differ shared libraries differ ``` For beginner work, the easiest method is often: ```text build locally mount the source directory into the container run the debugger inside the container ``` For more advanced work, run a debug server inside the container and connect from the host. #### Debugging WebAssembly WebAssembly debugging has a different shape. A Wasm module does not run directly on the operating system. It runs inside a host, such as a browser, Wasmtime, Node.js, or another runtime. That means you debug two things: ```text the Wasm module the host that loads and calls it ``` For a simple Zig Wasm function: ```zig export fn add(a: i32, b: i32) i32 { return a + b; } ``` Problems may occur in the Zig code, but they may also occur at the boundary: ```text wrong export name wrong argument types memory not exported pointer and length mismatch host reads memory incorrectly allocator ownership mismatch ``` For Wasm, inspect the interface carefully. Many bugs are not algorithm bugs. They are boundary bugs. #### Debugging Bare-Metal Targets Bare-metal debugging is lower-level than Linux or macOS debugging. There may be no process, no terminal, no filesystem, and no operating system. You often use hardware debugging tools such as: ```text JTAG SWD OpenOCD probe-rs vendor debug probes ``` The debugger talks to the chip through a hardware probe. In this world, you may inspect: ```text CPU registers memory addresses hardware registers stack pointer program counter interrupt state flash memory RAM ``` Source-level debugging may still be possible, but it depends on correct debug info, linker scripts, startup code, and target support. Bare-metal debugging requires more setup because the debugger needs to understand the exact chip and how to control it. #### Use Logging When Debugging Is Hard A debugger is useful, but logging is still valuable. For Linux-like targets, write to stderr: ```zig const std = @import("std"); pub fn main() !void { const stderr = std.io.getStdErr().writer(); try stderr.print("starting program\n", .{}); const value = 42; try stderr.print("value = {d}\n", .{value}); } ``` For embedded targets, logging may mean: ```text UART output LED blink patterns debug memory buffers semihosting RTT logs ``` A simple log can often find a problem faster than a full remote debugger setup. #### Check the Target Triple First Many cross-target debugging problems start with the wrong target. For example: ```bash zig build-exe main.zig -target x86_64-linux ``` will not run on an `aarch64-linux` device. Use the correct target: ```bash zig build-exe main.zig -target aarch64-linux ``` When something fails, verify: ```text CPU architecture operating system ABI libc choice dynamic library dependencies ``` A binary may fail to run because it was compiled correctly for Linux, but for the wrong CPU or wrong libc environment. #### Static vs Dynamic Linking Debugging can be affected by dynamic libraries. If your program depends on shared libraries, the target machine must have compatible versions installed. A program may fail before your code even starts if a required library cannot be loaded. Symptoms may look like: ```text No such file or directory error while loading shared libraries Exec format error Illegal instruction ``` These do not all mean the same thing. `Exec format error` often means the binary was built for the wrong architecture or operating system. A shared library error means the dynamic loader could not find or load a dependency. `Illegal instruction` may mean the CPU does not support an instruction used by the binary. #### Debug Build vs Release Build Debug builds are easier to inspect. Release builds are closer to production behavior. The difference matters. In debug mode: ```text variables are easier to inspect line stepping is clearer safety checks may be enabled performance is lower ``` In release mode: ```text code may be inlined variables may disappear instructions may be reordered performance is higher some safety checks may differ ``` When finding a logic bug, start with debug mode. When finding a performance bug, use release mode and profiling tools. #### Complete Example: Debug-Friendly Code This small example is easy to debug because the functions are separated and the values are visible: ```zig const std = @import("std"); fn multiply(a: i32, b: i32) i32 { return a * b; } fn compute(x: i32) i32 { const doubled = multiply(x, 2); const adjusted = doubled + 10; return adjusted; } pub fn main() void { const result = compute(16); std.debug.print("result = {d}\n", .{result}); } ``` Build for ARM Linux: ```bash zig build-exe main.zig -target aarch64-linux ``` Copy to target: ```bash scp main user@device:/home/user/main ``` Run with a debug server on the target: ```bash gdbserver :1234 ./main ``` Connect from the host: ```bash gdb ./main ``` Inside GDB: ```gdb target remote device:1234 break compute continue step print x print doubled print adjusted ``` This lets you inspect the program while it runs on the target machine. #### The Practical View Cross-target debugging is mostly about keeping four things aligned: ```text the binary the source code the target machine the debugger ``` The binary must match the target. The debugger must read the same binary that is running. The source paths must make sense. The target must provide a way to control or observe the program. For Zig beginners, start with simple debug builds and one target. Then add remote debugging. After that, learn container debugging, Wasm boundary debugging, and bare-metal hardware debugging.