# NVIDIA GH100 LibOS Kernel v3.1.0 — Architecture Documentation

## Overview

**Binary**: `kernel_gh100.elf`
**Architecture**: RISC-V 64-bit, Little-Endian (RV64IMAFDC)
**Compiler**: GCC
**Image Base**: `ram:0xffffffff92fff000`
**Size**: ~197KB in memory, 199 functions, 598 symbols
**Source Tree**: `/gpu_drv/uproc/os/libos-v3.1.0/kernel/`

The LibOS kernel runs on the NVIDIA H100 (GH100) GPU's internal RISC-V falcon microcontroller. It provides a microkernel-style OS with memory management, task scheduling, IPC ports, and hardware drivers. The kernel boots from M-mode, delegates to S-mode, and manages partitions that run user-space tasks. All 199 functions have been renamed and recovered into 26 C source files (~10K lines) organized by subsystem.

---

## Memory Map

| Region | Virtual Address | Size | Purpose |
|--------|----------------|------|---------|
| Kernel Text | `0xffffffff92000000` | 16 MB | Code + read-only data |
| Kernel Data | `0xffffffffa3000000` | 16 MB | BSS, globals, pools |
| INTIO (MMIO) | `0xffffffff60000000` | 2 MB | Hardware registers |
| Global Page | `0xffffffff70000000` | 16 KB | Shared boot configuration |
| Page Tables | `0xffffffffa3010000` | 8 KB | Root page table pages |

### Physical Memory Layout
- **Free page list**: `0xffffffffa3017000` — linked list of 4KB pages
- **Object pool regions**: Separate 4KB/8KB regions per pool type
- **Interrupt descriptor table**: `0xffffffffa300b000` — 0x310 bytes
- **Handle table free lists**: `0xffffffffa300c2d8` — 0x400 bytes
- **Per-hart timer state**: `0xffffffffa300c6d8` — 0x20 bytes per hart

### CSR Address Spaces
- **Standard RISC-V CSRs**: `csreg:0x000-0xfff` (sstatus, sepc, scause, stval, sscratch, etc.)
- **NVIDIA Custom CSRs**: `csreg:0x5ca-0x5d1` (GDMA engine control, timer config)
- **PMP Registers**: `csreg:0x3a0-0x3ef` (64 PMP entries — fine-grained memory protection)

---

## Boot Flow

```
Reset Vector (ram:0xffffffff92fff000)
  │
  ├─► KernelInit (ram:0xffffffff93002444)
  │     │
  │     ├─► InitializeCsrRegisters()     — set scounteren for U-mode counter access
  │     ├─► FenceAll()                   — memory barrier
  │     ├─► Check boot hart == 0x30      — must boot from M-mode
  │     ├─► Check partition == ROOT (3)  — verify root partition
  │     │
  │     ├─► Parse WPR shadow descriptors (up to 9 regions)
  │     │     ├─► Type 0x02: identity-mapped at 0xc000... offset
  │     │     └─► Others: aperture-mapped at 0x8000000000000000 (2MB aligned)
  │     │
  │     ├─► Initialize object pools (16+ pools)
  │     │     ├─► KernelObjectPoolInitialize(size, poolId, destructor)
  │     │     └─► KernelObjectPoolRegisterRegion(pool, base, count)
  │     │     Object sizes are ((raw + 7) & ~7) + 8 (aligned + 8-byte refcount header)
  │     │
  │     ├─► KernelAllocateMemoryPool()   — claim physical memory via buddy allocator
  │     ├─► KernelMemorySetAllocate()    — create kernel memory set (PoolId_MemorySetNode)
  │     ├─► KernelMemorySetInsert()      — add physical node to set's rb-tree
  │     │
  │     ├─► AllocateZeroedPage()         — pop from g_freePages, zero-fill, return
  │     ├─► Initialize page tables       — fill 0xffffffffa3010000 with root PTE
  │     │
  │     ├─► KernelAllocateAddressSpace(FALSE, &kernelAS)  — kernel AS (is_user=0)
  │     ├─► KernelAddressSpaceRegister(kernelAS, 0xffffffff00000000, 0xfffff000)
  │     ├─► KernelAddressSpaceAllocate(kernelAS, &textMap,   TEXT_BASE,  TEXT_SIZE)
  │     ├─► KernelAddressSpaceAllocate(kernelAS, &dataMap,   DATA_BASE,  DATA_SIZE)
  │     ├─► KernelAddressSpaceAllocate(kernelAS, &intioMap,  INTIO_BASE, INTIO_SIZE)
  │     ├─► KernelAddressSpaceAllocate(kernelAS, &globalMap, GLOBAL_BASE, GLOBAL_SIZE)
  │     │
  │     ├─► KernelAddressSpaceMapContiguous(intioMap, ..., INTIO_PHYS, INTIO_SIZE, RW)
  │     ├─► KernelAddressSpaceMapContiguous(dataMap, ..., data_phys, DATA_SIZE, RW)
  │     ├─► Map identity apertures from WPR shadow descriptors
  │     │
  │     ├─► InterruptControllerInit(hart_id)  — per-hart interrupt setup
  │     ├─► Initialize scheduler (segment tree, priority queue)
  │     │
  │     ├─► Map global page (0xffffffff70000000) — shared config with bootloader
  │     ├─► KernelPortAllocate(&partitionParentPort)
  │     ├─► KernelRootfsMountFilesystem(image_phys)
  │     ├─► Search for "init.elf" in rootfs via StringCompare()
  │     │
  │     ├─► KernelElfMap(userAS, initElf, kernelMemSet)
  │     │     └─► Per PT_LOAD segment: Allocate → Map → GDMA copy → BSS zero-fill
  │     │
  │     ├─► Init task setup:
  │     │     ├─► KernelTaskHandleTableCreate(0x80, &handleTable)
  │     │     ├─► KernelTaskCreate(HIGH, userAS, memSet, entry, stack=0x8000, &TaskInit)
  │     │     ├─► KernelPortAllocate(&rootToken)
  │     │     ├─► KernelTaskRegisterObject(TaskInit, &rootTokenHandle, rootToken, 0, 0)
  │     │     └─► KernelTaskRegisterObject(TaskInit, &outHandle, parentPort, PORT_SEND, 0)
  │     │
  │     ├─► Kernel server setup:
  │     │     ├─► KernelTaskHandleTableCreate(8192, &serverHT)
  │     │     ├─► KernelTaskCreate(NORMAL, kernelAS, memSet, KernelServerEntry, &kernelServer)
  │     │     └─► KernelTaskRegisterObject(kernelServer, &portHandle, kernelServerPort, PORT_ALL, 0)
  │     │
  │     ├─► TaskResume(TaskInit)         — mark init task as runnable
  │     ├─► TaskResume(kernelServer)     — mark server task as runnable
  │     │
  │     ├─► GDMA configuration (if bounce buffer present)
  │     │     ├─► Validate bounce buffer size >= 0x2000
  │     │     ├─► KernelAddressSpaceAllocate + MapContiguous for bounce buffer
  │     │     └─► Set g_gdmaEnabled = TRUE, g_gdmaChannelCount = 5
  │     │
  │     ├─► Root filesystem setup
  │     │     ├─► IpiCheckMessages() — drain any pending IPIs
  │     │     ├─► KernelRootfsMountFilesystem(image_phys)
  │     │     └─► Walk directory entries, validate each
  │     │
  │     ├─► GDMA engine initialization (at end of init)
  │     │     ├─► Program NV_CSR_GDMA_ENGINE = 8 (enable)
  │     │     ├─► Set SRC_ADDR = rootfs_va | 1, DST_ADDR = rootfs_phys
  │     │     ├─► Set SIZE = rootfs_size, CTRL = 0xc0018 (start)
  │     │     └─► Enable GDMA interrupt: STATUS=1, ERROR=0
  │     │
  │     ├─► Boot hart scheduler init
  │     │     ├─► Write boot hart timer page to per-hart array
  │     │     ├─► g_schedulerDeadline = time + 1500000
  │     │     ├─► StartTaskByPartitionId(boot_hart_id)
  │     │     └─► SchedulerYield()
  │     │
  │     └─► Enter primary hart WFI loop
  │           ├─► Enable timer + external interrupts (stvec |= 0x220)
  │           ├─► If g_gdmaPending: GdmaTransfer()
  │           ├─► WaitForInterrupt()
  │           └─► Loop forever (process DMA + WFI)
  │
  └─► Running: init task + kernel server
```

---

## Subsystem Architecture

### Memory Management

The memory management subsystem is organized in five layers, from physical pages up to virtual address spaces:

```
┌─────────────────────────────────────────────────────────────────┐
│                      AddressSpace                                │
│  Red-black tree of VA mappings + root page table pointer        │
│  KernelAllocateAddressSpace / KernelAddressSpaceAllocate         │
│  KernelAddressSpaceMapContiguous / AddressSpaceMapRegion         │
├─────────────────────────────────────────────────────────────────┤
│                       MemorySet                                  │
│  VMA-like address range tracker using rb-tree                    │
│  KernelMemorySetAllocate / KernelMemorySetInsert                 │
│  MemorySetSplitNode / ReleaseAddressRange                        │
├─────────────────────────────────────────────────────────────────┤
│                      PageTable (Sv39)                            │
│  3-level: L2 (1GB) → L1 (2MB) → L0 (4KB)                       │
│  PageTableMapEntry / UnmapPageRange / FreePageTableEntry         │
│  AllocateZeroedPage / TranslateVirtualToPhysical                 │
├─────────────────────────────────────────────────────────────────┤
│                      ObjectPool (Slab)                           │
│  16+ fixed-size pools with free lists + destructors              │
│  KernelObjectPoolInitialize / KernelObjectPoolRegisterRegion     │
│  KernelMemorySetAllocate / KernelObjectPoolRelease               │
├─────────────────────────────────────────────────────────────────┤
│                    MemoryPool (Buddy)                            │
│  Physical page management with buddy allocator                   │
│  KernelAllocateMemoryPool / FindAddressSpaceNode                 │
│  SplitPageEntry / MergeAdjacentPages                             │
└─────────────────────────────────────────────────────────────────┘
```

#### Physical Memory — Buddy Allocator

`KernelAllocateMemoryPool` claims a physical address range and organizes it into per-order free lists (buddy system):

1. Validates 4KB alignment of `physical_start`
2. Calculates page table overhead: `pages_needed = (size / 4096 + 1023) / 1024`
3. Calls buddy allocator to populate per-order free lists
4. Buddy allocation rounds down to power-of-2, respects alignment via `(-current & current)`

`KernelObjectPoolRelease` uses buddy allocator for returning pages:
- `FindAddressSpaceNode` — locate the node in the rb-tree by address
- `SplitPageEntry` — split a higher-order block into two buddies
- `MergeAdjacentPages` — coalesce freed buddies, walks up order levels

#### Object Pools — Slab Allocator

Each object pool (`KernelObjectPoolInitialize`) manages fixed-size objects:

```
ObjectPool descriptor (+0x28 bytes):
  +0x00: uint64_t objectSize      — ((raw_size + 7) & ~7) + 8 (aligned + 8 for refcount)
  +0x08: void*    destructor      — cleanup callback when refcount hits zero
  +0x10: void*    freeList        — head of doubly-linked circular free list
  +0x18: void*    freeListTail    — tail of free list
  +0x20: uint32_t totalAllocated  — count of live objects
  +0x24: uint32_t peakAllocated   — high-water mark
```

Every allocated object has an 8-byte header at `(object - 8)` storing the reference count. `KernelPortClose` decrements this; when it reaches zero, the destructor is called and the object returns to the free list.

`KernelMemorySetAllocate` auto-expands the pool: when the free list is empty, it calls `KernelElfLoadSegment` to map a new page and registers it via `KernelObjectPoolRegisterRegion`.

#### Memory Sets — Virtual Address Range Tracker

Memory sets use a red-black tree augmented with `max_free_subtree` for O(log n) free-range lookups:

- `MemorySetSplitNode` — splits a VA range at an offset, creating 1–2 new nodes from `PoolId_Shuttle`, inserts into rb-tree
- `ReleaseAddressRange` — marks node as unallocated, unmaps pages via `UnmapPageRange`, merges with predecessor/successor if free, propagates `max_free_subtree` updates to root via `RbTreeUpdateAllAncestors`
- `ReserveAlignedAddressRange` — 2MB-aligned allocation using `(start + 0x1fffff) & ~0x1fffff`
- `FindFreeVirtualRange` — walks rb-tree using `max_free_subtree` to find a gap of requested size

#### Page Tables — Sv39

`PageTableMapEntry` performs a 3-level Sv39 walk:
- L2 index: bits 38–30 (1GB entries)
- L1 index: bits 29–21 (2MB entries)
- L0 index: bits 20–12 (4KB entries)
- Missing intermediate levels are allocated via `AllocateZeroedPage` (pops from `g_freePages` linked list, zero-fills, returns)
- `UnmapPageRange` walks the page table for a VA range and frees entries

`TranslateVirtualToPhysical` uses an aperture-based system with 9 descriptors and an LRU-cached index:
1. Check cached aperture index first (fast path)
2. Linear scan all 9 descriptors if cache miss
3. Each descriptor defines a VA→PA offset for a memory region

#### Address Spaces

`KernelAllocateAddressSpace(is_user, &as_out)` creates a new address space with an rb-tree for VA range tracking and a root page table.

`AddressSpaceMapRegion` is the full map operation:
1. Reserve VA range via `KernelAddressSpaceAllocate`
2. Transition page states via `KernelPageStateTransition`
3. Map physical pages via `KernelAddressSpaceMapContiguous`
4. Load ELF segment data if present (via `KernelElfLoadSegment`)
5. Error cleanup path: unmaps and releases on failure

```

### Task / Scheduling

#### Task Structure

Recovered from trap save/restore code and KernelTaskCreate:

```
Task (pool 0, size 0x30 + 8 header):
  +0x00:  RbNode  tree_node        — rb-tree linkage (next, prev, color)
  +0x20:  uint64_t phys_page       — physical page mapping
  +0x28:  void*    run_queue       — scheduler run queue
  +0x30:  ...      task_state fields (pool 0 sub-object)
  +0x60:  uint8_t  state           — TaskState: 0=ready, 1=running, 2=blocked
  +0x7c:  uint8_t  started         — 1 if task has been started
  +0x90:  uint64_t ra              — saved return address (trap frame start)
  +0x98:  uint64_t sp              — saved stack pointer
  +0xa0:  uint64_t gp              — saved global pointer
  +0xa8:  uint64_t tp              — saved thread pointer
  +0xb0:  uint64_t t0              — saved temporaries (t0-t2)
  +0xc8:  uint64_t s0/s1           — saved frame pointer / s1
  +0xd8:  uint64_t a0-a7           — saved arguments (8 × 8 bytes)
  +0x118: uint64_t s2-s11          — saved callee-saved (10 × 8 bytes)
  +0x168: uint64_t t3-t6           — saved temporaries (4 × 8 bytes)
  +0x188: uint64_t sepc            — saved exception PC
  +0x190: uint64_t padding
  +0x198: uint64_t stval           — saved trap value
  +0x1a0: uint64_t sstatus         — saved status
  +0x1a8: uint64_t flags           — bit 0-1: FPU state
  +0x1b0: uint64_t fpu_state[8]    — FPU register save area
  +0x210: uint64_t runtime         — accumulated runtime (shifted)
  +0x218: uint64_t runtime_acc     — runtime accumulator
  +0x220: uint64_t last_scheduled  — timestamp of last schedule
  +0x228: uint8_t  priority_idx    — index into priority queue
  +0x438: void*    handle_table    — pointer to handle table
```

Task creation parameters (from `KernelTaskCreate`):
- `priority` — LIBOS_PRIORITY_HIGH (0x40) or LIBOS_PRIORITY_NORMAL (0x80)
- `address_space` — which VA space the task runs in
- `memory_set` — memory set for page allocations
- `entry_point` — initial PC
- `is_privileged` — kernel vs user mode
- `stack_size` — 0x8000 for init task
- `handle_table` — pre-created handle table
- `inherit_parent` — whether to inherit parent handles

#### Scheduler

```
┌──────────────────────────────────────────────────────────────┐
│                  Segment Tree (Priority Queue)                │
│  O(log n) find-highest-priority-ready-task                    │
│  SegmentTreeFindIndex() returns the index of the              │
│  highest-priority non-empty run queue                         │
├──────────────────────────────────────────────────────────────┤
│                  Multiple Priority Queues                     │
│  g_timerQueue     (a300a3d0) — expired timer queue            │
│  g_portOpQueue    (a300ac30) — pending port operation queue   │
│  g_partitionQueue (a300a3b0) — partition start queue          │
│  g_runQueue       (a300a3c0) — ready-to-run task queue        │
│  g_priorityQueue  (a300a3e0) — main priority queue            │
│  Each backed by segment tree + slot array                     │
│  g_timerSlots[]   (a300ac40) — task ptrs for timer slots      │
│  g_portOpSlots[]  (a300b110) — task ptrs for port op slots    │
│  g_taskSlots[]    (a300b000) — task ptrs for main run queue   │
├──────────────────────────────────────────────────────────────┤
│                  Priority Queue Update                         │
│  PriorityQueueUpdate(heap, index, value)                       │
│  Sets run queue [index] to value, re-heapifies                │
│  value=0xffffffffffffffff marks queue as empty                 │
└──────────────────────────────────────────────────────────────┘
```

Scheduling flow:
1. `SchedulerYield` — process all expired queues, then call `ScheduleNextTask`
   - Walk `g_timerQueue`: for each expired timer, remove from queue, `TaskResume`
   - Walk `g_portOpQueue`: for each pending port op, remove and dispatch
   - Walk `g_partitionQueue`: for each pending partition start, invoke handler
2. `ScheduleNextTask` — find highest-priority ready task via `SegmentTreeFindIndex`
   - If no task ready: `WaitForInterrupt` (WFI idle loop)
   - On context switch: update runtime counters, `TrapReturnRestoreRegisters`
   - Dispatches partition IPIs for inter-hart coordination
   - Monitors watchdog timer (`g_hartWatchdogTime`) for hang detection
3. `TaskSwitchHandler` — perform context switch to selected task
4. `TrapReturnRestoreRegisters` — restore sstatus/sepc/registers, `sret`

Timer interrupts (`TimerInterruptHandler`) drive preemptive scheduling by:
1. Calling `HandleTableFlush` on current task
2. Invoking `KernelPortClose` on completed ports
3. Dispatching `SoftwareInterruptHandler` for pending software interrupts
4. Calling `SchedulerYield`

`TaskResume(task)` marks a task as runnable and inserts it into the appropriate priority run queue, then updates the segment tree via `PriorityQueueUpdate`.

`SetCurrentTaskState(deadline)` blocks the current task (state=2) with an optional timer deadline. If deadline != -1, the task is also added to the timer queue.

`GetTaskPriority(task)` returns `(runtime >> 8) | (priority << 56)` for ready tasks, -1 for blocked tasks.

```

### Interrupt Handling

```
Trap Entry: TrapEntrySaveRegisters (ram:930001bc)
  │
  ├─► Save all registers to current task struct (via sscratch swap)
  ├─► Update runtime counters (runtime += current_time - last_scheduled)
  │
  └─► Dispatch based on scause:
        │
        ├─► Interrupts (scause bit 63 set):
        │     │
        │     ├─► Timer (scause=0x80000005):
        │     │     TimerInterruptHandler
        │     │       ├─► UpdateNextTimerDeadline
        │     │       ├─► SchedulerYield (preemptive scheduling)
        │     │       └─► ScheduleNextTask
        │     │
        │     ├─► Software (scause=0x80000001):
        │     │     SoftwareInterruptHandler
        │     │       ├─► DispatchSoftwareInterrupts
        │     │       └─► HandleIpiInterrupt → IpiMessageHandler
        │     │
        │     └─► External (scause=0x80000009):
        │           ExternalInterruptHandler
        │             ├─► DispatchExternalInterrupts
        │             ├─► FindInterruptHandlerByMask
        │             ├─► MaskAndAcknowledgeInterrupt
        │             └─► GdmaInterruptHandler (DMA completion)
        │
        ├─► Exceptions (scause < 0x80000000):
        │     ExceptionVectorHandler
        │       ├─► SyscallHandler (ecall from U-mode)
        │       │     ├─► SyscallDispatch (jump table at 0x9300c3b0)
        │       │     ├─► HandleSyscallPortSend
        │       │     ├─► HandleSyscallPortReceive
        │       │     ├─► HandleSyscallWait
        │       │     └─► SyscallMapMemory / SyscallPartitionConfig / etc.
        │       │
        │       ├─► HandleFatalException (bad addr, illegal instr, etc.)
        │       │     ├─► WriteCrashDump
        │       │     └─► KernelPanic or task termination
        │       │
        │       └─► TaskSwitchHandler (scause=0x80000008 from sret)
        │             └─► Context switch to next scheduled task
        │
        └─► Trap Return: TrapReturnRestoreRegisters (ram:930002a2)
              ├─► Restore sstatus, sepc from task struct
              ├─► Restore tp (thread pointer)
              ├─► Restore all general-purpose registers
              └─► sret (return from trap)

Helper functions:
  InterruptControllerInit(hart_id)  — per-hart interrupt setup (SBI calls)
  UpdateInterruptMask()             — enable/disable specific interrupt sources
  InterruptHandlerRemove(task, node) — unregister an interrupt handler from RB tree,
    updates partition interrupt mask, releases interrupt object if no handlers remain
  BitmapFindAndClearBit / BitmapSetBit — interrupt pending bit manipulation

INTIO interrupt registers:
  INTIO_EXT_INTR_STATUS   = 0xFFFFFFFF60000200  — external interrupt status
  INTIO_SW_INTR_STATUS    = 0xFFFFFFFF60055200  — software interrupt status
  INTIO_EXT_INTR_PENDING  = 0xFFFFFFFF70000F2C  — external interrupt pending
  INTIO_SW_INTR_PENDING   = 0xFFFFFFFF70000F30  — software interrupt pending
  INTIO_FB_IRQ_STATUS     = 0xFFFFFFFF60054000  — framebuffer IRQ status
  INTIO_FB_IRQ_CTRL       = 0xFFFFFFFF60054400  — framebuffer IRQ control
  INTIO_MBOX_IRQ_CLEAR    = 0xFFFFFFFF60000100  — mailbox IRQ clear
```

### IPC / Ports / Shuttles

The IPC subsystem uses a **port + shuttle** model. Ports are communication endpoints; shuttles carry messages between them.

#### Port Operations

`KernelPortOperation(src_handle, operation, arg1, arg2, flags, arg3, arg4)` is the main dispatcher:

1. Check if task has port capability (`port_caps & 0x100`)
2. Resolve source port by handle via `ElfGetProgramHeader` (handle table lookup with grant check)
3. If operation==0 and port has direct target: fast-path `PortReceiveEnqueue` + `CompletePortOperation`
4. Otherwise: resolve destination port by handle, validate address space ID against bitmask `0x491`
5. Dispatch to `PortEnqueueMessage` or return error

`KernelPortClose(object)` — reference-counted close:
- Decrement refcount at `(object - 8)` (8-byte header)
- When refcount reaches zero: call pool destructor, return to free list

Port close variants:
- `PortCloseLocked` — acquires spinlock before close (concurrent safety)
- `PortCloseWithTree` — walks port's rb-tree, closes all child ports
- `PortCloseWithFlush` — flushes all pending shuttles, aborts pending receives with error
- `ClosePortsByPartition` — rb-tree search by partition_id, calls `PortCloseWithFlush`

#### Shuttle State Machine

Shuttles are message carriers allocated from `PoolId_Shuttle` (size 0x68). Each shuttle transitions through these states:

```
State 0: IDLE           — shuttle is free / operation completed
State 1: RECEIVING      — waiting on PortReceiveShuttle
State 2: CALLING        — PortCallShuttle active (send+receive combined)
State 3: DMA_CALL       — DMA transfer in progress for call
State 4: WAITING        — HandleSyscallWait blocked
State 5: REPLY          — PortReplyShuttle pending
State 6: REPLY_COMPLETE — reply transfer done, awaiting ack
State 7: RECEIVING_DMA  — DMA receive in progress
State 8: ABORTED        — operation aborted (error)
```

Shuttle lifecycle:
```
AcquireShuttleObject(pool_id, type, addr, data, tag)
  │
  ├─► PortSendShuttle (type=3)    — enqueue send, set state=0 on completion
  ├─► PortReceiveShuttle (type=6) — create DMA receive shuttle, state=7
  ├─► PortCallShuttle (type=2)    — combined send+receive, state=2
  ├─► PortReplyShuttle (type=7)   — reply to sender, state=6
  │
  └─► ShuttleComplete(shuttle_node)
        ├─► Remove from list via ListRemoveInit
        ├─► Decrement refcount
        └─► KernelPortClose if refcount==0
```

#### Pending Operation Processing

Pending port and DMA operations use **double-buffered lists** with a toggle flag for concurrent safety:

`ProcessPendingPortOps`:
- Toggle `g_portPendingFlag` (0↔1) to switch active list
- Walk entries; dispatch by state:
  - State 2: `PortDmaSend`
  - State 4: `HandleSyscallPortSend` + `KernelPortClose`
  - State 6: Allocate shuttle, enqueue
  - State 8: `PortSendShuttle`

`ProcessPendingDmaOps`:
- Toggle `g_dmaPendingFlag` (0↔1)
- Walk entries; dispatch by state:
  - State 4: `PortReceiveShuttle`
  - State 1: `HandleSyscallPortReceive`
  - State 3: `PortCallShuttle`
  - State 5: `PortReplyShuttle`

Both are called from `ProcessPendingOperations`, which also runs `InitializeSecondaryHarts` and `GdmaWaitCompletion`.

```

### GDMA (GPU DMA Engine)

The GDMA engine has two programming interfaces: CSR-based (for simple boot-time transfers) and MMIO-based (for multi-channel runtime DMA).

**CSR Interface** (for boot-time / simple transfers):
```
NV_CSR_GDMA_ENGINE   (0x5ca): Engine select (0=off, 8=GDMA)
NV_CSR_GDMA_SRC_ADDR (0x5cb): Source physical address (with enable bit)
NV_CSR_GDMA_DST_ADDR (0x5cc): Destination physical address
NV_CSR_GDMA_SIZE     (0x5ce): Transfer size (8-byte units)
NV_CSR_GDMA_CTRL     (0x5cf): Control register (0xc0018 = start transfer)
NV_CSR_GDMA_STATUS   (0x5d0): Status register (1=running, 0=idle)
NV_CSR_GDMA_ERROR    (0x5d1): Error register
```

**MMIO Channel Interface** (for runtime multi-channel DMA):
```
Base: 0xFFFFFFFF60141000 + channel * 0x1000
Per-channel registers (stride 0x1000):
  +0x10: SRC_LO   — Source physical address [31:0]
  +0x18: SRC_HI   — Source physical address [63:32]
  +0x1C: DST_LO   — Destination physical address [31:0]
  +0x20: DST_HI   — Destination physical address [63:32]
  +0x28: CTRL     — Control register
  +0x30: XFER     — Transfer size / trigger
  +0x30: STATUS   — Status register (same offset, read)
  +0x3C: SUBMITTED  — Submitted transfer count
  +0x44: COMPLETED  — Completed transfer count

Control bits:
  Bit 0x09: START           — Start transfer
  Bit 0x30: IRQ_ENABLE     — Interrupt on completion
  Bits 0x05..0x0C: SRC_PERM — Source permission field
  Bits 0x15..0x1C: DST_PERM — Destination permission field

Transfer encoding:
  Bits 0x00fffffc: Size in bytes (must be 4-byte aligned)
  Bit  0x01000000: CHAIN   — Chain to next descriptor
  Bit  0x80000000: ENABLE  — Valid/enable bit
  Max single transfer: 0x00fffffc (~16MB)
```

`GdmaCopyMemory(dst_as, dst, src_as, src, size)`:
1. Check `g_gdmaEnabled` — if false, fall back to `MemoryCopy`
2. Validate **4-byte alignment** of src, dst, and size (return `LibosErrorAccess` if not)
3. Resolve physical addresses via `AdvanceMappingIterator` for both src and dst
4. Round-robin channel selection (`g_dmaChannel` increments mod `g_dmaChannelCount`)
5. Program MMIO registers: SRC→DST→SIZE with ENABLE|CHAIN flags
6. Returns immediately; completion is polled or interrupt-driven

`GdmaBounceBufferCopy` / `GdmaCopyViaBounceBuffer`:
- Bounce buffer at `g_gdmaBounceBuffer` (phys from global page offset 0x410)
- For unaligned transfers: copy to bounce buffer first, then DMA from aligned buffer
- Must be >= 0x2000 bytes (validated at boot)

`GdmaWaitCompletion()`: polls `NV_CSR_GDMA_STATUS` until engine idle.

`GdmaInterruptHandler()`: called on DMA completion interrupt, processes pending DMA ops via `ProcessPendingDmaOps`.

### ELF Loading

```
KernelElfMap(address_space, elf_offset, memory_set):
  │
  ├─► LibosBootFindElfHeader(g_rootFs, elf_offset)
  │     ├─► Walk rootfs directory tree
  │     ├─► Search for nodes with type==5 (regular file)
  │     └─► Validate ELF magic (0x7f454c46) at file offset 0
  │
  ├─► Validate ELF header (e_ident magic check)
  │
  ├─► For each PT_LOAD program header:
  │     ├─► KernelAddressSpaceAllocate(as, &mapping, p_vaddr, p_memsz)
  │     ├─► Derive permissions from p_flags (PF_R→Read, PF_W→Write, PF_X→Exec)
  │     ├─► KernelAddressSpaceMapContiguous(mapping, vaddr, paddr, size, perms)
  │     ├─► If p_filesz > 0: GdmaCopyMemory for segment data
  │     └─► If p_memsz > p_filesz: zero-fill BSS (MemoryFill)
  │
  └─► Return mapped entry point

KernelElfLoadSegment(mapping, offset, size):
  — Lower-level segment loader used by auto-expanding object pools
  — Maps pages and copies data for a single segment

LibosBootFindElfHeader(rootfs_base, node_offset):
  — Iterates rootfs directory entries
  — Uses StringCompare to match "init.elf"
  — Returns pointer to validated ELF header

KernelRootfsMountFilesystem(image_phys):
  — Maps rootfs image into kernel address space
  — Validates filesystem structure headers
  — Sets g_rootFs, g_rootFsPhys, g_rootFsSize globals
```

### Handle Table

Handle tables manage per-task references to kernel objects (ports, address spaces, memory sets, etc.):

```
HandleTable:
  +0x00: uint32_t capacity      — max handles (0x80 for init, 8192 for server)
  +0x08: HandleEntry[] entries  — array of (object_ptr, grant_flags) pairs
  +0x10: void*    freeList      — stack of free handle indices

HandleEntry (16 bytes):
  +0x00: void*    object        — pointer to kernel object (bit 63 = allocated marker)
  +0x08: uint64_t grant_flags   — access permissions for this handle
```

Operations:
- `KernelTaskHandleTableCreate(capacity, &ht_out)` — allocate page via `KernelPageStateTransition`, initialize free list chain from pool slot 0xb
- `KernelTaskRegisterObject(task, &handle_out, object, grants, flags)` — pop free slot, store object pointer + grant flags with **bit 63 set as allocated marker**, increment refcount
- `ValidateObjectAccess(handle)` — look up handle, check ASID bitmask **0x491** (grants ASIDs 0,4,7,10 + exception ASID 5), increment refcount on success
- `HandleTableUpdateEntry(task, index)` — close object, clear bit 63, chain slot into free list
- `HandleTableFlush(task)` — close exception frame handles, clear frame count
- `HandleTableClear(table)` — close all handles, set count to 0
- `HandleTableRelease(handle_table)` — decrement refcount; when zero, close all live objects and free physical pages

Grant flags (LibosGrantPort):
- Bit 0: receive access (LibosGrantPortSend = 2)
- Bit 1: send access
- Bit 2: all access (LibosGrantPortAll = 3)

The `ElfGetProgramHeader` function serves double duty as the handle table lookup — it resolves a handle index to its object pointer and grant flags, checking bit 63 for validity.

### Syscall Dispatch

`SyscallHandler` is invoked on ecall from U-mode. It calls `ExceptionVectorHandler` and `SoftwareInterruptHandler` for the syscall frame, then `SyscallDispatch` which uses a jump table at address `0x9300c3b0` to route to specific handlers:

```
Syscall Number │ Handler
───────────────┼──────────────────────────────
  0x00         │ SyscallMapMemory (low-level)
  0x01         │ SyscallUnmapMemory
  0x02         │ SyscallSetInterruptPriority
  0x03         │ SyscallRegisterInterrupt
  0x04         │ SyscallAllocateExceptionFrame
  0x05         │ SyscallAllocateObjectPool
  0x06         │ SyscallMemorySetInsert
  0x07         │ SyscallWriteConsole
  0x08         │ SyscallReadMemory
  0x09         │ HandleSyscallPortSend
  0x0a         │ HandleSyscallPortReceive
  0x0b         │ HandleSyscallWait
  0x0c         │ SyscallPartitionConfig
  0x0d         │ SyscallPartitionIpi
  0x0e         │ SyscallPartitionPortClose
  0x0f         │ SyscallEcall
  0x10         │ SyscallTriggerCrash
```

**SyscallMapMemory** has two variants:
- Low-level (0x00afa): validates ASID==10 and grant bits 0xc (RW), increments refcount, either grants immediately or queues task on wait list with `SetCurrentTaskState(timeout)`
- Full (0x07180): validates ASID==3 and grant bit 0, performs multi-handle lookup (dst/src/as handles), calls `AddressSpaceMapRegion` with proper permissions

`HandleSyscallPortSend(op)` acquires a shuttle object from `AcquireShuttleObject`, sets the message data pointer, and marks the operation as state 5 (REPLY). If no shuttle available, queues on the pending port operation list.

`HandleSyscallPortReceive(op)` acquires a shuttle, sets up DMA receive, validates address space access via `SyscallDispatch`, and marks the operation as state 2 (CALLING).

`HandleSyscallWait(handles, count, timeout)` iterates through handle array, checking each handle's state:
- Handle state 2 (RECEIVING): call `HandleTableFlush`, mark as done
- Handle state 1 (IDLE): `TaskResume` immediately via `ApplyTaskContextFromPort`
- Otherwise: increment refcount, add to wait list, set task blocked with deadline

`SyscallPartitionConfig(cmd, a1, a2, a3, a4, a5)` is a multiplexed syscall:
- cmd 0x29: notify root partition
- cmd 0x2c: complete secondary hart initialization

`SyscallEcall` performs a nested ecall to M-mode (for delegated operations).

`SyscallTriggerCrash` deliberately triggers an exception for testing/debugging.

```

### IPI (Inter-Processor Interrupts)

**INTIO IPI Registers:**
```
INTIO_IPI_BASE       = 0xFFFFFFFF60140000
INTIO_IPI_CTRL       = 0xFFFFFFFF6014000C
INTIO_IPI_MASK       = 0xFFFFFFFF60140010
INTIO_IPI_SEND_BASE  = 0xFFFFFFFF60141004  (per-hart, stride 0x1000)
```

**IPI Message Types** (dispatched via jump table at `0x9300c3f4`):

| Type | Name | Size | Description |
|------|------|------|-------------|
| 0 | IpiMessageHartStart | 0x18 | Start a secondary hart |
| 1 | IpiMessageDmaCopy | 0x10 | Request DMA copy |
| 2 | IpiMessageTimerSet | 0x18 | Set timer on target hart |
| 3 | IpiMessageInterrupt | 0x20 | Deliver interrupt to partition |
| 4 | IpiMessagePortOp | 0x08 | Pending port operation |
| 5 | IpiMessageDmaWait | 0x10 | Wait for DMA completion |
| 6 | IpiMessagePartitionCmd | 0x18 | Partition control command |
| 7 | IpiMessageNull | 0x10 | Sentinel / invalid |

Message header format (8 bytes):
```
struct MessageHeader {
    uint32_t kind;   // IpiMessageType
    uint32_t size;   // total message size including header
};
```

`IpiMessageHandler` reads the message page for the current hart (`g_ipiMessagePage`), validates each message header against the size table at `0x9300c690`, and dispatches to the appropriate handler via the jump table at `0x9300c3f4`.

```
IpiSend(target, message)          — write message to target hart's IPI mailbox
  │
  ├─► Write to INTIO IPI register
  └─► Trigger software interrupt on target hart

IpiMessageHandler()               — process incoming IPI messages
  │
  ├─► IpiCheckMessages()          — poll for pending IPI
  ├─► Validate message headers against size table
  ├─► Dispatch based on message type via jump table
  └─► ProcessPendingOperations()

SendIpiMessage(target_partition)  — high-level IPI to a partition
  └─► IpiSend(target_hart, &msg)

HandleIpiInterrupt()              — interrupt handler entry point
  ├─► SoftwareInterruptHandler()
  ├─► SendIpiMessage()
  ├─► ProcessPendingInterupts()
  ├─► IpiMessageHandler()
  └─► FlushPartitionChannels() / PortCloseWithFlush()

Secondary hart startup:
  SecondaryHartEntry (ram:93008bc8)
    │
    ├─► IpiMessageHandler()       — process any pending IPIs
    ├─► InitializeSecondaryHarts() — start any other pending harts
    ├─► SyscallPartitionConfig(0x2c, ...) — signal hart ready
    ├─► ProcessPendingOperations() — drain pending work
    └─► ecall to S-mode           — enter supervisor mode
```

### Partitions

Partitions isolate tasks into separate security domains:

```
KernelPartitionCreate(&partition_out, partition_id)
  ├─► Allocate partition object from PoolId_Partition
  ├─► Insert into partition rb-tree keyed by partition_id
  └─► Return partition pointer

AllocatePartitionChannel(tree, channel_id, &port_out)
  ├─► Allocate channel sub-object from pool 0x14
  ├─► Allocate channel port from pool 0x15
  ├─► Insert into partition's channel tree
  └─► Return port pointer

FlushPartitionChannels(channel_id) — close all channels matching ID
ClosePortsByPartition(partition_id) — rb-tree search + PortCloseWithFlush
```

### Kernel Server

The kernel server is a kernel-mode task (priority NORMAL) that handles deferred system operations:

```
KernelServerEntry (ram:93008f7a)
  └─► KernelServerMainLoop (ram:93001296)

KernelServerMainLoop:
  while (TRUE) {
    1. Flush console output if pending (SbiConsolePutchar)
    2. ProcessPendingOperations()
       ├─► ProcessPendingPortOps()     — shuttle state machine
       ├─► ProcessPendingDmaOps()      — DMA transfers
       ├─► InitializeSecondaryHarts()  — start pending harts
       └─► GdmaWaitCompletion()        — wait for DMA idle
    3. Process partition IPI bitmask
       ├─► For each pending bit: validate, send IPI to target
       └─► SyscallPartitionConfig(0x29) — notify root partition
    4. Update scheduler priority queue
       ├─► SegmentTreeFindIndex() — find next runnable task
       └─► PriorityQueueUpdate()  — re-prioritize
  }

KernelServerDispatch (ram:93003ad8) — handles incoming IPC requests on server port
  ├─► Read message from kernel server port
  ├─► Dispatch based on operation type
  └─► CompletePortOperation() — signal completion
```

The kernel server runs with `started = 0` (not resumable from `TaskResume`), meaning it enters its main loop directly from `KernelServerEntry` and never returns.

---

## Object Pool Table

| Pool ID | Raw Size | Aligned Size | Name | Destructor | Region Size |
|---------|----------|-------------|------|------------|-------------|
| 0 | 0x28 | 0x30 | Task State | HandleIpiInterrupt | 0x1000 (85 objects) |
| 1 | 0x00 | 0x08 | Syscall Stack | (none) | 0x1000 (512 objects) |
| 2 | 0x30 | 0x38 | Memory Set Node | (none — free list) | 0x1000 (68 objects) |
| 3 | 0x48 | 0x50 | Physical Page | KernelTaskDestroy | 0x2000 (64 objects) |
| 4 | 0x240 | 0x248 | Address Space | TimerInterruptHandler | 0x1000 (6 objects) |
| 5 | 0x20 | 0x28 | Task Run Queue | SoftwareInterruptHandler | 0x1000 (94 objects) |
| 6 | 0x48 | 0x50 | Exception Frame | ExceptionVectorHandler | 0x1000 (32 objects) |
| 7 | 0x28 | 0x30 | Port | TimerInterruptHandler | 0x1000 (85 objects) |
| 8 | 0x08 | 0x10 | Page Table Root | KernelPageStateTransition | 0x1000 (256 objects) |
| 9 | 0x40 | 0x48 | (reserved) | (none) | — |
| 0xb | 0x20 | 0x28 | Memory Pool | (none — free pages) | 0x1000 (94 objects) |
| 0xc | 0x18 | 0x20 | Interrupt Handler | (none) | 0x1000 (128 objects) |
| 0xd | 0x60 | 0x68 | Shuttle | HandleIpiInterrupt | 0x2000 (48 objects) |
| 0xe | 0x48 | 0x50 | Object Pool | (none) | 0x1000 (32 objects) |
| 0xf | 0x08 | 0x10 | Task Token | KernelPortDestroyAll | 0x1000 (256 objects) |
| 0x10 | 0x18 | 0x20 | Task Token 2 | (none) | 0x1000 (128 objects) |
| 0x11 | varies | varies | Partition | (rb-tree insert) | — |
| 0x12 | varies | varies | Partition Port | (close) | — |
| 0x13 | 0x238 | 0x240 | Partition Port 2 | TaskSwitchHandler | 0x1000 (6 objects) |
| 0x14 | 0x48 | 0x50 | Channel Sub-Object | (none) | — |
| 0x15 | 0x38 | 0x40 | Channel Port | (none) | — |

Aligned size formula: `((raw_size + 7) & ~7) + 8` — 8-byte alignment plus 8-byte reference count header.

---

## Configuration Constants (from Global Page)

The kernel reads boot configuration from the global page mapped at `0xffffffff70000000`:

| Offset | Name | Description |
|--------|------|-------------|
| 0x410 | tcmBounceBufferSize | GDMA bounce buffer physical address |
| 0x418 | tcmBounceBufferPhys | GDMA bounce buffer size (must >= 0x2000) |
| 0x420 | kernelArgs[0] | Boot argument 0 |
| 0x428 | kernelArgs[1] | Boot argument 1 |
| 0x430 | kernelArgs[2] | Boot argument 2 (boot hart ID) |
| 0x438 | kernelArgs[3] | Boot argument 3 |
| 0x440 | initConfig[0] | Init task config value |
| 0x448 | initConfig[1] | Init task config value |
| 0x450 | rootfsOffset | Root filesystem offset in image |
| 0x468 | rootfsSize | Root filesystem size |
| 0x478 | hasPartitionPort | Non-zero if partition port is active |
| 0x480 | globalPagePhys | Global page physical address |
| 0x488 | partitionVaddr | Partition VA mapping address |
| 0x4a0 | initPartitionId | Init task partition ID |
| 0xf28 | secondaryHartId | Secondary hart to start (0 = none) |
| 0xf2c | secondaryTimeBase | Secondary hart time base |
| 0xf30 | secondaryEntryPoint | Secondary hart entry point |

---

## WPR (Write-Protected Region) Shadow Descriptors

The kernel parses up to 9 WPR shadow descriptors from the boot configuration. Each descriptor:

```
struct WprShadowDescriptor {
    uint8_t  type;         // +0x00: 0x02 = identity-mapped, others = aperture
    uint8_t  apertureId;   // +0x00: aperture index (0-7)
    uint64_t physicalAddr; // +0x08: physical address
    uint64_t size;         // +0x10: region size
};
```

Identity-mapped regions (type=0x02) are mapped at fixed VA `0xc000...` offset.
Other regions are mapped starting at `0x8000000000000000` with 2MB granularity.

---

## Key Global Variables

| Address | Name | Type | Description |
|---------|------|------|-------------|
| `a3008000` | g_rootFs | void* | Mapped root filesystem base |
| `a3008008` | g_currentTask | Task* | Currently running task |
| `a3008938` | g_rootFsPhys | uint64_t | Root filesystem physical address |
| `a3008940` | g_rootFsSize | uint64_t | Root filesystem size |
| `a3008830` | g_gdmaBounceMapping | void* | GDMA bounce buffer mapping object |
| `a3009150` | g_zeroPage | void* | All-zeros sentinel page |
| `a300a000` | g_objectPools | ObjectPool[16] | Array of object pool descriptors |
| `a300a370` | g_portFreeTree | RbTree | Port free list (rb-tree) |
| `a300a7f0` | g_taskFreeList | ListHead | Task free list |
| `a300b000` | g_interruptTable | IDT[16] | Interrupt descriptor table |
| `a300b380` | g_pageTableRoot | void* | Root page table pointer |
| `a300c2d8` | g_handleFreeList | void** | Handle table free list |
| `a300c6d8` | g_hartState | HartState[N] | Per-hart timer/state |
| `a3010000` | g_pageTablePages | PageTable | Pre-allocated page table pages |
| `a3017000` | g_freePages | Page* | Free physical page linked list |
| `a3019120` | g_partitionPendingMask | uint64_t | Bitmask of partitions with pending IPI |
| `a30191c0` | g_kernelAddressSpace | AS* | Kernel address space |

---

## Complete Function Inventory (199 functions)

### Kernel Initialization (3)

| Address | Name | Source |
|---------|------|--------|
| `93002444` | KernelInit | full/init.c |
| `93000176` | InitializeCsrRegisters | full/init.c |
| `93003f5c` | FenceAll | full/init.c |

### Memory Management — Object Pools (5)

| Address | Name | Source |
|---------|------|--------|
| `9300545a` | KernelObjectPoolInitialize | mm/objectpool.c |
| `93004068` | KernelObjectPoolRegisterRegion | mm/objectpool.c |
| `930054f6` | KernelObjectPoolRelease | mm/objectpool.c |
| `9300542a` | KernelMemoryPoolAcquire | mm/memorypool.c |
| `93005b48` | InitTaskAllocateMemoryPool | mm/objectpool.c |

### Memory Management — Memory Pools / Buddy (5)

| Address | Name | Source |
|---------|------|--------|
| `93005022` | KernelAllocateMemoryPool | mm/memorypool.c |
| `93004976` | FindAddressSpaceNode | mm/memorypool.c |
| `9300499e` | SplitPageEntry | mm/memorypool.c |
| `9300480e` | MergeAdjacentPages | mm/memorypool.c |
| `93004a5a` | AdvancePageIterator | mm/memorypool.c |

### Memory Management — Memory Sets (4)

| Address | Name | Source |
|---------|------|--------|
| `93004f0c` | KernelMemorySetAllocate | mm/memoryset.c |
| `930051e2` | KernelMemorySetInsert | mm/memoryset.c |
| `9300573e` | MemorySetSplitNode | mm/address_space.c |
| `93005bf6` | SyscallMemorySetInsert | mm/memoryset.c |

### Memory Management — Page Tables (7)

| Address | Name | Source |
|---------|------|--------|
| `93006c8c` | PageTableMapEntry | mm/pagetable.c |
| `93004d08` | AllocateZeroedPage | mm/pagetable.c |
| `93007602` | FreePageTableEntry | mm/pagetable.c |
| `93009bb8` | UnmapPageRange | mm/pagetable.c |
| `93009b12` | GetPageTableEntry | mm/pagetable.c |
| `93004180` | WalkPageTable | mm/pagetable.c |
| `93009d9a` | KernelPageStateTransition | mm/pagestate.c |

### Memory Management — Address Spaces (9)

| Address | Name | Source |
|---------|------|--------|
| `93005a9c` | KernelAllocateAddressSpace | mm/address_space.c |
| `93004fce` | KernelAddressSpaceRegister | mm/address_space.c |
| `93005966` | KernelAddressSpaceAllocate | mm/address_space.c |
| `93006d8e` | KernelAddressSpaceMapContiguous | mm/address_space.c |
| `93006ff2` | AddressSpaceMapRegion | mm/address_space.c |
| `9300588c` | ReleaseAddressRange | mm/address_space.c |
| `9300594a` | ReleaseAddressSpaceMapping | mm/address_space.c |
| `93005836` | ReserveAlignedAddressRange | mm/address_space.c |
| `930040e4` | FindFreeVirtualRange | mm/address_space.c |

### Memory Management — Virtual/Physical Translation (4)

| Address | Name | Source |
|---------|------|--------|
| `93003fd0` | TranslateVirtualToPhysical | mm/identity.c |
| `9300490a` | VirtualToPhysical | mm/identity.c |
| `9300068c` | CopyWithAddressTranslation | mm/identity.c |
| `930040c4` | GetAddressSpaceId | mm/identity.c |

### Memory Management — Utility (3)

| Address | Name | Source |
|---------|------|--------|
| `93004cc0` | MemoryFill | common/mem.c |
| `93004d4e` | MemoryCopy | common/mem.c |
| `93004c9a` | CountTrailingZeros | common/bitops.c |

### Red-Black Tree (10)

| Address | Name | Source |
|---------|------|--------|
| `9300424e` | RbTreeSearch | common/rbtree.c |
| `9300427a` | RbTreeSearchByAddress | common/rbtree.c |
| `93003ed8` | RbTreeSearchByAddressRange | common/rbtree.c |
| `9300446a` | RbTreeInsert | common/rbtree.c |
| `930048d8` | RbTreeInsertByAddress | common/rbtree.c |
| `930044b2` | RbTreeDelete | common/rbtree.c |
| `930043a6` | RbTreeInsertFixup | common/rbtree.c |
| `93004302` | RbTreeRotateLeft | common/rbtree.c |
| `93004354` | RbTreeRotateRight | common/rbtree.c |
| `93004e26` | RbTreePredecessor | common/rbtree.c |
| `93003ef8` | RbTreeUpdateMaxSubtree | common/rbtree.c |
| `93003f3a` | RbTreeUpdateAllAncestors | common/rbtree.c |

### Linked List (4)

| Address | Name | Source |
|---------|------|--------|
| `9300037a` | ListAddTail | common/list.c |
| `93000386` | ListAddAfter | common/list.c |
| `930005b8` | ListRemoveInit | common/list.c |
| `9300035a` | ListIsSingular | common/list.c |

### Task Management (7)

| Address | Name | Source |
|---------|------|--------|
| `930072be` | KernelTaskCreate | full/task.c |
| `930056c8` | KernelTaskDestroy | full/task.c |
| `9300476c` | KernelTaskHandleTableCreate | full/task.c |
| `930052f0` | KernelTaskRegisterObject | full/task.c |
| `930042a6` | TaskResume | full/sched.c |
| `9300768c` | HandleTableRelease | full/task.c |
| `93000432` | StartTaskByPartitionId | full/init.c |

### Handle Table (5)

| Address | Name | Source |
|---------|------|--------|
| `9300637a` | HandleTableClear | full/task.c |
| `93006420` | HandleTableFlush | full/task.c |
| `93006680` | HandleTableUpdateEntry | full/task.c |
| `930063b0` | ValidateObjectAccess | full/task.c |
| `93009b94` | ElfGetProgramHeader | full/task.c |

### Scheduler (6)

| Address | Name | Source |
|---------|------|--------|
| `930018f2` | ScheduleNextTask | full/sched.c |
| `9300234a` | SchedulerYield | full/sched.c |
| `93000392` | PriorityQueueUpdate | full/sched.c |
| `930003cc` | SegmentTreeFindIndex | full/sched.c |
| `93000466` | GetTaskPriority | full/sched.c |
| `93000482` | SetCurrentTaskState | full/sched.c |

### IPC — Ports (14)

| Address | Name | Source |
|---------|------|--------|
| `930052c4` | KernelPortAllocate | full/sched/port.c |
| `93003694` | KernelPortOperation (lookup) | full/sched/port.c |
| `930037fc` | KernelPortOperation (dispatch) | full/sched/port.c |
| `930055f8` | KernelPortClose | full/sched/port.c |
| `930056a4` | KernelPortDestroyAll | full/sched/port.c |
| `9300570c` | KernelPortFlushPending | full/sched/port.c |
| `93006710` | PortCloseLocked | full/sched/port.c |
| `930080c8` | PortCloseWithTree | full/sched/port.c |
| `9300813c` | PortCloseWithFlush | full/sched/port.c |
| `930081ac` | ClosePortsByPartition | full/sched/port.c |
| `93000cb8` | CompletePortOperation | full/sched/port.c |
| `93006748` | ObjectCloseLocked | full/sched/port.c |
| `9300468a` | SpinLockAcquire | full/sched/port.c |
| `930046aa` | SpinLockRelease | full/sched/port.c |

### IPC — Shuttles (8)

| Address | Name | Source |
|---------|------|--------|
| `93007afc` | AcquireShuttleObject | full/sched/port.c |
| `93007cca` | PortSendShuttle | full/sched/port.c |
| `93007d38` | PortReceiveShuttle | full/sched/port.c |
| `93007d90` | PortReplyShuttle | full/sched/port.c |
| `93007de2` | PortCallShuttle | full/sched/port.c |
| `9300676c` | ShuttleComplete | full/sched/port.c |
| `93006888` | ShuttleListFlush | full/sched/port.c |
| `93007f7e` | PortDrainReceivers | full/sched/port.c |

### IPC — Port DMA (4)

| Address | Name | Source |
|---------|------|--------|
| `930083ee` | PortDmaSend | full/sched/port.c |
| `9300850a` | PortDmaReceive | full/sched/port.c |
| `93001e84` | PortEnqueueMessage | full/sched/port.c |
| `93008008` | PortReceiveEnqueue | full/sched/port.c |

### IPC — Pending Operations (3)

| Address | Name | Source |
|---------|------|--------|
| `93008aa0` | ProcessPendingPortOps | full/sched/port.c |
| `930081fc` | ProcessPendingDmaOps | full/sched/port.c |
| `93008b90` | ProcessPendingOperations | full/sched/port.c |

### IPC — Port High-Level (5)

| Address | Name | Source |
|---------|------|--------|
| `93001dea` | PortSendAndWait | full/sched/port.c |
| `93001e2a` | PortReceiveComplete | full/sched/port.c |
| `930064ee` | HandlePortSendReceive | full/sched/port.c |
| `93006582` | PortWaitComplete | full/sched/port.c |
| `93008038` | PortOperationComplete | full/sched/port.c |

### Interrupt Handling (10)

| Address | Name | Source |
|---------|------|--------|
| `93007916` | InterruptControllerInit | common/nvriscv-2.0/sbi.c |
| `9300770c` | TimerInterruptHandler | drivers/timer.c |
| `93006460` | SoftwareInterruptHandler | full/intr.c |
| `93002214` | ExternalInterruptHandler | drivers/extintr-v2.c |
| `93000ec2` | ExceptionVectorHandler | full/intr.c |
| `93008120` | SyscallHandler | full/intr.c |
| `93007c2c` | TaskSwitchHandler | full/intr.c |
| `930068b2` | InterruptHandlerRemove | full/intr.c |
| `93000fec` | HandleIpiInterrupt | full/ipi.c |
| `930064ae` | HandlePartitionInterrupt | full/partition.c |

### Interrupt Dispatch Helpers (6)

| Address | Name | Source |
|---------|------|--------|
| `9300213c` | DispatchExternalInterrupts | drivers/extintr-v2.c |
| `93002160` | DispatchSoftwareInterrupts | full/intr.c |
| `930004ee` | BitmapFindAndClearBit | common/bitops.c |
| `930042ec` | BitmapSetBit | common/bitops.c |
| `93000522` | FindInterruptHandlerByMask | full/intr.c |
| `9300055a` | ClearInterruptPending | full/intr.c |
| `930008f6` | UpdateInterruptMask | full/intr.c |
| `9300099c` | MaskAndAcknowledgeInterrupt | full/intr.c |

### SBI (Supervisor Binary Interface) (6)

| Address | Name | Source |
|---------|------|--------|
| `93000e10` | SbiCall | common/nvriscv-2.0/sbi.c |
| `9300779c` | SbiQueryMemoryRegion | common/nvriscv-2.0/sbi.c |
| `930077e2` | SbiSetTimer | common/nvriscv-2.0/sbi.c |
| `9300782a` | SbiConsolePutchar | common/nvriscv-2.0/sbi.c |
| `93007a4c` | SbiHartStart | common/nvriscv-2.0/sbi.c |
| `93000e50` | UpdateNextTimerDeadline | common/nvriscv-2.0/sbi.c |
| `93007870` | TimerSetComparator | common/nvriscv-2.0/sbi.c |

### GDMA (GPU DMA) (6)

| Address | Name | Source |
|---------|------|--------|
| `93009896` | GdmaCopyMemory | drivers/gdma.c |
| `93009a8c` | GdmaWaitCompletion | drivers/gdma.c |
| `93008c70` | GdmaTransfer | drivers/gdma.c |
| `93008d3e` | GdmaInterruptHandler | drivers/gdma.c |
| `93003b3e` | GdmaBounceBufferCopy | drivers/gdma.c |
| `93003b4a` | GdmaCopyViaBounceBuffer | drivers/gdma.c |

### IPI (Inter-Processor Interrupts) (4)

| Address | Name | Source |
|---------|------|--------|
| `9300859a` | IpiMessageHandler | full/ipi.c |
| `93008bf4` | IpiSend | full/ipi.c |
| `93009b3a` | SendIpiMessage | full/ipi.c |
| `93003c0c` | IpiCheckMessages | full/ipi.c |

### ELF Loading / Boot (4)

| Address | Name | Source |
|---------|------|--------|
| `93009e9c` | KernelElfMap | full/loader.c |
| `93009b5a` | LibosBootFindElfHeader | full/loader.c |
| `93004062` | KernelRootfsMountFilesystem | full/loader.c |
| `93004200` | StringCompare | full/loader.c |
| `93009ca2` | KernelElfLoadSegment | full/loader.c |

### Syscall Handlers (12)

| Address | Name | Source |
|---------|------|--------|
| `93005fbe` | SyscallDispatch | full/syscall.c |
| `93000afa` | SyscallMapMemory (low) | full/syscall.c |
| `93007180` | SyscallMapMemory (high) | full/syscall.c |
| `93000b88` | SyscallUnmapMemory | full/syscall.c |
| `93000c4e` | SyscallSetInterruptPriority | full/syscall.c |
| `93005dc2` | SyscallRegisterInterrupt | full/syscall.c |
| `93005e2a` | AllocateExceptionFrame | full/syscall.c |
| `93005e8a` | SyscallAllocateExceptionFrame | full/syscall.c |
| `93005b9e` | SyscallAllocateObjectPool | full/syscall.c |
| `930039f4` | SyscallWriteConsole | full/syscall.c |
| `93003a0a` | SyscallReadMemory | full/syscall.c |
| `93008d00` | SyscallEcall | full/syscall.c |
| `9300101e` | SyscallPartitionConfig | full/syscall.c |
| `9300117c` | SyscallPartitionIpi | full/syscall.c |
| `93001212` | SyscallPartitionPortClose | full/syscall.c |
| `93008e4c` | HandleSyscallWait | full/syscall.c |
| `93007be0` | HandleSyscallPortSend | full/syscall.c |
| `93007ea2` | HandleSyscallPortReceive | full/syscall.c |
| `9300184c` | SyscallTriggerCrash | full/syscall.c |

### Partitions (4)

| Address | Name | Source |
|---------|------|--------|
| `93005eb4` | KernelPartitionCreate | full/partition.c |
| `93005ca8` | AllocatePartitionChannel | full/partition.c |
| `930065d4` | FlushPartitionChannels | full/partition.c |
| `93005248` | AllocatePartitionPortNode | full/partition.c |

### Trap / Context Switch (4)

| Address | Name | Source |
|---------|------|--------|
| `930001bc` | TrapEntrySaveRegisters | full/trap.c |
| `930002a2` | TrapReturnRestoreRegisters | full/trap.c |
| `930003f0` | ApplyTaskContextFromPort | full/trap.c |
| `93000410` | ReadTaskSavedPc | full/trap.c |

### Kernel Server (4)

| Address | Name | Source |
|---------|------|--------|
| `93008f7a` | KernelServerEntry | full/server/server.c |
| `93001296` | KernelServerMainLoop | full/server/server.c |
| `93003a96` | KernelServerMainLoop (alt) | full/server/server.c |
| `93003ad8` | KernelServerDispatch | full/server/server.c |

### Hart Management (3)

| Address | Name | Source |
|---------|------|--------|
| `930082b0` | InitializeSecondaryHarts | full/hart.c |
| `930083a0` | InitializeHart | full/hart.c |
| `93008bc8` | SecondaryHartEntry | full/hart.c |

### Diagnostics / Crash (5)

| Address | Name | Source |
|---------|------|--------|
| `93000a68` | KernelPanic | full/diag.c |
| `93004e52` | KernelAssert | full/diag.c |
| `93001390` | WriteCrashDump | full/diag.c |
| `93001fa8` | HandleFatalException | full/diag.c |
| `93003cda` | HandleKernelPanic | full/diag.c |

#### Crash Dump Format

`WriteCrashDump` produces a structured crash dump starting at `0xffffffffa3008010` (max 0x800 bytes) with magic number `0x181dead`:

```
Crash Dump Layout:
  +0x00: uint64_t magic          — 0x181dead
  +0x08: uint64_t version        — build version from 0x9300c708
  +0x10: uint64_t partition_info — partition_id | hart_id encoding
  +0x18: uint64_t timestamp      — (time / 1e9) << 32 | 0x31000000
  +0x20: uint64_t type_field     — task priority | partition | flags
  +0x28: uint64_t exception_info — exception_code << 32 | hart_index
  +0x30: uint64_t return_addr
  +0x38: uint64_t extra_arg
  +0x40: uint64_t saved_sstatus
  +0x48: uint64_t sstatus
  +0x50: uint64_t stvec
  +0x58: uint64_t sip
  +0x60: uint64_t sepc
  +0x68: uint64_t stval
  +0x70: uint64_t scause
  +0x78: uint64_t sscratch
  +0x80: uint64_t exception_ctx
  +0x88: uint8_t  registers[0xF8] — full register file (x1-x31)
  +0x180: Stack trace frames      — (frame_count-1)<<22 | header, then return PCs
```

`KernelPanic` saves all 32 RISC-V registers + CSRs to `0xffffffffa3008840-0xffffffffa3008930`, then calls `HandleKernelPanic` which writes the crash dump and copies it to WPR via `GdmaCopyViaBounceBuffer`.

`KernelAssert` logs to a circular ring buffer at `g_assertBuffer` (a300a820). Each entry is `(level+1)` words: variadic args + a packed header word containing `(level+1) << 48 | boot_count << 56 | truncated_timestamp`. The first slot holds a running total of all entries written.

`HandleFatalException` validates the fault address, writes a crash dump, enqueues an error via `PortEnqueueMessage`, and sends an IPI to partition 3.

### Stack Trace (3)

| Address | Name | Source |
|---------|------|--------|
| `9300072a` | WalkStackFrame | full/diag.c |
| `93000874` | CollectStackTrace | full/diag.c |
| `93004156` | IsValidVirtualAddress | full/diag.c |

### Miscellaneous (5)

| Address | Name | Source |
|---------|------|--------|
| `930001b2` | WaitForInterrupt | full/sched.c |
| `930046ba` | InvokeAndReschedule | full/sched.c |
| `930046f4` | ProcessPendingInterupts | full/intr.c |
| `9300646e` | HandleExternalInterruptEvent | drivers/extintr-v2.c |
| `93009b02` | FindFirstSetBit | common/bitops.c |
| `93004b98` | AdvanceMappingIterator | mm/address_space.c |
| `93004c2c` | ConsumeMappingBytes | mm/address_space.c |
| `93004c6c` | ConsumeAndFlushMapping | mm/address_space.c |

---

## API Naming Convention

All kernel API functions follow the pattern `Kernel<Subsystem><Operation>`:
- `KernelAllocate*` — allocate/create an object
- `KernelAddressSpace*` — address space operations
- `KernelTask*` — task management
- `KernelPort*` — IPC port operations
- `KernelMemory*` — memory management
- `KernelObject*` — object pool management

Internal helpers use descriptive names without the `Kernel` prefix:
- `RbTree*` — red-black tree operations
- `List*` — linked list operations
- `Sbi*` — SBI call wrappers
- `Gdma*` — GDMA operations
- `Port*` — port internal operations
- `Handle*` — handle table operations
- `Syscall*` — syscall handlers

## Source File Mapping

Recovered source files map to the original NVIDIA source tree at `/gpu_drv/uproc/os/libos-v3.1.0/kernel/`:

```
recovered/                          — ~10K lines total, 199 functions
├── include/
│   ├── libos.h                    — Full API declarations (all 199 functions)
│   └── libos_types.h              — Type definitions, enums, constants
├── kernel/
│   ├── init.c          (290 lines) — KernelInit, boot sequence
│   ├── trap.c           (48 lines) — TrapEntrySaveRegisters, TrapReturnRestoreRegisters
│   ├── task.c          (589 lines) — KernelTaskCreate/Destroy, handle table ops
│   ├── sched.c         (663 lines) — SchedulerYield, ScheduleNextTask, priority queues
│   └── diag.c          (663 lines) — KernelPanic, KernelAssert, WriteCrashDump, stack trace
├── mm/
│   ├── memorypool.c    (802 lines) — KernelAllocateMemoryPool, buddy allocator, page state
│   ├── objectpool.c    (361 lines) — KernelObjectPoolInitialize/Release, slab allocator
│   ├── pagetable.c     (296 lines) — PageTableMapEntry, Sv39 walk, UnmapPageRange
│   ├── address_space.c (977 lines) — KernelAllocateAddressSpace, VA management, mapping iterators
│   └── identity.c      (191 lines) — TranslateVirtualToPhysical, aperture translation
├── drivers/
│   ├── sbi.c           (226 lines) — SbiCall, SbiSetTimer, InterruptControllerInit
│   ├── gdma.c          (429 lines) — GdmaCopyMemory, GdmaWaitCompletion, MMIO registers
│   ├── extintr.c       (580 lines) — ExternalInterruptHandler, InitializeSecondaryHarts
│   ├── timer.c         (140 lines) — TimerInterruptHandler, UpdateNextTimerDeadline
│   └── ipi.c           (318 lines) — IpiMessageHandler, IpiSend, message types
├── loader/
│   └── elf.c            (93 lines) — KernelElfMap, LibosBootFindElfHeader
├── server/
│   └── server.c        (146 lines) — KernelServerEntry, KernelServerMainLoop/Dispatch
├── sched/
│   ├── port.c         (1234 lines) — KernelPortOperation, shuttle state machine, DMA
│   ├── syscall.c       (576 lines) — SyscallDispatch, all syscall handlers
│   └── partition.c     (247 lines) — KernelPartitionCreate, AllocatePartitionChannel
└── common/
    ├── rbtree.c        (531 lines) — RbTreeInsert/Delete/Search, max_free_subtree
    ├── list.c           (56 lines) — ListAddTail, ListAddAfter, ListRemoveInit
    ├── bitops.c        (177 lines) — BitmapSetBit, BitmapFindAndClearBit, FindFirstSetBit
    └── mem.c           (323 lines) — MemoryFill, MemoryCopy, CopyWithAddressTranslation
```