From 70a1abc62f4d565b2345b150eff0d2613ead7d8c Mon Sep 17 00:00:00 2001 From: Ronald Caesar Date: Mon, 29 Sep 2025 23:30:37 -0400 Subject: [PATCH] docs: Remove mmio design doc The mmio will not be touchwd for a couple months while we focus on executing arm instructions. Once we get to that point this design doc will be rewritten. Signed-off-by: Ronald Caesar --- design_docs/DESIGN_DOC_MMIO.md | 107 --------------------------------- 1 file changed, 107 deletions(-) delete mode 100644 design_docs/DESIGN_DOC_MMIO.md diff --git a/design_docs/DESIGN_DOC_MMIO.md b/design_docs/DESIGN_DOC_MMIO.md deleted file mode 100644 index 4f577c7..0000000 --- a/design_docs/DESIGN_DOC_MMIO.md +++ /dev/null @@ -1,107 +0,0 @@ -## **Design Document: MMIO Subsystem** - -**Author:** GloriousTacoo, Lead Developer -**Status:** FINAL -**Version:** 1.0 -**Date:** 2025-09-20 - -*Disclaimer: This document was mostly written by AI. I'm not a good technical writer.* - -### **1. Problem Statement** - -A virtual machine is not merely a CPU core; it is a system. A system includes peripherals. In modern architectures, communication with these peripherals is overwhelmingly achieved via Memory-Mapped I/O (MMIO). A guest operating system's attempt to read from or write to a specific physical memory address is not a memory access, but a command to a virtual device. - -The MMIO subsystem is the central nervous system for this communication. It is the gatekeeper that intercepts these special memory accesses and dispatches them to the correct emulated device handlers. Its failure is not an option. A slow, buggy, or poorly designed MMIO dispatcher results in a slow, buggy, and fundamentally incorrect virtual machine. - -We require an MMIO dispatcher that is: -1. **Fast:** It sits on the most critical path of the PVM—memory access. Its overhead must be minimal to the point of being statistically insignificant for non-MMIO accesses. -2. **Correct:** It must unambiguously and correctly route every access to the one and only correct handler, or correctly identify it as a non-MMIO access. -3. **Rigid:** The registration process for MMIO regions must be strict, preventing common errors such as overlapping address spaces at system configuration time, not at runtime. -4. **Testable:** The entire subsystem must be verifiable in isolation, independent of the full PVM, to prove its correctness. - -### **2. Glossary** - -* **MMIO (Memory-Mapped I/O):** The use of a shared memory address space for both main memory and peripheral device registers. -* **Guest Physical Address (GPA):** The physical memory address as seen from within the guest virtual machine. This is the address space the MMIO subsystem operates on. -* **MMIO Region:** A contiguous, half-open range of GPAs, `[base, end)`, assigned to a single virtual device. -* **MMIO Handler:** A function pointer (or pair of function pointers for read/write) that implements the behavior of a virtual device when its MMIO region is accessed. -* **Dispatcher:** The core logic responsible for taking an incoming GPA and efficiently finding the corresponding MMIO handler. -* **MMIO Database (`mmio_db_t`):** The central data structure holding all registered MMIO regions and their associated handlers. -* **Structure of Arrays (SoA):** The chosen data layout for the MMIO database. Parallel arrays (`handlers`, `address_ranges`) are used instead of a single array of complex structs. This improves cache performance by ensuring memory accesses during the dispatch search are contiguous. - -### **3. Breaking Changes** - -* Any and all previous, ad-hoc mechanisms for device emulation are deprecated and will be removed. -* All virtual devices (e.g., UART, interrupt controller, timers) **must** register their memory regions through the `mmio_db_register` function during the PVM's initialization phase. No other mechanism for device interaction will be permitted. -* The system initialization sequence will be refactored to create and populate the `mmio_db_t` *before* the first guest instruction is executed. - -### **4. Success Criteria** - -* **Performance:** A dispatch call for a GPA that does *not* map to any MMIO region (the most common case) must complete in sub-microsecond time. The overhead should be dominated by the function call and a cache-hot binary search. -* **Correctness:** 100% of MMIO accesses must be routed to the correct handler. 100% of non-MMIO accesses must be correctly identified as such (`ENOT_HANDLED`). There is no tolerance for mis-routing. -* **Safety:** The `mmio_db_register` function must be provably correct. It must be impossible to register two overlapping MMIO regions. An attempt to do so will result in a fatal assertion failure during initialization. -* **Test Coverage:** The MMIO dispatcher and registration logic will have 100% unit test coverage for all code paths, including error conditions and edge cases. - -### **5. Proposed Design** - -The design is based on the principle of **early failure and fast paths**. We do the expensive, complex work of validation and organization once at startup, allowing the runtime dispatch path to be brutally simple and fast. The core of the system is the `mmio_db_t` structure, a purpose-built database optimized for one single operation: finding the handler for a given GPA. - -1. **Data-Oriented Structure:** The `mmio_db_t` will use a Structure of Arrays (SoA) layout. The `address_ranges` and `handlers` will be stored in separate, parallel `std::vector`s. When the dispatcher searches for an address, it will only scan through the `address_ranges` vector. This maximizes CPU cache-line utilization, as the search algorithm does not need to load and pollute the cache with handler function pointers, which are irrelevant until a match is found. - -2. **Immutable Database at Runtime:** The MMIO database is configured once during PVM initialization. After the first guest instruction is executed, the database is considered immutable. There will be no mechanism for runtime registration or de-registration of MMIO regions. This simplifies the design enormously by eliminating the need for complex locking or thread-safety mechanisms on the database itself. The dispatcher can read from it without any synchronization primitives. - -3. **High-Performance Dispatch:** The `mmio_db_dispatch_*` functions will use a binary search (`std::upper_bound`) on the `address_ranges` vector, which **must** be kept sorted by `gpa_base`. This provides logarithmic time complexity, `O(log N)`, where N is the number of MMIO regions. For the expected small number of regions (`MMIO_REGIONS` is 20), this is practically constant time and is the optimal search strategy. The search logic described in the header file's comments is sound and will be the basis of the implementation. - -4. **Strict and Unforgiving Registration:** The `mmio_db_register` function is the sole gatekeeper for the database. It is not designed for performance; it is designed for absolute correctness. Before inserting a new region, it will perform a linear scan of all existing regions to guarantee that the new region does not overlap with any of them. If an overlap is detected, it will return `EADDRESS_OVERLAP`. The caller (the PVM initializer) will treat this error as a fatal configuration flaw and will `PVM_ASSERT_MSG(false, ...)` to terminate immediately. Configuration errors must be caught at boot, not during guest execution. - -### **6. Technical Design** - -The implementation will be based on the provided header file, with the following clarifications and mandates. - -* **`mmio_db_t`**: The use of a custom arena allocator is approved. This ensures that all MMIO metadata resides in a single, contiguous block of memory, further improving locality of reference. -* **`mmio_db_register`**: - 1. The function will first iterate through the *entire* existing `address_ranges` vector. For each `existing_range` in the vector, it will check if `new_range.gpa_base < existing_range.gpa_end && new_range.gpa_end > existing_range.gpa_base`. If this condition is ever true, an overlap exists. Return `EADDRESS_OVERLAP` immediately. - 2. If no overlap is found, the function will find the correct insertion point to maintain the sort order of `address_ranges`. `std::upper_bound` can be used to find this position efficiently. - 3. The new `range` and `handler` will be inserted into their respective vectors at the calculated position. -* **`mmio_db_dispatch_read`/`write`**: - 1. Use `std::upper_bound` on `db->address_ranges` with a custom comparator that only compares the search GPA with the `gpa_base` of the `mmio_range_t` elements. - 2. This returns an iterator to the first element whose `gpa_base` is *greater than* the target GPA. - 3. If the iterator is the beginning of the vector, no preceding range exists, so it's a non-match. Return `ENOT_HANDLED`. - 4. Otherwise, decrement the iterator to get the candidate range (the one immediately before the one found by `upper_bound`). - 5. Perform the final check: `gpa >= candidate.gpa_base && gpa < candidate.gpa_end`. - 6. If it matches, retrieve the handler from the *parallel* `handlers` vector using the same index. Check if the required handler (`read` or `write`) is non-NULL. If it is NULL, return `EACCESS_DENIED`. Otherwise, call the handler and return its result. - 7. If the final check fails, the GPA is in a hole between regions. Return `ENOT_HANDLED`. - -### **7. Testing** - -The correctness of this subsystem is not a matter of luck; it will be proven. A dedicated unit test suite (`test_mmio.cpp`) will be created. It will have no dependencies on the rest of the PVM. - -* **Test Harness:** A mock `kvm_t` object will be created. A set of dummy handler functions will be implemented that, when called, simply set a flag and return a specific value, allowing the test to verify that the correct handler was indeed called. - -* **Registration Tests:** - 1. **Test Empty DB:** Test registration into a fresh, empty database. - 2. **Test Non-Overlapping:** Register several disjoint, non-adjacent regions (e.g., `[0x1000, 0x2000)`, `[0x5000, 0x6000)`) and assert that registration succeeds and the database remains sorted. - 3. **Test Adjacent:** Register two perfectly adjacent regions (e.g., `[0x1000, 0x2000)`, `[0x2000, 0x3000)`). This must succeed. - 4. **Test Overlap (Fatal):** Test every possible overlap condition and assert that `mmio_db_register` returns `EADDRESS_OVERLAP`: - * New region completely contains an existing one. - * Existing region completely contains the new one. - * New region overlaps the end of an existing one. - * New region overlaps the beginning of an existing one. - * New region is identical to an existing one. - 5. **Test Full DB:** If we define a hard limit like `MMIO_REGIONS`, test that attempting to register more than this limit fails correctly (this may require an assertion or a specific error code). - -* **Dispatch Tests:** - 1. **Setup:** Create a pre-populated database with several known regions (e.g., R1:[0x1000-0x1FFF], R2:[0x4000-0x4FFF], R3:[0x8000-0x8FFF]). R1 is read-only, R2 is write-only, R3 is read-write. - 2. **Test Hits:** - * Dispatch a read to the beginning, middle, and last byte of R1. Verify the R1 read handler is called. - * Dispatch a write to the beginning, middle, and last byte of R2. Verify the R2 write handler is called. - * Dispatch both reads and writes to R3 and verify correct handler invocation. - 3. **Test Misses (Holes):** - * Dispatch reads and writes to addresses *between* regions (e.g., `0x3000`, `0x6000`). Assert that the return code is `ENOT_HANDLED` and no handler is called. - * Dispatch to an address below all regions (e.g., `0x0`). - * Dispatch to an address above all regions (e.g., `0x9000`). - 4. **Test Access Denied:** - * Dispatch a *write* to the read-only region R1. Assert the return is `EACCESS_DENIED`. - * Dispatch a *read* to the write-only region R2. Assert the return is `EACCESS_DENIED`. - -These tests are not optional. They are the only way to gain confidence in this critical component. They will be integrated into the CI/CD pipeline and must pass before any change to this subsystem is even considered for review. subsystem is even considered for review.