Advanced Policy Concepts

Once you've mastered basic allocation and eviction, you can leverage USM's more advanced features to achieve state-of-the-art performance for specialized workloads. This guide introduces three powerful concepts discussed in the USM research paper.

1. Asynchronous Fault Handling for uThreads

The Problem: In systems that use user-level threading (uThreads), like Shenango or Junction, a major page fault that requires slow I/O (e.g., reading from disk) can be disastrous. The kernel thread (kThread) executing the faulting uThread will block, preventing other ready-to-run uThreads scheduled on that same kThread from making progress.

The USM Solution: An advanced USM policy can handle faults asynchronously.

• The usm_alloc function is triggered for a faulting uThread.
• Instead of blocking on I/O, the policy initiates the data transfer in the background.
• It immediately returns control to the LocalManager (the uThread scheduler).
• The LocalManager can then schedule a different, non-blocked uThread onto the kThread, keeping the CPU core busy with useful work.

As the paper shows, this strategy can improve the performance of co-located CPU-intensive tasks by up to 2.26x.

// Conceptual Asynchronous Policy
int async_aware_alloc(struct usm_event *event, struct usm_event *async_rq) {
    // Check if the fault requires slow I/O (e.g., it's a swap-in)
    if (is_swap_fault(event)) {
        // Don't block. Instead, populate the async_rq (asynchronous request)
        // structure to tell USM to handle this in the background.
        setup_async_swap_in(async_rq, event);

        // Return a special code to tell the LocalManager to schedule
        // the next uThread immediately.
        return USM_YIELD;
    }

    // ... handle fast-path, in-memory allocations synchronously ...
    return regular_alloc(event);
}

2. Application-Specific Hints

The Problem: An OS memory manager is fundamentally "blind." It has no understanding of the application's data. It might evict a critical index page from a database while keeping a rarely used data cache page in memory, simply because the data cache was touched more recently.

The USM Solution: USM provides a hinting library (lib_usm) that allows an application to send arbitrary, custom information directly to its memory policy.

Example Use Case: A Smarter Database Eviction Policy

1. Application Code: The database, when loading its index, uses lib_usm to "tag" that memory region.

   // Database Application Code
   index_buffer = mmap(...);
   usm_hint(index_buffer, "PRIORITY=HIGH; EVICTION=DENY");
   
   data_cache = mmap(...);
   usm_hint(data_cache, "PRIORITY=LOW; EVICTION=ALLOW");
   

2. Policy Code: The usm_evict_fallback function receives these hints. When choosing a victim page, it can now make an intelligent, application-aware decision.

   // Eviction Policy Code
   struct page* choose_victim_page() {
       // Iterate through used pages and check their hints.
       for each page in usedList {
           const char* hint = get_hint_for_page(page);
           if (hint && strcmp(hint, "EVICTION=ALLOW") == 0) {
               return page; // Found a low-priority victim!
           }
       }
       // ... fallback to other logic if no low-priority pages are found ...
   }
   

3. Flexible Huge Pages (FHP)

The Problem: Hardware huge pages (e.g., 2MB or 1GB) can improve performance by reducing TLB misses, but their large, fixed sizes can lead to significant internal fragmentation if an application doesn't need the full page.

The USM Solution: A policy can create its own "Flexible Huge Pages" of any arbitrary size (e.g., 64KB, 128KB).

• How it Works: The usm_alloc function, upon receiving the first page fault in a new memory region, doesn't just allocate one page. It preemptively allocates and maps N contiguous virtual pages.
• The Benefit: This drastically reduces the number of page fault events the kernel has to process, lowering overhead. The get_pages_prep function in policiesSet1.c is a real-world example of this batch-allocation logic.
• The Result: The paper shows this can improve the performance of fault-sensitive applications like Redis and memflt by 8-88%.

Explaining videos