XMemGuard Blocks Windows Shellcode Injection: Research Results and Deployment Limits

Why this matters to Windows defenders​

Shellcode injection is the fileless trick behind much commodity malware and many advanced persistent threats. An attacker with code running in one process gets a handle to another process, usually a trusted one. The attacker puts a payload into that process and redirects execution to it. Nothing needs to touch the disk.

The authors split the chain into three stages:

• Acquisition: the attacker gets an executable memory region in the target.
• Writing: the shellcode is copied into that region.
• Triggering: control flow is redirected to the shellcode.

The paper argues that most defenses watch the trigger stage. The authors say trigger techniques keep multiplying. Examples are remote threads and APCs, plus newer methods that use thread-pool work items, fibers, loader notifications and waiting-thread hijacking. Any detector that tries to list every trigger will eventually miss a variant. Writing alone proves little, because legitimate software writes into other processes too.

Acquisition is the steadier choke point. Every one of those triggers needs memory that is executable and attacker-controlled. XMemGuard watches for that.

The gap in existing Windows mitigations​

The paper describes the existing protections this way:

• DEP/NX: it stops a process from executing its own data pages. It does not stop the kernel from creating executable pages on request.
• Arbitrary Code Guard (ACG): the paper calls it an application-level, opt-in mitigation. A protected process cannot request new executable pages in its own address space. An unprotected remote process can still obtain executable memory in a target.
• CFG and CET: these constrain where control can go. They do not constrain how executable memory was acquired. The authors call them orthogonal to this problem.

When process A uses a call such as NtAllocateVirtualMemory or NtProtectVirtualMemory against process B, the kernel mainly checks that the caller holds the right handle and access rights. It does not ask whether the remote executable allocation makes sense. XMemGuard adds that question.

How XMemGuard works​

The design is described as preemptive intervention with deferred confirmation. It has two stages.

Stage one: silent permission downgrade​

• Interception point: the driver hooks the path every system call takes after KiSystemServiceRepeat but before the SSDT dispatch. It does this by swapping the HalCollectPmcCounters entry in the kernel's HalPrivateDispatchTable for its own detour function. The paper says this does not modify the SSDT or patch kernel code. It builds on the publicly demonstrated InfinityHook technique.
• What it watches: five system calls in three categories. These are allocating a private region with execute permission, adding execute permission to an existing region, and mapping a section view with execute permission. The paper names NtAllocateVirtualMemory, NtProtectVirtualMemory and NtMapViewOfSection. It also monitors the two Ex variants, NtAllocateVirtualMemoryEx and NtMapViewOfSectionEx, for forward compatibility. No sample in the dataset used the Ex calls.
• What it does: it removes the execute bit from the request and lets the call proceed. A request for PAGE_EXECUTE_READWRITE becomes PAGE_READWRITE. The call succeeds and the attacker gets an ordinary data page. The Bioengineer report adds that the driver returns STATUS_SUCCESS and reports the originally requested protection to memory queries, so the defense is hard to spot. I could verify the downgrade-and-proceed behavior in the paper text I saw. The success-code and query-spoofing details come from the news report.
• Filtering: same-process operations and cross-process requests that do not ask for execute are released early. The paper also describes a whitelist for trusted source and target pairs.
• Bookkeeping: the affected range is registered as a controlled region. The first time a target process gets one, the driver injects a monitoring component by kernel-mode manual mapping.

Stage two: catching the execution attempt​

If the attacker later tries to run code from that region, the CPU's NX protection raises a page fault. The monitoring component captures the exception at the user-mode exception-dispatch boundary and reports the faulting address to the driver. The driver matches it against its controlled-region records. It then applies the configured response. The Bioengineer report says that can mean ending the thread or process, or restoring execute permission to collect more attack data.

The paper's framing is that the permission change an attacker would normally notice at allocation time becomes a hardware exception that appears only at execution time.

An important nuance is that removing the execute bit is not the same as confirming malice. The downgrade is enforced up front. Confirmation arrives only if something tries to execute from the region.

The numbers​

Proof-of-concept detection​

The paper says neither FalconEye nor Starink'23 detected any PoolParty variant or any emerging injection sample. Di Pietro'25 missed PoolParty Start Routine, Thread Name-Calling and Waiting-thread hijacking. Their triggers fell outside its predefined rules. All three still acquired memory through ordinary allocation and protection calls, which XMemGuard intercepted.

This is a result on a hand-built set of 33 variants. It is not a guarantee against every injection technique.

In-the-wild samples​

The final evaluation set held 1,143 malware and 1,091 benign samples.

• Precision: 99.79 percent.
• Recall: 81.54 percent.
• F1: 89.74 percent, the highest accuracy, recall and F1 of the methods compared.
• Competitors: FalconEye and Di Pietro'25 both reached 100 percent precision, with recall of 66.84 and 60.89 percent.
• Unique catches: 122 malware samples were detected only by XMemGuard.
• Acquisition calls: NtAllocateVirtualMemory appeared in 90.03 percent of samples and was the only acquisition call in 76.99 percent. NtProtectVirtualMemory appeared in 17.59 percent. NtMapViewOfSection appeared in 1.40 percent, though one sample relied on it entirely.

The recall figure needs context. For 150 samples that other tools caught, XMemGuard intercepted the acquisition but counted no final outcome within the fixed 180-second window. Another 61 samples did not show cross-process executable-memory acquisition during the runs. So the authors read much of the gap as an observability issue, not a failure to see the calls. A skeptic could say a detector that does not confirm within three minutes has still not given you an alert. The enforcement did happen, though, because the region was already non-executable.

False positives​

The two false positives were draw.io and 7+ Taskbar Tweaker.

• draw.io: it behaves like an Electron-style multi-process app. It repeatedly spawns child processes of the same image and loads initialization code into them with RWX allocations and protection changes.
• 7+ Taskbar Tweaker: it injects its own code into explorer.exe.

At the syscall level these look like injection, and the authors say acquisition-stage evidence alone cannot tell them apart. The remaining 1,089 benign samples never triggered interception. The practical lesson is that any real deployment needs allowlisting and compatibility testing. That matters most for shell extensions, taskbar and UI customizers, and security products.

Performance​

The authors report that overhead is concentrated in cross-process memory-heavy work.

• Among seven application workloads, only Git clone showed a clear slowdown, at 4.81 percent.
• The PCMark 10 overall score fell 3.22 percent, and no category dropped more than 6.98 percent.
• The framework held about 344 KB of kernel pool with no controlled targets, plus 128 bytes per controlled region.
• A 24-hour run showed no bugcheck, no memory growth and no loss of detection.

These are results from the authors' test machine and workloads. They are not proof of production reliability.

What it does not cover​

The threat model assumes the attacker has user-mode code execution in a source process and a handle with suitable rights, such as PROCESS_VM_OPERATION and PROCESS_VM_WRITE. It assumes no kernel-mode capability and that PatchGuard stays effective.

Several techniques fall outside it:

• ROP and other code reuse: these create no new executable memory. The authors point to CET and CFG.
• Mockingjay: it reuses pre-existing RWX sections in signed DLLs. The authors say such DLLs are rare and shrinking, and they suggest constraining section permissions at image load time.
• Remote RW write plus in-process ROP: an attacker writes shellcode as read-write remotely, then uses an in-process ROP chain to call VirtualProtect. That makes execute permission inside the target. The authors say control-flow defenses should handle it. They add that a target with ACG enabled would refuse the in-process change.

The design is meant to sit alongside ACG, CFG and CET, not replace them.

Deployment reality check​

This is where an admin should slow down.

• Tested platform: Windows 10 22H2 x64 only. The paper notes that adapting to another build means re-deriving the interception locations. Do not assume it works on current Windows 11 builds.
• Kernel fragility: the technique redirects a function pointer in HalPrivateDispatchTable. The authors say no PatchGuard failure occurred in their experiments. They also warn that a future kernel change could alter the dispatch path or bring this table under integrity monitoring, which would need a different mechanism.
• Code availability: The GitHub repository is a research implementation. It builds as XMemGuard.sln in Release x64 with Visual Studio 2022, MSVC v143 and the Windows SDK/WDK. Deployment needs a log directory, and the DLL has to sit at a fixed path. That is a lab setup, not an enterprise package.
• Driver risk: hooking kernel paths in production brings stability, compatibility and support concerns. These are general industry considerations, not claims from the paper.

What admins can do today​

Nothing here requires XMemGuard. The paper's analysis points to existing controls:

1. Enable ACG where application compatibility allows. It blocks in-process dynamic code generation.
2. Use CFG- and CET-capable software and hardware where supported.
3. Monitor for cross-process RWX allocations and protection changes with your EDR. They are a strong injection signal, but expect benign hits from Electron-style apps and UI tweakers.
4. Keep an allowlist of known legitimate injectors, so a real alert stands out.

Bottom line​

XMemGuard's idea is simple: stop caring how the attacker triggers the shellcode, and refuse to hand over executable memory in the first place. The numbers on the authors' corpus are strong. Recall of 81.54 percent shows a real gap, and the tested configuration is a single Windows 10 build. This is a promising research direction, and it shows one thing worth watching in future Windows hardening. Treat it as research, not as a product you can deploy.