vGPU on Azure Local (and Hyper-V)

Table of Contents

Preface

There are basically three ways to get a GPU into a Hyper-V VM, and it’s worth knowing which one you’re actually reaching for before you start pasting commands off a blog (this one included):

  • DDA (Discrete Device Assignment): you rip the whole physical GPU out of the host and hand it, in one piece, to a single VM. Great performance, but it’s one card, one VM, and you lose live migration.
  • GPU-P (GPU Partitioning): you carve one physical GPU into several partitions and hand a slice to each VM. This is the SR-IOV-based feature that Azure Local and Windows Server 2025 actually want you to use, and on Azure Local it supports live migration. This is what we’re doing here.
  • vGPU: NVIDIA’s licensed flavor of the above. On the datacenter cards you install NVIDIA’s vGPU host driver (the nvgridswhci.inf you’ll see below), the host driver is what makes the partitions real, and the guests run a matching NVIDIA guest driver against a license.

So when I say “vGPU on Azure Local,” what I really mean is GPU-P on the host, backed by the NVIDIA vGPU host driver, with a slice pinned into each VM. That’s the combo I’ll walk through. My example splits one card into 4 partitions, so up to four VMs can each get an identical slice.

One thing to get straight before we start: the host prep is the same whether you’re on standalone Windows Server or Azure Local — install the driver, confirm the card’s partitionable, set the partition count. Where they part ways is how you hand a partition to a VM. On Azure Local that’s an Arc operation you do from the portal or Azure CLI; on Windows Server it’s the older Add-VMGpuPartitionAdapter PowerShell route. I’ll cover both, and call out which is which.

Pre-reqs

  • A GPU that actually supports partitioning. Datacenter NVIDIA cards (A2/A16-class Ampere and friends) do; check Get-VMHostPartitionableGpu returns something once the driver’s in.
  • Check Microsoft’s Azure Local GPU support matrix before you buy or build. It changes often and has some major gotchas. For example, the A10 and A40 can’t do DDA to Arc VMs or AKS, the T4 can’t do GPU-P at all, AKS Arc doesn’t support GPU partitions, and GPU VMs must be Generation 2. GPU-P also needs SR-IOV and virtualization enabled in each node’s BIOS.
  • Windows Server 2025 Datacenter or Azure Local. Server 2022 can only do DDA (whole-GPU pass-through) with NVIDIA — no GPU-P.
  • The NVIDIA vGPU host driver package (contains nvgridswhci.inf). This is not the GeForce/Studio driver off nvidia.com — it comes from the NVIDIA vGPU/licensing portal, and the host driver version has to line up with the guest driver version you’ll install later.
  • Server core / no display attached is fine — you don’t need the card driving a monitor.

Step 1 — Install the vGPU host driver

Point pnputil at the .inf and let it pull in the whole package. /subdirs lets it recurse into the extracted folder, /install binds it immediately, and /reboot because the display stack really does want a clean restart before it’ll behave.

pnputil /add-driver nvgridswhci.inf /subdirs /install /reboot

Run this from the directory where you extracted the host driver package. After the reboot the card should show up as an NVIDIA device with a real driver bound — which we’ll confirm next.

Step 2 — Confirm the host sees a partitionable GPU

First, the money question — does Hyper-V consider this card partitionable at all?

Get-VMHostPartitionableGpu

If the host driver is in and the card supports it, you’ll get one entry back per physical GPU, and the important field is Name — a big ugly \\?\PCI#... string. Hang onto that; it’s the exact value the next step wants.

Now, if you’ve got more than one NVIDIA device in the box (or you just want to map that \\?\PCI# string back to a real card), find the NVIDIA PnP devices. VEN_10DE is NVIDIA’s PCI vendor ID, so this filter catches every NVIDIA device present:

Get-PnpDevice -PresentOnly |
    Where-Object { $_.InstanceId -like "PCI\VEN_10DE*" } |
    Select-Object -Property FriendlyName, InstanceId |
    Format-List

That gives you the PnP InstanceId, e.g.:

PCI\VEN_10DE&DEV_25B6&SUBSYS_157E10DE&REV_A1\4&12F3C2E8&0&0008

Same GPU, just written as a PnP instance ID instead of the interface path. Note the 4&12F3C2E8&0&0008 chunk — that’s the bit that lets you line this up with the \\?\PCI#...4&12f3c2e8&0&0008#... string from Get-VMHostPartitionableGpu. Same card, two notations.

While you’re here, sanity-check that a driver is actually bound to it. Feed the InstanceId from above into Get-PnpDeviceProperty and pull the driver key:

Get-PnpDeviceProperty -InstanceId "PCI\VEN_10DE&DEV_25B6&SUBSYS_157E10DE&REV_A1\4&12F3C2E8&0&0008" |
    Where-Object { $_.KeyName -eq "DEVPKEY_Device_Driver" } |
    Select-Object -Property Data

If that comes back empty, the host driver didn’t bind — go back to Step 1. If it returns a driver key, you’re good.

Step 3 — Set the partition count

This is where you decide how many slices the card gets cut into. PartitionCount 4 splits it four ways. The -Name value is that \\?\PCI#... string from Get-VMHostPartitionableGpu — the same GPU as the PnP InstanceId above, just in device-interface form (that trailing {064092b3-...} GUID is the device-interface class; it’s how Hyper-V references the card).

Set-VMHostPartitionableGpu -Name "\\?\PCI#VEN_10DE&DEV_25B6&SUBSYS_157E10DE&REV_A1#4&12f3c2e8&0&0008#{064092b3-625e-43bf-9eb5-dc845897dd59}" -PartitionCount 4

Gotcha — keep the -Name string on ONE line. When you copy that path around, it’s stupidly easy to end up with a newline stuffed inside the quotes right before the closing ". If you do, the embedded newline becomes part of the string, it no longer matches any GPU, and the cmdlet either errors or silently does nothing. One line, no trailing newline, no leading space.

Pick your partition count based on how much you’re willing to give each VM — more partitions means each slice gets less VRAM and fewer engines. You can’t over-subscribe: 4 partitions means a hard maximum of 4 VMs sharing this card. You also can’t pick an arbitrary number — the card only allows specific counts. Check the ValidPartitionCounts field before you commit:

Get-VMHostPartitionableGpu | Format-List Name, ValidPartitionCounts, PartitionCount

On my card that comes back {16, 8, 4, 2, 1}, so 4 is legal. If you’re running a cluster, set the same count on every node — the config has to be homogeneous or partitions won’t place / migrate cleanly.

Step 4 — Assign a partition to a VM

This is where Azure Local and standalone Windows Server split.

Azure Local: assign it from the portal (or Azure CLI)

On Azure Local, don’t use Add-VMGpuPartitionAdapter. Azure Local VMs are Arc resources, so you attach the partition from Azure (portal or CLI), the same way you’d resize the VM, and Azure Local handles the partition plumbing.

Azure portal — for an existing VM:

  1. Open your Azure Local resource → Virtual machines → pick the VM.
  2. Under Settings, select Size.
  3. For GPU Setup, choose GPU Partition, then Save.

Azure Local shuts the VM down, attaches the partition, and restarts it for you. (Creating a new VM? There’s an Attach GPU → GPU Partition step in the create wizard that does the same thing.)

Azure CLI — Microsoft’s recommended method, and the one to use if you’re scripting a fleet. Attach after creation (--gpus GpuP is the partition flavor; GpuDDA is the whole-GPU passthrough flavor). I run az from PowerShell, so the line continuations below are backticks — swap them for \ if you’re in bash:

# VM has to be stopped first
az stack-hci-vm stop --name "JT2-AVDPAW-0" --resource-group "rg-azlocal"

az stack-hci-vm gpu attach `
    --resource-group "rg-azlocal" `
    --custom-location "cl-azlocal" `
    --vm-name "JT2-AVDPAW-0" `
    --gpus GpuP

You can also bake it in at creation time with az stack-hci-vm create ... --gpus GpuP, and tear it back off with az stack-hci-vm gpu detach.

A few Azure Local realities worth knowing before you start:

  • One partition per VM. You can’t stack multiple partitions onto a single Azure Local VM.
  • You don’t choose which partition. Partitions are auto-assigned. You tell it “give this VM a GpuP partition,” and Arc picks a free one.
  • You don’t hand-tune the slice here. No Min/Max/Optimal VRAM/encode/decode knobs — the slice size is whatever your host PartitionCount from Step 3 works out to. (In the CLI you can pass a partition size, and it maps to the minPartitionVRAM you’d see in Get-VMHostPartitionableGpu, but there’s no per-engine tuning like the standalone cmdlet gives you.)

That’s the whole assignment step on Azure Local. Skip ahead to In the guest.

Windows Server / standalone Hyper-V: Add-VMGpuPartitionAdapter

On plain Windows Server 2025 / standalone Hyper-V (no Arc), this is your tool, and it’s the only route that lets you hand-tune the per-partition resource split.

First, read what one slice is worth — don’t copy numbers from the internet. After Step 3 the driver reports the per-partition budget (VRAM, encode, decode, compute) as Min / Max / Optimal, derived from the partition count you chose. Double the count and each slice’s VRAM roughly halves:

Get-VMHostPartitionableGpu -Name "\\?\PCI#VEN_10DE&DEV_25B6&SUBSYS_157E10DE&REV_A1#4&12f3c2e8&0&0008#{064092b3-625e-43bf-9eb5-dc845897dd59}" |
    Format-List Name, PartitionCount, `
        MinPartitionVRAM, OptimalPartitionVRAM, MaxPartitionVRAM, `
        MinPartitionEncode, OptimalPartitionEncode, MaxPartitionEncode, `
        MinPartitionDecode, OptimalPartitionDecode, MaxPartitionDecode, `
        MinPartitionCompute, OptimalPartitionCompute, MaxPartitionCompute

At PartitionCount 4 on my card, that comes back like this:

PartitionCount          : 4
MinPartitionVRAM        : 3825205248
OptimalPartitionVRAM    : 3825205248
MaxPartitionVRAM        : 3825205248
MinPartitionEncode      : 9
OptimalPartitionEncode  : 9
MaxPartitionEncode      : 9
MinPartitionDecode      : 6
OptimalPartitionDecode  : 6
MaxPartitionDecode      : 6
MinPartitionCompute     : 512
OptimalPartitionCompute : 512
MaxPartitionCompute     : 512

Min is the guaranteed floor, Max the ceiling, and Optimal the driver’s recommendation. On a vGPU card they usually come back identical, because the partitions are a fixed, even split of the card. So the assignment below just sets Min = Max = Optimal to what the driver reported.

Now feed those numbers into Add-VMGpuPartitionAdapter. The VM has to be off for this — you can’t hot-add a partition adapter:

Add-VMGpuPartitionAdapter -VMName JT1-JUMP-D1 `
    -MinPartitionVRAM 3825205248 `
    -MaxPartitionVRAM 3825205248 `
    -OptimalPartitionVRAM 3825205248 `
    -MinPartitionEncode 9 `
    -MaxPartitionEncode 9 `
    -OptimalPartitionEncode 9 `
    -MinPartitionDecode 6 `
    -MaxPartitionDecode 6 `
    -OptimalPartitionDecode 6 `
    -MinPartitionCompute 512 `
    -MaxPartitionCompute 512 `
    -OptimalPartitionCompute 512

Or skip the transcription and let PowerShell fill in the live values. This version also keeps working if you re-partition the card later:

$gpu = Get-VMHostPartitionableGpu -Name "\\?\PCI#VEN_10DE&DEV_25B6&SUBSYS_157E10DE&REV_A1#4&12f3c2e8&0&0008#{064092b3-625e-43bf-9eb5-dc845897dd59}"

Add-VMGpuPartitionAdapter -VMName JT1-JUMP-D1 `
    -MinPartitionVRAM      $gpu.MinPartitionVRAM `
    -MaxPartitionVRAM      $gpu.MaxPartitionVRAM `
    -OptimalPartitionVRAM  $gpu.OptimalPartitionVRAM `
    -MinPartitionEncode    $gpu.MinPartitionEncode `
    -MaxPartitionEncode    $gpu.MaxPartitionEncode `
    -OptimalPartitionEncode $gpu.OptimalPartitionEncode `
    -MinPartitionDecode    $gpu.MinPartitionDecode `
    -MaxPartitionDecode    $gpu.MaxPartitionDecode `
    -OptimalPartitionDecode $gpu.OptimalPartitionDecode `
    -MinPartitionCompute   $gpu.MinPartitionCompute `
    -MaxPartitionCompute   $gpu.MaxPartitionCompute `
    -OptimalPartitionCompute $gpu.OptimalPartitionCompute

Run whichever form you like against each additional VM you want to give a slice to (just swap -VMName).

What those knobs actually mean:

  • VRAM is in bytes. 3825205248 is 3,648 MiB (about 3.56 GiB) per partition — the framebuffer each VM gets, and the number that matters most. Want more per VM? You don’t just raise this value; you lower the PartitionCount back in Step 3 so the driver hands out fewer, larger slices, then re-read the new figures.
  • Encode / Decode are the NVENC / NVDEC video engine allocations — matters a lot for AVD, RDP with GPU acceleration, and anything doing video.
  • Compute is the slice of the CUDA/compute engine.

Start the VM and check the adapter took:

Get-VMGpuPartitionAdapter -VMName JT1-JUMP-D1

In the guest

Attaching the partition is only half the job — the guest still needs a driver, and this is where the datacenter-vGPU path diverges from the consumer-GPU-P hack you may have seen floating around:

  • Datacenter / vGPU cards (what we’re doing): install the matching NVIDIA vGPU guest driver inside the VM — the version has to line up with the host driver from Step 1 — and then license it. No license, and the guest driver throttles/nags.
  • Consumer GeForce cards: there’s no vGPU guest driver, so the community trick is to copy the host’s driver files out of C:\Windows\System32\HostDriverStore\FileRepository\... into the guest’s System32\HostDriverStore. That works, but it’s the unlicensed hobbyist path — not what you’d run on Azure Local.

Dropping in the license token

Modern vGPU licensing (NVIDIA License System — cloud NLS or an on-prem DLS) hands you a client configuration token, a .tok file. Licensing the guest is literally just putting that file in the right folder inside the VM and restarting the driver service — there’s no license server address to type in.

The token has to go in this exact directory in the guest (create the folder if it isn’t there):

# Windows guest — drop the .tok file here:
C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken\

# Linux guest, for completeness:
# /etc/nvidia/ClientConfigToken/

Then kick the NVIDIA display service so it picks the token up (or just reboot the VM):

Restart-Service NVDisplay.ContainerLocalSystem

Gotcha — permissions on the token folder. The ClientConfigToken directory has to be readable by the NVIDIA service (NVDisplay.ContainerLocalSystem, which runs as SYSTEM). If you created the folder by hand and it inherited some locked-down ACL, the driver silently won’t find the token and stays unlicensed. Default inherited perms from Program Files are fine; just don’t get clever with it.

Once the guest driver’s happy and licensed, Device Manager in the VM shows the NVIDIA adapter, nvidia-smi reports a licensed state, and dxdiag / your app will actually use it.

Gotchas & things I’d double-check

  • The VM has to be stopped to attach a partition. On standalone Hyper-V you shut it down first; on Azure Local the portal/CLI stops and restarts it for you.
  • Host and guest driver versions must match. vGPU is strict about this — mismatched versions = the guest driver refuses to load. Re-do Step 1 and the guest driver together as a pair when you update.
  • Large-BAR / MMIO space. NVIDIA’s GPU-P guidance is to set the VM’s MMIO space before first boot (Set-VM -GuestControlledCacheTypes $true -LowMemoryMappedIoSpace 1GB -HighMemoryMappedIoSpace 32GB). I got away without it on this card, but if your guest driver throws a code 43, this is the first place to look.
  • Live migration. On Azure Local, GPU-P partitions can live migrate, but there’s fine print: the OS build has to be 26100.xxxx or newer, host and guests need NVIDIA vGPU software v18+ (and Azure Local 2504 or later), the CPUs need IOMMU DMA bit tracking (Sapphire Rapids / EPYC 7003 and newer — EPYC 7002 does GPU-P but can’t migrate it), and every node needs the same card, the same host driver, and a free partition on the target. Miss one of those and the VM has nowhere to land. It also falls back to compressed TCP for the migration, so expect it to be slower and heavier on host CPU than a normal live migration.
  • Homogeneous or bust. Every node in the cluster needs the same GPU make/model and the same PartitionCount. Mixed vendors, mixed models, or mismatched partition counts = unsupported, and placement/migration gets unhappy fast.
  • Don’t over-slice. Four partitions of ~3.5 GB is fine for jump boxes and PAWs. If you’re doing real GPU compute in the guest, fewer/bigger partitions beats more/smaller ones every time.

That’s the whole flow: host driver → confirm partitionable → set the count → attach a slice to the VM (portal/CLI on Azure Local, Add-VMGpuPartitionAdapter on Windows Server) → guest driver + license. Once it clicks it’s genuinely easy to live with, and being able to give a jump box or a PAW a real GPU slice — without dedicating a whole card to it — is a nice trick to have in the bag.