---
title: "Proxmox GPU Passthrough Guide: RTX 5060 Ti to Ollama VM"
status: "verified-working-procedure"
audience: "Proxmox admin / Websonora operations"
purpose: "Configure Proxmox so an Ollama VM can see and use an NVIDIA GeForce RTX 5060 Ti"
last_updated: "2026-07-03"
---

# Proxmox GPU Passthrough Guide: RTX 5060 Ti to Ollama VM

## Objective

Configure a Proxmox host so the NVIDIA GeForce RTX 5060 Ti is not used by the Proxmox host and is instead available for passthrough to an Ollama virtual machine.

Target architecture:

```text
Proxmox host
├── VM 1: Ollama / local LLM server
│   └── Gets NVIDIA GeForce RTX 5060 Ti through PCI passthrough
└── VM 2: Hermes / Godzilla
    └── Connects to Ollama over LAN/API
```

Important principle:

Install NVIDIA drivers inside the Ollama VM, not on the Proxmox host.

For passthrough, the Proxmox host should bind the GPU to:

```text
vfio-pci
```

not:

```text
nouveau
nvidia
```

---

## 1. Confirm the GPU exists on Proxmox

Run on the Proxmox host:

```bash
lspci | grep -i -E "nvidia|vga|3d"
```

Verified result from this setup:

```text
01:00.0 VGA compatible controller: NVIDIA Corporation GB206 [GeForce RTX 5060 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GB206 High Definition Audio Controller (rev a1)
```

This confirms Proxmox can see the GPU and its HDMI/audio function.

---

## 2. Confirm IOMMU is enabled

Run:

```bash
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
```

Verified working result included:

```text
DMAR: IOMMU enabled
DMAR: Intel(R) Virtualization Technology for Directed I/O
```

This means Intel VT-d/IOMMU is active and GPU passthrough is possible.

If IOMMU is not enabled, enable it in BIOS/UEFI:

- Intel systems: enable Intel VT-d
- AMD systems: enable AMD-Vi / IOMMU

Recommended BIOS options if available:

- Above 4G Decoding: Enabled
- Resizable BAR: Enabled or Auto
- Primary Display: Integrated graphics if available

---

## 3. Confirm current GPU driver state

Run:

```bash
lspci -nnk -s 01:00.0
lspci -nnk -s 01:00.1
lsmod | grep -E "vfio|nouveau|nvidia" || true
```

Before passthrough was ready, this setup showed nouveau loaded:

```text
nouveau
```

For passthrough, nouveau must not control the GPU.

Final desired result:

```text
Kernel driver in use: vfio-pci
```

for both:

```text
01:00.0
01:00.1
```

---

## 4. Do not install NVIDIA drivers on Proxmox host

For this architecture, do not install:

```bash
apt install nvidia-driver nvidia-smi
```

The Ollama VM needs the NVIDIA driver, not the Proxmox host.

The Proxmox host should only prepare VFIO passthrough.

---

## 5. Load VFIO modules at boot

Create the VFIO modules file:

```bash
cat > /etc/modules-load.d/vfio.conf <<'EOF'
vfio
vfio_iommu_type1
vfio_pci
EOF
```

Verify:

```bash
cat /etc/modules-load.d/vfio.conf
```

Expected:

```text
vfio
vfio_iommu_type1
vfio_pci
```

---

## 6. Bind the RTX 5060 Ti to VFIO

The verified PCI IDs for this RTX 5060 Ti setup were:

```text
GPU:   10de:2d04
Audio: 10de:22eb
```

Create the VFIO binding config:

```bash
cat > /etc/modprobe.d/vfio.conf <<'EOF'
options vfio-pci ids=10de:2d04,10de:22eb disable_vga=1
EOF
```

Verify:

```bash
cat /etc/modprobe.d/vfio.conf
```

Expected:

```text
options vfio-pci ids=10de:2d04,10de:22eb disable_vga=1
```

---

## 7. Blacklist host GPU drivers

This prevents Proxmox from grabbing the GPU before the VM can use it.

Create the blacklist file:

```bash
cat > /etc/modprobe.d/blacklist-gpu.conf <<'EOF'
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nvidia_uvm
options nouveau modeset=0
EOF
```

Verify:

```bash
cat /etc/modprobe.d/blacklist-gpu.conf
```

Expected:

```text
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nvidia_uvm
options nouveau modeset=0
```

---

## 8. Rebuild initramfs and reboot

Run:

```bash
update-initramfs -u -k all
reboot
```

Important: after reboot, Proxmox may no longer use the RTX 5060 Ti for local monitor output. Use the Proxmox web UI or SSH if needed.

---

## 9. Verify passthrough readiness after reboot

Run:

```bash
echo "=== GPU ==="
lspci -nnk -s 01:00.0

echo "=== GPU AUDIO ==="
lspci -nnk -s 01:00.1

echo "=== MODULES ==="
lsmod | grep -E "vfio|nouveau|nvidia" || true
```

Verified working result:

```text
=== GPU ===
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GB206 [GeForce RTX 5060 Ti] [10de:2d04] (rev a1)
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau

=== GPU AUDIO ===
01:00.1 Audio device [0403]: NVIDIA Corporation GB206 High Definition Audio Controller [10de:22eb] (rev a1)
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

=== MODULES ===
vfio_pci
vfio_pci_core
vfio_iommu_type1
vfio
```

Success criteria:

```text
01:00.0 GPU        Kernel driver in use: vfio-pci
01:00.1 GPU Audio  Kernel driver in use: vfio-pci
nouveau is not loaded
vfio modules are loaded
```

Note: it is okay if `Kernel modules:` lists `nvidiafb` or `nouveau`. That only means those modules are available. The important line is:

```text
Kernel driver in use: vfio-pci
```

---

## 10. Create the Ollama VM

Recommended Proxmox VM settings:

```text
OS: Ubuntu Server 24.04 LTS
Machine: q35
BIOS: OVMF / UEFI
CPU Type: host
Disk Controller: VirtIO SCSI
Network: VirtIO
RAM: 32GB+ recommended
CPU Cores: 8+ recommended
```

For larger local LLM models, allocate more RAM if available.

---

## 11. Add the RTX 5060 Ti to the Ollama VM

In the Proxmox GUI:

```text
Ollama VM → Hardware → Add → PCI Device
```

Select:

```text
01:00.0 NVIDIA RTX 5060 Ti
```

Enable these options:

- All Functions: checked
- PCI-Express: checked
- ROM-Bar: checked
- Primary GPU: checked

Because All Functions is enabled, Proxmox should include both:

```text
01:00.0 GPU
01:00.1 GPU Audio
```

If Proxmox shows the audio function separately, make sure it is included or add it separately.

---

## 12. Install NVIDIA driver inside the Ollama VM

Inside the Ollama VM, not on the Proxmox host:

```bash
sudo apt update
sudo apt install -y ubuntu-drivers-common
ubuntu-drivers devices
sudo ubuntu-drivers autoinstall
sudo reboot
```

After reboot inside the VM:

```bash
nvidia-smi
```

Expected result:

```text
GeForce RTX 5060 Ti
```

If `nvidia-smi` works inside the VM, GPU passthrough is successful.

---

## 13. Install Ollama inside the Ollama VM

Inside the Ollama VM:

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

Verify Ollama:

```bash
ollama --version
```

Run a test model:

```bash
ollama run llama3.2:3b
```

In another terminal inside the VM, verify GPU usage:

```bash
nvidia-smi
```

You should see GPU memory usage while the model is loaded or running.

---

## 14. Expose Ollama to the Hermes VM

Inside the Ollama VM, edit the Ollama service override:

```bash
sudo systemctl edit ollama
```

Add:

```ini
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
```

Reload and restart:

```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```

Verify Ollama is listening:

```bash
ss -tulpn | grep 11434
```

From the Hermes VM, test connectivity:

```bash
curl http://OLLAMA_VM_IP:11434/api/tags
```

Replace `OLLAMA_VM_IP` with the actual IP address of the Ollama VM.

---

## 15. Troubleshooting

### GPU is visible but not bound to VFIO

If this command:

```bash
lspci -nnk -s 01:00.0
```

shows:

```text
Kernel driver in use: nouveau
```

then Proxmox is still grabbing the GPU.

Check:

```bash
cat /etc/modprobe.d/blacklist-gpu.conf
cat /etc/modprobe.d/vfio.conf
cat /etc/modules-load.d/vfio.conf
update-initramfs -u -k all
reboot
```

### IOMMU not enabled

If this command:

```bash
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
```

shows no IOMMU enabled message, check BIOS settings and boot parameters.

For Intel systems, GRUB usually needs:

```text
intel_iommu=on iommu=pt
```

For AMD systems, GRUB usually needs:

```text
amd_iommu=on iommu=pt
```

### Proxmox host loses local display

This can happen after binding the GPU to VFIO. It is normal if the RTX 5060 Ti was the only display output. Manage Proxmox through:

- Proxmox web UI
- SSH

### NVIDIA driver should not be installed on Proxmox host

For VM passthrough, install NVIDIA drivers inside the Ollama VM only.

---

## Final Verified State

The Proxmox host is ready for GPU passthrough when this is true:

```text
01:00.0 RTX 5060 Ti GPU       Kernel driver in use: vfio-pci
01:00.1 RTX 5060 Ti Audio     Kernel driver in use: vfio-pci
IOMMU                         enabled
nouveau                       not loaded
vfio modules                  loaded
```

This setup was verified with:

```text
NVIDIA GeForce RTX 5060 Ti
GPU PCI ID:   10de:2d04
Audio PCI ID: 10de:22eb
Proxmox kernel: 6.17.2-1-pve
```