So, I had a situation where proxmox would not update the linux image correctly and discovered that one of my boot ZSF drives had failed. I found this kb on the fix for it and I have fine tuned it for my use case with GRUB. More info is also available here.

Determine the Proxmox Boot System

First it is important to determine the Proxmox boot system.

proxmox-boot-tool status

For me it was this

root@pve06:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
297A-7BD9 is configured with: grub (versions: 6.14.11-5-pve, 6.17.13-1-pve, 6.17.4-2-pve)
297A-D82E is configured with: grub (versions: 6.17.13-7-pve, 7.0.0-3-pve, 7.0.2-2-pve)

So yes I am using grub so I will need to flag that on the commands when I recreate the boot partition and also noticed that Proxmox is booting with uefi so that makes it really important to get this correct!

Also, it is helpful to get the current disk partition info by id.

ls -l /dev/disk/by-id/

Copy Partitions from Good Disk to New Disk

The first steps of copying the partition table, reissuing GUIDs and replacing the ZFS partition are the same. To make the system bootable from the new disk, different steps are needed which depend on the bootloader in use.

# sgdisk <healthy bootable device> -R <new device>
# sgdisk -G <new device>
# zpool replace -f <pool> <old zfs partition> <new zfs partition> 

Format and INIT ESP for Boot

With proxmox-boot-tool:

# proxmox-boot-tool format <new disk's ESP>
# proxmox-boot-tool init <new disk's ESP> [grub]

Note that in my case I will be adding the grub to this command.

Monitoring the Re-silvering

ZFS will require some time to complete the re-silvering process.

zpool list -v

That is it if anything changes I will update this article.