First, I really need to document the overall configuration of my home lab. This thing has grown into a major project over the last few years. Anyway, for now I wanted to document setting Proxmox cluster behind an HAProxy Load Balancer.

Config

3 node Proxmox cluster behind HAProxy Load Balancer running on pfSense.

Step 1 – Setup Backend

1. Click the Add button
2. Backend window will popup
3. Put in a name
   • 
4. Add the servers to the server list. Oh, Proxmox is ssl so remember to check that option for each server and set the weight I set all to 100 .
   • 
5. Add needed cookie settings by clicking the + symbol under the server list item
   • 
   • The addition options open like below
   • 
6. Once all servers are configured. Now configure load balancing
   • 
7. Now, Configure checks
   • 
8. Finally, complete the cookies option
   • 
9. All done, Click the save button at the bottom of the screen.

Step 2 – Configure the Frontend

1. Select the Frontend from HAProxy menu at the top of the page.
2. Next, click the Add button at the bottom of the frontend list.
3. Now, A new window should appear. Fill in the below items.
   • 
4. Add the associated address and port. In my case it is LAN so I have selected that below listening on 443. Note: If you have not configured Web GUI to be in HAProxy as well and to use http you will have issues using 443. See this article, Configure HAProxy To Serve pfSense GUI. You have been warned!!!
   • 
5. Now, add the pve url to the Access Control List as shown below.
   • 
6. Next, add the associated action item for above selection of pve.
   • 
7. Then, add  “forwardfor” option
   • 
8. Finally, complete the SSL Offloading started in step 4.
   • 
9. Press the Save button at the bottom of the window.

Step 3 – Apply all the settings for HAProxy

May be needed on new Installs

Enable HAProxy and set Maximum connections for new installs if other HAProxy configuration has been done this should already be configured.

Configure DNS

So, this section is very customized for what is configured in pfSense. So, be prepared to adapt as needed.

Step 1 – Set DNS Resolver

Available in, Services – > DNS Resolver

Remember, DNS much match the HAProxy access control setup in Step 2.5 above and IP must point to LAN address associated in Step 2.4 above.

My Current Config

Below is my current haproxy config file for those interested in using haproxy directly rather than through pfSense

# Automaticaly generated, dont edit manually.
# Generated on: 2022-10-29 10:56
global
	maxconn			1000
	log			/var/run/log	local0	info
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	uid			80
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	log-send-hostname		haproxy.log
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats refresh 5
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend wan-ad-alshowto-com
	bind			165.23.33.29:443 name 165.23.33.29:443   ssl crt-list /var/etc/haproxy/wan-ad-alshowto-com.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			matches-rtr	var(txn.txnhost) -m str -i rtr.alshowto.com
	acl			aclcrt_wan-ad-alshowto-com	var(txn.txnhost) -m reg -i ^([^\.]*)\.alshowto\.com(:([0-9]){1,5})?$
	acl			aclcrt_wan-ad-alshowto-com	var(txn.txnhost) -m reg -i ^alshowto\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	use_backend gui-lan-ad-server_ipvANY  if  matches-rtr aclcrt_wan-ad-alshowto-com
	use_backend gui-lan-ad-server_ipvANY  if   aclcrt_wan-ad-alshowto-com

frontend lan-ad-alshowto-com-copy
	bind			192.168.0.1:443 name 192.168.0.1:443   ssl crt-list /var/etc/haproxy/lan-ad-alshowto-com-copy.crt_list  
	mode			http
	log			global
	option			httplog
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			matches-rtr	var(txn.txnhost) -m str -i rtr.ad.alshowto.com
	acl			matches-pve1	var(txn.txnhost) -m str -i pve.ad.alshowto.com
	acl			matches-sw-10g-01	var(txn.txnhost) -m str -i sw-10g-01.ad.alshowto.com
	acl			matches-fjb	var(txn.txnhost) -m str -i fjb.ad.alshowto.com
	acl			matches-sw-1g-01	var(txn.txnhost) -m str -i sw-1g-01.ad.alshowto.com
	acl			matches-gs-01-lan	var(txn.txnhost) -m str -i gs01.ad.alshowto.com
	acl			aclcrt_lan-ad-alshowto-com-copy	var(txn.txnhost) -m reg -i ^([^\.]*)\.ad\.alshowto\.com(:([0-9]){1,5})?$
	acl			aclcrt_lan-ad-alshowto-com-copy	var(txn.txnhost) -m reg -i ^ad\.alshowto\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	use_backend gui-lan-ad-server_ipvANY  if  matches-rtr aclcrt_lan-ad-alshowto-com-copy
	use_backend server-pve1_ipvANY  if  matches-pve1 aclcrt_lan-ad-alshowto-com-copy
	use_backend sw-10g-01-lan-srv_ipvANY  if  matches-sw-10g-01 aclcrt_lan-ad-alshowto-com-copy
	use_backend fjb-srv_ipvANY  if  matches-fjb aclcrt_lan-ad-alshowto-com-copy
	use_backend sw-1g-01-srv_ipvANY  if  matches-sw-1g-01 aclcrt_lan-ad-alshowto-com-copy
	use_backend gs-01-lan_ipvANY  if  matches-gs-01-lan aclcrt_lan-ad-alshowto-com-copy
	use_backend gui-lan-ad-server_ipvANY  if   aclcrt_lan-ad-alshowto-com-copy

backend gui-lan-ad-server_ipvANY
	mode			http
	id			100
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	server			rtr-gui-srv 192.168.0.1:8080 id 101  

backend server-pve1_ipvANY
	mode			http
	id			102
	log			global
	cookie SERVERID insert nocache
	balance			roundrobin
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			pve1 192.168.0.31:8006 id 103 ssl cookie S1 check inter 1000  weight 100 verify none 
	server			pve2 192.168.0.33:8006 id 104 ssl cookie S2 check inter 1000  weight 100 verify none 
	server			pve3 192.168.0.35:8006 id 105 ssl cookie S3 check inter 1000  weight 100 verify none 

backend sw-10g-01-lan-srv_ipvANY
	mode			http
	id			106
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			sw-10g-01 192.168.0.2:443 id 107 ssl check inter 1000  verify none 

backend fjb-srv_ipvANY
	mode			http
	id			108
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			fjb-srv 192.168.0.3:80 id 109 check inter 1000  

backend sw-1g-01-srv_ipvANY
	mode			http
	id			110
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			sw-1g-01 192.168.0.4:80 id 111 check inter 1000  

backend gs-01-lan_ipvANY
	mode			http
	id			112
	log			global
	cookie SERVERID nocache
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			gs-01-lan 192.168.0.41:443 id 111 ssl cookie S1 check inter 1000  verify none

Troubleshooting

proxmox vm behind haproxy shell and console not working

This is what spurred this article for me. I found this post related to Proxmox and HAProxy. So, I knew cookies have to be my problem as initially I did not have cookies and all seemed to be working. Until, I clicked shell and it would not stay connected. That is when I knew I need something to keep track of the server this particular web session is served within.