Does anyone know how the private network heartbeat works specifically?
I've read articles and you must use a different IP/Subnet then the public network but without a gateway. I'm confused how the heartbeat will actually work if there is no requirement to use a VLAN/Gateway etc.
The lack of a default gateway on the private network exists for a
couple of reasons.
1) Nodes of a cluster should have low-latency link between all
members of the cluster. In general, you need gateways between IP
subnets because you are either provisioning subnets for administrative
purposes (client subnet vs. server subnet vs. perimeter subnet, etc.)
or because you have more devices than can fit on a single subnet. Now,
on a well-managed network, adding a gateway only increases the latency
by a tiny bit, but it does represent a potential point of congestion.
For a cluster, that congestion could be killer.
2) IP networking stacks in general don't handle the case of multiple
default gateways very well. Think about how IP routing works for a
minute by taking a look at the output of route print. That routing
table there helps the IP stack figure out which next hop (and which
interface) a packet needs to use, and it does it by matching from most
specific to least specific, modified by the metric (weight; lower is
higher priority) of the route if there's a tie based on IP
So, having two default gateways will confuse the computer -- two
"last match" destinations. What happens is that Windows typically will
assign one a higher metric, which means it won't get used. This rarely
leads to desirable conditions.
Example time: say I've got an internal network of 172.31.0.0/24 and
a cluster network of 192.168.0.0/24. My IP address are 172.31.0.16 and
192.168.0.16. Here's my routing table:
Network Destination Netmask Gateway Interface
0.0.0.0 0.0.0.0 172.31.0.1 172.31.0.16
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1
172.31.0.0 255.255.255.0 172.31.0.16 172.31.0.16
172.31.0.16 255.255.255.255 127.0.0.1 127.0.0.1
172.31.0.255 255.255.255.255 172.31.0.16 172.31.0.16
* 192.168.0.0 255.255.255.0 192.168.0.16 192.168.0.16
192.168.0.16 255.255.255.255 127.0.0.1 127.0.0.1
192.168.0.255 255.255.255.255 192.168.0.16 192.168.0.16
18.104.22.168 240.0.0.0 172.31.0.16 172.31.0.16
22.214.171.124 240.0.0.0 192.168.0.16 192.168.0.16
255.255.255.255 255.255.255.255 172.31.0.16 172.31.0.16
255.255.255.255 255.255.255.255 192.168.0.16 192.168.0.16
Default Gateway: 172.31.0.1
When this computer wants to send a packet using the private
(192.168.0.0/24) network, the cluster service knows that other node
has an IP address of 192.168.0.12, so that's the destination IP
address in the packet. When it gets to the IP stack, that destination
matches the line marked with an asterisk, so the IP stack routes it
out the appropriate interface. No second default gateway is needed;
the 172.31.0.1 gateway isn't part of the private network and doesn't
know how to route for it, and the second interface is part of the
So that's the easy example -- a single stretched subnet between the
two. What happens if you're using a stretched subnet with a second
subnet for the other node (say, 192.168.1.0/24)? Simple -- you don't
add a default gateway, you add a persistent static route:
route -p add 192.168.1.0 mask 255.255.255.0 192.168.0.1
This adds the following line to the routing table:
192.168.1.0 255.255.255.0 192.168.0.1 192.168.0.16
Windows still has a specific route entry in the table to guide the
packets to the correct next hop, without having to resolve between two
Hope this helps.