I'm digging into Geoclustering for our Linux web servers. This is partly to include DR and HA into a single solution. We're running RHEL on several hundred web servers now with mostly static content, some database and very little JBoss. I really just want to be able to cluster 4 servers, 2 per geographically distant datacenter. We want to be able to stay up and running with the failure of a single server, or with the failure of a whole datacenter. Hopefully there won't need to be any manual intervention.
I'm looking at either SAN replication (Pure Storage) or DRDB for storage replication, GFS2 filesystem. The clustering solution so far seems to be Red Hat Cluster Suite, most likely with some custom fencing, and/or the addition/substitution of boothd. I've used RHCS in the past with success for simple HA clusters.
I'm unsure how to go about the virtual IP for the cluster. I can use LVS or one of two types of physical load balancers (A10 and F5), but we still have an issue of the virtual IP being different between datacenters. I know of a couple products that do "DNS Failover", such as F5's GTM product, but even with TTLs set to 0, a lot of DNS servers don't honor that, and most browsers cache DNS internally regardless of TTL. The only other way I know of around this situation is to have the same virtual IP in both datacenters, and use multi-homed BGP routing. Our company does not have a BGP AS.
SUSE has a GeoCluster solution which uses either DNS or BGP for dealing with the virtual IP issue. We're on RHEL, and won't likely be switching anytime soon. How is everyone else working around these issues? Is my initial setup anywhere near viable?
[link][16 comments]