I have an infrastructure problem I need to solve and was looking for any suggestions on how you all might handle this. Basically the gist of the problem is that I have let's say 50 million image files that are critical to our business and I want to make sure they are always available to our SaaS web application and other processes.
Currently this data totals about 1.5TB and is stored in a VM on a replicated couple of servers & disks using Xen, DRBD and Debian. There is a WebService on the VM that handles the file push and pull operations. The data is replicated but backup is hard with so many files, and with a single VM I have no way to ever take the it down for maintenance. My plan was to move this off the Xen & DRBD cluster and onto the SAN & VMWare Cluster and when doing so, make it more resilient at the same time.
The way I see it my options might be these...
- SAN - > VMWare Cluster - Fault Tolerant VM with Data and WebService.
- SAN -> VMWare Cluster - VM with Data and exported via NFS -> 2 x VM's with NFS mounted and WebService
- SAN + NAS (for image data only) with data exported via NFS - > VMWare Cluster -> 2 x VM's with NAS-NFS mounted and WebService
- SAN + 3 Physical Node Gluster DFS (for image data only) with data exported via NFS - > VMWare Cluster -> 2 x VM's with Gluster DFS-NFS mounted and WebService
Option 1: Pretty much the same as I have now but on a SAN, which at least saves some headaches.
Option 2. Multiple VMs with web services to serve the data but still a single VM with the actual data on it.
Option 3. NAS for image storage, easier to maintain but not as fault tolerant as the SAN.
Option 4: Hardest to manage, best performance and reliability?
Also the SAN (Dell md3220i) doesn't export via NFS, only iSCSI.
Thanks,
David
[link][2 comments]