One of the web platforms we recently acquired (that I just finished migrating into our data centers) has a rather large MySQL database backing it.
By "rather large", I mean on the order of 70+GiB. All InnoDB, file per table. Single RW master, a handful of RO relay masters, and a number of RO slaves covering production, testing, and development environments. Performance doesn't seem to be a major issue currently, though part of that may be due to the very beefy systems that run this platform. The code is maintained by the development team who created the platform, and is very much under active development.
To give a comparison, another web platform I maintain covers a similar number of client sites, but has a more manageable database that clocks in at around 10GiB.
Naturally, this makes backing up or restoring the database take quite a long time. I could theoretically use mysqldump to grab all tables from one of my slaves (my typical MO with our other platform), but I never had a successful restore in any of my initial tests. It gets to about the 8-10 hour mark and dies for one reason or another.
With looming deadlines and only so much time to troubleshoot, I took the low road and used rsync to copy the binary log and database directories to our new hosts. This method works great and only takes 1-2 hours for a full backup and restore. They will then spin up cleanly as a relay master and begin synchronizing with the read-write master. Through manipulation of my.cnf, the new slaves can be designated as either a RO slave or a relay master as needed.
While I have successfully used this method of effectively 'cloning' the database onto a new system a number of times now, I wonder if there is a better way to accomplish this. Has anyone else dealt with large MySQL InnoDB databases? Got a better way to accomplish this? I'd be interested to hear what others do in a situation such as this.
TL;DR: 70GiB MySQL database - how would you approach it?
[link] [6 comments]