sick of tapes - good disk-based backup solution for small/medium businesses?
in [Storage]
|
Prev: looking for USB cable that supplies power "downstream"
Next: 2.5" drives for server instead of 3.5"?
From: Kremlar on 27 Apr 2010 16:12 I've had it with tapes and am trying to come up with a good disk-based backup solution for servers (running AD, Exchange, etc.). We currently use Symantec Backup Exec with mostly DLT-V4 drives, previously using Sony AIT. Experiencing too many early tape drive failures, bad tapes, bad tapes ruining drives (!), etc. What are others doing for disk-based backup solutions for small/medium size businesses? I don't mind sticking with Backup Exec. External USB drives plugged in via USB cable can work, but seems a bit hack to me. I'm envisioning some kind of caddy containing 2.5" or 3.5" SATA drives (to cover the electronics and make them a bit more durable) that inserts into either an external USB/eSATA base unit or internal drive bay. Something like this on NewEgg, but ideally I'd want a caddy for the drive that will protect the electronics as we often times need the ability to take backups off-site. http://www.newegg.com/Product/Product.aspx?Item=N82E16817998038&cm_re=2.5%22_disk_enclosure-_-17-998-038-_-Product We need the ability to rotate between 5+ hard drives, plus take at least one backup off-site. What are others doing?
From: Kremlar on 28 Apr 2010 05:39 1-2TB for a week's worth of data would be fine, maybe 200-300GB would be a full nightly backup. I prefer taking full backups every night. RSYNC sounds great, and I will check into it, but for off-site copies of at least 1 full backup (200-300GB), I was envisioning more of a removable media that can be physically taken off-site. Like a tape, or removable hard disk. "David Brown" <david(a)westcontrol.removethisbit.com> wrote in message news:4bd7f9ce$0$11807$8404b019(a)news.wineasy.se... > On 27/04/2010 22:12, Kremlar wrote: >> I've had it with tapes and am trying to come up with a good disk-based >> backup solution for servers (running AD, Exchange, etc.). >> >> We currently use Symantec Backup Exec with mostly DLT-V4 drives, >> previously using Sony AIT. Experiencing too many early tape drive >> failures, bad tapes, bad tapes ruining drives (!), etc. >> >> What are others doing for disk-based backup solutions for small/medium >> size businesses? I don't mind sticking with Backup Exec. External USB >> drives plugged in via USB cable can work, but seems a bit hack to me. >> >> I'm envisioning some kind of caddy containing 2.5" or 3.5" SATA drives >> (to cover the electronics and make them a bit more durable) that inserts >> into either an external USB/eSATA base unit or internal drive bay. >> >> Something like this on NewEgg, but ideally I'd want a caddy for the >> drive that will protect the electronics as we often times need the >> ability to take backups off-site. >> >> http://www.newegg.com/Product/Product.aspx?Item=N82E16817998038&cm_re=2.5%22_disk_enclosure-_-17-998-038-_-Product >> >> >> We need the ability to rotate between 5+ hard drives, plus take at least >> one backup off-site. >> >> What are others doing? >> >> > > What sort of data size are you looking at? It's important to know how > much data you have in total, how much changes in a given timeframe, and > also how much network bandwidth you have to an offsite backup. > > My preferred solution is to use an rsync-based backup (you can use > rsnapshot, dirvish, etc., to automate it, or write the scripts yourself). > When the server data is copied over to the backup system, only the changes > are actually transferred and stored. But for each backup run, you have a > snapshot of the server data at the backup time. Thus you can always go > back in time and see exactly what you had at any point - restoration of > data is a simple file copy. And if your network bandwidth matches your > data change per day, you can run offsite backups over the Internet > overnight automatically. > >
From: Kremlar on 28 Apr 2010 07:32 You're right, it is a hangover from my current mentality and something I need to open my mind about. How does RSYNC handle things like Active Directory and Exchange? "David Brown" <david(a)westcontrol.removethisbit.com> wrote in message news:4bd81273$0$11799$8404b019(a)news.wineasy.se... > On 28/04/2010 11:39, Kremlar wrote: >> 1-2TB for a week's worth of data would be fine, maybe 200-300GB would be >> a full nightly backup. I prefer taking full backups every night. >> >> RSYNC sounds great, and I will check into it, but for off-site copies of >> at least 1 full backup (200-300GB), I was envisioning more of a >> removable media that can be physically taken off-site. Like a tape, or >> removable hard disk. >> > > Taking full backups is a hangover from the sort of backup system you are > used to - it is unnecessary with an rsync style backup. I only ever do > full backups when I set up a new backup server, and even then it's really > an incremental backup but with nothing there from before. > > There are two reasons for doing full backups. One is when you consider > your backup media to be unreliable - with rsync backups you avoid that by > having the redundancy of at least two backup servers, and possibly using > raid on the servers (though three backup servers is better than two > servers with mirrored raid). > > The other reason for doing full backups is so that you can easily restore > lost data. Restoration of data is the critical part of any backup system, > though it is often overlooked or given only brief testing. > > When you are thinking in terms of "backup sets", such as making a full > backup once a week and incremental (or differential) backups every night, > it's clear that full backups are much easier for restoring data. This is > independent of the media - be it tape, removable hard drives, optical > disks, etc. > > With rsync backups as I describe them, you are keeping a full copy of all > your data as a normal directory on the backup server. Each time you run a > backup, you use rsync to update that copy from the main server - new or > changed files are copied over, and files that are deleted from the main > server are deleted on the backup copy. On the backup server you then make > a hardlinked snapshot copy of the sync directory ("cp -al") to a dated > directory - metadata for directories are copied, but file data is not as > the snapshot and the sync copies share the same hardlinked file. > > The result is that you have a series of directories on the backup server > that hold snapshots of the main server at given dates, and you can access > all the files directly as though you had multiple independent full > backups. Yet the backup transfer time, and the space requirements, are as > though you had done simple incremental backups each night. > > mvh., > > David > > > > >> >> "David Brown" <david(a)westcontrol.removethisbit.com> wrote in message >> news:4bd7f9ce$0$11807$8404b019(a)news.wineasy.se... >>> On 27/04/2010 22:12, Kremlar wrote: >>>> I've had it with tapes and am trying to come up with a good disk-based >>>> backup solution for servers (running AD, Exchange, etc.). >>>> >>>> We currently use Symantec Backup Exec with mostly DLT-V4 drives, >>>> previously using Sony AIT. Experiencing too many early tape drive >>>> failures, bad tapes, bad tapes ruining drives (!), etc. >>>> >>>> What are others doing for disk-based backup solutions for small/medium >>>> size businesses? I don't mind sticking with Backup Exec. External USB >>>> drives plugged in via USB cable can work, but seems a bit hack to me. >>>> >>>> I'm envisioning some kind of caddy containing 2.5" or 3.5" SATA drives >>>> (to cover the electronics and make them a bit more durable) that >>>> inserts >>>> into either an external USB/eSATA base unit or internal drive bay. >>>> >>>> Something like this on NewEgg, but ideally I'd want a caddy for the >>>> drive that will protect the electronics as we often times need the >>>> ability to take backups off-site. >>>> >>>> http://www.newegg.com/Product/Product.aspx?Item=N82E16817998038&cm_re=2.5%22_disk_enclosure-_-17-998-038-_-Product >>>> >>>> >>>> >>>> We need the ability to rotate between 5+ hard drives, plus take at >>>> least >>>> one backup off-site. >>>> >>>> What are others doing? >>>> >>>> >>> >>> What sort of data size are you looking at? It's important to know how >>> much data you have in total, how much changes in a given timeframe, >>> and also how much network bandwidth you have to an offsite backup. >>> >>> My preferred solution is to use an rsync-based backup (you can use >>> rsnapshot, dirvish, etc., to automate it, or write the scripts >>> yourself). When the server data is copied over to the backup system, >>> only the changes are actually transferred and stored. But for each >>> backup run, you have a snapshot of the server data at the backup time. >>> Thus you can always go back in time and see exactly what you had at >>> any point - restoration of data is a simple file copy. And if your >>> network bandwidth matches your data change per day, you can run >>> offsite backups over the Internet overnight automatically. >>> >>> >
From: Arno on 28 Apr 2010 14:28 David Brown <david(a)westcontrol.removethisbit.com> wrote: > On 28/04/2010 13:32, Kremlar wrote: >> You're right, it is a hangover from my current mentality and something I >> need to open my mind about. >> >> How does RSYNC handle things like Active Directory and Exchange? >> > I don't know - I use Linux on servers (and though I use samba for file > sharing, we don't use active directory at all). > Basically, rsync backups are about doing a snapshot of a directory. I > know that works perfectly well for the maildir directory I have for my > imap server (dovecot). The thing here is that a Linux installation expects to be backed up with tar/rsync/... while running. After all it is a server OS and backups without stopping services or reboots are just the way to do it. I have done backups/copies with tar of running systems for years under Linux and never had any problems. This is what a server OS gives you. The jokers in Redmont are (again) decades behind in this. I bet they do not even know that. > I gather Exchange uses some sort of mishmash of huge email store files, > "jet" databases, and other bits and pieces that are apparently designed > to make life difficult - it probably even stores important information > in the windows registry. This is one of the reasons why I don't > consider Exchange a good choice for an email server. But obviously you > are looking for a backup solution for the email server you have, not an > email server to suit the backup system! Exchange is a toy, for various reasons. That said, for exchange the only real choice is to stop the mail server for backup. Then you can use rsync again. This is not too bad actually, as the email system was not (fortunately!) designed by MS and has redundancy. If a mail server is unreachable, the sending MTA will try again, hence no mail lost. The only issue I see is if this server is also used outbound. In that case you probably need to either provide a failover or tell your users that there can be problems sending mail in the backup time window. > I doubt if a direct rsync copy from the relevant directories on the > email server would be successful. At best, backup and restore would be > like somewhat like giving the server an unexpected hard reset. > Theoretically Exchange can use its logs to recover from this and fix any > database corruption - I would not want to trust that in a real system. What is worse, you cannot be sure until you try it. And even then some backups may work and others may not. Not a good situation that speaks of lack of experience and foresight on the part of the software designers. I think arranging a backup time windown (e.g. 4:00-5:00) and then stopping exchange for a backup is your best bet. From what I hear this may also increase the stability of exchange in addition. Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno(a)wagner.name GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
From: Arno on 28 Apr 2010 17:57
David Brown <david.brown(a)hesbynett.removethisbit.no> wrote: > Arno wrote: >> David Brown <david(a)westcontrol.removethisbit.com> wrote: >>> On 28/04/2010 13:32, Kremlar wrote: >>>> You're right, it is a hangover from my current mentality and something I >>>> need to open my mind about. >>>> >>>> How does RSYNC handle things like Active Directory and Exchange? >>>> >> >>> I don't know - I use Linux on servers (and though I use samba for file >>> sharing, we don't use active directory at all). >> >>> Basically, rsync backups are about doing a snapshot of a directory. I >>> know that works perfectly well for the maildir directory I have for my >>> imap server (dovecot). >> >> The thing here is that a Linux installation expects to be backed up >> with tar/rsync/... while running. After all it is a server OS and >> backups without stopping services or reboots are just the way to do it. >> I have done backups/copies with tar of running systems for years >> under Linux and never had any problems. This is what a server OS >> gives you. The jokers in Redmont are (again) decades behind in >> this. I bet they do not even know that. >> > Backing up is easy - have you tested restoring? Yes. I use it as a mechanism to copy installations as well, and have successfully restored a lot of times. > I know for a fact that > this sort of backup is fine for dovecot and maildir storage - most Linux > mail servers are happy that way. But it might not work so well with, > for example, a database server. It will /probably/ work - but you have > to check it. Well, on issue is that you get a snapshot. Depending on what you want, this may actually not be the right ting. If you need more, you may have to stop services and applications before doing the backup. No, I have not done it on a database server. But they usually have their own backup mecahnism for the DB contents, typically some kind of dump/restore mechanism that can be used with the DB running. > In general, of course, you are right - Linux services are designed to be > robust in the light of failure, and typically store files in a way that > makes them easily recoverable. Using a simple file copy > (tar/rsync/etc.) for backup therefore mostly works fine. Windows style > servers are a different matter - they may store all sorts of stuff in > different places, and because you generally have a single huge program > trying to do everything, they don't have the necessary intermediate data > stored in files. Indeed. That is why I do not consider the windows style to be suitable for servers. >>> I gather Exchange uses some sort of mishmash of huge email store files, >>> "jet" databases, and other bits and pieces that are apparently designed >>> to make life difficult - it probably even stores important information >>> in the windows registry. This is one of the reasons why I don't >>> consider Exchange a good choice for an email server. But obviously you >>> are looking for a backup solution for the email server you have, not an >>> email server to suit the backup system! >> >> Exchange is a toy, for various reasons. That said, for exchange the >> only real choice is to stop the mail server for backup. Then you >> can use rsync again. >> > That should probably work. But in that case, I'd make a fast local copy > of the data before doing an offsite rsync. Agreed. Especially to keep downtime low. Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno(a)wagner.name GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans |