Backups
Backups would be nice. And there is an excess of hardware to do them to at home.
sacredchao to be master server and do backups to USB connected removable drives.
bitkeeper to power on as needed and get replicas of backup sets from sacredchao stored on local disks
honkingbigtapelibrary to power on as needed and get replicas of bitkeeper 's replicas.
Mom's Windows PC to run storage daemon and do backups to locally attached external USB drive. (Should be fun running a storage daemon under Windows, right?)
youngmi's linux box to also run storage daemon, back up to external USB drive and replicate backup sets to Greeley.
overoverkill? You decide.
deciding on bareos
Bareos is a fork of Bacula. Bacula does all kinds of nice things, but the development model seems to have gone the open core route. Which is fine, but I don't feel a great need to spend dollars on a Windows bacula client, when I can get the Bareos Windows client for free. Bareos also has a nifty feature that allows replications from one storage daemon to another, simplifying the off-siting strategy I'd like to implement.
bareos director setup
- Build bareos binaries for needed platforms:
- source distribution for Debianized sources is http://download.bareos.org/bareos/release/latest/Debian_7.0/
- Unable to find source packages for the fastlz library this references, I skipped it. Removed references to it from source tree, and made appropriate comment in debian/changelog.
- Install a director somewhere. sacredchao seems like a good choice.
- The director is in charge of things. It has the database of backups, job definitions, and all that sort of stuff.
- Support for three different databases is available. I'll take SQLite over PostgreSQL over MySQL, given the option. (No extra layer of database engine needed (just a tiny library), no additional services exposed to the network, very transparent (UNIX filesystem permissions) security model.)
- director package installation command was:
sudo dpkg -i bareos-director_13.2.2-7.1.fnord.0_amd64.deb bareos-database-common_13.2.2-7.1.fnord.0_amd64.deb bareos-database-sqlite3_13.2.2-7.1.fnord.0_amd64.deb bareos-common_13.2.2-7.1.fnord.0_amd64.deb bareos-database-tools_13.2.2-7.1.fnord.0_amd64.deb
Director configuration bits from /etc/bareos/bareos-dir.conf:
Director { # define myself
Name = fnord-dir # the organization is "fnord". The director is an organization-wide resource, name it a
QueryFile = "/usr/lib/bareos/scripts/query.sql"
Maximum Concurrent Jobs = 1
Password = "a big string from 'pwgen -sync 48'" # Console password ("pwgen -sync 48" made this)
Messages = Daemon
# remove comment in next line to load plugins from specified directory
# Plugin Directory = /usr/lib/bareos/plugins
DirAddresses = { # we have a number of these. Let's not listen to Internet-facing addresses
ipv4 = {
addr = 172.16.0.1;
}
ipv4 = {
addr = 10.255.248.1;
}
ipv6 = {
addr = 2001:470:ba93:10::1;
}
ipv6 = {
addr = 2001:470:1f0f:5be::1;
}
}
}
Trying to start the director by running the init script now fails. Some poking around found nothing recorded by syslog. Tracing the init script eventually got me to running the director by hand to look for output...
adj@sacredchao:/var/log$ sudo /usr/sbin/bareos-dir -t -v bareos-dir: dird.c:1087-0 Could not open Catalog "MyCatalog", database "bareos". bareos-dir: dird.c:1092-0 sqlite.c:172 Database /var/lib/bareos/bareos.db does not exist, please create it. 13-Jan 16:36 bareos-dir ERROR TERMINATION Please correct configuration file: bareos-dir.conf adj@sacredchao:/var/log$
That's actually quite instructive. Let's see about sorting that out. I really think I should call the database after the organization (fnord).
Edit the director config file, and fix up the Catalog section as follows:
# Generic catalog service
Catalog {
Name = FnordCatalog
# Uncomment the following lines if you want the dbi driver
# dbdriver = "dbi:postgresql"; dbaddress = 127.0.0.1; dbport =
#dbdriver = "postgresql"
dbdriver = "sqlite3"
dbname = "fnord-catalog" # was "bareos", but it ought to be named after the organization.
dbuser = "bareos"
dbpassword = ""
}
Also find any other references to "MyCatalog" and replace with "FnordCatalog". Run the director in test mode again, and look for output like this:
adj@sacredchao:/etc/bareos$ sudo /usr/sbin/bareos-dir -t -v bareos-dir: dird.c:1087-0 Could not open Catalog "FnordCatalog", database "fnord-catalog". bareos-dir: dird.c:1092-0 sqlite.c:172 Database /var/lib/bareos/fnord-catalog.db does not exist, please create it. 14-Jan 10:32 bareos-dir ERROR TERMINATION Please correct configuration file: bareos-dir.conf adj@sacredchao:/etc/bareos$
So, how to create the SQLite catalog database? There's a handy dandy script to be found in /usr/lib/bareos/scripts helpfully called "create_sqlite3_database". Run that while in the /var/lib/bareos directory. (It assumes you're root, but should also work as user bareos. Expect the chown at the end to fail, though.) Follow up by running /usr/lib/bareos/scripts/make_sqlite3_tables. Then rename /var/lib/bareos/bareos.db to /var/lib/bareos/fnord-catalog.db. Running the director daemon in test mode (-t) should now report no issues:
adj@sacredchao:/var/lib/bareos$ sudo /usr/sbin/bareos-dir -t -v adj@sacredchao:/var/lib/bareos$
storage daemon setup
Bareos's storage daemons accept the clients' files and put them on some sort of (hopefully) reliable long term storage medium.
My Debian package for the storage daemon is bareos-storage_13.2.2-7.1.fnord.0_amd64.deb. Support for tape libraries is in the bareos-storage-tape_13.2.2-7.1.fnord.0_amd64.deb package. Since sacredchao doesn't have tape attached, we'll just install bacula-storage for now and set up some USB attached disk for storage.
Since we renamed the director from "sacredchao-dir" to "fnord-dir", it is important to make a matching change in the storage daemon's config.
Important bits of storage daemon config:
Storage { # definition of myself
Name = sacredchao-sd # this name identifies a storage daemon inside the org. Name it after the serv
Maximum Concurrent Jobs = 20
# remove comment in next line to load plugins from specified directory
# Plugin Directory = /usr/lib/bareos/plugins
}
Director {
Name = fnord-dir # important that this matches the director's name. See http://doc.bareos.org/master/html/bareos-manual-main-reference.html#AuthorizationErrors
Password = "BIG_PASSWORD_FROM_pwgen_-sync_48" # Be sure to stick this in the director's config, too.
}
Device {
Name = FileStorage
Media Type = File
Archive Device = /var/lib/bareos/storage
LabelMedia = yes; # lets Bareos label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
This storage daemon has one storage device defined. It's called "FileStorage" and it writes files under /var/lib/bareos/storage. We'll want to get that on a drive that isn't a backup source sometime in the near future.
Clients? That's where all the data live.
We've got a working director to make decisions now. And a storage daemon to write our backups to. How about being able to suck the files out of a system? The piece that does that is called a file daemon. It runs on each client.
Install by installing the bareos-filedaemon package and update bareos-fd.conf to make the director's name and password match. Also update the bareos-dir.conf to put in a correct password for the file daemon.
Let the two jobs the director has currently defined (back up /usr/sbin and back up the catalog) run and come back in the morning.
Setting up file storage
So, sadly, last night's backups failed. The director was able to talk to the file daemon (client) and the storage daemon, but the storage daemon wasn't really ready to write to the FileStorage device. It seems that it must be labelled before being used. One uses the Bareos console to make a labeling operation happen. Install the bareos-bconsole package and edit the director's name amd password in /etc/bareos/bconsole.conf (check them against the director's config file) to enable the console to manage the director.
Fiddle about inside bconsole quite a bit now to create a working, writable backup target.
Add another client
Because that's where the files are. :)
Install bareos-common and bareos-filedaemon (and any prerequisites) on client system. Update /etc/bareos/bareos-fd.conf (on the client) to fix name and password for the director.
Then, on the director machine, add the client's details to bacula-dir.com. Add a Client block to define the let the director know the client exists. Also add a Job block to get an entry into the schedule. This latter will need a distinct name and Client = line.