Backups

Backups would be nice. And there is an excess of hardware to do them to at home.

sacredchao to be master server and do backups to USB connected removable drives.

bitkeeper to power on as needed and get replicas of backup sets from sacredchao stored on local disks

honkingbigtapelibrary to power on as needed and get replicas of bitkeeper 's replicas.

Mom's Windows PC to run storage daemon and do backups to locally attached external USB drive. (Should be fun running a storage daemon under Windows, right?)

youngmi's linux box to also run storage daemon, back up to external USB drive and replicate backup sets to Greeley.

overoverkill? You decide.

deciding on bareos

Bareos is a fork of Bacula. Bacula does all kinds of nice things, but the development model seems to have gone the open core route. Which is fine, but I don't feel a great need to spend dollars on a Windows bacula client, when I can get the Bareos Windows client for free. Bareos also has a nifty feature that allows replications from one storage daemon to another, simplifying the off-siting strategy I'd like to implement.

bareos director setup

• Build bareos binaries for needed platforms:
  • source distribution for Debianized sources is http://download.bareos.org/bareos/release/latest/Debian_7.0/
  • Unable to find source packages for the fastlz library this references, I skipped it. Removed references to it from source tree, and made appropriate comment in debian/changelog.
• Install a director somewhere. sacredchao seems like a good choice.
  • The director is in charge of things. It has the database of backups, job definitions, and all that sort of stuff.
  • Support for three different databases is available. I'll take SQLite over PostgreSQL over MySQL, given the option. (No extra layer of database engine needed (just a tiny library), no additional services exposed to the network, very transparent (UNIX filesystem permissions) security model.)
  • director package installation command was: sudo dpkg -i bareos-director_13.2.2-7.1.fnord.0_amd64.deb bareos-database-common_13.2.2-7.1.fnord.0_amd64.deb bareos-database-sqlite3_13.2.2-7.1.fnord.0_amd64.deb bareos-common_13.2.2-7.1.fnord.0_amd64.deb bareos-database-tools_13.2.2-7.1.fnord.0_amd64.deb

Director configuration bits from /etc/bareos/bareos-dir.conf:

 Director {                      # define myself
        Name = fnord-dir        # the organization is "fnord".  The director is an organization-wide resource, name it a
        QueryFile = "/usr/lib/bareos/scripts/query.sql"
        Maximum Concurrent Jobs = 1
        Password = "a big string from 'pwgen -sync 48'"   # Console password ("pwgen -sync 48" made this)
        Messages = Daemon

        # remove comment in next line to load plugins from specified directory
        # Plugin Directory = /usr/lib/bareos/plugins

        DirAddresses = {        # we have a number of these.  Let's not listen to Internet-facing addresses
                ipv4 = {
                        addr = 172.16.0.1;
                }
                ipv4 = {
                        addr = 10.255.248.1;
                }
                ipv6 = {
                        addr = 2001:470:ba93:10::1;
                }
                ipv6 = {
                        addr = 2001:470:1f0f:5be::1;
                }
        }
}

Trying to start the director by running the init script now fails. Some poking around found nothing recorded by syslog. Tracing the init script eventually got me to running the director by hand to look for output...

adj@sacredchao:/var/log$ sudo /usr/sbin/bareos-dir -t -v
bareos-dir: dird.c:1087-0 Could not open Catalog "MyCatalog", database "bareos".
bareos-dir: dird.c:1092-0 sqlite.c:172 Database /var/lib/bareos/bareos.db does not exist, please create it.
13-Jan 16:36 bareos-dir ERROR TERMINATION
Please correct configuration file: bareos-dir.conf
adj@sacredchao:/var/log$ 

That's actually quite instructive. Let's see about sorting that out. I really think I should call the database after the organization (fnord).

Edit the director config file, and fix up the Catalog section as follows:

# Generic catalog service
Catalog {
        Name = FnordCatalog
        # Uncomment the following lines if you want the dbi driver
        # dbdriver = "dbi:postgresql"; dbaddress = 127.0.0.1; dbport =
        #dbdriver = "postgresql"
        dbdriver = "sqlite3"
        dbname = "fnord-catalog"        # was "bareos", but it ought to be named after the organization.   
        dbuser = "bareos"
        dbpassword = ""
}

Also find any other references to "MyCatalog" and replace with "FnordCatalog". Run the director in test mode again, and look for output like this:

adj@sacredchao:/etc/bareos$ sudo /usr/sbin/bareos-dir -t -v
bareos-dir: dird.c:1087-0 Could not open Catalog "FnordCatalog", database "fnord-catalog".
bareos-dir: dird.c:1092-0 sqlite.c:172 Database /var/lib/bareos/fnord-catalog.db does not exist, please create it.
14-Jan 10:32 bareos-dir ERROR TERMINATION
Please correct configuration file: bareos-dir.conf
adj@sacredchao:/etc/bareos$

So, how to create the SQLite catalog database? There's a handy dandy script to be found in /usr/lib/bareos/scripts helpfully called "create_sqlite3_database". Run that while in the /var/lib/bareos directory. (It assumes you're root, but should also work as user bareos. Expect the chown at the end to fail, though.) Follow up by running /usr/lib/bareos/scripts/make_sqlite3_tables. Then rename /var/lib/bareos/bareos.db to /var/lib/bareos/fnord-catalog.db. Running the director daemon in test mode (-t) should now report no issues:

adj@sacredchao:/var/lib/bareos$ sudo /usr/sbin/bareos-dir -t -v
adj@sacredchao:/var/lib/bareos$

storage daemon setup

Bareos's storage daemons accept the clients' files and put them on some sort of (hopefully) reliable long term storage medium.

My Debian package for the storage daemon is bareos-storage_13.2.2-7.1.fnord.0_amd64.deb. Support for tape libraries is in the bareos-storage-tape_13.2.2-7.1.fnord.0_amd64.deb package. Since sacredchao doesn't have tape attached, we'll just install bacula-storage for now and set up some USB attached disk for storage.

Since we renamed the director from "sacredchao-dir" to "fnord-dir", it is important to make a matching change in the storage daemon's config.

Important bits of storage daemon config:

Storage {               # definition of myself
        Name = sacredchao-sd            # this name identifies a storage daemon inside the org.  Name it after the serv
        Maximum Concurrent Jobs = 20

        # remove comment in next line to load plugins from specified directory
        # Plugin Directory = /usr/lib/bareos/plugins
}

Director {
        Name = fnord-dir                # important that this matches the director's name.  See http://doc.bareos.org/master/html/bareos-manual-main-reference.html#AuthorizationErrors
        Password = "BIG_PASSWORD_FROM_pwgen_-sync_48"           # Be sure to stick this in the director's config, too.
}

Device {
        Name = FileStorage
        Media Type = File
        Archive Device = /var/lib/bareos/storage
        LabelMedia = yes;                   # lets Bareos label unlabeled media
        Random Access = Yes;
        AutomaticMount = yes;               # when device opened, read it
        RemovableMedia = no;
        AlwaysOpen = no;
}

This storage daemon has one storage device defined. It's called "FileStorage" and it writes files under /var/lib/bareos/storage. We'll want to get that on a drive that isn't a backup source sometime in the near future.

Clients? That's where all the data live.

We've got a working director to make decisions now. And a storage daemon to write our backups to. How about being able to suck the files out of a system? The piece that does that is called a file daemon. It runs on each client.

Install by installing the bareos-filedaemon package and update bareos-fd.conf to make the director's name and password match. Also update the bareos-dir.conf to put in a correct password for the file daemon.

Let the two jobs the director has currently defined (back up /usr/sbin and back up the catalog) run and come back in the morning.

Setting up file storage

So, sadly, last night's backups failed. The director was able to talk to the file daemon (client) and the storage daemon, but the storage daemon wasn't really ready to write to the FileStorage device. It seems that it must be labelled before being used. One uses the Bareos console to make a labeling operation happen. Install the bareos-bconsole package and edit the director's name amd password in /etc/bareos/bconsole.conf (check them against the director's config file) to enable the console to manage the director.

Fiddle about inside bconsole quite a bit not to create a working, writable backup target.