LVMify the disk
LVM? What's this all about, anyway?
LVM Linux's Logical Volume Manager. Wikipedia describes the Linux LVM here. Basically, it allows you to take a pool of one or more hard drives and create one or more virtual hard drives out of piecs of the real drives. These can then be resized at run-time, allowing the admin to grow (or sometimes) shrink filesystems while the machine is up and running. This page covers the moving of a hard-partitioned Seagate DockStar running Debian to an LVM-based setup.
Before LVM-ification
Got a backup? You could bork your machine pretty hard if something bad happens. You've been warned.
Here's how the DockStar's USB flash drive looks now with regard to filesystem free space and partitioning:
root@canopus:~# fdisk -l -u /dev/sda Disk /dev/sda: 4000 MB, 4000317440 bytes 64 heads, 32 sectors/track, 3815 cylinders, total 7813120 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9a198ead Device Boot Start End Blocks Id System /dev/sda1 * 2048 1050623 524288 83 Linux /dev/sda2 1050624 2099199 524288 83 Linux /dev/sda3 2099200 7813119 2856960 8e Linux LVM root@canopus:~# df Filesystem 1K-blocks Used Available Use% Mounted on rootfs 516040 311876 177952 64% / none 60492 36 60456 1% /dev /dev/sda1 516040 311876 177952 64% / tmpfs 62864 0 62864 0% /lib/init/rw tmpfs 62864 0 62864 0% /dev/shm tmpfs 62864 0 62864 0% /tmp root@canopus:~# cat /etc/fstab # /etc/fstab: static file system information. # # <file system> <mount point> <type> <options> <dump> <pass> /dev/root / ext2 noatime,errors=remount-ro 0 1 /dev/sda2 none swap sw 0 0 tmpfs /tmp tmpfs defaults 0 0 root@canopus:~#
The current situation
As you can probably tell, the only filesystem is on /dev/sda1 and /dev/sda2 is used for swap space. /dev/sda3 is not used for anything at this point.
The desired end state
I'd like to end up with /dev/sda1 used purely for system configuration and needed-for-boot files. Swap space, /var, /home, /tmp, and /usr will all live in LVM managed block devices. /dev/sda2 will be converted from swap space to a working backup copy of the current root filesystem and used for repairing the real system should the sysadmin suffer a severe bout of brain-damage and break the thing really bad.
Based on lots of use of du, I believe 512Mbytes for swap, 64Mbytes for /home, 128Mbytes for /tmp, 192Mbytes for /usr, and 192Mbytes for /var will be a good target configuration.
Creating an LVM physical volume
Some data structures need to be placed on a physical device that LVM uses. This is done with the pvcreate program. /dev/sda3 is where we'll put the LVM managed logical volumes, so pvcreate is invoked so (in test mode first, to be sure our command line is sane):
root@canopus:~# pvcreate --test --verbose --metadatatype 2 --pvmetadatacopies 2 --zero y /dev/sda3
Test mode: Metadata will NOT be updated.
Set up physical volume for "/dev/sda3" with 5713920 available sectors
Zeroing start of device /dev/sda3
Physical volume "/dev/sda3" successfully created
Test mode: Wiping internal cache
Wiping internal VG cache
root@canopus:~# pvcreate --verbose --metadatatype 2 --pvmetadatacopies 2 --zero y /dev/sda3
Set up physical volume for "/dev/sda3" with 5713920 available sectors
Zeroing start of device /dev/sda3
Physical volume "/dev/sda3" successfully created
root@canopus:~#
Creating a volume group
Now that there is a physical volume, we can create a volume group. We're specifying that this volume group will not be clustered, is an LVM2 volume group (the tools support the older LVM1 format, too), a physical extent size of 64Mibibytes (this is the size of the pieces that go into any given logical volume. Since the backing store for this volume group is about 3Gibibytes in size, a 64Mibibyte gives us 43 pieces of that to play with), a volume group name of vg00, and that /dev/sda3 is our physical backing device. Again, test mode is used to ensure sanity of command line before really creating the volume group.
root@canopus:~# vgcreate --test --verbose --clustered n --metadatatype 2 --physicalextentsize 64M vg00 /dev/sda3
Test mode: Metadata will NOT be updated.
Wiping cache of LVM-capable devices
Wiping cache of LVM-capable devices
Adding physical volume '/dev/sda3' to volume group 'vg00'
Test mode: Skipping archiving of volume group.
Test mode: Skipping volume group backup.
Volume group "vg00" successfully created
Test mode: Wiping internal cache
Wiping internal VG cache
root@canopus:~# vgcreate --verbose --clustered n --metadatatype 2 --physicalextentsize 64M vg00 /dev/sda3
Wiping cache of LVM-capable devices
Wiping cache of LVM-capable devices
Adding physical volume '/dev/sda3' to volume group 'vg00'
Creating directory "/etc/lvm/archive"
Archiving volume group "vg00" metadata (seqno 0).
Creating directory "/etc/lvm/backup"
Creating volume group backup "/etc/lvm/backup/vg00" (seqno 1).
Volume group "vg00" successfully created
root@canopus:~#
Making /dev/sda2 available
/dev/sda2 is currently used as swap space. It's the same size as /dev/sda1. I want to deactivate the swap space here and make a copy of the root filesystem. First, though, there should be a different swap space make available.
Making our first logical volume: /dev/vg00/swap
Creating a logical volume is done with the lvcreate command. You want to decide a few things before creating one: how big should it be? What name should it have? Which volume group does it belong to?
Here's what I did to make /dev/vg00/swap:
root@canopus:~# lvcreate --test --verbose --size 512M --name swap --zero y vg00
Test mode: Metadata will NOT be updated.
Setting logging type to disk
Finding volume group "vg00"
Test mode: Skipping archiving of volume group.
Creating logical volume swap
Test mode: Skipping volume group backup.
Found volume group "vg00"
Aborting. Failed to activate new LV to wipe the start of it.
Found volume group "vg00"
Unable to deactivate failed new LV. Manual intervention required.
Test mode: Wiping internal cache
Wiping internal VG cache
root@canopus:~# lvcreate --verbose --size 512M --name swap --zero y vg00
Setting logging type to disk
Finding volume group "vg00"
Archiving volume group "vg00" metadata (seqno 5).
Creating logical volume swap
Creating volume group backup "/etc/lvm/backup/vg00" (seqno 6).
Found volume group "vg00"
Creating vg00-swap
Loading vg00-swap table (254:0)
Resuming vg00-swap (254:0)
Clearing start of logical volume "swap"
Creating volume group backup "/etc/lvm/backup/vg00" (seqno 6).
Logical volume "swap" created
root@canopus:~#
Reconfiguring swap location
After this logical volume has been created, we'll put the swap space signature on it using mkswap, activate it with swapon, deactivate the old one (swapoff), and update /etc/fstab to save the changes across a reboot:
root@canopus:~# mkswap /dev/vg00/swap
mkswap: /dev/vg00/swap: warning: don't erase bootbits sectors
on whole disk. Use -f to force.
Setting up swapspace version 1, size = 524284 KiB
no label, UUID=76b70850-c3ab-4512-b55b-0545f2690130
root@canopus:~# swapon /dev/vg00/swap
root@canopus:~# swapoff /dev/sda2
root@canopus:~# sed -e'/^\/dev\/sda2/s/^/# /' /etc/fstab > /tmp/fstab
root@canopus:~# echo '/dev/vg00/swap none swap sw 0 0' >> /tmp/fstab
root@canopus:~# install -o root -g root -m 644 /tmp/fstab /etc/fstab
root@canopus:~# cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/root / ext2 noatime,errors=remount-ro 0 1
# /dev/sda2 none swap sw 0 0
tmpfs /tmp tmpfs defaults 0 0
/dev/vg00/swap none swap sw 0 0
root@canopus:~#
Just to be sure /dev/sda2 is not being used any more as swap space, have a look at /proc/swaps:
root@canopus:~# cat /proc/swaps Filename Type Size Used Priority /dev/dm-0 partition 524280 0 -1 root@canopus:~#
/dev/sda2 is no longer being used for swap space and we can now put a backup root filesystem there.
Make a backup root filesystem
Now that we've stopped using /dev/sda2 for swap, let's put our backup root FS there. We'll format it with mkfs.ext2, mount it someplace convenient, and copy our files over.
create an ext2 filesystem on /dev/sda2
root@canopus:~# mkfs.ext2 -T ext2 /dev/sda2
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
32768 inodes, 131072 blocks
6553 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=134217728
4 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
root@canopus:~#
populate the backup
I like to use /altroot for this purpose. My normal idiom for creating the directory underlying a mount point is to give it 000 (or d---------) permissions. Makes it obvious to me that something should be mounted there:
root@canopus:~# mkdir -m 0 /altroot root@canopus:~# mount /dev/sda2 /altroot root@canopus:~# cd / root@canopus:/# find . -xdev -depth -print0 | cpio -pdmv0a /altroot [...] root@canopus:~# umount /altroot
With this done, you can use /dev/sda2 as your root filesystem by changing the "root=/dev/sda1" kernel command line prarameter (set in the bootloader) to "root=/dev/sda2".
Move stuff to LVM
Now for the good stuff. Our swap space is on an LVM logical volume. Next we'll move /home, /tmp, /usr, and /var to LVM.
create more logical volumes
We need a place to put our filesystems. We've already decided on sizes, so let's use lvcreate to make them:
root@canopus:/# lvcreate --test --verbose --size 64M --name home --zero y vg00
Test mode: Metadata will NOT be updated.
Setting logging type to disk
Finding volume group "vg00"
Test mode: Skipping archiving of volume group.
Creating logical volume home
Test mode: Skipping volume group backup.
Found volume group "vg00"
Aborting. Failed to activate new LV to wipe the start of it.
Found volume group "vg00"
Unable to deactivate failed new LV. Manual intervention required.
Test mode: Wiping internal cache
Wiping internal VG cache
root@canopus:/# lvcreate --verbose --size 64M --name home --zero y vg00
Setting logging type to disk
Finding volume group "vg00"
Archiving volume group "vg00" metadata (seqno 8).
Creating logical volume home
Creating volume group backup "/etc/lvm/backup/vg00" (seqno 9).
Found volume group "vg00"
Creating vg00-home
Loading vg00-home table (254:1)
Resuming vg00-home (254:1)
Clearing start of logical volume "home"
Creating volume group backup "/etc/lvm/backup/vg00" (seqno 9).
Logical volume "home" created
root@canopus:/#
Repeat this for /tmp (128Mbytes), /usr (192Mbytes), and /var (192Mbytes).
create filesystems to hold our files
We've got logical volumes now. Let's put filesystems on them so we can keep our files there.
root@canopus:/# for lvol in /dev/vg00/{home,tmp,usr,var}; do mkfs.ext2 -t ext3 -T default ${lvol}; done
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
4096 inodes, 16384 blocks
819 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=16777216
1 block group
32768 blocks per group, 32768 fragments per group
4096 inodes per group
Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 29 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
8192 inodes, 32768 blocks
1638 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=33554432
1 block group
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 29 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
12288 inodes, 49152 blocks
2457 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=50331648
2 block groups
32768 blocks per group, 32768 fragments per group
6144 inodes per group
Superblock backups stored on blocks:
32768
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
12288 inodes, 49152 blocks
2457 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=50331648
2 block groups
32768 blocks per group, 32768 fragments per group
6144 inodes per group
Superblock backups stored on blocks:
32768
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 21 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
root@canopus:/#
Copy existing files to new filesystems
We've created filesystems to hold our stuff. Now we need to put our stuff in there. Here's a little shell script procedure to do just that. This script's output is quite long, se I'll just skip showing it.
root@canopus:/# for fs in home tmp usr var; do
mount -v /dev/vg00/${fs} /mnt;
cd /${fs};
find . -xdev -depth -print0 | cpio -pdmv0a /mnt;
cd /;
umount -v /mnt;
done
update /etc/fstab
Add the following lines to /etc/fstab. This ensures our new filesystems will be mounted and used the next time this machine reboots.
/dev/vg00/home /home ext3 defaults 0 0 /dev/vg00/tmp /tmp ext3 defaults 0 0 /dev/vg00/usr /usr ext3 defaults 0 0 /dev/vg00/var /var ext3 defaults 0 0
Also, comment out the line that lists /tmp on a tmpfs (RAM-backed) filesystem.
see if it worked
Now is time to reboot, log in again, and see if it's all working as expected. After rebooting, /home, /tmp, /usr, and /var should be mounted from LVM logical volumes. You'll see them listed as /dev/mapper/vg00-{home,tmp,usr,var} in the output of mount or df. If something isn't working correctly, removing the LVM entries from /etc/fstab and rebooting again will leave you with just the root filesystem, still intact, on /dev/sda1.