PowerEdge 1950 Debian install

From FnordWiki
Revision as of 22:24, 13 February 2012 by Adj (talk | contribs) (→‎IPMI)
Jump to navigation Jump to search

BIOS Settings

  • Turn on virtualization, IOMMU
  • Lots of options regarding console redirection over serial. Need to investigate.

IPMI

Watch for IPMI setup message during boot. Press Ctrl-E when prompted. Settings as follows: 

Static IP: 172.16.0.241
Netmask:   255.255.255.0
Gateway:   0.0.0.0

Reset user credentials.

TODO: Investigate crypto key stuff, alerts, other settings.

What is this good for? Well, 

ipmitool -H 172.16.0.241 -U Admin power on

will turn on the server from somewhere on the network. A network attached serial console should be possible, too. And 

ipmitool -H 172.16.0.241 -U Admin power off

will turn it off. 

ipmitool -H 172.16.0.241 -U Admin sol activate

will conect to a serial console where you can interact with GRUB and the Linux kernel console, and log in via a serial connection on /dev/ttyS1.

There are a bunch of other possibilities, too.

Updating firmware

Should be doable using the PXE boot firmware update procedure found elsewhere in this wiki. (Replace with a real link.)

Debian Install image locations

http://cdimage.debian.org/debian-cd/6.0.3/amd64/iso-cd/debian-6.0.3-amd64-netinst.iso

non-Free firmware for Ethernet cards

PowerEdge 1950 machines have 2 on-board Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet adapters. Firmware for these adapters is non-free and must be loaded to perform the Debian installation.

An ISO image is available at http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/6.0.3/amd64/iso-cd/ which should include the bits necessary to use these adapters during the OS installation. This is the Netinst ISO but it includes the non-free firmware. Saves on getting the non-free firmware installed using USB keys, floppies, or other media.

http://wiki.debian.org/Firmware has a discussion of issues surrounding the non-free firmware needed by these systems and its use in Debian.

Install process

Proceed as normal with keyboard, setting up network, hostname, etc.

Partition disks by using parted in a shell. Here's what sda looks like now: 

~ # parted /dev/sdb
GNU Parted 2.3
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit s                                                           
unit s
(parted) print                                                            
print
Model: ATA WDC WD6400AAKS-0 (scsi)
Disk /dev/sda: 1250263728s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start     End       Size      File system  Name                     Flags
 1      2048s     526335s   524288s                gulik_sda_efi_boot       bios_grub
 2      526336s   2623487s  2097152s               gulid_sda_mdraid_boot    raid
 3      2623488s  4720639s  2097152s               gulik_sda_mdraid_root    raid

(parted)

After this, we create and start up two software raid devices (the "missing" bit lets us create a software raid device before having all the actual pieces): 

~ # mdadm --create gulik_mdraid_boot --chunk 128 --level 1 --raid-devices=2 --run --name=gulik_mdraid_boot /dev/sda2 /dev/sdb2
~ # mdadm --create gulik_mdraid_root --chunk 128 --level 1 --raid-devices=2 --run --name=gulik_mdraid_root /dev/sda3 /dev/sdb3

Post-install tweaks

Prevent apt-get from automatically installing Recommended packages

Miminal software installation is good! Put the following into /etc/apt/apt.conf: 

APT {
        Install-Recommends "0";
}

USB drivers for the initramfs

Add the following to /etc/initramfs-tools/modules: 

hid
ehci_hcd
uhci_hcd

Run update-initramfs when done to add these modules to the pre-init runtime environment.

Add backports.org packages

Add the following to /etc/apt/sources.list: 

deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free

Install a recent kernel from backports.org

Not yet ready to build a custom kernel here, so install something current from backports.org instead. 

# linux-image-3.2.0-0.bpo.1-amd64 is the most recent kernel package available at backports.org
apt-get install -t squeeze-backports linux-image-3.2.0-0.bpo.1-amd64

Network console

Motivation

It's nice being able for fix problems on the machine without standing in front of it with a keyboard and monitor.

IPMI Serial Over LAN connects a serial port to the network. It can be talked to with ipmitool.

Serial access for the boot loader and tell Linux it has a serial console

Set following in /etc/default/grub: 

GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS1,57600n8"
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=57600 --unit=1 --word=8 --parity=no --stop=1"

http://www.cyberciti.biz/faq/howto-setup-serial-console-on-debian-linux/ has a good tutorial on this.

After editing /etc/default/grub, run /usr/sbin/update-grub to write a new bootloader configuration file (/boot/grub/grub.cfg for instance).

Note that the IPMI Serial over LAN redirection on the PowerEdge 1950 redirects COM2, which Linux calls /dev/ttyS1 and GRUB calls "serial unit 1".

Allow logins on the serial port

We need a getty to listen for a user on the serial port. Add the following to /etc/inittab: 

S1:2345:respawn:/sbin/getty -L ttyS1 57600

And run "/sbin/init Q" to tell init to re-read its configuration file (/etc/inittab).

Disk encryption, more software RAID, and LVM setup

End goal

  • Two physical drives (sda and sdb)
  • Boot from unencrypted, software RAID1 (mirrored) /boot partitions
  • initramfs does "cyptsetup luksOpen" on large partition 4 of each physical drive
  • mdadm is then run and assembles a software RAID 1 from the encrypted partitions
  • the assembled RAID 1 device contains a LUKS partition. This LUKS device contains an LVM physical volume.
  • Logical volumes are LUKS devices containing filesystems
  • three layers of crypto here:
    • LVM lvols are LUKS devices
    • LVM physical volume(s) is (are) LUKS devices
    • MD software RAID members are LUKS devices

Open questions

  • Hot to get passphrases to "cryptsetup luksOpen" ?
  • What's a reasonable set of stacked ciphers?

Procedure

  • Squeeze's cryptsetup and associated packages are woefully out of date. I have pulled in cryptsetup and its dependencies from unstable and will hope the lack of security updates does not bite me in the ass.
  • Create partition number 4 on each of the disks. These will be LUKS devices that will be assembled into a software RAID1.
  • Splat random bits onto these devices like so:
sudo dd if=/dev/urandom of=/dev/sda4 bs=1M
sudo dd if=/dev/urandom of=/dev/sdb4 bs=1M
  • Generate keys for these devices:
# 64 bytes of random data, printed in hex is 128 characters and 512 bits, which should be a decent sized passphrase.
# openssl might take a long, long, long time to get its RNG seeds from /dev/random, so be patient
# what's with the "printf %s $(...)" funny business?  It prevents a trailing newline being fed to gpg's stdin
printf %s $(openssl rand -hex -rand /dev/random 64) | gpg --symmetric -a -v --s2k-mode 3 --s2k-count 65011712 -o gulik_sda4_passphrase.gpg
printf %s $(openssl rand -hex -rand /dev/random 64) | gpg --symmetric -a -v --s2k-mode 3 --s2k-count 65011712 -o gulik_sdb4_passphrase.gpg
  • perhaps multiple keys should go into a single GPG encrypted file, one per line, with a key of the block device
  • Put a LUKS header on the devices:
sudo cryptsetup -c aes-xts-plain64:sha512 -h SHA512 -y -s 256 --iter-time 5000 luksFormat /dev/sda4
sudo cryptsetup -c aes-xts-plain64:sha512 -h SHA512 -y -s 256 --iter-time 5000 luksFormat /dev/sdb4
  • Add some modules to the initramfs. Add the following lines to the end of /etc/initramfs-tools/modules:
dm_mod
dm_crypt
cbc
aes_generic
aes_x86_64
cryptd
sha256_generic
sha512_generic
gf128mul
xts
  • Update /etc/crypttab so that the LUKS device is opened at boot time. So that mdadm can create the RAID1 array. So that cryptsetup can open the LVM volume group's physical volume. So that vgchange can activate vg00. So that cryptsetup can then open the LUKS encrypted logical volumes and make those available to the rest of the system. Put the following in /etc/crypttab:
gulik_mdraid_vg00_pv_0_member_0       sda4            none            luks
  • Run update-initramfs before rebooting so that the needed drivers are available
sudo update-initramfs -v -u
  • Create a software RAID device on our LUKS volumes:
sudo mdadm --create gulik_softraid_vg00_pv_0 --metadata=1.2 \
    --level=1 --raid-devices=2 --name gulik_softraid_vg00_pv_0 \
    /dev/mapper/gulik_softraid_vg00_pv_member_0 missing
# (We'll add another physical disk, LUKS format it, and add it to the MD array at some later time.)
Retrieved from "https://wiki.fnord.greeley.co.us/mediawiki/index.php?title=PowerEdge_1950_Debian_install&oldid=108"