View source for A10 AX3000 firewall conversion


=== console ===
Serial console with 8P8C modular connector on front of box.  Works with Cisco console cables.  (Yay for that.)  9600bps, 8n1.

=== Hardware inside ===
{|
 !Processor
 |1x Intel Xeon X5670
 |-
 !RAM
 |24 Gibytes ECC RAM
 |}

=== It wants to boot ACOS! ===
Not much chance to hit the keyboard escape.  If you're not having any luck with that, pop the top cover (3 screws on each side, and one at the rear edge of the top side, near the center) and pull the CMOS battery, wait, and reverse the procedure.  The BIOS settings seem to be considered good if the machine boots at least as far as finding the boot loader from the hard drive.

=== Base OS install ===
<code>debootstrap</code> to a drive using another system.  There is room for a pair of thin 2.5 inch SATA disks, so plan ahead and make a RAID.  Firmware is BIOS without an UEFI option, but GNU GRUB can boot an GPT (EFI usual) partitioned disk on a BIOS system just fine.

==== put random bits onto the disk ====
Assuming the disk appears as '''<code>sdg</code>''' on the host system:
 $ sudo shred -n 1 -v /dev/sdg
 [wait however long is needed]

==== disk partition table ====
Now that random bits are all over the disk, let's partition it.  I like GNU parted for this.

 $ '''sudo parted /dev/sdg'''
 (parted) '''unit s'''
 (parted) '''mktable gpt'''
 (parted) '''mkpart bios_grub 2048 2099199'''
 (parted) '''set 1 bios_grub on'''
 (parted) '''mkpart mdraid_boot 2099200 4196351'''
 (parted) '''mkpart mdraid_vg00_pv_0 4196352 167772160'''
 (parted) '''p'''
 Model: WDC WDBN CE2500PNC-WRSN (scsi)
 Disk /dev/sdg: 488397168s
 Sector size (logical/physical): 512B/512B
 Partition Table: gpt
 Disk Flags:
 
 Number  Start     End         Size        File system  Name              Flags
  1      2048s     2099199s    2097152s                 bios_grub         bios_grub
  2      2099200s  4196351s    2097152s                 mdraid_boot
  3      4196352s  167772160s  163575809s               mdraid_vg00_pv_0
 
 (parted) '''q'''

==== RAID! ====
Well, half a RAID.  The new drive has 3 partitions on it.  First will be used to keep some GNU Grub bootloader bits handy.  Next is a Linux software RAID (MD) device for /boot.  And at the end comes another RAID partition that will hold a Linux LVM physical volume which will be the backing store for the '''<code>vg00</code>''' volume group.  So here we're making 2 RAID devices each missing their second halves.  (Maybe we'll get another disk in there eventually.)

 $ '''sudo mdadm --create warthog-0-md_boot --metadata=1.2 --name=warthog-0-md_boot --bitmap=internal --level=1 --raid-devices=2 /dev/sdg2 missing'''
 mdadm: array /dev/md/warthog-0-md_boot started.
 $ '''sudo mdadm --create warthog-0-vg00_pv_0 --metadata=1.2 --name=warthog-0-vg00-pv-0 --bitmap=internal --level=1 --raid-devices=2 /dev/sdg3 missing'''
 mdadm: array /dev/md/warthog-0-vg00_pv_0 started.

==== Logical volume manager bits and bobs ====
I'm all old school.  I like LVM.  I really like LVM.

 $ '''sudo pvcreate --verbose --metadatatype 2 --pvmetadatacopies 2 /dev/md/warthog-0-vg00_pv_0'''
     Wiping internal VG cache
     Wiping cache of LVM-capable devices
     Wiping signatures on new PV /dev/md/warthog-0-vg00_pv_0.
     Set up physical volume for "/dev/md/warthog-0-vg00_pv_0" with 163444736 available sectors.
     Zeroing start of device /dev/md/warthog-0-vg00_pv_0.
     Writing physical volume data to disk "/dev/md/warthog-0-vg00_pv_0".
   Physical volume "/dev/md/warthog-0-vg00_pv_0" successfully created.
 $ '''sudo vgcreate --verbose --metadatatype 2 --vgmetadatacopies 2 --physicalextentsize 256M warthog-0-vg00 /dev/md/warthog-0-vg00_pv_0'''
     Wiping internal VG cache
     Wiping cache of LVM-capable devices
     Wiping signatures on new PV /dev/md/warthog-0-vg00_pv_0.
     Adding physical volume '/dev/md126' to volume group 'warthog-0-vg00'
     Archiving volume group "warthog-0-vg00" metadata (seqno 0).
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 1).
   Volume group "warthog-0-vg00" successfully created
 $ 

Volume group created.  On to some logical volumes to hold filesystems to hold things:

 $ sudo lvcreate --verbose --name root --size 1024M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 1).
     Creating logical volume root
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 2).
     Activating logical volume warthog-0-vg00/root.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/root.
     Creating warthog--0--vg00-root
     Loading warthog--0--vg00-root table (254:8)
     Resuming warthog--0--vg00-root (254:8)
     Wiping known signatures on logical volume "warthog-0-vg00/root"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/root" with value 0.
   Logical volume "root" created.
 $ sudo lvcreate --verbose --name swap --size 4096M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 2).
     Creating logical volume swap
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 3).
     Activating logical volume warthog-0-vg00/swap.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/swap.
     Creating warthog--0--vg00-swap
     Loading warthog--0--vg00-swap table (254:9)
     Resuming warthog--0--vg00-swap (254:9)
     Wiping known signatures on logical volume "warthog-0-vg00/swap"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/swap" with value 0.
   Logical volume "swap" created.
 $ sudo lvcreate --verbose --name tmp --size 4096M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 3).
     Creating logical volume tmp
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 4).
     Activating logical volume warthog-0-vg00/tmp.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/tmp.
     Creating warthog--0--vg00-tmp
     Loading warthog--0--vg00-tmp table (254:10)
     Resuming warthog--0--vg00-tmp (254:10)
     Wiping known signatures on logical volume "warthog-0-vg00/tmp"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/tmp" with value 0.
   Logical volume "tmp" created.
 $ sudo lvcreate --verbose --name var --size 8192M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 4).
     Creating logical volume var
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 5).
     Activating logical volume warthog-0-vg00/var.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/var.
     Creating warthog--0--vg00-var
     Loading warthog--0--vg00-var table (254:11)
     Resuming warthog--0--vg00-var (254:11)
     Wiping known signatures on logical volume "warthog-0-vg00/var"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/var" with value 0.
   Logical volume "var" created.
 $ sudo lvcreate --verbose --name usr --size 4096M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 5).
     Creating logical volume usr
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 6).
     Activating logical volume warthog-0-vg00/usr.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/usr.
     Creating warthog--0--vg00-usr
     Loading warthog--0--vg00-usr table (254:12)
     Resuming warthog--0--vg00-usr (254:12)
     Wiping known signatures on logical volume "warthog-0-vg00/usr"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/usr" with value 0.
   Logical volume "usr" created.
 $ sudo lvcreate --verbose --name home --size 4096M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 6).
     Creating logical volume home
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 7).
     Activating logical volume warthog-0-vg00/home.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/home.
     Creating warthog--0--vg00-home
     Loading warthog--0--vg00-home table (254:13)
     Resuming warthog--0--vg00-home (254:13)
     Wiping known signatures on logical volume "warthog-0-vg00/home"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/home" with value 0.
   Logical volume "home" created.
 $ sudo lvcreate --verbose --name opt --size 1024M warthog-0-vg00
     Archiving volume group "warthog-0-vg00" metadata (seqno 7).
     Creating logical volume opt
     Creating volume group backup "/etc/lvm/backup/warthog-0-vg00" (seqno 8).
     Activating logical volume warthog-0-vg00/opt.
     activation/volume_list configuration setting not defined: Checking only host tags for warthog-0-vg00/opt.
     Creating warthog--0--vg00-opt
     Loading warthog--0--vg00-opt table (254:14)
     Resuming warthog--0--vg00-opt (254:14)
     Wiping known signatures on logical volume "warthog-0-vg00/opt"
     Initializing 4.00 KiB of logical volume "warthog-0-vg00/opt" with value 0.
   Logical volume "opt" created.
 $ 

And some filesystems now that we have logical volumes:
 $ '''for dev in /dev/warthog-0-vg00/*; do sudo mkfs.ext4 -Fq ${dev}; done'''
 $

"Aha!" you exclaim.  "You just put an ext4 FS on the swap device."  Yes, I did.  To fix that...

 $ '''sudo mkswap /dev/warthog-0-vg00/swap'''
 mkswap: /dev/warthog-0-vg00/swap: warning: wiping old ext4 signature.
 Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
 no label, UUID=4443088b-d2dd-4cbb-8b78-cd85b06115fc
 $ 

==== Almost ready for the install ====
We need a place to put all the newly created filesystems so that debootstrap can write to them.  Let's do that and mount the things...

 $ sudo mkdir -m 0 /media/warthog-0
 $ sudo mount /dev/warthog-0-vg00/root /media/warthog-0
 $ sudo mkdir -v -m 0 /media/warthog-0/{boot,dev,home,opt,proc,run,sys,tmp,usr,var}
 mkdir: created directory '/media/warthog-0/boot'
 mkdir: created directory '/media/warthog-0/dev'
 mkdir: created directory '/media/warthog-0/home'
 mkdir: created directory '/media/warthog-0/opt'
 mkdir: created directory '/media/warthog-0/proc'
 mkdir: created directory '/media/warthog-0/run'
 mkdir: created directory '/media/warthog-0/sys'
 mkdir: created directory '/media/warthog-0/tmp'
 mkdir: created directory '/media/warthog-0/usr'
 mkdir: created directory '/media/warthog-0/var'
 $ sudo mount /dev/md/warthog-0-md_boot /media/warthog-0/boot
 $ sudo mount /dev/warthog-0-vg00/home /media/warthog-0/home
 $ sudo mount /dev/warthog-0-vg00/opt /media/warthog-0/opt
 $ sudo mount /dev/warthog-0-vg00/tmp /media/warthog-0/tmp
 $ sudo chmod 1777 /media/warthog-0/tmp
 $ sudo mount /dev/warthog-0-vg00/tmp /media/warthog-0/tmp
 $ sudo mount /dev/warthog-0-vg00/usr /media/warthog-0/usr
 $ sudo mount /dev/warthog-0-vg00/var /media/warthog-0/var
 $ sudo mount -t proc none /media/warthog-0/proc 
 $ sudo mount -t sysfs none /media/warthog-0/sys

==== Put an OS in it ====
Wherein debootstrap does its thing...

 adj@new-optiplex:/media$ sudo debootstrap --include=grub-pc,lvm2,mdadm,ssh,sudo,vim-nox,linux-image-4.9.0-6-amd64,man-db,less stretch warthog-0/
 I: Retrieving InRelease 
 I: Retrieving Release 
 I: Retrieving Release.gpg 
 I: Checking Release signature
 I: Valid Release signature (key id 067E3C456BAE240ACEE88F6FEF0F382A1A7B6500)
 I: Retrieving Packages 
 I: Validating Packages 
 [... many things elided ...]
 I: Configuring systemd...
 I: Configuring ca-certificates...
 I: Configuring initramfs-tools...
 I: Base system installed successfully.
 $ 

==== Additional necessary tweaks ====
Make sure /dev, /proc, and /sys are available inside the target root directory:

 $ sudo mount --bind /dev /media/warthog-0/dev
 $ sudo mount -t proc /media/warthog-0/proc
 $ sudo mount -t sys none /media/warthog-0/sys

Link /etc/mtab to /proc/self/mounts so that df and other things wanting to know about mounted filesystems work:

 $ sudo ln -s ../proc/self/mounts /media/warthog-0/etc/mtab

Enable serial console for the boot loader and the Linux kernel:

 $ sudo sed -i.orig -e 's/^GRUB_TIMEOUT=5$/GRUB_TIMEOUT=20/g' /media/warthog-0/etc/default/grub
 $ sudo sed -i -e 's/^GRUB_CMDLINE_LINUX_DEFAULT="quiet"$/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0,9600"/g' /media/warthog-0/etc/default/grub
 $ echo 'GRUB_TERMINAL=serial' | sudo tee -a /media/warthog-0/etc/default/grub
 $ echo 'GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1"' | sudo tee -a /media/warthog-0/etc/default/grub

==== set its host name ====
 $ echo warthog-0 | sudo tee /media/warthog-0/etc/hostname

==== a list of filesystems is important, too ====
Else we get a panic at boot.

 $ cat << _EOF_
 /dev/warthog-0-vg00/root					/		ext4		errors=remount-ro		0	0
 /dev/warthog-0-vg00/home					/home		ext4		defaults			0	0
 /dev/warthog-0-vg00/opt					/opt		ext4		defaults			0	0
 /dev/warthog-0-vg00/tmp					/tmp		ext4		defaults			0	0
 /dev/warthog-0-vg00/usr					/usr		ext4		defaults			0	0
 /dev/warthog-0-vg00/var					/var		ext4		defaults			0	0
 /dev/warthog-0-vg00/swap					none		swap		sw				0	0
 UUID=75dfffb8-1ab6-494d-a1bc-d70d30a135ba	/boot		ext4		defaults			0	0
 __EOF__ | sudo tee -a /media/warthog-0/etc/fstab
 $

==== make it bootable ====
Enter the chroot, put grub onto the new drive:
 $ '''sudo chroot /media/warthog-0 /bin/bash --login'''
 root@new-optiplex:/# '''grub-install /dev/sdg'''
 Installing for i386-pc platform.
 device node not found
 device node not found
 device node not found
 device node not found
 device node not found
 device node not found
 device node not found
 device node not found
 grub-install: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..
 grub-install: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..
 device node not found
 Installation finished. No error reported.
 root@new-optiplex:/# update-grub

==== make a user with admin privileges ====
Because it will be nice to say "root is locked" and still be able to log in, right?
 root@new-optiplex:/# '''adduser --home /home/itops --shell /bin/bash --uid 1000 --ingroup itops --gecos "IT operations account" itops'''
 Adding user `itops' ...
 Adding new user `itops' (1000) with group `itops' ...
 Creating home directory `/home/itops' ...
 Copying files from `/etc/skel' ...
 Enter new UNIX password: 
 Retype new UNIX password: 
 passwd: password updated successfully
 root@new-optiplex:/# '''adduser itops sudo'''
 Adding user `itops' to group `sudo' ...
 Adding user itops to group sudo
 Done.
 root@new-optiplex:/#
Clean up /etc/passwd, /etc/group, and related files:
 root@new-optiplex:/# '''pwck -s'''
 root@new-optiplex:/# '''grpck -s'''

==== un-mount all the stuffs ====
 root@new-optiplex:/# '''umount -av'''
 root@new-optiplex:/# '''exit'''
 $ '''sudo umount /media/warthog-0'''
 $ '''sudo vgchange -a n warthog-0-vg00'''
 $ '''sudo mdadm --stop /dev/md/warthog-0-md_boot'''
 $ '''sudo mdadm --stop /dev/md/warthog-0-vg00-pv-0'''

== Make it go! ==
Remove drive from installation host system.
Install in A10 Networks chassis
Re-install CMOS battery
Close up the box
Attach serial console  (Cisco console cables work for this)
Configure terminal program (Kermit or similar)
Apply power
Watch for GRUB menu after 
Let Linux boot
If all goes well, there will be a friendly getty asking for a user name (log-in with account created above)

== Make it network ==
Create an /etc/network/interfaces.d/enp21s0 file with these contents:
 allow-hotplug enp21s0
 auto enp21s0
 iface enp21s0 inet dhcp

(enp21s0 is just what the new network interface name scheme calls the "MGMT" port on the front the chassis)

== Make it less loud ==
It really is quite loud.  Obnoxiously so.  Let us see about making it less so.

 $ sudo apt-get install lm-sensors fancontrol
 $ sudo sensors-detect
 [...]
 $ sudo pwmconfig
 [go through the all the menus]
 $

No luck with this.  Will have to see if ACOS does anything to moderate the fan speeds and if it can be pulled into a less specialised Linux distro.

There is a BIOS setting that allows control of the fans by the OS.  This needs to be set appropriately.  Afterwards pwmconfig kinda works.

== Lots of network interfaces, I need a map! ==
{|
 !Front panel name
 !Linux name
 |-
 |MGMT
 |enp21s0
 |-
 |1
 |
 |-
 |2
 |
 |-
 |3
 |
 |-
 |4
 |
 |-
 |5
 |
 |-
 |6
 |
 |-
 |7
 |
 |-
 |8
 |
 |-
 |9
 |
 |-
 |10
 |
 |-
 |11
 |
 |-
 |12
 |
 |-
 |13
 |
 |-
 |14
 |
 |-
 |15
 |
 |-
 |16
 |
 |-
 |17
 |enp18s0f0 (10Gbits/sec SFP+)
 |-
 |18
 |enp18s0f1 (10Gbits/sec SFP+)
 |-
 |19
 |enp19s0f0 (10Gbits/sec SFP+)
 |-
 |20
 |enp19s0f1 (10Gbits/sec SFP+)
 |}

This looks like it's missing some stuff, doesn't it?  Well, need to get the drivers for the Ethernet bypass ports sorted out.  (Because things can't be too easy.)

'''Important Note:''' The stock Debian stretch 4.9 Linux kernel was unable to get the links on the 10GbE ports to come up.  Upgrading to a 4.16 backports kernel had good results.

== Other Debian packages you will want to install ==
* less
* pciutils
* iucode-tool intel-microcode (these come from contrib and non-free)

== Ethernet bypass port frustrations ==
Along with the fans...  The 1000baseT ports numbered 1 through 8 are being recalcitrant.  In the ACOS image, there is a custom driver for the chips called e1000_a10e.  As far as I can tell, the A10 ACOS software doesn't really care about LAN bypass stuff at all and pretty much just turns on all the ports as separate units.  (The LAN bypass idea here is that if power to the system goes away, a pair of ports can be electrically bound together and let traffic flow as though it were just a cable.  A possibly nice feature in a firewall.  But the hardware isn't standard and has to be dealt with.)

But...  [https://www.lannerinc.com/ Lanner] (who makes the system for A10) does provide source for working with the LAN bypass modules.  Watch this space for updates.

Lanner provides a hopefully useful manual here: https://www.lannerinc.com/category/1202-network-appliances?download=1720

'''Update 2019-01-11:''' No forward progress on this, unfortunately.  I have verified that the 1000baseT cards have Lanner Gen 2 bypass modules and should be controllable with the Lanner <code>wdbp</code> or <code>wd_bp</code> program.  I have not had success getting them to leave bypass mode.  :(  I **do** have an email correspondence with a Lanner rep and have asked for a stock (not A10 custom) BIOS image for the machine, which **should** be able to tweak the bypass settings at POST time.

In the mean time, the eight 1000baseT ports and the eight GigE SFP ports are unused, the Comcast/Xfinity cable modem is connected to the switch and the Comcast/Xfinity WAN VLAN is passed to the A10 device on one of its 10G SFP+ ports.