Nortel 55x0
Recently acquired 3 Nortel/Avaya BayStack ERS 5520-48T-PWR switches. They are pretty nifty.
Why an ERS 55x0?
- The price is right. These can be had on eBay for < $50 including shipping at present.
- Solid build. They have a reputation for sitting in wiring closets powering large numbers of desktop phones for years on end.
- Well liked by the /r/homelab crowd on Reddit
- Firmware can be found with a little searching. (These are no longer supported, and firmware is only available with a support contract if you want to go through Avaya channels. But recently released updates have made their way to sites across the Internet. Look for files with names like
5xxx_636017s.imgand5xxx_60021_diags.bin - Stackable. There are 2 40Gbps stack ports located at the rear of each switch. So they say "80Gbps stacking included". Stacking cables can be found for well under $20 on eBay at present.
- POE to keep the phones and APs happy without additional power injectors in the network.
- Way cheaper than an ERS 5650 line switch. The 5650s are the successor product. Better everything. But relatively few of them shipped. And sellers on eBay seem to want at least 5x as much for them right now. 5650s have redundant AC power options, faster stacking connection, and are still getting software updates.
Why to avoid this one
- It's not whisper quiet. (It's not super loud, but you can't have a quiet conversation someone if you have one running on your desk.)
- It has a reputation for being power hungry. No Kill-a-Watt so I can't measure it.
- It's not a Cisco or Juniper or ProCurve or something else that will translate 1-for-1 to an enterprise switching certification. But they're capable and can do pretty much anything a mid-high end 48 port 1000baseT switch should do.
- It isn't even remotely black. All the other switches I've had in recent memory have been black or grey. This is a light grey/putty color.
A funny thing
This physical box for these switches is very nearly identical to the Force10 S50s I have. So much so that the Force10 rack mount ears fit. Yay for reference designs.
Get an old switch up to date
Oh, if only it were as simple as apt-get update; apt-get dist-upgrade. Alas, these run an customized embedded vxWorks that seems to be derived from an example OS provided by Broadcom, maker of most of the smarts inside these switches.
List of needed thingies:
- Console cable. Straight through 9-pin works fine. A null modem cable is not needed. Put another way, the switch's console port is set up as an RS-232 DCE. A PC's RS-232 port is a DTE port. DTE talks to DCE over a straight through connection. And there is much rejoicing.
- Terminal emulator. I like C-Kermit.
- A collection of Nortel/Avaya software images. (See above for hints on finding these. Also note that they're popular with the HomeLab crowd on Reddit
- Enough network infrastructure to have a functional TFTP server.
Firmware update overview
The latest (and most likely last) software release for these switches is v6.3.6. I can't quite find a consistent name for the OS it runs, but that may be due to insufficient amounts of time googling. The first of the 3 switches I upgraded was running an early v6.0 release. Careful notes were not kept as that switch was upgraded. Here follows the process for switch number 2.
- Attach console cable
- fire up kermit. 9600bps, 8 data bits, no parity, one stop bit. (9600-8n1) No hardware or software flow control.
- Apply power to switch
- Interrupt bootstrap by pressing Esc a few times after "
Starting.." appears. Be presented with a nice menu of options. This switch is running newer diag firmware than the first one. Diag image is "Diagnostic Version 6.0.0.10 Jan 28 2010, 11:58:40" and the agent image in the flash is "Vers: 6.1.5.015". - Wipe switch config by selecting option "
i" from the menu. - boot the switch firmware by starting the "agent" code. This is option "
a" or "b" on the menu. - Type Ctrl-Y when prompted after the big NORTEL banner (This may be a big AVAYA banner instead, depending on the firmware on the switch)
- Enter an IP address that can reach the TFTP server when the switch's quick config wizard prompts for it.
- continue to the menu
- drop to the CLI from the the menu
- gain privileges using the
enablecommand - determine what versions of firmware are installed.
show sys-infois your friend here. This switch (s/n LBNNTMJPT4080V) has diags version 6.0.0.10 and agent code v6.1.5.015 running. I have seen a number of references skipping lots of versions when installing updates is not a good idea, so I'll proceed from the 6.1.5 code currently running to 6.2.0 to 6.3.0 to 6.3.6. - Determine if a "diags" update is needed. v6.2.0 includes diags 6.0.0.10, the same version currently installed. No need to re-flash this, so install the 6.2.0 agent code. Put the appropriate .img file in the root of the TFTP server's tree and execute
download address 192.168.20.1 secondary image 5xxx_620009s.imgto download and flash the new agent code. There is a handyimage-if-newerversion of this command, but it needs an image in the correct bank to compare against.image-if-newer 5xxx_620009s.imgwill report "Image version is not new" if there is no secondary image in the flash. - Let switch reboot and move on to the v6.3.0 update. This one has a new diags image file: 5xxx_60015_diags.bin. The diags image is supposed to be updated before the agent image so we'll do "
download address 192.168.20.1 diag 5xxx_60015_diags.binand let the switch reboot. Follow withdownload address 192.168.20.1 secondary image-if-newer 5xxx_630013s.img. - Last firmware step to bring us to the final 6.3.6 release:
download address 192.168.20.1 diag 5xxx_60021_diags.binfollowed bydownload address 192.168.20.1 secondary image-if-newer 5xxx_636017s.img. - confirm software versions with
show bootandshow system
Basic configuration items
Desired end state
- Start from clean slate
- Switch will not do layer 3 forwarding (no IPv4 or IPv6 forwarding from the switch)
- VLANs as descriped in VLANs and subnet
- DHCP proxying/forwarding back to sacredchao
- Minimal PoE enabled (only one phone and one PoE AP in service right now. Might grow to twice that much soon)
- Put unknown devices on the guest VLAN
- NTP client turned on for switch(es)
- SSH server turned on for switch(es)
- Telnet access turned off for switch(es)
- Reasonable passwords set
wipe the config
Put the factory blank config back onto the switch from the CLI:
5520-48T-PWR>enable 5520-48T-PWR#restore factory-default Warning the switch/stack will be reset to factory default configuration, Do you wish to continue (y/n) ? y Switch is now resetting to factory default configuration . . .
Switch will reboot with a clean config. The config can also be wiped using the Nortel button on the front panel or from the diags menu when first powered on.
Got logins?
It might be nice to have the switch ask for a password when a user connects to it. We'll set up two users, user and admin. Passwords are prompted for, but these lines are overwritten on the terminal so aren't recorded here.
5520-48T-PWR>enable 5520-48T-PWR#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 5520-48T-PWR(config)#username user ro % Invalid password length - must be from 10 to 15 characters in length 5520-48T-PWR(config)#username user ro % Minimum 2 upper,2 lowercase letters,2 numbers,2 chars like !@#$%^&*() require d 5520-48T-PWR(config)#username user ro % Minimum 2 upper,2 lowercase letters,2 numbers,2 chars like !@#$%^&*() require d 5520-48T-PWR(config)#username user ro % Minimum 2 upper,2 lowercase letters,2 numbers,2 chars like !@#$%^&*() require d 5520-48T-PWR(config)#username user ro 5520-48T-PWR(config)#
Note that it took several tries to get select a password that met the complexity requirements. And now for the privileged user:
5520-48T-PWR(config)#username admin rw 5520-48T-PWR(config)#
Let!Me!L00k! and L3t!M3!Chang3! are possible passwords that do meet the complexity requirements.
Important Note: Providing a username and password is not required on the console unless the serial-security configuration setting is enabled.
Clock setting
Pretty simple, really:
5520-48T-PWR>enable 5520-48T-PWR#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 5520-48T-PWR(config)#sntp server primary address 10.10.0.1 5520-48T-PWR(config)#sntp enable 5520-48T-PWR(config)#sntp sync-interval 1 5520-48T-PWR(config)#^Z 5520-48T-PWR#write memory
And now, every hour, the switch will do a quick SNTP query to 10.10.0.1 and set its clock based on the response.
I can haz VLANs?
Creating a VLAN and adding it to a port these switches is a multi-part process:
- Create the VLAN
- (optionally) give it a name/description
- determine if a port will carry this port tagged or untagged
- connect the VLAN to a port
More concretely, a new server has arrived. It's an HP(E) Proliant DL380e with a dedicated system management (ILO) Ethernet connection and 4 on-motherboard 1000baseT connections. Let's find 2 free switch ports, connect one to the ILO (hardware management VLAN) and the other to a newly created "garage network" VLAN (it's where servers' OS get installed) for GigE port 1 on the back of the server. Looking for a block of ports near each other that are not running at the moment...
5520-48T-PWR#show interfaces link-down
Status Auto Flow
Port Trunk Admin Oper Link LinkTrap Negotiation Speed Duplex Control
---- ----- ------- ---- ---- -------- ----------- -------- ------ -------
1 Enable Down Down Enabled Enabled
2 Enable Down Down Enabled Enabled
3 Enable Down Down Enabled Enabled
4 Enable Down Down Enabled Enabled
5 Enable Down Down Enabled Enabled
6 Enable Down Down Enabled Enabled
7 Enable Down Down Enabled Enabled
8 Enable Down Down Enabled Enabled
9 Enable Down Down Enabled Enabled
12 Enable Down Down Enabled Enabled
13 Enable Down Down Enabled Enabled
14 Enable Down Down Enabled Enabled
15 Enable Down Down Enabled Enabled
17 Enable Down Down Enabled Enabled
18 Enable Down Down Enabled Enabled
20 Enable Down Down Enabled Enabled
21 Enable Down Down Enabled Enabled
24 Enable Down Down Enabled Enabled
25 Enable Down Down Enabled Enabled
26 Enable Down Down Enabled Enabled
27 Enable Down Down Enabled Enabled
28 Enable Down Down Enabled Enabled
29 Enable Down Down Enabled Enabled
31 Enable Down Down Enabled Enabled
32 Enable Down Down Enabled Enabled
33 Enable Down Down Enabled Enabled
36 Enable Down Down Enabled Enabled
37 Enable Down Down Enabled Enabled
38 Enable Down Down Enabled Enabled
39 Enable Down Down Enabled Enabled
40 Enable Down Down Enabled Enabled
41 Enable Down Down Enabled Enabled
42 Enable Down Down Enabled Enabled
43 Enable Down Down Enabled Enabled
44 Enable Down Down Enabled Enabled
45 Enable Down Down Enabled Enabled
46 Enable Down Down Enabled Enabled
47 Enable Down Down Enabled Enabled
48 Enable Down Down Enabled Enabled
5520-48T-PWR#
That collection of ports numbered 24-29 looks like a good possibility. A visual inspection of those show that ports 25-28 have no cables in them. So we're going to make port 25 the ILO port and attach it to the hardware management VLAN (99) and port 26 will be attached to the "server build" VLAN (199). Confirming that ports 25-28 don't have anything special done to them so far...
5520-48T-PWR#show vlan interface info 25-28
Filter Filter
Untagged Unregistered
Port Frames Frames PVID PRI Tagging Name
---- -------- ------------ ---- --- ------------- --------------
25 No Yes 1 0 UntagAll Port 25
26 No Yes 1 0 UntagAll Port 26
27 No Yes 1 0 UntagAll Port 27
28 No Yes 1 0 UntagAll Port 28
5520-48T-PWR#
Put names on ports 25 and 26 like so:
5520-48T-PWR#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 5520-48T-PWR(config)#interface fastEthernet 25 5520-48T-PWR(config-if)#name "HP DL380e s/n MXQ51303SS iLO" 5520-48T-PWR(config-if)#interface fastEthernet 26 5520-48T-PWR(config-if)#name "HP DL380e s/n MXQ51303SS GigE #1" 5520-48T-PWR(config-if)#^Z
And confirm they are there:
5520-48T-PWR#show interfaces names 25-26 Port Name ---- ---------------------------------------------------------------- 25 HP DL380e s/n MXQ51303SS iLO 26 HP DL380e s/n MXQ51303SS GigE #1 5520-48T-PWR#
Next up is putting the iLO port on VLAN 99, untagged. VLAN 99 already exists, so we need to connect port 25 to VLAN 99, make VLAN 99 port 25's primary VLAN, and remove any other VLANs from port 25:
5520-48T-PWR#show vlan id 99 Id Name Type Protocol PID Active IVL/SVL Mgmt ---- -------------------- -------- ---------------- ------- ------ ------- ---- 99 infra mgmt Port None 0x0000 Yes IVL Yes Port Members: 1-2,10,16,18,45-48 Total VLANs: 1 5520-48T-PWR#
Changing the port and VLAN config like so...
5520-48T-PWR#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 5520-48T-PWR(config)#vlan ports 25 pvid 99 5520-48T-PWR(config)#vlan ports 25 filter-unregistered-frames enable 5520-48T-PWR(config)#vlan ports 25 tagging untagPvidOnly 5520-48T-PWR(config)#vlan members remove 1 25 5520-48T-PWR(config)#vlan members add 99 25 5520-48T-PWR(config)#^Z 5520-48T-PWR#
A little bit of explantion for the above:
- set port 25's Primary VLAN id to 99
- make sure that 802.1Q tagged frames arriving at this port that do not match a VLAN this port is a member of are dropped
- make sure that VLAN 99 frames are untagged when they leave the port
- remove port 25 from VLAN 1
- add port 25 to VLAN 99
With this, port 25 should see only traffic on VLAN 99, and it should all be untagged.
In this next bit of switch configuration, VLAN 199 is created and port 26 configured to carry it untagged
5520-48T-PWR#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 5520-48T-PWR(config)#vlan create 199 name "server build" type port cist 5520-48T-PWR(config)#vlan ports 26 pvid 199 5520-48T-PWR(config)#vlan ports 26 filter-unregistered-frames enable 5520-48T-PWR(config)#vlan ports 26 tagging untagPvidOnly 5520-48T-PWR(config)#vlan members remove 1 26 WARNING: STP configuration may be lost on selected ports. You may need to recon figure the ports manually. 5520-48T-PWR(config)#vlan members add 199 26 5520-48T-PWR(config)#^Z 5520-48T-PWR#show vlan id 199 Id Name Type Protocol PID Active IVL/SVL Mgmt ---- -------------------- -------- ---------------- ------- ------ ------- ---- 199 server build Port None 0x0000 Yes IVL No Port Members: 26 Total VLANs: 1 5520-48T-PWR#
The only significant difference from the port 25 on VLAN 99 configuration is that VLAN 199 is actually created as the first configuration change command.