Mellanox SX6036

Console access

Serial port

There is an 8P8C modular (RJ45) connector on the right side of the port side of the switch. This connector uses the Cisco pinout and a common Cisco console cable works great for driving it.

Serial settings are 9600 bits per second, 8 data bits, no parity bits, 1 stop bit (9600 8N1)

Default credentials

User admin.

Password admin.

Resetting the password

In the lower left corner of the port side of the switch, there is a small hole labelled "RST". Holding the button inside with a paperclip or similar tool for 15 seconds will reset the switch and erase any password set for the admin user.

Firmware updates

Updating firmware on these switches is a protracted process. Firmware images are available on the public internet, but some searching is required. To bring a switch from a 3.2.x release up to the final 3.6.8012 release, locate the following images on the web and save them to an SSHable or HTTP(S)able location near you:

• image-PPC_M460EX-3.4.3002.img
• image-PPC_M460EX-3.5.1006.img
• image-PPC_M460EX-3.6.1002.img
• image-PPC_M460EX-3.6.8012.img
• image-PPC_M460EX-SX_3.3.3400.img
• image-PPC_M460EX-SX_3.3.5006.img
• image-PPC_M460EX-SX_3.4.0012.img

New firmware image flashing procedure

This is a privileged operation, so first off gain elevated privileges...

enable

image fetch http://172.17.0.17/~adj/image-PPC_M460EX-SX_3.3.3400.img

image install image-PPC_M460EX-SX_3.3.3400.img location 2 progress track verify require-sig

reload

Wait for U-Boot to prompt about a menu, and send a Ctrl-B before the countdown reaches 0. Select option 2 (no need for a newline) to boot the newly flashed image. Iterate through each of the newer images, alternating location 1 and location 2 in the image install ... command.

When updates version installations are finally complete, it might be good to ensure that 3.6.8012 is flashed to both locations 1 and 2.

Useful CLI commands

Find MAC address of 1000baseT management interface 0

Good for setting up the DHCP server's config for this switch. For some reason, this command needs elevated privileges. Grrr.

enable

show interfaces mgmt0 | include HW

Find system serial number without reading the physical tag

This is useful for license key generation. And inventory purposes.

show inventory

Find system host ID

Also can be used for license key generation. Looks suspiciously like the mgmt0 MAC address...

show version | include Host

Licensing features

Mellanox's SwitchX2 ASIC is capable of performing Infiniband and Ethernet switching. Mellanox the company likes to lock some of these features behind licensing keys. Which is annoying. But with enough effort, is a surmountable obstacle

The license generation tool

Inside the MLNX-OS distribution images (those files with names like image-PPC_M460EX-3.6.8012.img) there exists a gzipped tarball containing the MLNX-OS filesystem. Inside the tarball is a directory called /opt/tms/bin. And inside that directory, one can find the license key generation program, genlicense. The MLNX-OS filesystem image is a PowerPC Linux distribution that can be run from a Qemu emulated big-endian PowerPC machine, or natively on big-endian PowerPC hardware. We can use this to generate feature licenses until the cows come home.

Our first license, shell access from the switch CLI

There is a hidden CLI command, _shell which will cause the CLI to execve("/bin/bash", ...). This command is privileged (must be run from enable mode) and locked behind a license key. The following is a working CLI session to install a key for this feature:

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-1 login: admin
Password: passwordgoesherebutisnotechoed
Last login: Sun Feb 19 23:25:30 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal 
mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # show licenses
License 1: LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
   Feature:          RESTRICTED_CMDS_GEN2
   Description:      Access to restricted system functionality
   Valid:            yes
   Active:           yes
mellanox-sx6036-rack-1 [standalone: master] #

Ethernet features: Ethernet, Ethernet L2, and Ethernet L3

We will generate these from the switch as we already have shell access and can use the included genlicense tool.

Running the following from the switch's shell will print a key for a non-hardware-locked, non-date-locked license to activate the switch's full Ethernet capabilities:

[admin@mellanox-sx6036-rack-0 ~]# cd /opt/tms/bin
[admin@mellanox-sx6036-rack-0 bin]# ./genlicense 2 EFM_SX m2l0n%0x9 -o 53 true -o 51 true -o 58 true
LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X
[admin@mellanox-sx6036-rack-0 bin]#

A quick explanatory diversion is in order here. There are 2 types of keys. genlicense's first option above specifies key type 2. EFM_SX is the Generic SX license type. By itself it does nothing, but all of the -o flags specify further features to be activated. A complete list of these can be generated by running /opt/tms/bin/genlicense 2. There are 14 of these additional options, and their meaning is not well documented. The magic word m2l0n%0x9 can be found by watching for strlen() calls in the process flow of /opt/tms/bin/dumplicense when run under ltrace.

And it can be installed in the switch's configuration database by leaving the shell (which will disconnect the terminal session), logging in again, running enable to get elevated privileges, and using the license install configuration command like so:

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-1 login: admin
Password: 
Last login: Sun Feb 19 23:31:51 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X
License was installed successfully. Please wait 1 minute before further configurations.
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] #