Mellanox SX6036
Console access
Serial port
There is an 8P8C modular (RJ45) connector on the right side of the port side of the switch. This connector uses the Cisco pinout and a common Cisco console cable works great for driving it.
Serial settings are 9600 bits per second, 8 data bits, no parity bits, 1 stop bit (9600 8N1)
Default credentials
User admin.
Password admin.
Resetting the password
In the lower left corner of the port side of the switch, there is a small hole labelled "RST". Holding the button inside with a paperclip or similar tool for 15 seconds will reset the switch and erase any password set for the admin user.
Firmware updates
Updating firmware on these switches is a protracted process. Firmware images are available on the public internet, but some searching is required. To bring a switch from a 3.2.x release up to the final 3.6.8012 release, locate the following images on the web and save them to an SSHable or HTTP(S)able location near you:
- image-PPC_M460EX-3.4.3002.img
- image-PPC_M460EX-3.5.1006.img
- image-PPC_M460EX-3.6.1002.img
- image-PPC_M460EX-3.6.8012.img
- image-PPC_M460EX-SX_3.3.3400.img
- image-PPC_M460EX-SX_3.3.5006.img
- image-PPC_M460EX-SX_3.4.0012.img
New firmware image flashing procedure
This is a privileged operation, so first off gain elevated privileges...
enable
image fetch http://172.17.0.17/~adj/image-PPC_M460EX-SX_3.3.3400.img
image install image-PPC_M460EX-SX_3.3.3400.img location 2 progress track verify require-sig
reload
Wait for U-Boot to prompt about a menu, and send a Ctrl-B before the countdown reaches 0. Select option 2 (no need for a newline) to boot the newly flashed image. Iterate through each of the newer images, alternating location 1 and location 2 in the image install ... command.
When updates version installations are finally complete, it might be good to ensure that 3.6.8012 is flashed to both locations 1 and 2.
Useful CLI commands
Find MAC address of 1000baseT management interface 0
Good for setting up the DHCP server's config for this switch. For some reason, this command needs elevated privileges. Grrr.
enable
show interfaces mgmt0 | include HW
Find system serial number without reading the physical tag
This is useful for license key generation. And inventory purposes.
show inventory
Find system host ID
Also can be used for license key generation. Looks suspiciously like the mgmt0 MAC address...
show version | include Host
Licensing features
Mellanox's SwitchX2 ASIC is capable of performing Infiniband and Ethernet switching. Mellanox the company likes to lock some of these features behind licensing keys. Which is annoying. But with enough effort, is a surmountable obstacle
The license generation tool
Inside the MLNX-OS distribution images (those files with names like image-PPC_M460EX-3.6.8012.img) there exists a gzipped tarball containing the MLNX-OS filesystem. Inside the tarball is a directory called /opt/tms/bin. And inside that directory, one can find the license key generation program, genlicense. The MLNX-OS filesystem image is a PowerPC Linux distribution that can be run from a Qemu emulated big-endian PowerPC machine, or natively on big-endian PowerPC hardware. We can use this to generate feature licenses until the cows come home.
Our first license, shell access from the switch CLI
There is a hidden CLI command, _shell which will cause the CLI to execve("/bin/bash", ...). This command is privileged (must be run from enable mode) and locked behind a license key. The following is a working CLI session to install a key for this feature:
Mellanox MLNX-OS Switch Management mellanox-sx6036-rack-1 login: admin Password: passwordgoesherebutisnotechoed Last login: Sun Feb 19 23:25:30 on ttyS0 Mellanox Switch mellanox-sx6036-rack-1 [standalone: master] > enable mellanox-sx6036-rack-1 [standalone: master] # configure terminal mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1 mellanox-sx6036-rack-1 [standalone: master] (config) # exit mellanox-sx6036-rack-1 [standalone: master] # show licenses License 1: LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1 Feature: RESTRICTED_CMDS_GEN2 Description: Access to restricted system functionality Valid: yes Active: yes mellanox-sx6036-rack-1 [standalone: master] #
Ethernet features: Ethernet, Ethernet L2, and Ethernet L3
We will generate these from the switch as we already have shell access and can use the included genlicense tool.
Running the following from the switch's shell will print a key for a non-hardware-locked, non-date-locked license to activate the switch's full Ethernet capabilities:
[admin@mellanox-sx6036-rack-0 ~]# cd /opt/tms/bin [admin@mellanox-sx6036-rack-0 bin]# ./genlicense 2 EFM_SX m2l0n%0x9 -o 53 true -o 51 true -o 58 true LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X [admin@mellanox-sx6036-rack-0 bin]#
A quick explanatory diversion is in order here. There are 2 types of keys. genlicense's first option above specifies key type 2. EFM_SX is the Generic SX license type. By itself it does nothing, but all of the -o flags specify further features to be activated. A complete list of these can be generated by running /opt/tms/bin/genlicense 2". There are 14 of these additional options, and their meaning is not well documented. The magic word m2l0n%0x9 can be found by watching for strlen() calls in the process flow of /opt/tms/bin/dumplicense when run under ltrace.
And it can be installed in the switch's configuration database by leaving the shell (which will disconnect the terminal session), logging in again, running enable to get elevated privileges, and using the license install configuration command like so:
Mellanox MLNX-OS Switch Management mellanox-sx6036-rack-1 login: admin Password: Last login: Sun Feb 19 23:31:51 on ttyS0 Mellanox Switch mellanox-sx6036-rack-1 [standalone: master] > enable mellanox-sx6036-rack-1 [standalone: master] # configure terminal mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X License was installed successfully. Please wait 1 minute before further configurations. mellanox-sx6036-rack-1 [standalone: master] (config) # exit mellanox-sx6036-rack-1 [standalone: master] # write memory mellanox-sx6036-rack-1 [standalone: master] #