Mellanox SX6036

Console access

Serial port

There is an 8P8C modular (RJ45) connector on the right side of the port side of the switch. This connector uses the Cisco pinout and a common Cisco console cable works great for driving it.

Serial settings are 9600 bits per second, 8 data bits, no parity bits, 1 stop bit (9600 8N1)

Default credentials

User admin.

Password admin.

Resetting the password

In the lower left corner of the port side of the switch, there is a small hole labelled "RST". Holding the button inside with a paperclip or similar tool for 15 seconds will reset the switch and erase any password set for the admin user.

Firmware updates

Updating firmware on these switches is a protracted process. Firmware images are available on the public internet, but some searching is required. To bring a switch from a 3.2.x release up to the final 3.6.8012 release, locate the following images on the web and save them to an SSHable or HTTP(S)able location near you:

• image-PPC_M460EX-3.4.3002.img
• image-PPC_M460EX-3.5.1006.img
• image-PPC_M460EX-3.6.1002.img
• image-PPC_M460EX-3.6.8012.img
• image-PPC_M460EX-SX_3.3.3400.img
• image-PPC_M460EX-SX_3.3.5006.img
• image-PPC_M460EX-SX_3.4.0012.img

New firmware image flashing procedure

This is a privileged operation, so first off gain elevated privileges...

enable

image fetch http://172.17.0.17/~adj/image-PPC_M460EX-SX_3.3.3400.img

image install image-PPC_M460EX-SX_3.3.3400.img location 2 progress track verify require-sig

reload

Wait for U-Boot to prompt about a menu, and send a Ctrl-B before the countdown reaches 0. Select option 2 (no need for a newline) to boot the newly flashed image. Iterate through each of the newer images, alternating location 1 and location 2 in the image install ... command.

When updates version installations are finally complete, it might be good to ensure that 3.6.8012 is flashed to both locations 1 and 2.

Useful CLI commands

Find MAC address of 1000baseT management interface 0

Good for setting up the DHCP server's config for this switch. For some reason, this command needs elevated privileges. Grrr.

enable

show interfaces mgmt0 | include HW

Find system serial number without reading the physical tag

This is useful for license key generation. And inventory purposes.

show inventory

Find system host ID

Also can be used for license key generation. Looks suspiciously like the mgmt0 MAC address...

show version | include Host

Licensing features

Mellanox's SwitchX2 ASIC is capable of performing Infiniband and Ethernet switching. Mellanox the company likes to lock some of these features behind licensing keys. Which is annoying. But with enough effort, is a surmountable obstacle

The license generation tool

Inside the MLNX-OS distribution images (those files with names like image-PPC_M460EX-3.6.8012.img) there exists a gzipped tarball containing the MLNX-OS filesystem. Inside the tarball is a directory called /opt/tms/bin. And inside that directory, one can find the license key generation program, genlicense. The MLNX-OS filesystem image is a PowerPC Linux distribution that can be run from a Qemu emulated big-endian PowerPC machine, or natively on big-endian PowerPC hardware. We can use this to generate feature licenses until the cows come home.

Our first license, shell access from the switch CLI

There is a hidden CLI command, _shell which will cause the CLI to execve("/bin/bash", ...). This command is privileged (must be run from enable mode) and locked behind a license key. The following is a working CLI session to install a key for this feature:

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-1 login: admin
Password: passwordgoesherebutisnotechoed
Last login: Sun Feb 19 23:25:30 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal 
mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # show licenses
License 1: LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
   Feature:          RESTRICTED_CMDS_GEN2
   Description:      Access to restricted system functionality
   Valid:            yes
   Active:           yes
mellanox-sx6036-rack-1 [standalone: master] #

Ethernet features: Ethernet, Ethernet L2, and Ethernet L3

We will generate these from the switch as we already have shell access and can use the included genlicense tool.

Running the following from the switch's shell will print a key for a non-hardware-locked, non-date-locked license to activate the switch's full Ethernet capabilities:

[admin@mellanox-sx6036-rack-0 ~]# cd /opt/tms/bin
[admin@mellanox-sx6036-rack-0 bin]# ./genlicense 2 EFM_SX m2l0n%0x9 -o 53 true -o 51 true -o 58 true
LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X
[admin@mellanox-sx6036-rack-0 bin]#

A quick explanatory diversion is in order here. There are 2 types of keys. genlicense's first option above specifies key type 2. EFM_SX is the Generic SX license type. By itself it does nothing, but all of the -o flags specify further features to be activated. A complete list of these can be generated by running /opt/tms/bin/genlicense 2. There are 14 of these additional options, and their meaning is not well documented. The magic word m2l0n%0x9 can be found by watching for strlen() calls in the process flow of /opt/tms/bin/dumplicense when run under ltrace.

And it can be installed in the switch's configuration database by leaving the shell (which will disconnect the terminal session), logging in again, running enable to get elevated privileges, and using the license install configuration command like so:

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-1 login: admin
Password: 
Last login: Sun Feb 19 23:31:51 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-EFM_SX-5M11-5K11-5T11-88A1-BBD0-JP82-X
License was installed successfully. Please wait 1 minute before further configurations.
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] #

Switch configuration commands

Some of these things will need licenses first.

I can haz Ethernet?

Install the Ethernet key as shown above and confirm it is working:

mellanox-sx6036-rack-1 [standalone: master] # show system capabilities
IB: Supported, L2, Adaptive Routing
Ethernet: Supported, L2, L3
GW: Not supported
Max SM nodes: 648
IB Max licensed speed: FDR10
Ethernet Max licensed speed: 56Gb
mellanox-sx6036-rack-1 [standalone: master] #

It says we can do Ethernet. Which is cool. But it does not have the Infiniband-Ethernet gateway feature licensed. And this is required to enable the switch to run in VPI mode. VPI is "Virtual Protocol Interconnect" or Mellanox's way of saying "We can do Infiniband and Ethernet on a single switch." So another license is needed. Here's a key for that:

mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # license install LK2-EFM_SX-5N21-488A-182A-UQXB-Y6
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # show system capabilities 
IB: Supported, L2, Adaptive Routing
Ethernet: Supported, L2, L3
GW: Supported
Max SM nodes: 648
IB Max licensed speed: FDR10
Ethernet Max licensed speed: 56Gb
mellanox-sx6036-rack-1 [standalone: master] # 

We will need to do this next thing to convert the switch from an Infiniband-only switch to an Infiniband-and-Ethernet switch:

mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # system profile vpi-single-switch
Warning - confirming will cause system reboot and all configuration will be deleted
Type 'yes' to confirm profile change: yes

Allow time for the switch to reboot and log in again. When it next starts up, run the command show system profile to confirm that it is in VPI mode:

mellanox-sx6036-rack-1 [standalone: master] > show system profile

Profile: vpi-single-switch

mellanox-sx6036-rack-1 [standalone: master] >

And finally, let's set some ports to Ethernet mode:

mellanox-sx6036-rack-1 [standalone: master] > show ports type
InfiniBand: 1/1 1/2 1/3 1/4 1/5 1/6 1/7 1/8 1/9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31 1/32 1/33 1/34 1/35 1/36 
mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # interface ib 1/19-1/36
mellanox-sx6036-rack-1 [standalone: master] (config interface ib 1/19-1/36) # shutdown
mellanox-sx6036-rack-1 [standalone: master] (config interface ib 1/19-1/36) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # port 1/19-1/36 type ethernet
mellanox-sx6036-rack-1 [standalone: master] (config) # interface ethernet 1/19-1/36
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/19-1/36) # no shutdown
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/19-1/36) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show ports type
Ethernet:   1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31 1/32 1/33 1/34 1/35 1/36 
InfiniBand: 1/1 1/2 1/3 1/4 1/5 1/6 1/7 1/8 1/9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 
mellanox-sx6036-rack-1 [standalone: master] #

Switch management items

Password for the admin user

Factory defaults are probably not a good idea, so let's tighten things up a bit.

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # username admin password the_password_goes_here 
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # 

IP addressing for the management interface

The following will configure the switch to retrieve its IP address, subnet mask, router IP address, name server info, and hostname from DHCP server on the management interface's local network.

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # interface mgmt0
mellanox-sx6036-rack-1 [standalone: master] (config interface mgmt0) # dhcp
mellanox-sx6036-rack-1 [standalone: master] (config interface mgmt0) # dhcp hostname
mellanox-sx6036-rack-1 [standalone: master] (config interface mgmt0) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # 

NTP

Let's add a couple of NTP servers to the config and discipline our poorly behaved battery backed clock:

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # ntp server 172.16.10.2
mellanox-sx6036-rack-1 [standalone: master] (config) # ntp server 172.17.0.17
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show ntp

NTP is administratively            : enabled
NTP Authentication administratively: disabled 

Clock is synchronized:
  Reference: 172.16.10.2
  Offset   : -0.006 ms

Active servers and peers:
  172.16.10.2:
    Conf Type          : serv
    Status             : sys.peer(*)
    Stratum            : 3  
    Offset(msec)       : -0.006
    Ref clock          : 45.45.184.14   
    Poll Interval (sec): 64  
    Last Response (sec): 8   
    Auth state         : none    

  172.17.0.17:
    Conf Type          : serv
    Status             : pending    
    Stratum            : 2  
    Offset(msec)       : 0.429
    Ref clock          : 198.60.22.240  
    Poll Interval (sec): 64  
    Last Response (sec): 7   
    Auth state         : none    

mellanox-sx6036-rack-1 [standalone: master] #

Ethernet goodies

LLDP

Turn it on and test it out like so:

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # lldp
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show lldp local
LLDP: enabled

Local global configuration
    Chassis sub type: Mac Address
    Chassis id: 00:02:c9:6d:4d:b0
    System Name: mellanox-sx6036-rack-1
    System Description: SX6036,MLNX-OS,SWv3.6.8012
    Supported capabilities: B,R             
    Supported capabilities enabled: B
mellanox-sx6036-rack-1 [standalone: master] # show lldp interfaces ethernet 1/36
TLV flags
    PD: port-description
    SN: sys-name        
    SD: sys-description 
    SC: sys-capabilities
    MA: management-address
    ETS-C: ETS-Configuration
    ETS-R: ETS-Recommendation
    AP: Application Priority
    PFC: Priority Flow Control
    CEE: Converged Enhanced Ethernet DCBX version
    MED-CAP: Media Capabilities
    MED-NWP: MED-Network Policy

-----------------------------------------------------------------------------------
Interface    Receive   Transmit  TLVs                                              
-----------------------------------------------------------------------------------
Eth1/36      Enabled   Enabled   PD, SN, SD, SC, PFC, AP, ETS-C, ETS-R             
mellanox-sx6036-rack-1 [standalone: master] # show lldp interfaces ethernet 1/36 remote

Eth1/36

Remote Index: 1
Remote chassis id: cc:4e:24:83:4b:36 ; chassis id subtype: Mac Address (4)
Remote port-id: cc:4e:24:83:4b:6c ; port id subtype: Mac Address (3)
Remote port description: 40GigabitEthernet1/2/6
Remote system name: brocade-icx6610-48p-rack-1
Remote system description: Not Advertised
Remote system capabilities supported: B,R  ; enabled B,R
Remote Management Addresses:
SubType        Address
------------------------------------
IPv4        172.16.10.17

No Remote PFC entry

No Remote ETS entry

No Remote Application Priority entry

mellanox-sx6036-rack-1 [standalone: master] #

And enjoy getting to know all the things about your neighbors.

Multiple Spanning Tree Protocol

Wherein we avoid broadcast loops. Because they are bad, even if they do make for lots of blinken lights. Our Brocade switches are running MSTP and, fortunately, MLNX-OS supports that on our SX6036es as well. Setting the switch to MSTP mode is pretty simple:

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # spanning-tree mode mst
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show spanning-tree

Switch                     : ethernet-default
Spanning tree protocol mst : enabled
Spanning tree force version: 3

Root ID:
  Priority           : 32768
  Address            : 00:02:c9:6d:4c:b0   
  Cost               : 0
  Port               : Eth1/36
  Hello Time (sec)   : 2
  Max Age (sec)      : 20
  Forward Delay (sec): 15

  MST00:
    Bridge is executing the mstp compatible Spanning Tree Protocol

Bridge ID:
  Priority           : 32768
  Address            : 00:02:c9:6d:4d:b0   
  Hello Time (sec)   : 2
  Max Age (sec)      : 20
  Forward Delay (sec): 15

L: Loop Inconsistent
R: Root Inconsistent
G: BPDU Guard Inconsistent

--------------------------------------------------------------------------
Interface         Role         Sts              Cost      Prio   Type     
--------------------------------------------------------------------------
Eth1/36           Root         Forwarding       500       128    normal   

mellanox-sx6036-rack-1 [standalone: master] #

VLANs

Create our standard assortment of VLANs like so:

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 10 name Management
mellanox-sx6036-rack-1 [standalone: master] (config vlan 10) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 100 name "WAN 0 (Comcast 44th Avenue)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 100) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 101 name "WAN 1 (Allo)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 101) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 102 name "WAN 2 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 102) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 103 name "WAN 3 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 103) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 104 name "WAN 4 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 104) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 105 name "WAN 5 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 105) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 106 name "WAN 6 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 106) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 107 name "WAN 7 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 107) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 108 name "WAN 8 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 108) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 109 name "WAN 9 (unused)"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 109) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 1000 name "Internal Services"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 1000) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 1001 name "Internal clients"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 1001) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 1002 name "Ceph front-end"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 1002) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 1003 name "Ceph back-end"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 1003) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # vlan 3900 name "Guest network"
mellanox-sx6036-rack-1 [standalone: master] (config vlan 3900) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show vlan
----------------------------------------------------------------------
VLAN    Name                    Ports
----------------------------------------------------------------------
1       default                 Eth1/19, Eth1/20, Eth1/21, Eth1/22, Eth1/23,
                                Eth1/24, Eth1/25, Eth1/26, Eth1/27, Eth1/28,
                                Eth1/29, Eth1/30, Eth1/31, Eth1/32, Eth1/33,
                                Eth1/34, Eth1/35, Eth1/36
10      Management              
100     WAN 0 (Comcast 44th Avenue)
101     WAN 1 (Allo)            
102     WAN 2 (unused)          
103     WAN 3 (unused)          
104     WAN 4 (unused)          
105     WAN 5 (unused)          
106     WAN 6 (unused)          
107     WAN 7 (unused)          
108     WAN 8 (unused)          
109     WAN 9 (unused)          
1000    Internal Services       
1001    Internal clients        
1002    Ceph front-end          
1003    Ceph back-end           
3900    Guest network           
mellanox-sx6036-rack-1 [standalone: master] #

And now, to assign all those VLANs as tagged on port 1/36:

mellanox-sx6036-rack-1 [standalone: master] > enable
mellanox-sx6036-rack-1 [standalone: master] # configure terminal
mellanox-sx6036-rack-1 [standalone: master] (config) # interface ethernet 1/36
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # switchport mode trunk
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # switchport trunk allowed-vlan 10
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # switchport trunk allowed-vlan add 100-109
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # switchport trunk allowed-vlan add 1000-1003
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # switchport trunk allowed-vlan add 3900
mellanox-sx6036-rack-1 [standalone: master] (config interface ethernet 1/36) # exit
mellanox-sx6036-rack-1 [standalone: master] (config) # exit
mellanox-sx6036-rack-1 [standalone: master] # write memory
mellanox-sx6036-rack-1 [standalone: master] # show vlan
----------------------------------------------------------------------
VLAN    Name                    Ports
----------------------------------------------------------------------
1       default                 Eth1/19, Eth1/20, Eth1/21, Eth1/22, Eth1/23,
                                Eth1/24, Eth1/25, Eth1/26, Eth1/27, Eth1/28,
                                Eth1/29, Eth1/30, Eth1/31, Eth1/32, Eth1/33,
                                Eth1/34, Eth1/35
10      Management              Eth1/36
100     WAN 0 (Comcast 44th Avenue)Eth1/36
101     WAN 1 (Allo)            Eth1/36
102     WAN 2 (unused)          Eth1/36
103     WAN 3 (unused)          Eth1/36
104     WAN 4 (unused)          Eth1/36
105     WAN 5 (unused)          Eth1/36
106     WAN 6 (unused)          Eth1/36
107     WAN 7 (unused)          Eth1/36
108     WAN 8 (unused)          Eth1/36
109     WAN 9 (unused)          Eth1/36
1000    Internal Services       Eth1/36
1001    Internal clients        Eth1/36
1002    Ceph front-end          Eth1/36
1003    Ceph back-end           Eth1/36
3900    Guest network           Eth1/36
mellanox-sx6036-rack-1 [standalone: master] #

Unlocking the bootloader

When reset or just powered on, the SX6036's bootloader does preset the human on the console a 5 second countdown timer and an opportunity to choose some other options than loading the currently active software image:

Mellanox MLNX-OS

Default image: 'PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc'
Press Enter to boot this image, or 'Ctrl B' for boot menu

Booting default image in:  0 


Mellanox MLNX-OS Boot Menu:

*  1: PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc
   2: PPC_M460EX 3.6.8010 2018-08-20 18:04:16 ppc
   u: USB menu (if USB device connected) (password required)
   c: Command prompt (password required)

   Choice: 

Irritatingly, I have been unable to locate the password required for command prompt access. Fortunately, some kind denizens of the internet have shared a way to change (even remove!) this password from a running MLNX-OS. I found this information at https://forums.servethehome.com/index.php?threads/solved-mellanox-sx6012-u-boot-password-removal-without-bash-access.33484/

Let's record this for posterity:

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-2 login: admin
Password: 
Last login: Sun Apr 12 16:57:02 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-2 [standalone: master] > enable
mellanox-sx6036-rack-2 [standalone: master] # configure terminal
mellanox-sx6036-rack-2 [standalone: master] (config) # boot bootmgr password 7 ""
mellanox-sx6036-rack-2 [standalone: master] (config) # exit
mellanox-sx6036-rack-2 [standalone: master] # write memory
mellanox-sx6036-rack-2 [standalone: master] # show bootvar

Installed images:
  Partition 1:
    version: PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc

  Partition 2:
    version: PPC_M460EX 3.6.8010 2018-08-20 18:04:16 ppc

Last boot partition             : 1
Next boot partition             : 1
Serve image files via HTTP/HTTPS: no

No boot manager password is set.

Image signing              : trusted signature always required
Admin require signed images: yes

Settings for next boot only:
  Fallback reboot on configuration failure: yes (default)

mellanox-sx6036-rack-2 [standalone: master] # 

Note the line saying "No boot manager password is set." A switch that has not had this treatment will report "Boot manager password is set." instead.

First try was not successful, though. I may continue poking at this again in the future.

Second attempt

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-2 login: admin
Password: 
Last login: Sun Apr 12 17:37:09 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-2 [standalone: master] > _shell
% Unrecognized command "_shell".
Type "?" for help.
mellanox-sx6036-rack-2 [standalone: master] > enable
mellanox-sx6036-rack-2 [standalone: master] # _shell
[admin@mellanox-sx6036-rack-2 ~]# stty rows 25 columns 160
[admin@mellanox-sx6036-rack-2 ~]# /opt/tms/bin/mddbreq /config/db/initial set modify - /system/bootmgr/password string ""
[admin@mellanox-sx6036-rack-2 ~]# eetool -a bf -s UBPASSWD=""
[admin@mellanox-sx6036-rack-2 ~]# exit
logo

Mellanox MLNX-OS Switch Management

mellanox-sx6036-rack-2 login: admin
Password: 
Last login: Sun Apr 12 17:56:18 on ttyS0

Mellanox Switch

mellanox-sx6036-rack-2 [standalone: master] > enable
mellanox-sx6036-rack-2 [standalone: master] # show bootvar

Installed images:
  Partition 1:
    version: PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc

  Partition 2:
    version: PPC_M460EX 3.6.8010 2018-08-20 18:04:16 ppc

Last boot partition             : 1
Next boot partition             : 1
Serve image files via HTTP/HTTPS: no

No boot manager password is set.

Image signing              : trusted signature always required
Admin require signed images: yes

Settings for next boot only:
  Fallback reboot on configuration failure: yes (default)

mellanox-sx6036-rack-2 [standalone: master] #

And this time, success!

Mellanox MLNX-OS

Default image: 'PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc'
Press Enter to boot this image, or 'Ctrl B' for boot menu

Booting default image in:  0 


Mellanox MLNX-OS Boot Menu:

*  1: PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc
   2: PPC_M460EX 3.6.8010 2018-08-20 18:04:16 ppc
   u: USB menu (if USB device connected)
   c: Command prompt

   Choice: c
Entering command prompt
=> ?
?       - alias for 'help'
askenv  - get environment variables from stdin
autoscr - run script from memory
base	- print or set address offset
bdinfo  - print Board Info structure
boot    - boot default, i.e., run 'bootcmd'
bootd   - boot default, i.e., run 'bootcmd'
bootelf - Boot from an ELF image in memory
bootm   - boot application image from memory
bootmenu- Run boot menu
bootp	- boot image via network using BOOTP/TFTP protocol
bootstrap - program the I2C bootstrap EEPROM
bootvx  - Boot vxWorks from an ELF image
cmp	- memory compare
coninfo - print console devices and information
cp	- memory copy
crc32	- checksum calculation
dcache  - enable or disable data cache
dhcp	- boot image via network using DHCP/TFTP protocol
echo    - echo args to console
eeprom  - EEPROM sub-system
envreset- Reset the environment to the defaults
erase   - erase FLASH memory
exit    - exit script
ext2load- load binary file from a Ext2 filesystem
ext2ls  - list files in a directory (default /)
fatinfo - print information about filesystem
fatload - load binary file from a dos filesystem
fatls   - list files in a directory (default /)
fdt     - flattened device tree utility commands
flinfo  - print FLASH memory information
getdcr  - Get an AMCC PPC 4xx DCR's value
getidcr - Get a register value via indirect DCR addressing
go      - start application at address 'addr'
help    - print online help
icache  - enable or disable instruction cache
icrc32  - checksum calculation
iloop   - infinite loop on address range
imd     - i2c memory display
iminfo  - print header information for application image
imls    - list all images found in flash
imm     - i2c memory modify (auto-incrementing)
imw     - memory write (fill)
imxtract- extract a part of a multi-image
inm     - memory modify (constant address)
interrupts - enable or disable interrupts
iprobe  - probe to discover valid I2C chip addresses
irqinfo - print information about IRQs
itest	- return true/false on integer compare
loadb   - load binary file over serial line (kermit mode)
loads   - load S-Record file over serial line
loady   - load binary file over serial line (ymodem mode)
loop	- infinite loop on address range
loopw	- infinite write loop on address range
md	- memory display
mdc	- memory display cyclic
mii     - MII utility commands
mm	- memory modify (auto-incrementing)
mtest	- simple RAM test
mw	- memory write (fill)
mwc	- memory write cyclic
nand    - NAND sub-system
nboot   - boot from NAND device
nfs	- boot image via network using NFS protocol
nm	- memory modify (constant address)
pci     - list and access PCI Configuration Space
ping	- send ICMP ECHO_REQUEST to network host
printenv- print environment variables
protect - enable or disable FLASH write protection
rarpboot- boot image via network using RARP/TFTP protocol
reginfo - print register information
reset   - Perform RESET of the CPU
run     - run commands in an environment variable
saveenv - save environment variables to persistent storage
setdcr  - Set an AMCC PPC 4xx DCR's value
setenv  - set environment variables
setexpr - set environment variable as the result of eval expression
setidcr - Set a register value via indirect DCR addressing
showvar- print local hushshell variables
sleep   - delay execution for some time
test    - minimal test like /bin/sh
tftpboot- boot image via network using TFTP protocol
usb     - USB sub-system
usbboot - boot from USB device
version - print monitor version
=> version

U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82 ppc (Dec 20 2012 - 17:53:54)
=> reset

Look! It runs U-Boot. This should not be a huge surprise. Enjoy poking around inside the pre-Linux environment if you like.