EMC branded Mellanox SX60xx switch

Used enterprise IT stuff is stupid cheap in the US. Stupid cheap like 18 port FDR (56Gbits/sec) Infiniband switches shipped to your door for 6.67USD per port. Only a couple of issues with these:

• They run a very limited EMC OS instead of the somewhat more open Linux-based Mellanox MLNX-OS
• No VPI (Ethernet over Infiniband, more or less)
• No subnet manager (facilitates IP over Infiniband)
• Management by telnet ("SSH? That's so not going to happen!")
• It's not Linux

The devices at FnordNet are Mellanox SX6018s wearing EMC colors. (Mellanox ships switches with blue fronts and EMC switches are black.) On the connector side are:

• 18 QSFP 56Gbits/sec Infiniband connectors
• 2 8P8C modular 1000baseT Ethernet connectors (Labelled MGT)
• 1 USB connector
• 1 8P8C modular connector for the serial console (labelled CONSOLE)
• a small recessed reset button switch (small hole on the left side of the connector panel)

Mellanox makes a number of similar devices with the same generation of silicon:

• SX6005 and SX6012 -- 1U, half width, 12 QSFP connectors arranged in two rows of six. SX6005 is the unmanaged version. SX6012 is the managed switch with the subnet manager and single port Ethernet connectivity
• SX6015 and SX6018 -- 1U, full width, 18 QSFP connectors in one row of 18. SX6018 is the managed version with a subnet manager and dual Ethernet management ports
• SX6025 and SX6036 -- 1U, full width, 36 QSFP connectors in two rows of 18. SX6025 is unmanaged. SX6036 is the managed version.

All of these are built on the same Infiniband silicon -- Mellanox SwitchX-2. The management functions on the SX6012, SX6018, and SX6036 are done by an embedded Linux system running on a PowerPC M460EX series CPU attached to the SwitchX-2 silicon over a PCI interface of some sort.

Putting MLNX-OS on it

As mentioned above, I desire to have a real OS on these switches. Found some leads over on https://forums.servethehome.com/ which said conversion is possible, but "PM me for more details." The following is a translation of the nice gentleman's "Mellanox Switch Conversion Guide" document into the procedure I have used.

Pre-requisiteses

(Yes. I know I misspelled that.)

• console cable for switch (Cisco cables work fine)
• serial comms program (C-Kermit and I get along just fine. Others may use screen. Or minicom. Or Teraterm. Or HyperTerminal. Or for people with long memories, even {COMMO} or Telix or Qmodem.)
• Willingness to root around typing stuff into a text console fiddle with unsupported softwares
• A MLNX-OS binary distribution for the switch in question. Try:
  • http://www.mellanox.com/downloads/Software/image-PPC_M460EX-3.6.1002.img or
  • http://www.mellanox.com/downloads/Software/image-PPC_M460EX-3.6.2102.img or
  • http://www.mellanox.com/downloads/Software/image-PPC_M460EX-3.6.3004.img
• a hex editor (we'll be picking apart some binary archives at some point)
• some time
• some willingness to violate software licenses
• attention to detail: breaking the U-Boot image inside the switch is likely to result in a need for I2C tools to make things work again. This is not something I want to deal with.