Cisco UCS Rack servers: Difference between revisions

From FnordWiki
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 94: Line 94:
* Select <code>Yes</code> from the menu.
* Select <code>Yes</code> from the menu.
* Return to the top level of the <code>AVAGO &lt;Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)&gt; Configuration</code> menu.
* Return to the top level of the <code>AVAGO &lt;Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)&gt; Configuration</code> menu.
* Save configuration and reboot server.
* Save configuration, reboot server, and enter the BIOS setup screen one more time (press <code>F2</code>).
* From main BIOS config menu screen, arrow right to Advanced
* Arrow down to <code>AVAGO &lt;Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)&gt; Configuration</code> and press Enter
* Arrow down to <code>View Server Profile</code> and press Enter
* Arrow down to <code>Controller Management</code> and press Enter
* Arrow down to <code>Advanced Controller Properties</code> and press Enter
* Arrow down to <code>JBOD Mode</code> and set to <code>Enabled</code> if it is not already so.
* Save changes and reset the system one more time
From here, operating system installation can commence. Or storage device firmware updates. Or whatever...

Latest revision as of 00:52, 18 March 2026

Acquired a Cisco UCS C220 M5 on 2026-03-10. Here are some notes.

Network based management

CIMC, the Cisco Integrated Management Controller. On the machine I have obtained, this was not configured to use the dedicated management Ethernet interface on the back of the server. This is easily corrected while the machine starts up. Pressing F8 from a USB keyboard during the server's power on self test sequence will bring up the CIMC configuration screen. From there, the dedicated NIC option, IPv4 option, and DHCP enabled can be selected to do the expected things. After those are selected, press the <F1> key and fix the dedicated NIC port properties if the need any adjustment.

The CIMC is accessible over https and SSH. At first login (at least by HTTPS), a new password must be set. None of Dell's just warning "Heh, you have the default credentials. Not recommended!" warning stuff.

Firmware updates

As acquired, server s/n WZP22040X50 is running BIOS version C220M5.3.1.2b.0.1025170315 and CIMC version 3.1(2c). Cisco's firmware update process is heavier weight than some other vendors'. More like HP(E)'s Service Pack for Proliant than Dells "get iDRAC to download update images from an FTP or web server." An ISO image must be downloaded and booted from. (This is maybe avoidable. And if is, updates will be made to document the process here.) As we are running 3.1 release code now, download the final update ISO image, 3.1(3k). This is free-as-in-beer with an account on Cisco's support site. No $$$ required. I have not read every line of the release notes to understand whether entire release trains can be skipped in the upgrade process.

One thing that is nice about getting the account created: they have a notification system a customer can sign up for when sends an email when new firmware is released. With options for weekly, daily, or monthly notications.

After the ISO is downloaded, open the KVM on the server's CIMC and attach it to the server using the appropriate item in the "Virtual Media" menu. Reboot server and press the <F6> key at the appropriate time to invoke the boot device selection menu. From there, boot from the Cisco vKVM attached ISO image. Wait patiently. After several minutes, you might catch Linux kernel boot messages. And shortly after that, something about the Cisco UCS Host Upgrade Utility Version 3.1.3k doing its work. Then wait some more. The Host Update Utility is not very good about giving progress updates. At least not this version. After 5-10 minutes an "accept the license" screen will appear. Do the obvious. And then, the easy thing is to use the "Update All" button and get all updates applied. Wait patiently some more. But at least we get some progress reports as the actual updates are applied. After all desired updates are complete, select "Exit" The CIMC will reboot if its code was updated. And the server will as well.

If the paranoia is very strong with you, Host Upgrade Utility ISO can be booted from a second time, and all updates verified. The boot and utility loading are just as excruciatingly slow as the actual update, but the verification itself goes swiftly.

First thing of note with going from CIMC 3.1(2c) to 3.1(3k) is that the CIMC's SSH server now supports more modern SSH host keys. No more need for -o HostKeyAlgorithms=+ssh-rsa when pointing an SSH client at the CIMC.

Also, following the release notes, jumping straight from 3.1(2c) to the most recent 4.3(2.260007) is almost certainly not a thing. I have been doing it in increments so far: 3.1(2c) -> 3.1(3k) -> 4.0(4n) -> 4.1(3n) -> 4.2(3l)

4.1(3n) includes some sort of cloud management functionality which I probably don't want. It is called "Cisco Intersight." It will get its own section.

Cisco Intersight Cloud Management

Yeah, what I said just above. I am pretty sure I don't want this. Do not need this server's BMC phoning home to the Cisco mothership. Need to do some reading about turning it off.

It turns out to not to too difficult. Kinda hidden, but here's how I did it:

  1. Log into CIMC web UI. You will be looking at the Chassis / Summary page.
  2. Click on the icon in the top left corner. It is pretty much a three bar "hamburger" menu with a right-pointing arrow over its top.
  3. At the bottom of the just-exposed navigation pane, select the Admin / Device Connector link.
  4. On the right side of the Device Connector page, find the "Settings" link and follow it.
  5. Toggle Device Connector off, save settings, enjoy some peace about not having your server watching you quite as much.

The above is what I did on CIMC 4.1(3n). It may be different in newer releases. It may also be re-activated after further firmware updates. So make a note to check this one after more of those.

Doing things with the CIMC

Restarting the CIMC

The web UI has a "CIMC Reboot" link toward the top right of the main page. At least while running CIMC 4.3 code. From an SSH session, do the following after logging in:

C220-WZP22040X50# scope cimc
C220-WZP22040X50 /cimc # reboot
This operation will reboot the Cisco IMC.
Continue?[y|N]y

and wait for it to restart.

Serial over LAN

This is a bit different than Dell's iDRAC or HP(E)'s iLO. It gets its own dedicated SSH port. From an SSH connection to the CIMC, the configuration can be listed out like so:

C220-WZP22040X50# show sol detail
Serial Over LAN:
    Enabled: no
    Baud Rate(bps): 115200
    Com Port: com0
    SOL SSH Port: 2400
C220-WZP22040X50#

It does need to be explicitly enabled. Again, to do that from the CIMC SSH interface (there are maybe BIOS config and CMIC web interface toggles for this, too) do this:

C220-WZP22040X50# scope sol
C220-WZP22040X50 /sol # set enabled yes
C220-WZP22040X50 /sol *# commit
C220-WZP22040X50 /sol # exit
C220-WZP22040X50#

And that is done, so connect to the SSH server on port 2400. With OpenSSH, that is done by specifying -p 2400 on the ssh command line. Something like:

$ ssh -p 2400 admin@172.16.10.160
admin@172.16.10.160's password: 
CISCO Serial Over LAN:
Press Ctrl+x to Exit the session

CISCO Serial Over LAN exited
Connection to 172.16.10.160 closed.
$

Mass storage devices, RAID, JBOD, and such

The ECS C220 M5 has a Broadcom/LSI MegaRAID Tri-Mode SAS3516 attached to the disk backplane in the front of the chassis.

I prefer running my disk interfaces in JBOD (just a bunch of disks) or pass-through mode, exposing the individual physical storage devices to the operating system, and using OS tooling to manage redundancy. As this server arrived running VMware ESXi, an OS without those data redundancy management features, the server came using the SAS3516 RAID controller set up to present 2 virtual disks to the OS. Setting the controller to JBOD mode is possible, though. To get to the MegaRAID configuration without a running OS equipped with storcli and related tools, restart the server, and enter the BIOS setup interface by hitting F2 before the boot loader is started. This can be done on the VGA console, CIMC virtual VGA console, or even the serial console. From the main screen, right arrow over to the Advanced screen and then down to the AVAGO <Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)> Configuration menu option. From there, navigate down to Set Factory Defaults, select it, and reboot server. Then enter the BIOS setup screens before control is passed to the boot loader again. This time through, the existing virtual drives will be deleted:

  • From main BIOS config menu screen, arrow right to Advanced
  • Arrow down to AVAGO <Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)> Configuration and press Enter
  • Arrow down to View Server Profile and press Enter
  • Arrow down to Virtual Drive Management and press Enter
  • A list of virtual drives is now presented. For each virtual drive in the list:
    • Press Enter to see the Virtual Drive Details
    • Press Enter again, to open the Operation menu
    • Arrow down to Delete Virtual Drive and press Enter
    • Select the Go option presented beneath the now selected Delete Virtual Drive operation.
    • Change the Confirm setting in the next menu to Enabled.
    • Select Yes in the menu.
    • Select OK to return to the list of existing virtual drives.
  • When the list of virtual drives is empty, return to the top level of the AVAGO <Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)> Configuration menu.
  • Select Configure from the menu.
  • Select Make JBOD from the menu.
  • Select Check All from the menu. This option is displayed after the list of physical drives the controller has found.
  • Select OK from the menu.
  • Move up and change the Confirm setting to Enabled
  • Select Yes from the menu.
  • Return to the top level of the AVAGO <Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)> Configuration menu.
  • Save configuration, reboot server, and enter the BIOS setup screen one more time (press F2).
  • From main BIOS config menu screen, arrow right to Advanced
  • Arrow down to AVAGO <Cisco 12G Modular Raid Controller with 2GB cache (max 16 drives)> Configuration and press Enter
  • Arrow down to View Server Profile and press Enter
  • Arrow down to Controller Management and press Enter
  • Arrow down to Advanced Controller Properties and press Enter
  • Arrow down to JBOD Mode and set to Enabled if it is not already so.
  • Save changes and reset the system one more time

From here, operating system installation can commence. Or storage device firmware updates. Or whatever...