UCS Manager: Difference between revisions
| Line 23: | Line 23: | ||
=== Basic Fabric Interconnect configuration === |
=== Basic Fabric Interconnect configuration === |
||
I might have lucked out on the first one I received, but it booted normally and started the interactive system setup dialog. It is all pretty basic. Set a password for the <code>admin</code> user and an IP address, netmask, IPv4 gateway address, and DNS server address for the management Ethernet interface (again, on the front). That's pretty much it. |
I might have lucked out on the first one I received, but it booted normally and started the interactive system setup dialog. It is all pretty basic. Set a password for the <code>admin</code> user and an IP address, netmask, IPv4 gateway address, and DNS server address for the management Ethernet interface (again, on the front). That's pretty much it. |
||
=== Clock discipline is good === |
|||
To set the time in the FI. Or just get it close. NTP will come. Eventually. |
|||
ucs-fi-6248up-0-A# '''scope system''' |
|||
ucs-fi-6248up-0-A /system # '''scope services''' |
|||
ucs-fi-6248up-0-A /system/services # '''set clock Mar 22 2026 02 53 35''' |
|||
Sun Mar 22 02:53:35 UTC 2026 |
|||
ucs-fi-6248up-0-A /system/services # |
|||
And now for the NTP: |
|||
ucs-fi-6248up-0-A# '''scope system''' |
|||
ucs-fi-6248up-0-A /system # '''scope services''' |
|||
ucs-fi-6248up-0-A /system/services # '''create ntp-server 172.16.10.2''' |
|||
ucs-fi-6248up-0-A /system/services* # '''commit-buffer''' |
|||
ucs-fi-6248up-0-A /system/services # '''create ntp-server 172.16.10.3''' |
|||
ucs-fi-6248up-0-A /system/services* # '''commit-buffer''' |
|||
ucs-fi-6248up-0-A /system/services # '''show ntp''' |
|||
NTP Servers: |
|||
Name: 172.16.10.2 |
|||
Name: 172.16.10.3 |
|||
ucs-fi-6248up-0-A /system/services # |
|||
=== Software upgrade time! === |
=== Software upgrade time! === |
||
Revision as of 02:59, 22 March 2026
There are a few pieces of Cisco UCS equipment in the data center:
- One UCS 5108 blade enclosure, accommodating up to 8 B200 server blades of various generations
- 2 UCS-FI-6248UP fabric interconnects, which sit between UCS servers or blade chassis and the rest of the network
- 1 UCS C220 M5 server
Cisco UCS Manager gets installed on the Fabric Interconnects and does Cisco proprietary magic things. This document is being written to capture knowledge of installation, upgrades, and server management tasks.
How to get
Much (all?) of Cisco's UCS software is available directly without a support contract. But a customer account is required to download. Registration is free as in beer. I have had one since July 2021 and have not been troubled with Cisco spam or any sort of "are you a real person at a real company?" validation.
As this is written, UCS Infrastructure and UCS Manager Software is the place to get it. As I have some older, end-of-life, end-of-support fabric interconnects (62xx series) which are not supported on the current 6.0 release train, I have instead selected the 4.2(3p) version for download. This comes as a 1Gbyte-ish ucs-k9-bundle-infra.4.2.3p.A.bin file.
Untested Fabric Interconnect, what do we do?
5ish years ago (summer 2021), I acquired three of these 6248UP FIs. And I think I got one of them configured for use on the management network and updated to then-current software. I'm not sure which of the three that was. And I am sure I did no documentation at the time. So here we are with new notes.
Console and power are at the front?!?!
So, this is weird. Console and power connections are at the front side of the chassis. All of the connectivity is at the rear as it should be. There is sufficient room on the back for a couple of C14 power connectors and 2 more 8P8C modular connectors for management LAN and serial console. But they are at the front. So cabling is kinda sorta extra fun. sigh
Console serial cable is the usual Cisco modular pinout. Signal on pins 3 and 6. 9600 bps, 8-n-1 are the serial port settings. Nothing unusual there.
Mystery solved regarding the connections at front of the chassis. My FIs have 32 ports for traffic, but there is (well, was, back when this was a current product) a 16 port expansion module available. See https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-6200-series-fabric-interconnects/data_sheet_c78-675245.html for some diagrams.
Basic Fabric Interconnect configuration
I might have lucked out on the first one I received, but it booted normally and started the interactive system setup dialog. It is all pretty basic. Set a password for the admin user and an IP address, netmask, IPv4 gateway address, and DNS server address for the management Ethernet interface (again, on the front). That's pretty much it.
Clock discipline is good
To set the time in the FI. Or just get it close. NTP will come. Eventually.
ucs-fi-6248up-0-A# scope system ucs-fi-6248up-0-A /system # scope services ucs-fi-6248up-0-A /system/services # set clock Mar 22 2026 02 53 35 Sun Mar 22 02:53:35 UTC 2026 ucs-fi-6248up-0-A /system/services #
And now for the NTP:
ucs-fi-6248up-0-A# scope system
ucs-fi-6248up-0-A /system # scope services
ucs-fi-6248up-0-A /system/services # create ntp-server 172.16.10.2
ucs-fi-6248up-0-A /system/services* # commit-buffer
ucs-fi-6248up-0-A /system/services # create ntp-server 172.16.10.3
ucs-fi-6248up-0-A /system/services* # commit-buffer
ucs-fi-6248up-0-A /system/services # show ntp
NTP Servers:
Name: 172.16.10.2
Name: 172.16.10.3
ucs-fi-6248up-0-A /system/services #
Software upgrade time!
Again, on the first one I have tried, let's log in as admin and see what's running:
Cisco UCS 6200 Series Fabric Interconnect
ucs-fi-6248up-0-A login: admin Password: Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (c) 2009, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://www.opensource.org/licenses/lgpl-2.1.php ucs-fi-6248up-0-A# show version System version: 2.2(1c) ucs-fi-6248up-0-A#
And, more verbosely (for reasons that are beyond me, "brief" mode shows more output?):
ucs-fi-6248up-0-A# show version brief System version: 2.2(1c) Package-Vers: 2.2(1c)A ucs-fi-6248up-0-A#
So that's really very super dooper old. I downloaded a copy of the 4.1(3i) code in August 2022. Off do so some reading to see if I can skip all the in-between versions.
Did not find anything authoritative in the release notes (but I only looked very briefly.) So, we can do it stepwise. Hopefully. Process looks something like this:
ucs-fi-6248up-0-A# scope firmware
ucs-fi-6248up-0-A /firmware # download image tftp://172.17.0.18/ucs-k9-bundle-infra.2.2.8m.A.bin
ucs-fi-6248up-0-A /firmware # show download-task
Download task:
File Name Protocol Server Userid State
--------- -------- --------------- --------------- -----
ucs-k9-bundle-infra.2.2.8m.A.bin
Tftp 172.17.0.18 Downloaded
ucs-fi-6248up-0-A /firmware #
Yes, you read that right. I delivered the software image to the FI over TFTP. SSH (scp and sftp) are supported protocols, but could not connect to an OpenSSH server on Debian 13. This might be better with newer FI firmware, but for now, I already have a TFTP server handy for netbooting other machines in the environment.
And from here, the firmware bundle can actually be installed on the Fabric Interconnect like so:
ucs-fi-6248up-0-A# scope firmware
ucs-fi-6248up-0-A /firmware # scope auto-install
ucs-fi-6248up-0-A /firmware/auto-install # install infra infra-vers 2.2(8m)A
This operation upgrades firmware on UCS Infrastructure Components
(UCS manager, Fabric Interconnects and IOMs). Do you want to proceed?
(yes/no):yes
Triggering Install-Infra with:
Infrastructure Pack Version: 2.2(8m)A
ucs-fi-6248up-0-A /firmware/auto-install # acknowledge primary fabric-interconnect reboot
ucs-fi-6248up-0-A /firmware/auto-install* # commit-buffer
ucs-fi-6248up-0-A /firmware/auto-install # show fsm status expand
FSM Status:
Affected Object: sys/fw-system/fsm
Current FSM: Deploy
Status: In Progress
Completion Time:
Progress (%): 87
FSM Stage:
Order Stage Name Status Try
------ ---------------------------------------- ------------ ---
1 DeployWaitForDeploy Success 0
2 DeployResolveDistributableNames Skip 0
3 DeployResolveDistributable Skip 0
4 DeployResolveImages Skip 0
5 DeployActivateUCSM Success 0
6 DeployPollActivateOfUCSM In Progress 1
7 DeployUpdateIOM Pending 0
8 DeployPollUpdateOfIOM Pending 0
9 DeployActivateIOM Pending 0
10 DeployPollActivateOfIOM Pending 0
11 DeployActivateRemoteFI Pending 0
12 DeployPollActivateOfRemoteFI Pending 0
13 DeployWaitForUserAck Pending 0
14 DeployActivateLocalFI Pending 0
15 DeployPollActivateOfLocalFI Pending 0
ucs-fi-6248up-0-A /firmware/auto-install # Connection to 172.16.10.176 closed.
Somewhere along the way, the SSH server hung up on me. But that seems like the sort of thing that should happen as the FI's OS is being upgraded. Progress can also be tracked on the serial console port like so:
ucs-fi-6248up-0-A# scope firmware
ucs-fi-6248up-0-A /firmware # scope auto-install
ucs-fi-6248up-0-A /firmware/auto-install # show fsm status expand | no-more
FSM Status:
Affected Object: sys/fw-system/fsm
Current FSM: Deploy
Status: In Progress
Completion Time:
Progress (%): 98
FSM Stage:
Order Stage Name Status Try
------ ---------------------------------------- ------------ ---
1 DeployWaitForDeploy Success 0
2 DeployResolveDistributableNames Skip 0
3 DeployResolveDistributable Skip 0
4 DeployResolveImages Skip 0
5 DeployDownloadImages Skip 0
6 DeployCopyAllImagesToPeer Skip 0
7 DeployInternalBackup Success 0
8 DeployPollInternalBackup Success 2
9 DeployActivateUCSM Skip 0
10 DeployPollActivateOfUCSM Success 0
11 DeployUpdateIOM Success 0
12 DeployPollUpdateOfIOM Success 0
13 DeployActivateIOM Success 0
14 DeployPollActivateOfIOM Success 0
15 DeployActivateRemoteFI Skip 0
16 DeployPollActivateOfRemoteFI Skip 0
17 DeployWaitForUserAck Skip 0
18 DeployPollWaitForUserAck Success 0
19 DeployActivateLocalFI Success 0
20 DeployPollActivateOfLocalFI In Progress 2
ucs-fi-6248up-0-A /firmware/auto-install #
At some point, the FI will reboot itself.
Confirm new code is running like so:
ucs-fi-6248up-0-A# show version System version: 2.2(8m) ucs-fi-6248up-0-A# show version brief System version: 2.2(8m) Package-Vers: 2.2(8m)A ucs-fi-6248up-0-A#