Complete Guide to Nexus Checkpoint & Rollback Feature. Fast Recovery from Nexus Misconfiguration. Nexus 9000, 7000, 5000, 3000

The Cisco NX-OS checkpoint feature provides the capability to capture at any time a snapshot (backup) of the Cisco Nexus configuration before making any changes. The captured configuration (checkpoint) can then be used to roll back and restore the original configuration.

The NX-OS checkpoint and rollback feature is extremely useful, and a life saver in some cases, when a new configuration change to a production system has caused unwanted effects or was incorrectly made/planned and we need to immediately return to an original/stable configuration.

With Catalyst IOS switches we would manually remove or restore IOS commands but with the Nexus NS-OS checkpoint-rollback feature this is a much faster and safer process that can be executed even by an authorized user with basic experience/knowledge on Nexus switches.

Key Topics:

• NX-OS Checkpoint & Rollback Limitations - Guidelines
• Understanding and Configuring the NX-OS Checkpoint Feature
• How to Backup Nexus Checkpoints – Exceeding the 10 Checkpoint Limit
• Rollback Configuration – Checking Differences Between ‘Running Config’ & ‘CheckPoint’
• Summary

Finally, we must point out that basic knowledge of the Cisco NX-OS is required for this article.

Additional Reading:

• Introduction to Nexus Family – Nexus OS vs Catalyst IOS
• NEXUS NX-OS: Useful Commands, CLI Scripting, Hints & Tips, Python Scripting and more
• Cisco Nexus 7000 Module Shutdown and Removal Procedure
• The Complete Cisco Nexus vPC Guide. Features & Advantages, Design Guidelines, Configuration, Failure Scenarios, Troubleshooting, VSS vs vPC

NX-OS Checkpoint & Rollback Limitations - Guidelines

The Checkpoint and Rollback feature has the following main configuration guidelines and limitations:

• The maximum number of checkpoints supported is equal to ten (10).
• Checkpoints are stored in an internal repository not accessible by the common user.
• Checkpoints are persistent and synced between redundant supervisors.
• It is not possible to apply or import the checkpoint file from another Nexus switch. Checkpoints can only be used on the device they were created on.
• Only one user at a time can perform a checkpoint, rollback, or copy of the running configuration to the startup configuration.
• Checkpoints are cleared from the system’s database after executing the write erase or reload command (switch reload).
• Checkpoints can be manually cleared by running the clear checkpoint database command. The checkpoints saved to the bootflash are not affected by the aforementioned commands.
• Checkpoints are only local to the NX-OS switch.
• Rollback using files stored in bootflash is supported only if it has been created using the checkpoint command.
• Checkpoint names must be unique. You cannot overwrite previously saved checkpoints. If attempting to overwrite existing checkpoints the user will receive the following error: ERROR: ascii-cfg: Checkpoint Name already exists (err id 0x405F002B)
• Checkpoints are local to a virtual device context (VDC) for the Nexus 7000.
• Rollback is not supported in the storage VDC for the Nexus 7000
• Rollback is not supported on the Nexus 5000 after enabling the FCoE feature. System will generate the following error after enabling the FCoE feature: ERROR: FCOE is enabled. Disabling rollback module.

Understanding and Configuring the NX-OS Checkpoint Feature

Working with the Nexus Checkpoint feature is a very easy process and the commands used follow a logical order allowing the easy usage of this fantastic feature.

It’s important to always create a checkpoint before you begin making changes to the existing configuration of your Nexus switch. The command below shows the creation of our first checkpoint named Checkpoint-1 along with the optional description parameter (max 80 characters) allowing the easy identification of the checkpoint:

N5k-UP(config)# checkpoint Checkpoint-1 description *** Testing the checkpoint feature ***
.....Done

Once the checkpoint has been created, we can easily confirm its creation and details by issuing the show checkpoint summary command as shown below:

N5k-UP(config)# show checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) Checkpoint-1:
Created by admin
Created at Mon, 08:10:29 22 May2023
Size is 15,568 bytes
Description: *** Testing the checkpoint feature ***

Note how the system not only provides all the necessary details about the recently created checkpoint but also shows which system user created it (admin). This detail is particularly important if the system is managed by multiple admins or engineers.

The NX-OS checkpoint feature doesn’t stop surprising thanks to the system’s intelligent features. Users can actually configure their Nexus switch to automatically generate checkpoints when specific changes occur in its running configuration. This awesome capability minimizes the risk for network downtime due to NX-OS key features misconfigurations and helps ensure there is always a valid snapshot that we can rollback to in case someone forgot to create a checkpoint before applying their changes!

Further capabilities built into this great feature allow the system to automatically create a checkpoint when aconfigured expiration period, e.g. the 120 days trial grace period of a license, has been exceeded. Reasons that could trigger automated system checkpoints are highlighted below: 

• License expiration of a feature
• Disabling a feature with the no feature command
• Removing an instance of a Layer 3 protocol

The system-generated checkpoint name convention has the format system-fm-feature. To help illustrate this automated feature we attempted to disable the VRRP feature on our Nexus 5000 therefore triggering the system to create a checkpoint. First we confirm the VRRP feature is enabled by issuing the show feature | include vrrp command then disable it and then verify it has been disabled:

N5k-UP(config)# show feature | include vrrp
vrrp 1 enabled
N5k-UP(config)# no feature vrrp

vrrp 1 disabled

N5k-UP# show checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) Checkpoint-1:
Created by admin
Created at Mon, 08:10:29 22 May2023
Size is 15,568 bytes
Description: *** Testing the checkpoint feature ***
System Checkpoint Summary
--------------------------------------------------------------------------------
2) system-fm-vrrp:
Created by admin
Created at Mon, 11:31:41 22 May2023
Size is 15,581 bytes
Description: Created by Feature Manager.

Notice that the system now shows a second checkpoint system-fm-vrrp which did not previously exist. This second checkpoint was created automatically by the Nexus as soon as we disabled the vrrp feature.

Multiple checkpoints can be created to save different versions of the running configuration, however as previously noted, there is a limit of ten (10) checkpoints a user can create. When this limit is reached a warning message to overwrite the oldest checkpoint is shown:

N5k-UP# checkpoint Checkpoint-11
Checkpoints limit reached, this will overwrite the oldest checkpoint,
Continue? (y/n) [n]

How to Backup Nexus Checkpoints – Exceeding The 10 Checkpoint Limit 

Those who require the ability to store more than 10 checkpoints can store checkpoint files in the bootflash (internal compact flash memory). This is a useful methodology to safely store checkpoint files as they won’t be erased with the write erase command or reboot of the Nexus switch.

To store the checkpoint to the bootflash simple use the checkpoint file bootflash: command and append the name to be used for the checkpoint file:

N5k-UP# checkpoint file bootflash:Checkpoint-11
.Done
N5k-UP# dir | grep 11
15568 May 22 11:54:53 2023 Checkpoint-11

The user and system checkpoint database can be manually cleared using the clear checkpoint database command. However, the checkpoint files stored at the bootflash are not affected by the clear checkpoint database command as displayed below:

N5k-UP# clear checkpoint database
...Done
N5k-UP# show checkpoint summary
N5k-UP#
N5k-UP# dir | grep 11
          15568 May22 11:54:53 2023 Checkpoint-11

Rollback Configuration – Checking Differences Between ‘Running Config’ & ‘CheckPoint’

The rollback feature allows us to apply a checkpoint backup configuration of the Cisco NX-OS switch at any point without having to reload the switch. When executed, rollback will compare the running-configuration with the checkpoint and make the necessary changes to the running configuration so that they become identical. Network-admin user privileges are required to configure rollback.

To test the rollback feature on our Nexus 5000 we created a checkpoint (Checkpoint-1) and configured a description for interfaces E1/10-E1/14. We then will attempt to rollback to the initial checkpoint created (Checkpoint-1) which should remove the descriptions from interfaces E1/10-E1/14:

N5k-UP(config)# interface ethernet 1/10-14
N5k-UP(config-if-range)# description *** TESTING CHECKPOINT FEATURE ***

We can review the differences between the running-config and a checkpoint before applying the rollback command by executing the show diff command. It is always recommended to use the show diff command and review the configuration changes before applying the checkpoint configuration file:

N5k-UP(config)# show diff rollback-patch running-config checkpoint Checkpoint-1
Collecting Running-Config
#Generating Rollback Patch
!!
interface Ethernet1/14
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/13
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/12
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/11
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/10
no description *** TESTING CHECKPOINT FEATURE ***
exit

The next rollback command options are provided for the Nexus 5000, Nexus 7000 and Nexus 9000 Series:

• Atomic: This is the default rollback type and applies the rollback file only if no errors occur
• Verbose: This option displays the execution log and allows the user to see the applied configuration

N5k-UP(config)# rollback running-config checkpoint Checkpoint-1 ?
<CR>
atomic Stop rollback and revert to original configuration (default)
verbose Show the execution log

In addition, the Nexus 7000 and Nexus 9000 Series support the following extra rollback options:

• Best-effort: Implement a rollback and skip any errors
• Stop-at-first-failure: Implement a rollback that stops if an error occurs

The Nexus 3000 supports only the atomic rollback option.

Finally the rollback mechanism has been successfully applied and… 

N5k-UP(config)# rollback running-config checkpoint Checkpoint-1 verbose
Collecting Running-Config
Generating Rollback patch for switch profile
Rollback Patch is Empty
Note: Applying config parallelly may fail Rollback verification
Collecting Running-Config
#Generating Rollback Patch
Executing Rollback Patch
========================================================
`config t `
`interface Ethernet1/14 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/13 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/12 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/11 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/10 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
========================================================
Generating Running-config for verification
Generating Patch for verification

At this point it seems like the rollback to the selected checkpoint was successful. We can verify this by checking to see if there is any description on Ethernet interfaces 1/10-14:

N5k-UP# sh interface ethernet 1/10-14 | include description
N5k-UP#

The rollback configuration test has been completed successfully.

Finally, to rollback using a checkpoint file located in the system’s bootflash we simply specify its location as shown below:

N5k-UP# rollback running-config file bootflash:///Checkpoint-11

Summary

This article serves as a configuration guide for the Nexus NX-OS checkpoint and rollback feature.  We have covered the main limitations and guidelines of this powerful Nexus NX-OS feature, demonstrated how to use the checkpoint & rollback feature and showed how to save checkpoints so they are not lost during a switch reboot or write erase. Finally, it is recommended that the configuration rollback procedure be used for managing change controls and notas a long term configuration management solution.