User Tools

Site Tools


supermicro_ipmi_v2.0_on_ubuntu

This is an old revision of the document!


Running IPMI on Linux

What is IPMI?

IPMI is standard which allows remote server management, primarliy developed by Intel. IPMI cards, known as Baseboard Management Cards (BMCs) are primitive computers in their own right and are operational all the time, so long as the server has a power source. The server itself does not need to be powered on, or the operating system operational for the BMC to work.

The primary benefits of IPMI are:

  • View server chassis and motherboard sensor output remotely, such as chassis status and intrusion detection.
  • Ability to remotely power on, power off, reboot the server and flash the identification light.
  • Ability to set up a console on a serial port and have the BMC redirect that console over a network port, which in cooperation with BIOS level console redirection, gives you the ability to view the BIOS, bootloader, bootup and shutdown procedures and console output should the machine hang or lock up, just as you would if you were interacting with the machine locally. This is called Serial Over Lan (SOL) and is available in IPMI v2.0 as a standard and using non-standard proprietary methods in v1.5.

IPMI Revisions

There are currently 3 IPMI revisions:

  • IPMI v1 is the first revision and is obsolete.
  • IPMI v1.5 is the current minimum version and does not feature SOL, though it is available in non-standard implementations.
  • IPMI v2 features most of v1.5, minus some obsolete functions, plus some new functions including SOL.

SOL is desirable as it allows you to interact with the server as though it were local in cases where the operating system locks up and SSH or telnet access are not available due to the operating system being inoperable. IPMI v1.5 still allows to you to power the system on and off and view sensor output.

Installation

This guide covers the installation of IPMI tools on Dell 1425 servers and Supermicro servers with a PDSMi+ motherboard. The instructions will be relevent for other server models, but I make no promises.

For Supermicros servers, open up the server chassis and make a note of the MAC address on the IPMI port, you'll need this later.

In the BIOS, set the concole redirection to COM2, which is the BMC console port, it doesn't phsically exist). Disable BIOS redirection after POST, choose a baud rate (19200 is recommended a lot), a terminal type (vt100 works ok for me) and leave the other settings as they are.

Supermicro BMCs require you to boot from a CD and use their tool to flash the BMC prior to use, refer to the FTP link below and go up a few directories to get the latest IPMI CD image (the one shipped with the server caused me problems). Burn the image to a CD and boot from it. Use the utility to install the firmware for your IPMI version and then your motherboard version. I had to guess whether to use RCMP+ or RCMP. I have the Supermicro AOC-IPMI20-E and it wouldn't work with RCMP+, even though it will upload a firmware for you. It worked after I reflashed it with the RCMP version. When the process finished, use the ipnmac utility while still booted from the CD, by typing ipnmac (if you're not in the right directory you'll have to navigate using cd and dir commands to find it). Give it a unique IP address from any other interface on the machine or on your network. The BMC needs to be network addressable in its own right. Also give it the MAC address you took from the IPMI socket on the motherboard, not the one written on the LAN ports as suggested by the official Supermicro docs.

Once this step is done, hit CTRL-Alt-Del and remove the CD.

For Dell 1425 servers (and probably other Dells), hit Alt-3 when prompted to enter the BMC setup and give it some unique network settings and some user settings (make your passwords secure!).

You can boot now into Linux, or from a Linux installer CD.

Note on network configuration:

Both Dell 1425s and Supermicro PDSMi+ motherboards use the first LAN port to redirect the IPMI traffic over when using SOL. For this reason, whether you intend to use SOL or not, it is a good idea to use the first LAN ports, almost certainly eth0, as your local network interface, rather than an Internet visible interface.

Once booted into Linux, install openipmi on all machines with a BMC and ipmitool on every machine from which you wish to run IPMI commands locally or to send IPMI commands to a remote machine. Red Hat, FC and SUSE users will have to download the IPMItool packages from the website if they're not in your package-shallow distros ;). OpenIPMI is not required on machines without a BMC to send ipmi commands to remote machines. You only need OpenIPMI where you want to run IPMI commands locally, manage the BMC locally from the OS (which you want to do if you have a BMC in the machine) or I presume to do console redirection over the BMC.

For kernel 2.6.x

modprobe ipmi_msghandler modprobe ipmi_devintf modprobe ipmi_si type=kcs ports=0xca8 regspacings=4

If ipmi_si won't load, look at the output of dmidecode for the base address of your IPMI BMC and then use that base address for the ports=<base address> module option.

Kernel 2.4 people will have to follow the Debian IPMI instructions, as you're living in a world I haven't encountered with IPMI. You should note that ipmi_si_drv is called something slightly different its module parameters are named slightly differently. You will also have to make your own device node probably, as documented in the Debian instructions.

If this works without errors, add the modules and options to /etc/modules.conf

If you then cat /proc/devices, you should see your IPMI device listed and ls -l /dev/ipmi0 should show your device node.

IPMI Commands

You should then be able to run some IPMI commands locally (probably need to be root as root owns the device node):

root@gw01:~# ipmitool -I open chassis power status Chassis Power is on root@gw01:~# ipmitool -I open chassis status System Power : on Power Overload : false Power Interlock : inactive Main Power Fault : false Power Control Fault : false Power Restore Policy : always-off Last Power Event : Chassis Intrusion : inactive Front-Panel Lockout : inactive Drive Fault : false Cooling/Fan Fault : false Sleep Button Disable : allowed Diag Button Disable : allowed Reset Button Disable : allowed Power Button Disable : allowed Sleep Button Disabled: true Diag Button Disabled : true Reset Button Disabled: true Power Button Disabled: true

Command dissection:

ipmitool -I open chassis power status

-I open - use the local openipmi interface. chassis - run a command from the chassis set of commands. power - run a command relating to power. status - show the status of the chassis power status.

Thats pretty simple.

Assuming you get a local response, then try a remote command from another machine, you won't need to be root as you are talking to the BMC directly, rather than through the OS.

adam@ns0:~$ ipmitool -I lan -H 192.168.10.123 -U root -a chassis power status Password: Chassis Power is on adam@ns0:~$ ipmitool -I lan -H 192.168.10.123 -U root -a chassis status Password: System Power : on Power Overload : false Power Interlock : inactive Main Power Fault : false Power Control Fault : false Power Restore Policy : always-off Last Power Event : Chassis Intrusion : inactive Front-Panel Lockout : inactive Drive Fault : false Cooling/Fan Fault : false Sleep Button Disable : allowed Diag Button Disable : allowed Reset Button Disable : allowed Power Button Disable : allowed Sleep Button Disabled: true Diag Button Disabled : true Reset Button Disabled: true Power Button Disabled: true

Command dissection:

ipmitool -I lan -H 192.168.10.123 -U ipmiadmin -a chassis power status

-I lan - Run the command over the LAN. -H 192.168.10.123 - -H means host followed by an IP address or hostname. -U root - -U specifies a user, followed by the username set up on the BMC, not a local or remote OS user. -a - prompt for password. chassis power status - as per the local command dissection.

If that all works for you then you can a woop and a holler.

IPMItool has a familiar UNIX or Cisco IOS syntax, which allows you to complete the command as you go along. If you type in an incomplete command, IPMItool will provide you with a list of options that can be used to complete your command.

Simply typing ipmitool -I open will offer the top-level sub-commands, which you can then choose from to add to your command.

root@gw01:~# ipmitool -I open No command provided! Commands: raw Send a RAW IPMI request and print response i2c Send an I2C Master Write-Read command and print response lan Configure LAN Channels chassis Get chassis status and set power state event Send pre-defined events to MC mc Management Controller status and global enables sdr Print Sensor Data Repository entries and readings sensor Print detailed sensor information fru Print built-in FRU and scan SDR for FRU locators sel Print System Event Log (SEL) pef Configure Platform Event Filtering (PEF) sol Configure IPMIv2.0 Serial-over-LAN isol Configure IPMIv1.5 Serial-over-LAN user Configure Management Controller users channel Configure Management Controller channels session Print session information sunoem OEM Commands for Sun servers shell Launch interactive IPMI shell exec Run list of commands from file set Set runtime variable for shell and exec root@gw01:~# ipmitool -I open user User Commands: summary [<channel number>] list [<channel number>] set name <user id> <username> set password <user id> [<password>] disable <user id> enable <user id> test <user id> <16|20> [<password]> root@gw01:~# ipmitool -I open chassis Chassis Commands: status, power, identify, policy, restart_cause, poh, bootdev root@gw01:~# ipmitool -I open chassis power chassis power Commands: status, on, off, cycle, reset, diag, soft

…And so on.

Beware, powering the server off or rebooting does not do a graceful shutdown like the OS would. They are the same as holding in the power button or pressing the reboot button. Supermicro offer a graceful shutdown daemon for Windows and Linux which will respond to the graceful showdown and reboot commands provided by Supermicro's graphical Java IPMI control application IPMIview.

More details to come on actually setting IPMI up, this is simply a terse beginning to the article.

IPMI Standards

IPMI Software

IPMI Software Documentation

IPMI Howtos

Vendor IPMI Documentation

Dell IPMI howto Contains useful sample commands.

Other IPMI Documentation

Cern IPMI presentation PDF containing demonstration of IPMI usage, including reference to a version of IPMItool patched by Supermicro.

Supermicro FTP server, Linux IPMI directory (version number in URL subject to change)

Todo

  • Tidy up command and output examples' formatting, they are currently all over the place
  • Include glossary of terms and IPMI version details
  • Include relevent info instead of referring to other docs in case they disappear, such as kernel 2.4 info from Debian howto
  • Cover serial consoles, grub, innitab etc
  • Cover IPMIView and Graceful Showdown Daemon
  • Link to Supermicro docs, tools and CD images more accurately
supermicro_ipmi_v2.0_on_ubuntu.1168621651.txt.gz · Last modified: 2016/11/25 22:38 (external edit)