| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
ipmi_on_linux [2008/04/22 15:16]
adam |
ipmi_on_linux [2016/11/25 22:38] (current)
|
| ===== Running IPMI on Linux ===== | ===== Running IPMI on Linux ===== |
| | |
| |
| ==== What is IPMI? ==== | ==== What is IPMI? ==== |
| |
| |
| IPMI is standard which allows remote server management, primarliy developed by Intel. IPMI cards, known as Baseboard Management Cards (BMCs) are primitive computers in their own right and are operational all the time, so long as the server has a power source. The server itself does not need to be powered on, or the operating system operational for the BMC to work, it just needs a power source to be connected to the server. | IPMI is standard which allows remote server management, primarily developed by Intel. IPMI cards, known as Baseboard Management Cards (BMCs) are primitive computers in their own right and are operational all the time, so long as the server has a power source. The server itself does not need to be powered on, or the operating system operational for the BMC to work, it just needs a power source to be connected to the server. |
| |
| The primary benefits of IPMI are: | The primary benefits of IPMI are: |
| * Ability to set up a console on a serial port and have the BMC redirect that console over a network port, which in cooperation with BIOS level console redirection, gives you the ability to view the BIOS, bootloader, bootup and shutdown procedures and console output should the machine hang or lock up, just as you would if you were interacting with the machine locally. This is called Serial Over Lan (SOL) and is available in IPMI v2.0 as a standard and using non-standard proprietary methods in v1.5. | * Ability to set up a console on a serial port and have the BMC redirect that console over a network port, which in cooperation with BIOS level console redirection, gives you the ability to view the BIOS, bootloader, bootup and shutdown procedures and console output should the machine hang or lock up, just as you would if you were interacting with the machine locally. This is called Serial Over Lan (SOL) and is available in IPMI v2.0 as a standard and using non-standard proprietary methods in v1.5. |
| |
| Essentially, IPMI will save you from a few hundred to over a thousand GBP instead of buying a remote power control unit and SOL will save you the same instead of buying an IP KVM. | Essentially, IPMI will save you from a few hundred to over a thousand GBP instead of buying a remote power control unit and SOL will save you the same amount again over buying an IP KVM. |
| |
| |
| |
| * IPMI v1.0 - Autonomous access, logging and control. IPMI messaging command sets, sensor data records and event messages. Access through system interfaces like memory mapped IO, I2C bus etc. | * IPMI v1.0 - Autonomous access, logging and control. IPMI messaging command sets, sensor data records and event messages. Access through system interfaces like memory mapped IO, I2C bus etc. |
| * IPMI v1.5 - Ability to send IPMI messages to BMC over LAN, LAN alerting. No SOL as part of standard but some vendor specific SOL implementations. | * IPMI v1.5 - Ability to send IPMI messages to BMC over LAN, LAN alerting. No SOL as part of the standard's specification but some vendor specific SOL implementations. |
| * IPMI v2.0 - Serial Over LAN enabling console redirection, access control, enhanced authentication, packet encryption using RCMP+, SMbus interface. | * IPMI v2.0 - Serial Over LAN enabling console redirection, access control, enhanced authentication, packet encryption using RCMP+, SMbus interface. |
| |
| IPMI version 2.0 is desirable as it allows you to use SOL to get a remote console on the server as though it were local in cases where the operating system locks up and SSH or telnet access are not available due to the operating system being inoperable; v 2.0 also allows you to encrypt the contents of the IPMI packets to protect the passwords sent to remote systems. IPMI v1.5 still allows to you to power the system on and off and view sensor output. Both 2.0 and 1.5 are in common usage and are both still sold on new servers. | IPMI version 2.0 is desirable as it allows you to use SOL to get a remote console on the server as though it were local in cases where the operating system locks up and SSH or (heaven forbid) telnet access are not available due to the operating system being inoperable. v2.0 also allows you to encrypt the contents of the IPMI packets sent to remote systems and so protects the BMC passwords and your commands on the network. IPMI v1.5 still allows to you to power the system on and off and view sensor output, but does not support packet encryption (and therefore sends your BMC password over the network in plain text) and does not support SOL in any standardised way. Both 2.0 and 1.5 are in common usage and are both still sold on new servers. |
| |
| ==== Glossary ==== | ==== Glossary ==== |
| modprobe ipmi_msghandler | modprobe ipmi_msghandler |
| modprobe ipmi_devintf | modprobe ipmi_devintf |
| | modprobe ipmi_si |
| | |
| | If ipmi_si won't load, you may need to use: |
| | |
| modprobe ipmi_si type=kcs ports=0xca8 regspacings=4 | modprobe ipmi_si type=kcs ports=0xca8 regspacings=4 |
| |
| If ipmi_si won't load, look at the output of dmidecode for the base address of your IPMI BMC and then use that base address for the ''ports=<base address>'' module option. The default ports option is ''0xca2'' so if your BMC is at that address according to ''dmidecode'', then you don't need this option. A SLES 10 user tells me that they did have to specify the ports value 0xca2 on an HP DL380 G5, so perhaps it isn't always the default. | Or look at the output of dmidecode for the base address of your IPMI BMC and then use that base address for the ''ports=<base address>'' module option. The default ports option is ''0xca2'' so if your BMC is at that address according to ''dmidecode'', then you don't need this option. A SLES 10 user tells me that they did have to specify the ports value 0xca2 on an HP DL380 G5, so perhaps it isn't always the default. |
| |
| Kernel 2.4 people will have to follow the Debian IPMI instructions, as you're living in a world I haven't encountered with IPMI. You should note that ipmi_si is called ipmi_si_drv and its regspacings option is called si_regspacings. You may also have to make your own device node if you're not using devfs, as documented in the Debian instructions. After modprobing the relevent modules successfully, ''ls -l /dev/ipmi0'' to see if you have a device node before trying to create one. | Kernel 2.4 people will have to follow the Debian IPMI instructions, as you're living in a world I haven't encountered with IPMI. You should note that ipmi_si is called ipmi_si_drv and its regspacings option is called si_regspacings. You may also have to make your own device node if you're not using devfs, as documented in the Debian instructions. After modprobing the relevent modules successfully, ''ls -l /dev/ipmi0'' to see if you have a device node before trying to create one. |
| |
| If this works without errors then Fedora/RH/Centos people are set, Ubuntu/Debian people need to add the modules and options to /etc/modules.conf or maybe add the modprobe commands to /etc/init.d/local if you have no other way. | If this works without errors then Fedora/RH/Centos people are set, Ubuntu/Debian people need to add the modules and options to /etc/modules or maybe add the modprobe commands to /etc/init.d/local if you have no other way. |
| |
| If you then ''cat /proc/devices'', you should see your IPMI device listed and ''ls -l /dev/ipmi0'' should show your device node. | If you then ''cat /proc/devices'', you should see your IPMI device listed and ''ls -l /dev/ipmi0'' should show your device node. |
| | |
| |
| ==== Setting Up Serial Consoles ==== | ==== Setting Up Serial Consoles ==== |
| You should have set up your BIOS for console redirection earlier, so now we will do the bootloader and init. | You should have set up your BIOS for console redirection earlier, so now we will do the bootloader and init. |
| |
| To allow your bootloader to redirect over the BMC's serial port, edit /boot/grub/grub.conf, sometimes known as /boot/grub/menu.list. | To allow your bootloader to redirect over the BMC's serial port, for Grub v1, edit /boot/grub/grub.conf, sometimes known as /boot/grub/menu.list add the following lines to grub.conf or menu.list: |
| | |
| | serial --unit=1 --speed=19200 --word=8 --parity=no --stop=1 |
| | terminal --timeout=10 serial console |
| |
| Disable splash screens by commenting out anything starting with splash outside of the OS boot menu section and remove any splash options from your kernel lines. A text console can't display them. | Disable splash screens by commenting out anything starting with splash outside of the OS boot menu section and remove any splash options from your kernel lines. A text console can't display them. |
| |
| Add ''console=tty0 console=ttyS1,19200n8r'' to the end of your kernel lines, so it should look something like the following: | To make kernel messages output over your BMC, add ''console=tty0 console=ttyS1,19200n8r'' to the end of your kernel lines, so it should look something like the following: |
| |
| title Ubuntu, kernel 2.6.15-26-amd64-server | title Ubuntu, kernel 2.6.15-26-amd64-server |
| savedefault | savedefault |
| boot | boot |
| | |
| | For Grub2, look [[http://linux.xvx.cz/2009/08/debian-with-grub2-and-serial-connection/|here]] |
| |
| This gives you a serial console on the second serial port, which should be your BMC's serial port, at 19200Kb per second, which should match what you chose in the BIOS. The order of the console options above is important. The last listed will be the system's default console which will display the boot messages and kernel errors. This means that after halfway through the boot process, during shutdown and when there are kernel errors, only the serial console will see the messages. As you want to work remotely, this is the way it has to be. You can't have more than one default console. You can however interrupt grub at boot time and edit the kernel line for a single boot with different parameters if you need to see the default console locally but remember that they will persist until you reboot, which means that if you reboot but still want to see them locally, you'll have to interrupt and edit grub at boot time again and also, to see them remotely again, you need to remember to reboot. | This gives you a serial console on the second serial port, which should be your BMC's serial port, at 19200Kb per second, which should match what you chose in the BIOS. The order of the console options above is important. The last listed will be the system's default console which will display the boot messages and kernel errors. This means that after halfway through the boot process, during shutdown and when there are kernel errors, only the serial console will see the messages. As you want to work remotely, this is the way it has to be. You can't have more than one default console. You can however interrupt grub at boot time and edit the kernel line for a single boot with different parameters if you need to see the default console locally but remember that they will persist until you reboot, which means that if you reboot but still want to see them locally, you'll have to interrupt and edit grub at boot time again and also, to see them remotely again, you need to remember to reboot. |
| | |
| This gives you a serial console on the second serial port, which should be your BMC's serial port, for the BIOS as configured earlier, the bootloader and init. Again, change 19200 to whatever you chose in your BIOS. | This gives you a serial console on the second serial port, which should be your BMC's serial port, for the BIOS as configured earlier, the bootloader and init. Again, change 19200 to whatever you chose in your BIOS. |
| | |
| | Users of Ubuntu (and presumably Debian and it's other derivative distributions) can read [[https://help.ubuntu.com/community/IPMI]] for assistance with all of this stuff. I guess it will be useful for everybody else too, if you can recognise where it is Ubuntu specific. |
| |
| ==== IPMI Commands ==== | ==== IPMI Commands ==== |
| |
| You should now be able to search for the host's IPMI device address in IPMIView and log in with the username and password of ADMIN. You should add a new user, change it's user level to administrator and delete the ADMIN user. | You should now be able to search for the host's IPMI device address in IPMIView and log in with the username and password of ADMIN. You should add a new user, change it's user level to administrator and delete the ADMIN user. |
| | |
| | |
| |
| |
| === IPMI Howtos === | === IPMI Howtos === |
| |
| [[http://buttersideup.com/docs/howto/IPMI_on_Debian.html|Debian IPMI howto]] | [[https://help.ubuntu.com/community/IPMI|Ubuntu IPMI documentation]] New for Ubuntu 8.04 |
| | |
| | [[http://buttersideup.com/docs/howto/IPMI_on_Debian.html|Older Debian IPMI howto]] |
| |
| [[http://www.ecst.csuchico.edu/~dranch/LINUX/IPMI/ipmi-on-linux.html|Supermicro IPMI on Linux]] Contains errata on the Supermicro Docs | [[http://www.ecst.csuchico.edu/~dranch/LINUX/IPMI/ipmi-on-linux.html|Supermicro IPMI on Linux]] Contains errata on the Supermicro Docs |
