Rocks Linux Node Install Error, Boots to Grub

We had a problem recently with one of our nodes that required replacing the motherboard.  After replacing the motherboard, I was unable to install the node on our cluster again.  I would run “insert-ethers” on the head node and then turn on the compute node.  Insert-ethers would detect a new device, however the node would never request a kickstart file.

After many different attempts to remedy the problem, I was finally able to get the node to instal.  This post will describe the steps I took in order to bring life back to the node.

Ensure the node is set to PXE boot

1. The first thing you should do is enter the BIOS of the troublesome node.
2. In the BIOS, make sure that an onboard LAN port is enabled and ensure that the Onboard LAN option is set to PXE.
3. In our BIOS these options were both under the “Advanced” tab.  Within this tab, enter the “Advanced PCI/PnP Setting” (See image below).  I apologize for the poor image quality.
   
4. Now that we know LAN is enabled and set to use PXE, we need to make sure that the network (LAN) is set to be the first boot device.
5. Still in the BIOS, move to the BOOT tab (might be different in different BIOS’s) and enter the Boot Device Priority option.
6. Make sure that 1st boot device is set to your Network option (see image below).   Again, I apologize for the poor image quality.
7. One all of these changes have been made, press F10 to save and exit.  Now just watch what happens when the node boots.  If you are lucky, this will solve your problems.

If the node still boots to the GNU GRUB command prompt, this most likely means that there is a problem in the local OS.  By default, Rocks tries to run some code locally and then switch to PXE boot.  The best option is to wipe the local hard drive on the node using something like badblocks.

Wipe the local hard drive

1. Remove the hard drive from the node, and insert it as a secondary hard drive on a separate node (see image below).  On our cluster, removing the hard drive was simple.  All that was required was to push a tab on the front of the node and pull the hard drive out.  The quality of this picture is even worse than the others.  The area was not well lit and i was using my phone.
2. After placing the hard drive in a new node, access the node either remotely or locally log into the system.
3. Type “fdisk -l” to make sure that the node was detected.  On our cluster, each node’s main hard drive was ‘/dev/sda’ and the secondary hard drive is ‘/dev/sdb’.
4. If you see the secondary node, the next step is to wipe the drive.  Do this by using something like badblocks.

   badblocks -wvf /dev/sdb

5. This only needs to be run for a few minutes in order to wipe the beginning of the drive.
6. After you have wiped the drive, replace it into the troublesome node and run insert-ethers again.

After wiping the hard drive, i now got a new error message when trying to boot the node.  The error message was:

Reboot and Select proper Boot device or Insert Boot Media in selected Boot device and press a key

When I was attempting to get the node to boot, making the changes mentioned previously about the BIOS was the last step I took.  The message about selecting a proper boot device was caused by the fact that I was not performing a PXE boot.  Switching the BIOS to PXE boot first solved the problem for me.

Force PXE Boot

If you still are unable to PXE boot, the only other possible solution I can think of is to force a PXE boot.  When the node is booting up, and you first get information posted to the monitor, press F12 to force a PXE boot.

Hopefully one of these solutions works!