HP Microserver and WOL

Update: See article here

 

They just don’t seem to work. I’ve spent an annoying hour or so trying to get WOL to work with an HP Microserver – no joy whatsoever. I assumed it must be my code until I tried it on a few other machines but they worked just fine.

Now most of my machines are Realtek whereas HP are using Broadcom (as do the Dells). I’m not saying there’s anything wrong with Broadcom, but whenever I have a weird network problem they have a habit of being at the heart of it. Is it my magic packet? As far as I know it’s supposed to be 48-bits of ‘1’ followed by sixteen copies of the MAC address. Does it need a secure-on password? If so, how come you can’t set one in the BIOS.

I’ve asked an HP server expert: “Update the BIOS”. Perhaps, but these are brand new machines of an established design. They either turn on when they receive the packet, or they don’t work, and I can’t believe HP didn’t test them. Then again…

I’m told that these do support WOL on Windows, but not if you’re running anything else. On the face of it this is bonkers. Why should the OS the powered-off drive affect anything. The machine is off; the OS isn’t running. Well here’s a theory – before Windows shuts off it puts something in a register on the Broadcom chip to leave it in a WOL state. With the wrong drivers this doesn’t happen. Setting it in the BIOS doesn’t help, because it’s erased by the OS driver. The BIOS doesn’t restore it as the power is killed, but Windows hits the registers differently.

Unfortunately Broadcom doesn’t seem keen on releasing the documentation needed to write proper drivers to anyone other than Microsoft. Is this my imagination? Everyone else publishes the reference material, but Broadcom – I can’t find it.

If anyone can throw light on this one, please do. I’m still looking.

Update

Fitting a Realtek-based NIC in the Microserver and using that instead solves the problem. WOL just works. If you’re going to order one, remember it’s PCIe, not PCI, and that you really need one with a low-profile bracket option because a full-height card won’t fit.

 Further Update: See article here

10 Replies to “HP Microserver and WOL”

  1. Frank, great news for anyone running FreeBSD based solutions such as my favoured NAS4Free:

    kern/177184: patch for bge network driver to enable wake on lan
    Thu Mar 21 10:30:01 UTC 2013
    http://lists.freebsd.org/pipermail/freebsd-bugs/2013-March/052180.html

    So now OSes that WOL on microservers & a bucket load of other HP & Dell kit are:
    Windows, Linux, VMWare ESXi & FreeBSD.

    Illumos/Solaris 11 still don’t

    So I’ve just pulled the Realtek that I fitted last week, to allow some bare-metal tinkering, & have NAS4Free 9.1.0.1.775 WOLed & imported the 4 disk ZFS raid-z pool ;)
    So this setup is now pretty low powered:
    1 NIC – the onboard Broadcom HP NC107i PCIe Gigabit Server Adapter
    4 WD Green disks
    1 2GB USB generic UMD for NAS4Free to boot from.

    Next on the list is to apply the modified BIOS enabling AHCI on the eSATA port for support of my SATA port multiplier boxes. This should also work then in the latest NAS4Free. ;)

    1. Applied the `tweaked’ BIOS to enable AHCI for all 6 ports off the AMD SouthBridge:

      ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
      ahcich0: at channel 0 on ahci0
      ahcich1: at channel 1 on ahci0
      ahcich2: at channel 2 on ahci0
      ahcich3: at channel 3 on ahci0
      ahcich4: at channel 4 on ahci0
      ahcich5: at channel 5 on ahci0
      pmp0 at ahcich4 bus 0 scbus4 target 15 lun 0
      pmp0: ATA-0 device
      pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes)
      pmp0: 5 fan-out ports

      So now I have 4 disks in RAID-Z in HP caddies internally & 4 disks in a port multiplier box, all easily pull-able.

  2. I’ve found that powering down the microserver via the vSphere client leaves the host in a state where it will respond to WOL requests. However, when using the following command in the shell:

    esxcli system shutdown poweroff -d 10 -r test

    The host no longer responds to WOL packets.

    Annoying when you want to be able to script the powering down and starting of the host and its VMs.

    I haven’t found a way to get round this, so if anyone has any bright ideas…

    1. That’s obviously a VM Ware utility, but I suspect you’re having the same issue as I’ve been getting with UNIX. Did VM Ware borrow the code from BSD? Well Microsoft often does, so why not.

      Since posting this I’ve taken a better look at the Broadcom hardware interface and it appears that you need to turn it to “WOL mode” before shutting the system down. I think I know that the BSD drivers actually set the bits to mask all interrupts from within the chip in order to prevent it generating an IRQ (of any kind) while it’s shutting down. I’ve seen this recommended in early Broadcom documentation, and it’d be pretty standard practice. The problem is, of course, that once the OS has shut down the IRQ remains disabled – hence WOL can’t tell anyone about it.

      HP blames the OS – after all, it works on Windows doesn’t it? I think “not quite” – surely the BIOS should set the IRQ mask on the in-built LAN card regardless of what the OS did while shutting down. I have to confess I don’t know how easy it would be to do this once the power-shut-down has been requested, but why would you set it in the BIOS configuration and then fail to act on it? Just because the Windows driver can be configured to leave the bit alone, it’s no excuse for HP have a BIOS setting that doesn’t work.

      1. I’m not sure what the base platform was that VMware used, but I think the OS is certainly the problem – to use my shell command I had to place vSphere in to maintenance mode (which you don’t have to do with the remote GUI).

        I have since modified my shutdown script to exit maintenance mode as soon as the shutdown script has run and before the machine starts to shutdown. I am now able to wake the machine with a wol packet following this scripted shutdown process.

        My thinking is that in maintenance mode, the OS does not enable wol mode on the nic so as to stop the machine being powered on accidentally. Thinking about it, this would be desirable if you were opening up the machine – you wouldn’t want it to accidentally turn on whilst swapping out a drive.

    1. I like your thinking, but no dice. I assume you mean bge0 (the Broadcom interface – I think Linux gives them the generic ehtn names, but BSD has a different name for each manufacturer’s card). On BSD, the -h optoon simply halts the system. You need -p for an actual hardware powerdown.

      While it’s in the halt state I don’t know of anything that’s able to reboot, although I must admit I’d never tried sending it a WOL. It didn’t work; even if it did it wouldn’t help. The only way out of that one is the power switch.

      Looking at the source code for the driver, IIRC, it downs the interface the same way on a powerdown or a simple down. It sets the mask to disable all interrupts as its final act. I really ought to modify the driver, and I think there’s even a comment there to say this is to prevent spurious IRQs from a downed interface. If it was simply a matter of not masking the WOL interrupt with the rest I’d have thought someone would have done it already. But perhaps not!

      Unfortunately the Microservers don’t have “proper” PCI slots, and I’ve not got a PCIe Ethernet card to see if this only applies to the Broadcom, but I suspect it’s a Broadcom driver issue.

  3. I’ve since dug out the source code for the Broadcom NIC driver and spotted something that could be fouling it all up when the machine shuts down. Windows probably doesn’t bother to turn off chip interrupts before powering down, whereas everything else does. I’ve also stumbled upon a Broadcom device driver’s guide; it seems they’re seeing the light.

    More to come when I’ve tried fixing the driver.

    1. You are correct that it’s all about the state the driver sets before the shutdown. The driver has to set the NIC to switch to the standby power source to keep the NIC listening, it does this for the core of the NIC & the PHY. Without this the NIC is powered off completely when the system powers off. the driver also sets up filters for the PHY/MAC to spot the magic that you are setting it to WOL on. Linux does support WOL too for this NIC, that in-turn allows VMWare ESXi to too. I abstract the hardware to allow WOL with ESXi 5 (free) & then use RDM to make the disks available to FreeNAS natively SATA to SCSI translated by ESXi. That gives me a FreeNAS server booting off an ISO held in a 5th disk which does WOL. The cost is that ESXi takes an overhead & doesn’t support its Datastore being on a UMD, so another spindle to feed :( on the +ve side I get a host that can also run copies of FreeBSD & windows/Linux as I desire.

      1. Hi Paul, Thanks for that. I haven’t had time to look into this any further yet. Your creative solution is still boggling my mind a bit!

        As I’ve found the line in the driver that’s doing the damage I should really try to fix it, but it’s not quite irritating enough yet. I also wonder why no one else has done it.

        Other solutions that are a bit quicker to implement would be to use a smart network PDU, use something locally to drive a relay across the “on” button. Probably the easiest idea is to bung a Realtek NIC in there – a cheap and nasty would do as it only needs to be used to switch the thing on (although it would take up an extra port on the switch).

Leave a Reply to Oliver Cancel reply

Your email address will not be published. Required fields are marked *