TFTP server in bridged VirtualBox VM not working with client specific configuration

I recently had to setup a network boot environment for some embedded devices. I decided to use an Ubuntu 14.04 in a VirtualBox VM with the virtual network interface bridged to the physical ethernet interface, to be able to move the server to different computers.

The DHCP server works flawlessly and each machine gets its address assigned correctly every time I try it. The TFTP server, however, has kind of a non-deterministic behavior. Sometimes I am able to boot at least the default configuration (stated in the default file), sometimes even this fails. In any case I was not able to use a client specific configuration file (01-aa-bb-cc-...).

Because I could not find a reason for this behavior I copied the very same configuration files to my physical PC, installed the same packages - and it worked out of the box.

I also found out that other virtual machines have no problem booting from the virtual TFTP server and it also does not matter if the connection between the virtual TFTP server and the VMs is made via bridging to the same physical adapter or via connection to the same host-only network.

By the way, both the virtual TFTP server and the physical PC have no firewall installed! I am testing with tftpd-hpa and isc-dhcp-server which are taken from the ubuntu repositories. Following you see my configuration.

DHCP server:

authoritative;
allow booting;
subnet 192.168.1.0 netmask 255.255.255.0 {
    range 192.168.1.50 192.168.1.60;        
    default-lease-time 600;
    max-lease-time 7200;
    option broadcast-address 192.168.1.255;
    option subnet-mask 255.255.255.0;
}

next-server 192.168.1.1;
filename "/pxelinux.0";

host ap_client
{
  hardware ethernet xx:xx:xx:xx:xx:xx; //here I use my real MAC of course with lower case letters where required
  fixed-address 192.168.1.11;
}

TFTP server:

/var/lib/tftpboot/pxelinux.cfg/default

LABEL      localhdd1
LOCALBOOT  0

/var/lib/tftpboot/http://ift.tt/1EuRrC6

LABEL      memtest
KERNEL     memtest/mt86plus

Do you have any idea why it is not working with a VirtualBox VM?