Not sure what this was ever trying to do with selenium. As xfvb allocates X11 displays, the port number it will allocate will be 6000+displayNumber so using the reserved port as a displayNumber will not actually cause the reserved port to be used (but the one 6000 greater than the reserved port)
All fun breaks loose when you get one of the ports within 6000 of 65535... which seems to happen a lot on MacOSX.
As a result I am removing the selenium profile from this integration test, as I cannot see that it was actually testing what it should have been testing.