Well the Hololens is AR which is totally different than VR.
I wouldn't be surprised to see the Oculus Rift or the HTC Vive work to some extent on the ONE. Both companies have a partnership with MS so it's not out of the question. The X1 has weakish hardware specs so the games might not be that impressive compared
to some super duper uber computer.
Yea the Hololens is AR and needs a super computer to even run at the moment. I don't see it ever coming to Xbox One with the already dated hardware it has. It might come out on the Next Box or Xbox Two...whatever they are going to call it.