If I understand correctly, the SIFT algorithm(s?) that Photosynth and other Structure From Motion utilities use desires differences of 30 degrees or less between photos for a successful feature match. Being that the Bird's Eye View/oblique photos are generally taken at 90 degree intervals, it seems like trying to synth the North, South, East, and West Bird's Eye shots together is a bit of a lost cause at the moment.
What I thought of ages ago and still haven't tried is getting all the North facing Bird's Eye shots for a certain grid and trying to synth them all together. This should work for any direction, but I'd like to try one compass direction at a time, essentially resulting in four synths for any given grid based soley on Bird's Eye View shots.
I'd love to use the original Bird's Eye images with original metadata intact, but a (tedious) workaround would be to use the Microsoft Photo Calibration Tool to reverse engineer the EXIF orientation.