Photosynth currently hosts two different types of imagery:
1) photosynths released 2008 (multiple photos taken from many different positions, looking at the same object or set of objects, resulting in an underlying sparse 3D reconstruction of the object or scene called a point cloud and additionally the ability to move all the way through an environment moving from photo to photo)
2) panoramas added 2010 (multiple photos taken from one single position, looking at the same object or set of objects, resulting in a single fused photo which is simple to pan through, but contains no depth information for the scene)
In other words, photosynths are essentially what you're after and have been since 2008, except that:
a) not every position will be a panorama, but a single photo and
b) instead of manually linking from one photo to another, you instead take the photos in such a way that they can be automatically linked by Photosynth.