By using the first frame in a video or some other chosen reference frame, movies could be incorporated into the photosynth montage. Clicking on the image would play the video in a frameless player withinn the montage. The movie would behave in EXACTLY the same way any still image does (positioned by the reference frame) except that it is a movie. The trigger to play the movie might be selecting it or movie key frmes might have discrete and simple player controls.
As an extension to this idea it would be possible to use multiple reference points in the video to smoothly transision betweeen the photosynth montage and the video based on the viewing angle and nearby images.
Lets suppose a moveie has 4 referecne points supported by other stills. As the movie play and reaches the new referece frame the photosynth montage wouuld re-align to the new reference frame. The surrounding images would change as if the video key frames were slide show images.