Thanks for the tweet Nate, Some time ago I took apart the photosynth silverlight control and embeded in my own custom Silverlight app, it was a Bing Maps embedded concept before it had been implimented - its no longer running as the host, Azure CTP, is now closed down.
The basic concept is that Silverlight runs on the client, so you can get the Photosynth dlls, redgate reflector allows you to see how to use it, however this is not a supported control, good for a concept but you would not use in a commerical application. I had to fix the code everytime a change was maded here.
Also as compiled dlls you can't use these on WP7.
The only current solution is to use the embed link, on a multi-touch tablet the control is very cool.
Love to see a supported Silverlight control, My guess is that the "supported" part is the difficult piece?