Get all your questions answered about our latest Photosynth Technical Preview.
Even before panorama hosting was introduced on Photosynth.net in 2010, I've wanted to make synths constructed entirely of spherical panoramas.
What technical challenges are stopping this from happening?
Although not precisely the same, Joscelin Trouwborst just made a suggestion for a Walk-Pano Synth type here:
I think he's suggesting that the input photos at each position be kept separate (as in the new parallax panoramas) rather than stitching them at each point and having a multiple stitched-pano synth, but anyone interested in this suggestion should probably support his too.
I am not sure Nathan, if we are talking about the same or different things. Here is my biggest walk and pan synth.
Could you please elaborate on the subject and point out the difference if any.
From that synth, I think I'm understanding you correctly.
In other words:
You and I both want to be able to make a 'Walk' style synth where at any point along the path the user can stop and look all the way around.
Since the new synth types do not offer any sort of stitching multiple images into a fused panorama, I conclude that you mean to keep all the photos separate from each other as the new parallax panorama type does.
What I have always hoped since spherical panoramas were a supported upload type here on Photosynth.net is that I could take photos to make standard ICE spherical panos, but instead of uploading them individually as standalone panos, put all of them into a single synth (since they were taken with that purpose in mind) and move between them as they are projected onto the point cloud/mesh.
In other words, I want a way to make a Streetside or Street View synth where each vantage point in the synth is a full spherical ICE panorama.
I actually would love to see all four synth types accept multiple ICE panoramas instead of multiple ordinary photos.
I think that a 'Spin' or 'Wall' style synth could be equally interesting using multiple stitched/fused panos.
It may sound redundant (and I don't want the entire thread dismissed because someone disagrees with this last part) but I would even be curious to see a 'parallax panorama' style synth where each input shot is a spherical pano (although I admit that hardly anyone would shoot these). This would give you the feeling of standing in one place but being able to lean back and forth a bit while maintaining the ability to have a continuous unbroken field of view at any point.
Multi-spherical panos would be fantastic, but I don't even see support for single spherical panos in the technical preview. It seems that all of the synth types are confined to one degree of freedom -- fwd, back; left, right; etc. Are there plans for spherical synths with the new tech so that I can look right, left, up and down?
That said, I'm really enjoying the new Photosynth so far!
@Toddard, the new 'Expert Shooting Guide' (found at http://photosynth.net/preview/help ) says the following on page 5 under 'Current Limitations':
"This new Photosynth technology has a few current constraints: ... The new Photosynth has no equivalent yet to a full panoramic sphere where you can look up and down as well as left and right. Keep using our apps or Microsoft ICE if you want to capture the ceiling!"
That said, in another discussion here http://photosynth.net/discussion.aspx?cat=00581351-82d8-438d-a37b-7eadb3fb4991&dis=75761a80-5879-413c-9e60-0b19becf4579 , the Photosynth team said, "We definitely are planning on fully supporting arbitrary resolution stitched panos in WebGL."
My understanding is growing. We should distinguish panorama Synth, parallax panorama Synth, spherical Synth and the multi forms of those. The concept of a spherical Synth includes the concept of a panorama Synth. (The panorama Synth could be regarded as an incomplete spherical Synth.) Whereas a spherical Synth and a parallax panorama Synth are something completely different. Current camera apps (WP, Android, iOS) support spherical Synths only. The introduction of Walk, Spin and Wall to these apps is great, a quantum leap forward to get Photosynth mainstream and obtain real world imagery to enhance Bing Maps.
Back to the multi forms. The multi-sphere Synth first. A multi-sphere Synth should have scenery overlap. The synther should recognize these scene overlaps and create in each sphere gateways to one or more other spheres at the optimal point in the sphere scenery.
I can imagine the following.
Camera apps could have an advanced option for multi forms of synthing, the multi-sphere Synth to be one of them. Or, a different pro app( for a price).
The web create function could have an advanced function too. For the multi-sphere Synth type it should distinguish in multiple selections for the different spheres within one creation process.
The Synth viewer would show in a sphere view clickable/touchable small circles for the gateways to the other spheres in the synth with a small, 'distant' view of the target sphere. Clicking/touching would navigate to the target sphere.
Analogous to the above multi-Sphere Synth, one could think of the following too.
'Wall multi-Sphere'. E.g. walking along a water line and looking around once in a while.
At this moment I cannot envision other combinations to work well, unless gateways could be created manually.
These multi-form, composed Synths would be for artistic, professional purpose and not to be put on Bing Maps, because on Bing Maps Synths should be elementary and linked to a geo location.
Bing Maps experience. I can imagine the Bing Maps experience to include a composed Synth experience of Walks, Walls, Spheres and Spins. If Walks and Walls would come across or near a Sphere or a Spin, then they could show an entry gateway to the Sphere or Spin. If Spheres and Spins are close to a Walk or Wall, then they could show an exit gateway to the Walk or Wall.
I would completely and utterly reject calling a single spherical panorama any kind of synth at all, since it does not spatially link three or more unique viewpoints.
I just call that a 'panorama', 'spherical panorama', 'cube panorama' or 'stitched panorama' as a way to differentiate.
A photosynth composed of multiple spherical panos instead of multiple ordinary photos is still just a photosynth to me. It doesn't change the path I take through a scene.
Trying to give it a different name is like giving different names to photosynths made from landscape photos or portrait photos or mixed aspect ratio photos. To me, that's overthinking it, although I might nickname it a 'panosynth' and the vast majority of everyone out there will just call it 'Street View' or 'Streetside'.
I would say the same thing for a photosynth composed of Lytro camera light fields. It's still just an original photosynth or a Spin, Walk, Wall, or (parallax) Panorama synth.
@Nathan. You are right. But, Photosynth has become a more generic term with the possibility to upload panorama's and the creation of spheres with the WP Photosynth apps. If you would put the use of terminology aside, I bet you understand what I am talking about. Maybe you could think of better terms.
I was imagining the above, but I came across this today
Yeah, I saw that on the same day that Photosynth announced their Technical Preview.
From what I saw in the video, though, it was more of a slideshow of spherical panos, though.
I didn't see any spatial transitions between the panos.
Sure Nate, but for the mainstream, Google got a pretty awesome thing out working. That added to the fact, that their Google Maps is well regarded in the market place. Now what is Microsoft's plan to catch up and take over. Certainly not with the timelines you have referred to earlier.
Can this serve as an example https://www.google.com/maps/views/view/101388107725448550582/photo/1BAMfcQshQoAAAAGOuk0IA?gl=ca&heading=161&pitch=90&fovy=75.
No, not really.
I'm sorry if the title is not worded the most clearly, but what I am asking for is the ability to make photosynths out of multiple spherical panoramas rather than multiple ordinary digital photos.
What Google is currently offering is the ability to present multiple spherical panoramas in a slideshow with no spatial transition between them and no automated image matching which would provide the information necessary to spatially transition between them.
What I am asking for is nothing short of being able to upload multiple spherical panoramas taken throughout a space (a living room, my own street, a public park, a garden, a cathedral, etc.) and have Photosynth automatically align them spatially, recover the geometry seen in multiple panoramas, and provide spatial transitions between these panoramas (ideally while projecting the panoramas onto the recovered geometry).
In short I want nothing short of the ability to shoot our own Streetside.
I agree Nate, I've always wanted more of a video game 3D feel, in the old games they would force you onto a "PATH" then you could free look around at will. If you took two walks forward and back, then two walls couldn't the processor then "SYNTH" these all together creating a "FREE LOOKING VIEW" at any point along the timeline or at least at each matching keyframe.
On a side note I thought I read PS team saying they might give "KEYFRAME" control or access, if this happened we could place spherical panoramas at selected keyframe for a free look feel. This could also include spins, path choice options or filters. We could then add special effects and integrate the video processing side of the world in to that spherical panoramas and map in 3D space.
Matched up with a CAD, Direct 3D engine or something like VUE you could merge the two and have Real/Virtual experience.
Side note Googles Product requires you pay an approved photographer.
What if you took 5 gopro(cameras) and put them on a frame set them to overlap the required amount then take each photo at the same time - 5 at once for each keyframe. I assume this is googles approach. The current process wouldn't like this but it seems like it's a short jump from where we are to there. I honestly thought after seeing PSv1 that's where the next step would be but I love PSv2 and understand research isn't dictated it's a ride you experience. I hope photo tourism is still the main goal however. I sold this idea to a lot of people years ago and would like to make good on that. With drones and cheaper tech we can do it we just need the management tools and processing from Microsoft.
Hey, Joscelin, I missed the ability to click on the ground the first time around, so yeah, that's actually getting a good bit closer to what I wanted than I'd thought, given the coverage that was shot at that location.
Sorry about my misunderstanding.
The spatial relationships have been recovered, but the transitions between panos still leave something to be desired with no option for the photography to be projected onto the recovered geometry in any way (and any way to look at the geometry apart from the photography).
Google would do well to tour all the contributed panos at a location, rather than leave it to the user to discover as I myself looked around for a while and after trying the different panos on the map and seeing no spatial transition between them (another area for improvement) I finally clicked the thumbnail in the lower right corner that leads to the next set of panos (which doesn't exactly help users discover everything at the current location).
Hello Nate, no hard feelings ;-). I guess you have been exploring the Photo Sphere Constallations of Alexis Jemus https://www.google.com/maps/views/profile/101388107725448550582?gl=ca.
Have you noticed these white chevrons (hooks) in a photo sphere? Same as in Street View. These symbolize gateways to other photo spheres. Orient a sphere so that the chevron points straight ahead and then click it. It will give you the best transition experience.
Right, there are spheres on Google Maps which are not part of a constellation. The distinction between Street View and the crowd sourced views are unclear too.
Please note that Photo Sphere Constellation experience will differ when accessed from Google Maps or the creator's Google Views page. The above link points to a creators page. On a Views page there will be single Spheres and Constellations of Spheres. Constellations can be recognized by a symbol in the top right of the thumbnail. After clicking it, the Constellation opens. One can navigate from sphere to sphere in the constellation by clicking the chevrons. In the lower left of the sphere view you will see a map area. The red dots in this map represent the spheres in the constellation. One can click a red dot to access that sphere in the constellation.
Relevant to our discussion here, might be the discussion I started in the Google product forum http://productforums.google.com/d/msg/maps/W38APAvvzPY/Ax5hqRge4RIJ
Also referred to in the Google+ Google Maps Views community https://plus.google.com/113333350094278871459/posts/fLPHgpGbfn6
Ah, I forgot. To access spheres on Google Maps, enable Browse Street View images. Blue dots represent spheres. It is unclear where there are Constellations. Or,open the Views bar at the bottom, showing thumbnails of photos, spheres and Photo Tours in the view port. Hovering over them will show the position on the map. Clicking them will open.
Yep, once I looked at the constellation you linked to again this morning in Opera, I saw the ellipse on the ground and the chevrons.
Part of the problem is that in IE 11, the ellipse and chevrons do not appear to render, at least on the laptop I'm currently using.
I also like to use the cursor keys on the keyboard to turn and advance or retreat, depending on which direction one is turned to and where surrounding panoramas are.
I actually love that the panoramas uploaded are linked to Street View panoramas, but once I'm into a Street View pano, I find that the chevrons don't appear to lead back to user contributed panos very readily.
It's a bit vexing to me to not be able to move back and forth between a Street View pano and a Constellation pano repeatedly via a simple keyboard command to better see how they match.
Bing did show linking user content (panos + a walk) and Streetside nearly three years ago: http://bit.ly/rwwlinking I'm hoping that will return.
Yes Nate, Google mentions that it is possible to link up a user Sphere to Street View and thereby a whole Constellation. But it also mentions that navigation is from the user content to Street View only and not from Street View to user content. I do not like that either.
What I like even less is that they are heading for full spheres and will turn down all that is less than a sphere as soon as they manage to get the acceptance process polished for it. The reason given by one of their program managers is that they feel Map users are used to Street View's ful Spheres. I feel a clear distinction between Street View and crowd sourced content is better. A Sphere could be watermarked with the contributors credentials. I feel that the crowd has much material in their archives which might be used to augment map Experience, but may not be full Sphere. I see no reason to rule it out.
For example Photosynth material see http://photosynth.net/discussion.aspx?cat=00581351-82d8-438d-a37b-7eadb3fb4991&dis=9ebf8cea-8684-4508-86a8-a5b9c5b632cf