Discussion about the now-retired Photosynth Apps for Windows Phone and iOS.
Hi, i was checking some other photosynths and saw some with over 50mb of data. It can zoom very well and you can see details from far objects, but i'm trying to do it on my iphone and all my photosynth have +/-5mb
How could i make the higher resolution one?
The measurement that you see is actually megapixels, rather than megabytes. 1 megapixel just means 1 million pixels and when you're dealing with jpg compression, the file size (what megabytes measures) will vary wildly, based on the subject matter.
To stitch and upload high resolution panoramas to Photosynth, you need to use one of two programs to do the stitching:
1: Microsoft Research Image Composite Editor (ICE):
2: Adobe Photoshop for Windows + a panorama publishing plugin from the ICE team:
To upload the panoramas from either of the above apps, you'll also need Photosynth's app for Windows, available here: http://photosynth.net/create.aspx
I really need to app to have the options for high q. I did a test with the app and without and with the app my image was 6.9MP without same phone it was 109MP... I need to be able to either set the app to HQ or the app needs to same every image used so it can be reprocessed into HQ. I also need those photos for later pre-possessed.
I agree, I miss the shooting of high resolution images. I am willing to wait longer while they are stitching and while uploading. When seeing the result on a desktop then I don't want to compromise on resolution since, as the panoramas look now, I prefer normal images. The resolution is much below my camera's resolution. Publishing such photos - with the need for an additional plugin (Silverlight) - makes no sense to me. What about using the full potential of the technologies we get?
I agree too. I want an option to shooting higher resolution 360 degree panorama. I know that there is a limitation in memory required for processing full resolution image from the phone but if it is capable to stitch higher resolution than 5-6 MP, maybe 10-12 MP, would be nice.
1) I guess it would be possible to pre-stitch lower resolution images on the mobile device to get a preview that is optimized for the smaller screen and lower computing capacities. But in the background, the originals persist and will be transferred to the server in full resolution with the transformation and composition metdata, so that, on the server, a full-resolution panorama can be synthed with the help of this precomputed data that does not need to be adapted that much.
2) Additionally, Silverlight as a necessary plugin for viewing should be removed to watch the panoramas. This can be done with W3C standard compliant APIs like canvas now.
Microsoft, do you hear us? We love this software!
What you need to understand is that the bottlenecks in getting Photosynth's mobile panorama app to create high resolution panoramas are:
1: the shooting mode that makes the app so popular is running 3D graphics, doing realtime image feature extraction and matching, keeping all shots captured so far in RAM to match to the video feed of the viewfinder, and doing this all in a freeform manner that allows you to shoot up to a full sphere in any order you want. This limits the resolution of the camera video mode being used by the app because there is so much else going on on the phone.
2: the speed at which full resolution images can be written to your phone's internal storage memory is too slow to keep the viewfinder video and 3D graphics running constantly and smoothly. If you pay attention to other apps which take full resolution images, they pretty much all black out the viewfinder video for a moment while saving the image. This would ruin the app's shooting mode.
In case it wasn't clear from what I said above, the reason that the final panorama is not high resolution is not because Photosynth's mobile panorama app is taking full resolution shots with your mobile device's camera and then shrinking them and stitching the smaller photos or alternately stitching the full resolution photos and then shrinking the final panorama (as there would be no sense in that).
What is the case is that the app captures lower resolution shots to begin with, due to hardware limitations.
If you'd like to hear this from the Photosynth team themselves from back in 2011, read this short Twitter conversation: http://bit.ly/psmpr
Photosynth's mobile panorama app originally debuted with the iPhone 3GS being the minimum hardware that could run it.
The iPhone 4, 4S, 5 as well as Windows Phone 7 and 8 added the capability for using a gyroscope to allow the placement of shots where the image matching doesn't have any image features to match (such as cloudless blue sky, blank white walls, etc.) but although they had more RAM and faster CPUs and GPUs, we haven't really seen the app scale up to this better hardware.
The bottleneck may remain in the speed of getting high resolution imagery from the camera sensor to internal storage in a fast manner while keeping the shooting mode running smoothly.
The limit is not spending more time to stitch higher res output like AutoStitch but rather finding a way to take high resolution input shots while preserving the app's interactive shooting mode's experience.
Beyond the stitching there is also the perspective correction for spherical projection and transforming that into a cube map. In ICE on Windows, this can be incredibly memory, power, and time intensive on higher resolution panoramas and ICE is programmed for an environment with considerably more memory than our phones have (in addition to unlimited electricity). I'm sure that Photosynth doesn't want to create a scenario where to complete the stitching of your panorama, you need to plug your phone into the wall so that it doesn't run out of power mid-stitch.
This is simply food for thought. Occipital's 360 Panorama app uses a higher resolution video mode because their shooting mode is more limited to strip panoramas and you are confined to: start shooting and pan in one direction until you complete a loop.
Apple's built in panorama app in iOS 6 is very much the same sort of thing, only even more limited range with only flat results.
In reply to your second point above:
"2) Additionally, Silverlight as a necessary plugin for viewing should be removed to watch the panoramas. This can be done with W3C standard compliant APIs like canvas now."
The Photosynth team did create a first version of an HTML5/CSS3 panorama viewer for mobile Safari back in the summer of 2011, although it lacked features of their Silverlight panorama viewer such as loading higher resolution tiles on each face of the pano's cube map, displaying highlights, loading 'Share' link virtual camera coordinates, etc. http://photosynth.net/discussion.aspx?cat=c76bfe71-d042-4aec-b49b-b48b67d7be96&dis=23657389-3f14-4585-aac7-1eb8d392b017
Now that Internet Explorer 10 supports CSS3:3D Transforms, Photosynth's HTML5 panorama viewer also works on Windows Phone 8.
You may also be interested to know that in 2011 Photosynth team members also worked on building viewer tech in multiple graphics technologies as a part of Bing's Read/Write World project (which went dark in early 2012 - videos from the team talking about what their vision was available at http://bit.ly/readwriteworld ).
One sign of some of that work is the HTML5 pano viewer on Bing's homepage every two weeks or so - both times that I saw one, featuring 360 Cities panoramas.
David Gedye from Photosynth has also said on Twitter that they are working on rewriting their viewer in HTML5.
The important thing is that the panorama viewer not lose features, quality, or performance when switching from Silverlight to an HTML5 viewer, but their architect, Blaise Agüera y Arcas also said in 2011 that they were pushing toward HTML5 as a single point where they could pour their efforts into to get the most bang for their buck, as it were. See his comment on his blog post here: http://styleisviolence.com/photosynth-app/#comment-341
Other things to consider in the move to HTML5:
:: Photosynth was not originally about panoramas at all, so I doubt we'll see the site switch to HTML5 for the default viewer until photosynths (in addition to panoramas) can be rendered acceptably.
:: Photosynth needs browsers to support CSS3:3D for their HTML5 viewers, most likely, and being Microsoft they probably don't ship them until most Microsoft customers can use them. Internet Explorer 10 has not yet shipped for Windows 7 and will never ship for Windows Vista or Windows xp.
Thank you, Nathanael for this great amount of information. It was clear to me that it would slow down Photosynth to save high resolution images in the background while stitching a panorama for a smaller screen in the foreground. I guess it will be possible with stronger mobile hardware to save full (or at least higher) resolution images in the background while (nearly) continuously shooting images in the foreground. The problem for me is that the great interface looses its value compared to a simple application that takes a longer time and results in a high-res panorama (whether it's computed on mobile, desktop or server). For me and other panorama shooters the quality of the end result is more important than the shooting experience, although it helps a lot to position the shot. As I said, I find Photosynth tremendesly inspiring from the point of view of its strength but to make use of the great viewer, I want to be able to zoom in and see more details.
So it's about saving higher (not full resolution images in the background) if you have stronger hardware (think of dual core processors) while computing a smaller synth in the foreground, after that sending the higher resolution images to the server to recompute them with the same metadata for the desktop. I am willing to wait to get great results. Take a look at KitCam that allows you to continue sooner with shooting by processing the saving in the background. Else I will make single shots and stitch them together manually on my desktop by circumventing Photosynth just to get full resolution.
It's clear that W3C APIs do not work in all browsers and Silverlight can stay as a fallback solution.
"What is the case is that the app captures lower resolution shots to begin with, due to hardware limitations. " -> This was pretty clear to me and makes sense. See my proposal above to do the computing two times, the second time based on the meta data of the first.
The higher resolution images are just saved and uploaded separately afterwards and will be computing input only on the server. I guess this does not need that much computing power. (You see I am idealistic ...)
No, I agree with you.
During the time that the app was out for iPhone and those of us who use Windows Phone were still waiting for our version of the app I made the case many times that you get much better results by just using one's own camera app and using ICE on Windows proper.
If you look at that Twitter convo that I linked to above, I'm also the one that asked Photosynth back in 2011:
"for your mobile apps, I request an option to use it only for shooting purposes, export full res shots, + stitch on desk/laptop." (which is close to what you're asking - process in the cloud or during a longer stitch on-device)
to which the response from David was:
"Good request, but harder than it sounds.: Writing full res to flash memory while smoothly tracking is tough."
And that led me to subsequently label the mobile pano app as a great pair of training wheels for panorama newbies, but not appropriate for anyone who cares about end results, much to my chagrin.
Thank you for you great information Nathanael. I know shooting full res to flash memory while snoothly tracking is quite hard for old phone hardware with entry level processor and little working memory. But I wonder that modern hardware smart phone can get a little bit higher resolution as the new cpu with multi cores and flash memory seem to work faster as technology go on. Photosynth mobile is a very great Apps for 360 degree panorama. It free me from packing up ton of panorama shooting stuffs around with me. It stitched all images automatically without spending time adding stitching points like desktop pano program. I love it and now I change from iPhone4 to iPhone 5 not the Samsung Galaxy S3 or Note 2 because I can use my hand to take a 360 degree panorama with it. I hope that the Photosynth team can make this great Apps more suitable for higher res 360 panorama. I don't want full res but I want a bit more if it's possible.
just my two cents..
is there no way for us mobile users to take the pics and then upload the pictures to photosynth where its stitched on the server thus maintaining a better resolution and not relying on the mobile tech to stitch. The only downside is uploading heavy images over data if no wifi is available. which, could be counter-acted by a setting to not stitch immediately and upload when you want to, like when wifi is available.
so, take pics > upload > server stitch > view/download completed photosynth.
If you take the photos with your phone's regular camera app (or just a regular camera), you can take photos for a *panorama* (that's what Photosynth's mobile app makes - not photosynths) then you can use Microsoft ICE ( http://bit.ly/microsoftice ) or Adobe Photoshop ( http://bit.ly/pstops ) on Windows to stitch a full resolution panorama and upload it to your Photosynth account.
I know that the above information is long, but the upshot is that when a phone is running Photosynth's mobile app's shooting mode, there is so much processing being done to track and align the images in real time that it is not possible to capture higher resolution input shots while keeping the viewfinder continuously on and smooth.
Like others in this thread, I would have expected the mobile app to increase the resolution of the input shots captured on WP8 since WP8 handsets have better CPU + RAM, but regrettably it doesn't seem like it has.
In other words, the choking point is not primarily the mobile stitching tech or the amount of RAM on phones compared to Windows PCs (where Microsoft ICE runs, for example).
The choking point is writing higher or full resolution images to the mobile's RAM while keeping the camera continuously on and keeping all captured shots thus far aligned in the viewfinder.
At present, the only two options available to use are:
1) Shoot high resolution input shots for a panorama manually with a regular camera or your mobile's camera app.
Stitch them on a Windows PC with Microsoft ICE or Adobe Photoshop for a full resolution panorama and upload it to your Photosynth account.
2) Shoot low resolution input shots with Photosynth's mobile panorama app with a fully interactive shooting mode that helps you remember what you've shot + what you still need to.
Stitch them in Photosynth's mobile app for the convenience of mobile stitching, uploading, sharing, and viewing.
If I am understanding David Gedye's statement correctly, then the only real solution for both:
1) using an interactive shooting mode like Photosynth's current mobile app uses and
2) saving high resolution images to then stitch (regardless of where they get stitched)
is to build dedicated hardware into all Windows Phones that is specifically there to handle video processing without impacting the CPU/GPU/ and has access to its own dedicated flash memory which is of higher speed (if necessary) than the phone's main flash memory.
Think of this, similar to how the Playstation 4 has dedicated hardware built in for Gaikai's video streaming.
It's just going to take the Windows Phone team waking up to the reality that they need onboard dedicated video processing hardware and insisting that all hardware manufacturers provide it for WP8.5 devices in order for a new generation of augmented reality apps to be written.
Nathanael: There is a 3rd option, I have a lumia 822 - 8mp camera and 64gb 2nd drive which I store all images. With WinPhones we all have a skydrive account to sync our images to, I have a folder "Pano" that I sync to from the phone to skydrive and also to my home Win8 desktop. I take a 3x8 array of images and want to stitch them. So from my phone I would like an app ("ICE") that would look for files in the Pano folder, stitch them (with standard settings) and send an image back to my phone to either send to Photosynth.net or discard. If skydrive will not allow processing of images on their servers or the photosynth group would not want the same, then I would want to have the app interface with the same images on my win8 desktop.
When I say send image back to phone, I mean a low res equivalent of the stitching to review for quality of stitch, not the entire pixal array, then if acceptable the final full pixal stitch would be uploaded to Photosynth.net.
What do you guys think about a hybrid approach: Low res stitch on the phone, but upload full frames and get a hi res stitch done in the cloud a minute or so after upload? Note that hi-res uploads would be too big do over 3G, and probably bigger than you'd want to do over 4G, so the scenario is more like: upload later when you are on Wifi. Thoughts?
@David, I'm definitely game, if that's technically possible for you guys to do now and you're willing to commit to the server side compute.
This would move the mobile app from being useful as a pano viewer to useful as a serious amateur's pano creation tool.
I must say that I still find single panos to be less than compelling and would like to see steps taken toward true photosynth style linking of nearby panos and projecting them onto recovered geometry from nearby synths or even stereo pairs of panos, but I'll take what you're able to give us.
The hybrid approach, if achievable, would definitely make this a better panorama product. I personally do not bother to shoot individual shots and use ICE, as I am often in a rush when taking photos and the photosynth auto capture method is really helpful.
Another way I have increased panorama resolution, when on holidays, is by taking wide panoramic video sweeps and use ICE to stitch into a panorama later on. This has the advantage of also having a video as a keepsake, but has the disadvantage that it only works in one direction, not 360.
Hi, I use photosynth to create panoramic shots with my iphone, however, although each photo on the iphone is usually 2mb, the panoramica photos created are only about 400kb and therefore offer minimum level detail (if I were to print the photo, the quality and size would be very low). I did not find a setting to change it to a higher resolution. Am I a missing somtething that can help me get the best quality panaromic shots possible?
I have an iphone 4s and an iphone 4.
No, at this time Photosynth's mobile panorama still only captures a very low resolution images from the video stream that you see in the center of the screen when capturing.
Stitching these low resolution thumbnails into a panorama results in a low resolution panorama.
This was originally done because of limited RAM on phones.
The Photosynth team is currently working on Photosynth 2 which you preview via their beta website at http://photosynth.net/preview/
After they get the website finished, they will be updating their mobile app to shoot full resolution photos to upload to the Photosynth 2 website to create one of the four new types of photosynths.
You can learn about Photosynth 2's four new types at http://photosynth.net/preview/about and learn how to shoot them at http://photosynth.net/preview/help
I do not know what the Photosynth team's plans are for shooting stitched panoramas with their new mobile app when it is finished.