Get all your questions answered about our latest Photosynth Technical Preview.
I love the new Photosynth but also HATE IT!
Details are critical when doing your Photosynthv2 so please plan! I've wasted hours and hours testing. Here are some details to help you not fail.
Walking - Walks
Time Lapse Apps = 1 photo per second at a rate of one-two feet per step (1 Yard is too much) You must speed up your frames p/sec or slow down. In other words while walking at a normal pace you'd need 2-4 photos per second. This is painfully slow if you want to do a large area.
Walking - Walls
Time Lapse Apps = 1 photo per second at a rate of 1/2-One foot per step (1.5-3 feet is way too much I've found and I failed often)
Driving - Walks
Time Lapse Apps failed %100 - While driving you must have closer to 3-10 photos per second at a speed of 15-50 miles per hour. Walls failed 100% after only a few frames. A must be 7-15 frames per second.
Flying/Skydiving - Walks
I've not tested yet but need help with an estimate on the frames/sec I'll have to reach
I also need help with equipment or software that can take more frame/sec than time lapse which stops at 1/sec.
I would like to use this for skydiving and in a plane but need more information.
Can a GoPro do more frames per second that time lapse without making it in to a video? or Are there any Mobile Apps which can?
If I have to can I take it in video format then pull out every so many frames. This would be a pain to do and I assume wouldn't work.
How were the Photosynth's over the mountains done and with which equipment?
You can see my examples and failures here I've deleted a lot of these that didn't work. Over 1200 photos worth.
Other Important Information
It's important to keep the camera in the same x/y and rotation in each frame. any movement is felt by the viewer. You shouldn't do this just walking I've found. I think the best options seems to be a tripod on wheels with a camera mount that doesn't move.
Mounted to a car, bike or other stable object would also work.
Hey -- thanks for these great tips. I agree with you about most of these, and in particular, having a high-end camera (so that you can set shutter speed at 1/500th of a second and *keep walking* as you click 1-2 times per second) is the fast and pleasant way to capture a walk. With a lower-end camera or a phone, you really need to pause for every shot, and that certainly becomes tedious for anything more than 10 shots.
Just some thoughts from one long time synther to another:
:; Your emphasis right now seems to be focused solely on capturing synth photos only at the speed at which you would otherwise be moving through the scene - perhaps not speedwalking and certainly driving slower than you normally would, but still in pretty constant motion.
I suppose that I can see where you're coming from, but as far as the typical photographer is concerned, this is shooting quite a bit quicker than normal. It does take time to get your coverage but I don't find it terribly taxing.
You're going to be fighting motion blur if you insist at shooting at this speed, as the Photosynth team points out above.
Unless you've got a higher quality camera (and have plenty of light or a whopping big sensor with correspondingly large photo sites) you just won't be able to capture enough light fast enough to avoid blur and that will lead to less than optimal matching.
:: As for flying and skydiving, my instinct is that as long as you have a good view past the fuselage and are up sufficiently high that your photo rate won't need to be firing all that quickly if you're just after the ground below you.
Your field of view is going to be wide open, so assuming that you're shooting with a lens that takes advantage of that, as long as you keep pointing in the same direction during freefall, things ought to link just fine, I'd think.
The nearer you are to a subject, the more often you need to fire, meaning fewer photos at higher altitudes.
The same would apply to shooting a wall synth moving down a street and slowing to get good overlap on trees or signs which you want to be reconstructed.
If you're shooting aimed at the horizon, though, while moving forward, rather than down (and especially at lower altitudes), then yes you'll need to fire quite rapidly to get adequate overlap.
You might want to keep an eye on UAV and KAP synths.
:: I asked twitter about time lapse apps today. Not too many replies at the moment but from what I'm told you're probably going to want a dedicated camera rather than any mobile app.
I knew that the Microsoft Research guys made Blink, but from what I can tell, there's no easy way to export all of the captured stills to your camera roll, so it's not really suitable for synthing purposes.
Chris ( http://photosynth.net/userprofilepage.aspx?user=christo27 ) suggested ProShot https://twitter.com/CB_27/status/417825309942358016 | http://bit.ly/winphoneproshot which boasts a 5 frames per second burst mode.
I talked a little to the ProShot devs, though, and it maxes out at 15 consecutive shots (3 seconds of continuous 5fps shooting) and each shot will only be 5 megapixels apiece due to current hardware limitations. https://twitter.com/natelawrence/status/417826380793933825
:: As far as GoPros' cameras' still shot burst modes http://gopro.com/cameras claims the following:
$200 buys you the HERO3 White Edition = 5 megapixel shots @ 3 frames per second
$300 buys you the HERO3+ Silver Edition = 10 megapixel shots @ 10 frames per second
$400 buys you the HERO3+ Black Edition = 12 megapixel shots @ 30 frames per second
There's no explicit mention of a limit on these shooting modes. I would hope that because they're primarily video cameras that they could keep up the pace until the memory card is full, but that's not explicitly stated.
:: As far as just taking video, I know that the Photosynth team has always discouraged it because of the lower resolution of video frames and usually the much lower bit rate (compressing the details out of the image even if it is technically 1920 x 1080 pixels) but that might not be a bad option for you to at least explore and see whether the motion blur will be too big a detracting factor.
It really oughtn't be difficult for you to convert your video to an image sequence with QuickTime Pro, VirtualDub, FFmpeg, or something similar and specify the frame rate you want to output highest quality JPGs at.
The other alternative is to simply export all of the frames to the highest quality JPGs and then use the view options in Windows Explorer / File Explorer / My Computer to arrange the folder of JPGs in the number of columns or rows of thumbnails or list view that gives you the desired number of frames over a specified period of time + then select a single column/row.
:: As far as using a tripod to shoot walks (well, any of the synth types, really), when you're allowing the synth to play in slideshow mode, it really doesn't matter because the photos are always presented at a "World Up" orientation, but yes, as soon as you pause the slideshow, the viewer window will align the top of the browser to the top of the photo, so if the photo was taken off level, then you will feel this shift every time that you stop scrolling through the synth.
From my perspective, the original Photosynth viewers kept the scene oriented "World Up" by default instead of defaulting to "Photo Up" every time you stop moving, which is just... vastly superior in my eyes.
The reason why things are currently the way they are in the regard of switching to "Photo Up" every time you come to a stop is a technology limitation because the Seadragon team was disbanded over three years ago now and this new viewer is being written from the ground up using many third party technologies, including an open source clone of Seadragon that only works in 2D, rather than 3D as the original did.
You can read an earlier discussion with me and the Photosynth team on the issue here:
I was talking to the OpenSeadragon guys about 3D support for this very feature at the beginning of 2013 because of its importance to the quality of the playback. https://github.com/openseadragon/openseadragon/issues/68
Photosynth is solving everything necessary for high quality video stabilization, so if the viewer does its job a tripod isn't necessary.
:: Going back to video for a moment, I've made some synths from some 480i aerial video shot of my childhood neighborhood and it synthed reasonably even though it is far from the ideal test imagery, so I think that anything you capture with a GoPro should do just fine as long as you have a good amount of frames where each part of the scene that you care about is seen more than 3 times before it moves out of view and the path is continuous without any significant breaks in time or position..
By the way, you asked about how the photosynths over the mountains were done.
If you were asking about David Breashears' Everest synths from around Mt. Everest for GlacierWorks http://explore.glacierworks.org/en/#everest/synths the Photosynth web log mentions him "rig[ging] a high altitude helicopter with an array of full frame cameras".
Here's a talk he gave last year: http://research.microsoft.com/apps/video/default.aspx?id=171318
David Gedye from Photosynth also has his own synths of Mt. Rainier and Mt. St. Helens and he tells a little about the shooting of them in the description when you pull up the Info pane: http://photosynth.net/preview/view/3c9b1418-2d6c-4f35-b3e8-053227e54efc
Great just what I wanted to hear. Gopro black should be here tomorrow and ill start testing with cars then plane.
This is a GoPro aerial shots taken every second http://photosynth.net/preview/view/ad1291d2-675c-4a52-afd2-f5ea12255fe5
I am very happy with the outcome. Looking forward to playing more.
I was trying to find a quote from David Breashears about the new synths when I was writing my reply the other night and couldn't remember where I'd read it, but it was on the Bing web log entry about Photosynth's preview: http://bit.ly/thenewphotosynth
It doesn't give much insight into the shooting other than that it was done from a helicopter, though, so the Photosynth web log entry quoted above is actually a bit more informative.
I was watching Larry Larsen's video interview with David Gedye again today and thought that this comment on shooting frequency for walks was relevant to this conversation: http://channel9.msdn.com/posts/A-Photosynth-2-Overview-with-David-Gedye#time=03m22s
The entire video is informative, though.
My small scale tests and fails. http://photosynth.net/preview/users/DontPanic but learning.
The motion blur has been a real pain, the video exports failed due to PS not being able to find focal point, which I know on the go promise 3mm but I haven't figured out how to manually change it. I had to clear the date taken. Its really easy to tilt more than 2degrees and mess stuff up. Why can't PS fix motion blur if it knows the images before and after, date taken, 3d path and others, any other ideas for fixing or helping.
DontPanic, after looking through some of your synths, it appears part of the problem is we do not yet have undistortion parameters for the Hero3+, just the Hero3.
From some initial experimentation, it looks like these parameters might be the same as the Hero3 (or at least very close). Compare the following.
We'll continue to experiment and hopefully add these parameters in the next update.
Wow, much better, could you please re-process my others. I spent hours reshooting these due to blur and many failures. Video exports had the worst failures which I understand was also due to blur. Is there any options to get earlier or increased beta access to the processor maybe a connect.microsoft. You can check my profile for my many years of testing and participation.
These are just small scale tests so I'm building the project in my head as I shoot it. I've found I do the turns wrong on walks.
My concerns are I will have access to a plane and some once in a life-time events that are high expense, high risk of failure and not easily repeatable so I want to make sure they look as good as they can. My concerns with not using strait video for capture of the event is the need for that missing frame or two we didn't get or motion blur in what we did. I totally understand this isn't the intended usage and it's all a research project so I appreciate your help
I also like that yours is in 16:9 or whatever and no side bars. It looks much better.
Also when I do export videos I'm using adobe prem elements, or the adobe encoder in cs6. I tried media encoder but couldn't find an jpeg export. What are my best case settings? Can't get time taken, how does PS want files named.
Any feedback from PS team on this.
While driving a car, a gopro Hero3+ set to timelapse at 1 frame/.5secs going 20-30 mph seems to work pretty well if you're doing scenery (not a up-close forest) but mountains or distance. When you get to close terrain it seems closer to 10-15mpg works better.
This doesn't apply to roundabouts which I've failed at so far.
Sorry Dontpanic, what's the question for us?
@Photosynth Team, here are Don't Panic's questions that I'm seeing for you:
1: Would it be possible for you to reprocess all of his GoPro Hero 3+ synths with updated undistortion parameters as you did in the test above (so that he can avoid the duplicate uplods, presumably).
2: Is there any possibility of getting deeper access to the beta (perhaps advanced options for the synther) - potentially through Microsoft Connect, since Don't Panic has a history of testing there.
3: Do you have any advice for shooting something expensive and not easily repeatable (such as a skidive) with GoPro? Specifically:
a) Would you advise shooting video over burst mode stills for continuity or would this introduce too much motion blur?
b) Is there something to be done to avoid motion blur even when shooting stills in this situation?
4: To achieve the appropriate photo aspect ratio as seen in your resynth, what software and/or settings ought he be using?
You should really hire Nate and I to travel and take photos...
Here's one that looks amazing but failed half way, I'm not sure if I should spend the time resorting or what.
Stupid spell check, reshoot.
Here is a 5-10MPH GoPro mounted with a timelapse of 1/.5sec - looks pretty good.
---Photosynth Team could you batch my entire collection to fix any Hero3+ issues?
Please check how many I've done and have a lot more coming.
We love your stuff. We'll be releasing a small update to support GoPro Hero3+ as early as next Monday if testing goes OK tomorrow. We don't have an automatic way to reprocess all your synths, so you'll need to re-upload the ones you care about. Will post on this thread when the update is live.
http://photosynth.net/preview/view/7503e7a0-2fcb-49cb-8991-3b316f10b414 - did good, but this on http://photosynth.net/preview/view/ec007661-0122-4c95-b594-d067d094bca6 had an issue I don't know why.
Also could we mark on the Bing.com map the route we took to help the process or match up the content? This would be very easy to do even with my phone while doing the shoot. I'm still using the GoPro here but I assume location information could be provided.
Oh my, I just uploaded a mass of content, any estimate on when a management interface will be ready, the current preview system is going to be hell to manage all this and keep it strait. I currently have 5 or 6 stuck waiting for processing that I can't clear out.
It's hard for me to know which ones would be helped by redoing. Could you give me an idea of which onces you think might do better.
I have a high failure rate on video export content at 5 frames/sec is that too many photos?
Here are some examples I can't figure out why they fail with so much overlap.
Not video exported but still had issues
OK. Yesterday we pushed out a new lens configuration file that now specifically handles the GoPro Hero3+. Please rerun your most interesting sets and tell us if this improves things.
Would very much like to see this one reprocessed... http://photosynth.net/preview/view/ec007661-0122-4c95-b594-d067d094bca6
Are the GoPro3 changes working for everyone now?? I'm seeing issues with grass quite frequently should I up the photo rate?
Doh I should have read the comment two up, sorry trying a new one right now.
Wow there seems to be quite a difference, I like!
This last one looks great, drone or plane? Video or photos?
So where all of my Grand Mesa National Forest before or after update?
@DontPanic drone and photos
@DontPanic. We have no bulk-reprocess capability yet. You'll need to reprocess them yourself if you want to see what improvements the updated lens model provides.
Loved your reprocessed "downtown Grand Junction" http://photosynth.net/preview/view/2e62b41f-359e-4f61-b887-e9674c1347e6
@Photosynth Team, are you saying Don't Panic's Grand Mesa National Forest uploads from January 21 were processed prior to the update?
@DontPanic. I'm thinking of a blog post about synthing from GoPro images, and I'd like to feature that Downtown Grand Junction "wall" I just referenced. Can you tell me roughly how fast the car (I assume) was moving, what your shutter speed, and frequency of shots was? Thanks!
I think Mike on our team hand-reprocessed one of these sets for DontPanic. Not sure which one, but our update to include a lens model for the Hero3+ went live late in the afternoon (Pacific) of 1/20, so that should be the dividing line.
I'd love to help anyway I can.
I was going 5-10MPH goal was 8 MPH, I used time lapse mode at 12 MP, .5 second per frame in GoPro's settings or 2 frame/second technically. I used a suction cup mount rated for 150MPH great device.
You can use a slower MicroSD card like 40MB but for 2K+ video you must use a High Speed MicroSD card something like 80MB/sec.
South Main street
Going East on Main street
Grand Mesa National Forest
Thanks! Do you know what your shutter speed was? Your images look really crisp, so I'm expecting something like 1/1000 of a second.
I've never seen anything on the GoPro for shutter speed at all. So If I'm wrong and some knows it there please let me know, but at this point I think it's hard set and can't be changed.