Topic: 3D Modeling from Photosynth: FOR NOOBS

Report Abuse Report Abuse
michaeldenis (Over 1 year ago)

pretty amazing stuff, right?

Now, how does a n00b like me get it done?  Stay around to find out..
michaeldenis (Over 1 year ago)
Nathanael wrote,

"As far as a tutorial, the closest thing that comes to mind is Josh Harle's web log entry here:

Also, feel free to ask the crowd over at Ólafur Haraldsson's for pointers.

Be aware that to perform dense reconstruction of great detail or a large area (covered by hundreds of photos, that is - 20 photos from the air of a large area/object naturally won't pose a problem), you'll need a 64-bit operating system to support a large amount of RAM and you will, of course, also need the large amount of RAM. The alternative to using all the cameras at once in PMVS2 is to split them into smaller groups via CMVS, but so far CMVS only works with Bundler output, rather than Photosynth output and even Josh and Henri's toolkits which simplify Bundler usage considerable have yet to succeed for me."
michaeldenis (Over 1 year ago)
To address the last paragraph first, I have 4 gigs RAM right now on my Windows 7 64 bit build.  How much more RAM should I get?  Would 10 gigs be good enough?  Would 16 gigs be best?  What would you prefer to have?

Ok, my specs aside, all this talk of PMVS2 and CMVS is foreign to me.

I'm watching Josh's video right now.. Thanks
Nathanael (Over 1 year ago)
Right. Henri's PhotosynthToolkit actually includes Pierre Moulon's Windows port of PMVS2, which is what you linked to (the original PMVS2 was written by Yasutaka Furukawa while he was at University of Washington to work with Noah Snavely's Bundler). Henri actually modified Pierre's port to develop a 64-bit port of PMVS2 which is what is included in today's PhotosynthToolkit and SfMToolkit.

Just so I'm being clear, if you've used PhotosynthToolkit, you've used PMVS2 via the scripts that make up the PhotosynthToolkit.

If you'd like to understand PMVS2 settings a bit more that adjust what resolution it will look at your photos at, etc. etc., see the original PMVS2 documentation here:

There's also a brief restatement of them in this thread:
Nathanael (Over 1 year ago)
The basic concept is this: you use a bundle adjuster to perform a sparse reconstruction of the scene - figuring out just enough to determine the positions of the cameras. This is what Bundler, Photosynth, and 

other photogrammetry programs do.

After you know the positions of the cameras and have figured them out in the most efficient and fast way possible, you can use other techniques (multi-view stereo) to determine much more of the scene's 

stationary geometry without needing to spend any time during the process figuring out where the cameras are. This is what programs like PMVS are designed to do, although PMVS2 still only outputs point 


See these two videos for explanation from Blaise:
(Original Kelvingrove point cloud viewable here: )
Nathanael (Over 1 year ago)
Given a point cloud, you can transform it into a mesh and then project the photos onto the polygonal surface, although this goes beyond what PhotosynthToolkit provides. Greg Downing of xRez Studio and 

Josh Harle have both demonstrated using Photosynth output to create a high resolution panorama stitched from photos taken from completely different positions (which transcends what traditional 

panorama stitchers have been able to cope with). See these topics for more details: 
>> Greg's demonstration: 
>> Josh's demonstration: 
Nathanael (Over 1 year ago)
Kean Walmsley of Autodesk (who wrote a Photosynth importer for AutoCAD 2011 last year) has also shown a sneak peek at Autodesk's upcoming update for their experimental Photo Scene Editor:

Naturally, the more dense a point cloud you can recover, the higher quality your subsequent meshes can be, therefore although it would be possible to create a mesh from Photosynth's sparse point cloud, create a mesh with it, and texture it, it is far better to use something like PMVS2 to recover as much geometry as possible before meshing. That said, Photosynth is one of the best bundle adjusters around as far as calculating the cameras' positions.
Nathanael (Over 1 year ago)
Where things break down is that multi-view stereo wants to load all the parts of the photos which are shared by them into memory to compare at the same time (I'm not positive on the specifics of how it operates, but certainly the idea is to compare all the photos at once, rather than comparing each possible pair of photos in a set of photos, but only two at a time, as Photosynth and Bundler do). If you've pushed Bundler or Photosynth to the limits of sparse reconstruction, it's a safe bet that that same set of photos will fail dense reconstruction on your machine.
Nathanael (Over 1 year ago)
Yasu must have encountered this frustration, himself, as he developed CMVS to slice Bundler's camera system into smaller chunks so that PMVS2 could work on less data at a time which in turn allows you to finish the job with your same amount of RAM - just one piece at a time. Since the positions of all of the pictures in the small groups of photos that CMVS chooses come from the same Bundler solution, all of the PMVS2 dense point clouds should fit back together perfectly to then view in a point cloud viewer as a complete single object.
Nathanael (Over 1 year ago)
As I mentioned above and as Josh Harle and Henri himself have pointed out is that, for now, using a Photosynth to PMVS2 workflow (as PhotosynthToolkit does) puts more of a limit on how much you can do in a single pass than with a Bundler to CMVS to PMVS2 workflow such as Henri's SfMToolkit and Josh's Bundler Photogrammetry Package. That said, Henri did create a 64-bit version of PMVS2, so as long as you have the RAM, the Photosynth to PMVS2 workflow remains a good option because of Photosynth's speed, accuracy, and ease of use compared to Bundler.

As to how much RAM a 64bit copy of Windows 7 can use, apparently it varies by edition. See: The comments on the article seem to indicate that current motherboards do not yet commonly support 192GB of RAM.
Nathanael (Over 1 year ago)
When John OBrien from Soul Solutions got his new computer!/soulsolutions/status/15706365309755392 I challenged him to give PMVS2 a go with it!/natelawrence/status/16392657387393025 and he reported that even letting PMVS2 examine the 143 photos from this synth: at a quarter their original resolution, it used 23 of his 24 GB of RAM!/soulsolutions/status/20383527145054209 +!/soulsolutions/status/20740721992536064 .
Nathanael (Over 1 year ago)
While we're on the topic, I should bring up, though they also caution users to submit smaller groups of photos (50 photos is pushing their recommended limit, though I did throw them a 130 curve ball). They return a polygonal model that has been textured with the original photos as seen in the upcoming version of Photo Scene Editor and Greg and Josh's demos. Hypr3D will compress your model so that the download is small for internet viewing, but if you ask them nicely they seem very willing so far to provide a download of the full quality version of your model.
michaeldenis (Over 1 year ago)
Hey Nathan, that's some excellent background, explanation, and synthesis.  Thanks.  I'll work through your work here throughout the day.

Last night, after we connected, I followed that video walk-through until I reached a step that required a lot of processing (the final Photosynth Toolkit step).  That step requires using PMVS2, and I think it failed for me.  I am not sure that I started it up correctly or "aimed it" at the right files.  Steps 1-3 of the Photosynth Toolkit are workable, assuming one has the video you linked to me previously and which I link to above.

Once again:  Josh Harle's video is clutch

Anyhow, great write-up, I just stepped in from a morning of X-country skiing, so you've given me a lot to consider.  I recognize the system-related limitations of Photosynth all too well.  Darn limits!  192 Gb of RAM!!!!!!  Yowza!  But on the other hand, I suppose we do have a demand...
Nathanael (Over 1 year ago)
=) I hear you on the RAM. I sadly only have 4GB, myself, so have been unable to push dense reconstruction like I've done with sparse. I continue to hope for official Photosynth support for dense reconstruction, but until after Silverlight gets some real graphics muscle (read: Silverlight 5 this fall), that probably isn't likely to happen.

Out of curiosity, what photo set are you testing with? Is it something that you have publicly published here on the site? My recommendation would be to begin small. 

If PMVS2 fails at first, try increasing the 'Level' variable by one number. This will use half the resolution of the input photos. The default value is '1' which is half of the original. Changing the value to '2' will make PMVS2 examine the photos at half the size as '1' - so 1/4 the original resolution and so on. If you push this too far, though, PMVS2's 'dense' reconstruction will be less dense than Photosynth's sparse if it starts using thumbnails of the photos.
michaeldenis (Over 1 year ago)
Interesting... Thanks for the "Level" tip.  I'll keep a sharp eye on that value.

Oh, and I'll struggle to find one of my photosynths that is small enough to work, but I'll go look now.
michaeldenis (Over 1 year ago)
This one, maybe:

Although at 37 photos it's huge by the standards outlined.  I may try doing it up overnight with that higher value under "Level".
michaeldenis (Over 1 year ago)
OK!  I got it to work!

Here's a n00b time for future n00bs: Use "4" as Levels in order to cut the processing time by 100x

In any case, things advance.  I just exported my first Photosynth.
michaeldenis (Over 1 year ago)
Check this out:

That's what can be done by following this process... pretty cool!
Nathanael (Over 1 year ago)
Nice work, Michael! I'm looking forward to seeing where you take this, going forward.

Also, as an extension of the talk around the 'Level' setting for PMVS2, though '1' is the default setting, changing this value to '0' will force PMVS2 to use the original resolution when generating its dense point cloud, which is useful for extracting maximum detail when using smaller numbers of photos.
michaeldenis (Over 1 year ago)
Thanks, Nathanael. As you can tell, I used a high "levels" value (4) in order to hasten the processing time.  As I understand it, the 4 translates to 1/4, meaning that the synth was at quarter resolution, which is pretty bad.

I have found myself needing to be judicious with my usage of my computer for long projects because as a one-man operation I need to move in so many directions.  Processing for four hours something that may or may not work is just ridiculous at this stage of my development.  Once I get things down or have paying clients (lol), then I'll increase the quality.

I went out skiing this morning and my friend kindly allowed me to shoot a quick synth of him.  I'm very interested to test up another one of these things.

Is Photosynth still on the fritz though? I keep stalling out at 80-90% through the process in the "Publishing Synth" phase.. :(  I may go and reprocess that "Cave of the DC" synth, which itself is only 1/25th of the entire 1200+ shoot..
michaeldenis (Over 1 year ago)
Reprocess it for Meshlab, that is... The bigger photosynth I'll need to wait on, or, better yet, process it piece-by-piece, in many small photosynths, and then combine it later in Meshlab, like a puzzle.
Nathanael (Over 1 year ago)
As to PMVS2's 'Level' setting, I was under the impression that it was more of a square root type operation, i.e.:
0 = examine exact number of pixels in images provided
1 = half width and height of images provided (1/4 number of original pixels)
2 = quarter width and height of images provided (1/16 number of original pixels)
3 = eighth width and height of images provided (1/64 number of original pixels)

I totally understand about cutting processing time while you're still working out your workflow and the tools, though.

On a happy note, it looks like our synths that were stuck in publishing got through. I see your skiing friend. 

Regarding publishing hanging, when you reach the 'publishing' phase, all calculations are complete and all data has been uploaded, leaving only a data verification and unzipping step to be performed on the Photosynth servers, so at that point it is safe to close the synther or begin new synths. Verify this with Photosynth's log file.
michaeldenis (Over 1 year ago)
Huh, that last one is a pro-tip.  :)

My friend, in that synth, suffers from poor lighting conditions.  Half his face didn't pointcloud out, it seems.  I will probably reshoot using another object, bah!

You could be quite right on the levels thing.  The math would be somewhat similar with such low numbers.  In fact, when I used 4, it was zippy, which suggest a exponetial increase.

I am re-running that same synth using level (1).  It's taking a goodly amount of time to process.
michaeldenis (Over 1 year ago)
Still processing..
michaeldenis (Over 1 year ago)
I've had great success in using the PhotosynthToolkit4 technique for exporting photosynths.  My process power still lags somewhat, but by making the jump at level3 and placing the job outside my Users/... folder while processing, I've managed to create several point clouds.

I will release more video as I have more results.
iNveridux1 (Over 1 year ago)
OK~! I've got stuff to show:

Also check out the end of the following video; its final credit sequence shows a couple of "Made in Photosynth" models:
iNveridux1 (Over 1 year ago)
Here's a couple of "pro" tips for using Photosynth Toolkit 4 in Windows 7 64bit:

1. Always output your synths to a folder that has ONE-word titles

For a long time, I experienced a lot of errors, probably because Photosynth Toolkit can't deal with a space; it's beta-ish after all.

NateLawrence (Over 1 year ago)
I just happened across an interesting video from Josh on his Vimeo channel: 

Masks with the PhotoSynth Toolkit 4 - tutorial
michaeldenis (Over 1 year ago)
whoa, that's awesome!  I'll test that out as soon as I finish uploading my latest video. for more info on "Sui Genesis," my latest and totally photosynthetic short-video piece.