Topic: Points cloud from videos ?

Report Abuse Report Abuse
Pierre_P (Over 1 year ago)

I need to create virtual 3D representations of objects, typically form points’ clouds. I am very interested in Photosynth but so far I cannot create a points cloud detailed enough for my work requirements. 

I just discovered the link between Ice and Photosynth, and that Ice can now create panoramas from videos. I was pretty excited about it cause I intuitively thought that Ice could extract “as much pictures from a video as necessary” (undreds…) so to create the best representation it can from a 3D object I’m interested in (with 100% Synthy), then upload it on Photosynth and extract the points cloud. . I was wrong. Ice does not work in 3D, and it attempts to make a 2-D panorama of my object rather than assemble the pictures in 3-D (I have been walking around the object while taking a video)… I figured what I need is Photosynth to work from videos. Is it possible ? Does it seems like I am missing something?

Nathanael (Over 1 year ago)
Greetings, Pierre!

Yes, ICE ( ) is only for 2D stitching of panoramas.

People have suggested being able to deliver video files into Photosynth ever since the day that it was first released back on 2008 August 20th both on the original forum as well as the "New Feature Suggestions/Requests" forum ( search that page for 'video' with F3 or Ctrl+F ).

The best thing to do to use video frames in Photosynth currently is to use a program such as VirtualDub ( ) or SUPER ( ) (or possibly mEncoder or Handbrake also... haven't tested these recently) to render the frames of your video to individual .jpg files which you can then drag into Photosynth.
Nathanael (Over 1 year ago)
I will warn you that a common downside of using video frames in photogrammetry/structure from motion/computer vision applications such as Photosynth is that due to video compression and camera motion during capture, often video frames will be too blurry or have too much of the visual texture of objects removed in compression for Photosynth to have enough image features to lock onto to form an excellent point cloud.

Having given you that proviso, though, I would still tell you to at least try and see what you get.

Regarding 100% synthy, this is honestly a measurement of how well all the images linked to each other and how possible it is to navigate from one to another, rather than a quality assessment of the point cloud. 

For example, this synth is very effective for giving a sense of what it is like to be on location, but given the small number of photos, you won't be loading it because of its incredible point cloud: (Open in IE)
Pierre_P (Over 1 year ago)
Hello, Nathanael !

Thank you for your detailed and useful answer.

I wonder why Photosynth’s developers did add this feature (i.e. ability to work from video) since – as you say – it has been suggested for so long… Anyway. 
I already try extracting a number of pictures from a video, but I did it manually, I’ll have a look at the softs you mention, thank you.   

Regarding your remarks (compression of the video and camera motion), I have the possibility to capture video with as much care as needed (turning around the object very slowly for example, trying to have the more “stable” video as possible..) and I use a GoPro camera (HD)… So I think I can try to minimize those problems… And since my internet connection is so bad, I have to compress the pictures extracted from the video myself, so to be able to upload many of them on Photosynth :(
Pierre_P (Over 1 year ago)
The synthy is not indeed the only parameter I’m interested in (since it does not translate the “quality” or detail level of the point cloud), but I also need the point cloud to be a realistic representation of the objects: e.g. I should be able to compute accurate volume estimates of the objects, so I need the images to be perfectly linked in 3D… (I don’t know if I make myself clear here…)

Well, I will have a look at your softs ! Thank’s again,

Nathanael (Over 1 year ago)
Pierre, if your GoPro videos can be opened in QuickTime, QuickTime Pro is one more program that can open a video file and export the frames as an image sequence, although it does cost money when compared to VirtualDub, etc. which are free.