Forum : Photosynth Lounge

Photogrammetric, Oblique Image Stitching, Pets Dressed in Clothes Photos… this is a place to chat and share stories with your fellow Photosynthers. Not all topics have to be about photography, this is a place to relax and chat about whatever you fancy.

Topic: Yes there is a way to vastly improve your synth

Report Abuse Report Abuse
jimcseke (Over 1 year ago)
Dear fellow synthers’
This comment may easily exceed the 900 word limit so please be patient with me.
It has been almost 2 months , since I last made any comments within this forum. 
From my very first day, (Thank you Nathanael) I have loved every aspect of being part of this community. 
Our common love of the art this amazing technology produces, is responsible for bringing us all together. 
Without this, I would have never had the good fortune to get to know any of you.
I have loved technology and art for as long as I can remember. 
My love of art, has always been a part of my life.
Within this community, I have found both together. I was able to express myself again.
Art is a very personal thing. 
And even if I was not able to share my works, the satisfaction comes from within.
jimcseke (Over 1 year ago)
I wanted to mention all of the above before I began to explain my absence.

As some of you may remember, I was one of many of you who had a bit of an interest in improving the final synth. 
In particular, the point cloud. I know that Nathanael was right there with me, weighing the pros and cons of certain possibilities.
I had tried almost every kind of digital edit that I could throw at a set of photographs. 
Some of them actually had a positive effect. But they were not consistent. What worked for one, was disaster for another.
And I know that I was not the only person trying to find a way to create a synth with a higher degree of predictability.
Even though some of  my completed synths displayed the words “100% synthy”, I still did not know what to expect in the final viewing.
jimcseke (Over 1 year ago)
If some of you remember, some of my last attempts to find a way to change the meaning of 100% synthy, was an edit I called  “The Cseke Cheat Zoom”.
Respectfully, I did not seriously intend for any type of edit to be given my name. I would have been kind of cool, but honestly, you could have called it the “Crazy Zoom Cheat” as far as I was concerned.
But at the same time, the “Zoom Cheat” was definitely a factor in my experiments which eventually led me to writing this comment to you today.
What I had discovered, was that I was able to affect the photographic and point cloud synth, with this edit. 

But I also had some extreme disasters, which almost made me admit that the synth process is what it is, and that is that!
I had tried different combinations of rotational zooms on one of my favourite collection of photographs, which were of the 2 Inuksuik (plural source: Wikipedia) 
In one particular experim
jimcseke (Over 1 year ago)
In one particular experiment, most of the little Inuksuk had a pretty decent looking  point cloud. But as I rotated the image, I saw that his (or her) central body had exploded outwards in one direction.
It looked painful. 
My stomach sank, and I closed the viewer. 
This “explosion” occurred after many positive results. And at this point I was very tired and ready to admit defeat.
That is when I had one of those “wait a minute”  moments.
I reopened the viewer and looked at the explosion differently this time.
Sometimes, some of our best ideas come from our worst “failures"
This was not a failure, it was an indicator.
It indicated a definite direction of miscalculation on my part.
If I could make the little guy explode in one direction, I know that I could make it explode in the other direction. Or better yet, not have the little guy explode at all.
jimcseke (Over 1 year ago)
So I retraced all of my editing steps one by one for this particular test and tried to find the common factor that causes the point cloud to do what it does. 
Long story short I found the common factor(s).
And convince you even more of what I know to be true I want all of you to be aware of something I have had no choice but to keep quiet about.
The second that I had realized I had indeed find a common factor, I stopped all of my public testing and consulted with an Intellectual Lawyer.
My moment of realization, then became an Intellectual Property, and as of April 19th 2010, was acknowledged in the U.S. Patent and Trademark Office in Arlington, Virginia as a provisional patent.
Title: Photosynthesizing Method and Apparatus.
Trust me, this was not an easy road for me. And I did not just pluck this out of thin air.
jimcseke (Over 1 year ago)
My attorney, who happens to be a very decent human being, has informed me that I am free to tell you all about my experiments and eventually successful methods. 
I have spent a long time on this, and I could probably write a short novel about what I went through to get to this point. There were many times that I wanted to simply give up.
I will think twice about jumping into something like this again if I ever get the chance. 
Believe me, it was no picnic. But I want to spare all of you, my tendency to digress.
A short explanation and then I will tell you all what I have discovered.
I was going to upload a collection of screen captures and try to amaze you with some of my synths, but I a truly exhausted from this experience.So I decided to simply tell you what you need to do, and let you decide for yourself. My photography skills leave a lot to be desired, and I think many of you, because of your skills will produce even more amazing results.
jimcseke (Over 1 year ago)
With what I am about to share with you, you will discover for yourself a new freedom in synth photography. 
100% synthy will take on a new meaning.
Scene transistions will be totally in your control.
Accurate depth of field.
Self occlusion and repeating patterns can actually be put in a category of good and  interesting ideas for synths. 
Again, I am really tired so I am trying to get to the main subject as soon as I can.

In short what I discovered was that not one of the 200 or so photographs I had take had a common reference marker between them.
For example:
If in one photograph, a certain tree was leaning in frame at about 18 degrees, that very same tree would almost certainly not be leaning in the exact way, referentially speaking. It could be leaning in the completely opposite direction, or worse still laying almost on it’s side.
jimcseke (Over 1 year ago)
I bicubic sampled rotated every single image of that synth, starting with one photograph, in which there was a tree that I knew was at one point of my photo session, almost perfectly vertical.
I used that first photograph and referenced every single following photograph to that tree. But the tree obviously was not in every picture, so I had to find a new reference marker which was vertical in the last picture of the tree still partially visible. Sometime the new reference marker was a blade of grass. At this point I could feel my eyes starting to bleed.
Long story short again, it worked. 
The point cloud was not spectacular but it was pretty special and a sure sign that I was finally on the right track.
I knew that nobody in their right mind would want to do what I did,  just to get a decent looking point cloud. I knew this was the right road but I needed a shortcut.
jimcseke (Over 1 year ago)
Where could I find a constant reference, other than trees or blades of grass or sides of buildings. I needed a visual reference I could count on to be accurate and also appear in each photograph so that I could rotate the images accordingly.
I have been in the construction field since I was about 16 years old.
And there is a very old and very reliable tool (much like myself) that has been used as a reference for a very long time. And it is not so much the tool, but the physics behind the reliability of this tool. 
The tool is called a Plumb Bob. I just happen to have one, and it did appear in one of my earlier synths. 
But the real world physical, force which makes the plumb bob so reliable and accurate is of course gravity.
So, I made a bunch of synths using a plumb bob. They turned out pretty good but the only problem was that eventually the plumb bob and line became part of the point cloud. Not a desirable effect at all.
jimcseke (Over 1 year ago)
How can I have an out of frame reference which takes advantage of the reliability of gravity. 
Long story short, a small construction level secured to the underside of my camera, so that the level bubble is easy to see before I take each picture.
It worked wonderfully, but was very awkward.   
The resulting point cloud was so dense and beautiful I almost cried man tears. But the joy and confirmation I felt was far too overwhelming.
Okay, I am now legally at liberty to share with you, the photographic method you can use as early as tomorrow, to start to see what I already know.
I live in Kelowna, British Columbia, Canada eh? And one of my favorite places to shop for those impossible to find items is called Princess auto.
You will find stuff there that nobody else carries, and the prices are dirt cheap.
jimcseke (Over 1 year ago)
Go to a hardware store and buy a small trailer bubble level. Better yet buy 2.
I used the other on the side of my camera to get into those hard to get to areas. Some of these trailer levels come with sponge sticky tape which will allow you to attach the level to your camera in area where it does not obstruct any of your ability to take photographs comfortably.
Each photograph must be taken when the bubble is centered.
Trust me the synther will appreciate the reduced processing it has to do.
“World up” will indeed be “Photo up” ( maybe out by a couple of degrees depending on your eye hand coordination) 
Long story short, it does not end there. You can also implement a type of level which is called a surface level which gives both horizontal and vertical reference at the same time. The standard trailer level only references in the same manner as a plumb bob.
jimcseke (Over 1 year ago)
The interesting thing about a plumb bob or any other type of levelling device is that, each gravitational line produce by the little wonders points directly to the center of the earth, and no 2 of these lines emanate from the earth in a parallel to each other.
They radiate away from each other like pins in a pin cushion. So each one is unique.  
What I have described to you is only the basics of my once Intellectual Property. 
You will find that you will be able to edit your synths in ways you never thought possible. Your camera will turn into a  flying machine, creating scene transitions you never thought possible. 
Your synths will now become almost completely editable. 
You can create highlight from a set of picture taken without the intention of creating highlight using the same bicubic zoom I discussed.
jimcseke (Over 1 year ago)
Again, I could upload a bunch of videos on the internet, but I truly am really tired of cutting and pasting and editing and screen capture, and I think that you need to do for yourself to prove for yourself that what I have just divulged, is completely true.
I want continue synthing, and because of what I was doing, that was completely impossible.
So I will wait until I see what you are able to do with what I have just explained and then I just want to return as a regular member and be part of this site and “the gang” once again. 
At this moment, I feel the large weight I have been carrying is finally gone.
Thank you all for taking the time to read this comment.
I won’t bore you any longer. So that is it. The end.
(Please excuse any grammatical errors or nonsensical sentences because I am really tired)
Jim Cseke
jimcseke (Over 1 year ago)
In my tired state, and in my haste to post this comment, I completely forgot to mention the fact that really shiny objects are absolutely synthable using my methods. I actually synthed the glare coming off of a 25 cent(CDN)coin . 
Have fun!
michaeldenis (Over 1 year ago)
hi jimcseke, good to see you!

First off, welcome back and congratulations on your provisional patent!  I'm impressed by your efforts, and happy to learn that you continued working on your projects and ideas since the interesting discussions you and Nathanael had a couple months ago exploring the nature of capturing pointclouds.

From what I'm reading here, you've strongly recommended "leveling" as a method for improving the quality of connections between images in photosynth.  I recognize that this statement is an extreme distillment of what you've written and considered here.

My first question is: Are you targeting synths made for "Area Viewing" or for synths made for "3D Point Cloud Production?"  To illustrate the difference, my first link here directs to the former type and the second link to the latter.
jimcseke (Over 1 year ago)
Hey Michaeldenis, nice to read from you again.
First of all, thank you for the nice welcome back.
And as far as the provisional patent goes, I am so glad that I am done with that part of it. The "getting there" is not as glamorous as one might think. If I am ever in this same situation again, I will think twice before diving in again.
But this was one of those situations that kept pulling me along and forcing me to dig deeper with each new experiment. You simply cannot walk away from questions that you know you will eventually answer in time.
I am not sure what you meant when you wrote "extreme distillment".
The "leveling" you mention, is only a factor, because the earth's gravitational pull is the only reference provider (that I can think of) which is always available and completely accurate. By attaching the trailer level to the camera, you are forcing the camera to hold true on at least one plane. As a result, all images are photo up and world up.
jimcseke (Over 1 year ago)
Here is a quick scenario. 15 photograph, all taken orbitally of a subject.
No attention is paid to the orientation of the camera, and in some cases images are close to being upside down. If you happened to take a photograph with your camera turned on it's side completely, it is recommended that you "right up" the image before upload. Now up load those images and cross you fingers and hope for a decent synth.
If an image oriented on it's side needs to be "right up" before upload, why is the same logic not used for all images? I am not "leveling", I am orienting the camera to an readily available and consistant reference marker. Some of my earlier experiments involved using a plumb bob. Completely accurate, readily available but read my comments in the forum, and you'll see why the plumb bob is very impractical and required digital editing of each image.
So getting back to the 15 image scenario, this time you have the trailer level attached to the camera.
michaeldenis (Over 1 year ago)
So, to restate the theory, creating synths from a series of single-plane leveled photographs are a better methodology because doing so will...

I understand that this methodology will improve the viewing experience, but I think you're saying more here.  Thanks for clarifying for me.
jimcseke (Over 1 year ago)
All of the 15 images are "right up" in the true sense. 
Now, using logic and realizing that the synther is not a citizen of the "real world" and has no concept of up or down. Which of the 2 scenarios do you think will produce the better more accurate photographic and point cloud synth.
By taking as many photographs as possible from as many different angles as possible, throwing them at the synther to process, you basically get what you pay for. 
If on the other hand you take the photograph as I suggest,(and trust me,it is not as cumbersome as it looks) you will be handing the synther all of your images in a nice neat little package for it to process. Photo up will be world up. If you wrote a report for school, and you dropped it before handing it to your teacher. All of the pages are in complete disarray, upside down, backwards, out of page order see what happens when you try handing it in as is.
jimcseke (Over 1 year ago)
by using the level as a force reference, you are handing the synther a set of images that you did not drop on the ground and hand in as is.
because of the neatness of you images, the synther will smile at you and probably give you an "A" for your extra effort. And trust me again, there really is not a lot of extra effort involved.
This method works for all manner of synthing. With a higher degree of predictability and editability. I paid about $6.00 canadian for my 2 trailer levels and about $5.00 for my surface level.
I strongly recommend that you go out and purchase those item, and see for yourself. I think that you will be pleased with the results.
I am putting myself on the line by making these statements, so please trust me and try it for yourself. You have nothing to lose except maybe about $10.00
jimcseke (Over 1 year ago)
Thanks again for the nice welcome back michaeldenis.
Sincerely Jim
jimcseke (Over 1 year ago)
To anyone else who is interested in my methods, please at least try this for yourself. I have released only one example for public viewing. And I did so, because it has a special meaning for me personally. It is not my best effort, but it was my first undeniable success. I will not publicly release any others until I see that others have tried what I have suggested. Any public "Befores and afters"..."withs or withouts" examples would only be a waste of my time and yours. I felt the best way to prove my methods work, is to have you either prove or disprove this for yourself. I am the only one with anything at stake.I could be making a complete fool of myself for all to see, and you have absolutely nothing to lose by at least trying it for yourself.
Nathanael.Lawrence (Over 1 year ago)
Hi, Jim, welcome back. I was a bit worried that I had stepped on your toes last time we talked when you went silent after being so active before.

I hope you don't mind my delayed response as I just got back home and online after being out of town this weekend.

As I read your description above, I must admit that I, like Michael, was wondering if I was really fully understanding what you were saying when, after talks of patenting, new freedom of editing (I am still unclear how this is true), etc. everything came down to what seems to be able to be summed up by saying, "Orient all your photos to a common edge and|or direction." and "One simple way of achieving this is to use a level attached to your camera.".

I certainly am very interested to see a contrasting pair of synths - both with photos shot from the exact same positions, but one using (near) perfectly leveled shots and one using casually handheld shots, but no more crooked than one would normally take.
Nathanael.Lawrence (Over 1 year ago)
Although that sounds skeptical, I do plan to test your method, if not tonight, then tomorrow. 

I had my doubts last time as to whether rotated crops would really yield more matched image features than systematically tiled versions of an image against itself, but it turned out that because your rotations were more similar in scale to each other that the cores of your spirals actually did have more points. My theory was that if I gave the synther the exact same pixels that you should see every single feature present in the image matched to itself, resulting in a point in the pointcloud apiece. I also thought that identical pixels would surely cause more matches than ones that had been jumbled by image rotation, but based on some preliminary tests that I did, it seems the synther actually prefers slightly altered versions of an image feature to an exact duplicate.

Nathanael.Lawrence (Over 1 year ago)
From this standpoint - this idea that the synther likes *nearly* identical versions of the same image feature (you could also say that it prefers minimal change between iterations of an image feature to tightly match - this is seen by people understanding that taking more photos along a path is more likely to cause matches than fewer photos) your method of aligning all images to a given vector or edge, (seemingly) seeks to increase likelihood of matches through minimising the differences in appearance of different versions of an image feature by alignment, rather than by capturing intermediate views between two positions.
Nathanael.Lawrence (Over 1 year ago)
As to the entire, "Photo up = World up!" line of reasoning, this is very true, although, apart from minimising the difference in appearance between iterations of image features as noted above, the benefits of this are not clear to me. Certainly I do not anticipate it hurting anything... at all, but whether it makes anything significantly better, I will withhold judgement until I can actually see a side by side comparison.
Nathanael.Lawrence (Over 1 year ago)
Perhaps the feature matching is helped by the features being pre-aligned, but I think that the primary purpose of the photos being uploaded right side up is simply for the synther to correctly detect 'up'. In this sense, since most people try to keep the ground level in their shots by nature, although all the photos will be a little off, they should average out. This means that as far as playback of the photos goes, the objects in existing synths shouldn't see any major difference to a synth constructed with level shots, unless someone habitually leans to one side or the other in addition to only taking photos that face two of the four cardinal directions.

Even though 'level' at different positions in a synth do follow different rays or vectors to the centre of the earth, the area that a typical synth covers makes this a practically negligibly small difference.
Nathanael.Lawrence (Over 1 year ago)
Again, I have to stress that I don't think there can be any downside to taking super level shots. I'm just not sure that there will be a huge benefit other than your shots not being tipped slightly on their side to keep the contents right side up as is the case with more carelessly framed shots.

Alright. Talk is cheap. I'll shut up until I have actual results to share.
aridolan (Over 1 year ago)
The benefit of using levels is exceptionally high for panoramas, and since ICE began to synth its panos I found using levels essential for successful work. I found small bubble level in ebay for $2.90 each, and I glued two of them, at right angle to each other, on the inexpensive ball head of my tripod (I removed the plastic holder before gluing). There is no doubt that it helps in this type of shooting.

Here is the link for these levels:

As to the advantage for normal Photosynths, I join Nathanael in his curiosity regarding this issue, and I also plan to do such a test myself.
jimcseke (Over 1 year ago)
Hey Nathanael, I just re-read, one of your comments. I want to clear something up as soon as possible. Although giving the synther a slightly different version of an image in the form of a crop which has been bicubically sampled does add, more points to the point cloud, this is not the correct way to add "artificial zooms or extra images from originals". Since I so quickly left this site to continue my testing, one of the most important things to realize about the cropping edit, is that it is not as straight forward as one might think. I now have a fairly good understanding as to why and why not certain types of crops work. And by adding crops which have been randomly rotated, you are in reality, getting away from the forced reference method of photography.
I have not applied randomly rotated images to my synths for quite some time.
I have discovered that the actual requirements a crop for a given situation are dependant on a more calculated approach.
jimcseke (Over 1 year ago)
I have developed about 3-4 different types of crops, which in in their own way serve a purpose with the type of synth that you hope to edit. These calculated crops are part and parcel of my methods. There are crops I use specifically to increase the point cloud density. The increased number of points are not artificial. They are actual point revealed to the synther, which it adds to the final synth. I have also developed a crop which will allow you to, for example:
Start with the minimun of 3 original images.
By using another of my cropping methods, you can pick and choose highlights, as in the "Highlights" which we display with our synths. These "highlights" must be cropped a certain way in order for the synther to accept the new image a part of the synth. And these are working highlights. They actually become part of a navigatable synth.They are part of the actual photographic and point cloud synth in the true sense.
jimcseke (Over 1 year ago)
These added highlight can also be edited.
I wil describe a synth I produced. I orbitally photographed some of my gardening tool. Rake shovel etc... I could not include my lawn mower,because someone had stolen it. 
Using the images from the orbital capture, I highlighted each item so that they functioned exactly as though I had photographed them to be highlight.
I have used this type of edit many times and it does indeed work beautifully. The garden tool synth started with 38 original images. I wanted to increase it's point cloud density and I also wanted to highlight each of the tools.
The end result was 185 artificial images producing a perfectly functioning photgraphic and point cloud synth. Just to be double sure, I captured the points and viewed the results in a render program. I was happy to see that even the render program accepted what was processed and ever bit of editing I applied to this synth worked perfectly.
jimcseke (Over 1 year ago)
I have edited a 3 photograph synth of the interior of an old pickup truck in the same manner, and was very pleased to see that this had exactly the same outcome as the garden tool synth. In that particular synth 3 images became 36, and again it functioned exactly as a 36 image synth should. Photographically, point cloud and point capture.
This editability is a direct result of my photographic approach. Not only does the synther's job become alot easier because of the forced reference photography, your job or editing or if you have some type of batch editor, will also become much easier as well. You can now trust that all of the images are oriented to the same external reference marker (gravity) and from that you can edit in a more calculated fashion. Take a close look at the images you take using my methods. I know that you love a challange. Look for something, anything which is common in all of the images.
jimcseke (Over 1 year ago)
Once you discover this not so obvious common factor, you will see exactly what I mean. Hey, I spent a lot of time on all of this. And there were actually some enjoyable moments. This was one of them. I have always loved solving puzzles (not jig-saw, I hate those things), logic problems, and I love a mental challanges.
You might pick up on it immediately or It might take a while to see. And as a couple of Internet buddies, I want you to experience a little sense of the fun of discovery that I had with this project. So I will leave that with you. If you don't see it, I will not think any less of you. I once spent almost an entire year tring to figure out one of those stupid I.Q. puzzles. One day I just happen to look at it a little differently and there was the answer. I when I discovered what the answer was, I realized this had nothing to do with intelligence at all. It was simply looking at something differently on particular day.
jimcseke (Over 1 year ago)
If you become absolutely stumped, let me know and I will ease your suffering.
Your Friend Jim
eliscio (Over 1 year ago)
Wow, over a year ago and I completely missed this conversation!

Has there been any further discussion or development in the photographic documentation area?  Any further testing on this?


Nathanael (Over 1 year ago)
Hey there, Eugene, 

This discussion was the sequel to this one:
(The last few comments in this link are after this thread and include an older incomplete reply from me to the linked discussion in the form of 'Dense Point Cloud Construction Theory'.docx

Jim also posted here:

If you search the lounge for his username, you'll actually find that he posted quite a bit on this topic (to the point of spamming the forum and other people's synth comments), seeking to get recognition for his ideas. 

I always maintained that his assertion (that stabilizing the 'roll' (in aviation terminology) of the camera would generate a sculpturally more accurate point cloud) was false.
Nathanael (Over 1 year ago)
I tried to explain to him the logic that the increased density that he saw in some of his (now deleted) tests was because the cropping was allowing the synther to examine portions of the photos at greater resolution, rather than due to any rotating of the camera (or lack therof). 

I also tried to explain to him that accurate depth calculated for any given point could only be generated by the different arrangements of keypoints that truly unique vantage points afford via parallax and that those differing arrangements would remain identical, regardless of the roll of the camera from any given vantage point, but he refused to listen or even try to shoot a set of synths: one with the camera level and one with the camera held normally (or even intentionally crooked) and compare the point clouds of the crooked set and the perfectly level set to prove or disprove his ideas.
Nathanael (Over 1 year ago)
I did attempt a small test of Jim's leveling ideas with these two small test synth sets: 

Flower Box: 

Intentionally Crooked:

You can let me know if you see a difference in the accuracy of the shape of the point clouds, but I didn't see it when I looked at them, but neither was I expecting to. Clearly the point clouds are oriented differently, but anyone who has made a few synths should have predicted that. 

I am not opposed to taking level photos but I don't see a point cloud benefit and structured photography is certainly not Jim's invention.
Nathanael (Over 1 year ago)
After he asked the Photosynth team about his ideas on Get Satisfaction and didn't receive a public reply, he grew discouraged. 

He also posted a few things over at Gary Mortimer's Photosynth fansite on Ning (where my above tests were posted) but has since deleted all of his topics and comments there.
Nathanael (Over 1 year ago)
In his farewell entry on Ning, Jim said that the lack of validation of his ideas from his peers and the Photosynth team was too discouraging for him and that all the time he had poured into his tests were taking a toll on his relationship with his wife and he needed to step away from Photosynth. 

He was an energetic and friendly guy and I wish that other people were as active on the forums and comments as he was, but especially in 2010 he unfortunately began to believe that he knew more about how Photosynth worked than everyone else in the community (even the Photosynth team) and in his quest for validation made something of a nuisance of himself. 

If you're interested in talking to him about his ideas, though, I think that he would love to have someone other than me listen to what he has to say. His email address is the same as his Photosynth username at
You need to be Signed In to add a comment. (Are you new? Sign Up for a free account.)