It seems like it would have to redo the entire synth every time you added or removed a photo (that is, it would have to compare all the image features or keypoints to each other all over again... it could skip finding image features for all the photos that are staying in the synth when new ones are added). It would definitely be cool to have the position of all the photos be held in 3D and animate to their new positions when more photos are added.
A related idea that I've had for a long time now is to have the option to enable people to watch when I'm uploading a new synth. The 3D data is the very last thing to get calculated and uploaded so you couldn't do anything too amazing but you could watch as new photos flow into the collection, upload the grouping data from the image feature matching that happens right before scene reconstruction to watch all the photos uploaded so far group appropriately, + then at the end watch as they all fly into place in 3D. Awesome!