Nathanael really hit the nail on the head with his reply. One of these was created in October, and one in February of this year. The more recent one got more matches from the synther, but as you pointed out, they are not great matches, and this causes the point cloud to be larger, but worse. It looks like there are at least three different coordinate systems that are joined poorly with the later synth.
Great subject matter by the way. The synther loves this sort of thing. To get this to be 100% synthy you probably need to be a little more robotic in the way you take your photos. One or two complete panos per room, shots taken every 3 feet (coming and going) works pretty well.
Here's a great example: http://photosynth.net/view.aspx?cid=a2c8398d-1d5d-4c6b-b36a-51d9e2e222b6