Well to be fair, the ellipse when hovering over the ground plane used to be much smaller without the chevron inside of it pointing toward other panos. That's just an interface update.
The assumption that Google has made in the past is that whenever I click on the walls or ceiling I mean to look closer at that part of the pano, but when I click the surrounding floor, I mean to move in that direction. That is probably true in more cases than not although today I seem to be moved to the nearest pano in that direction no matter where I click which seems overly sensitive.
We'll have to wait for connected or linked synths (and panos?) to see what Photosynth's interface for surrounding imagery looks like, but showing indicators for surrounding geotagged/geoaligned images or other points of interest from the map is totally expected (depending on where you're viewing the synth/pano - i.e. on Photosynth.net or on Bing Maps or embedded on a personal website).