There's no control over the panning speed in ICE panoramas when you're using the 'Play' button.
Is this what you're talking about or are you referring to the speed of panning when you're using the mouse to pan around the panorama? Are you aware of the two different mouse control schemes in panoramas? The default one is that of using your mouse cursor to grab a particular part of the panorama and drag it around, however if you click the small bracket icon just to the left of the center compass graphic at the bottom of the panorama viewer (it should say 'Change Panning Mode' when you hover your mouse over it), you will be able to click and hold your mouse button down and drag it around to glide over the panorama. The farther you move your mouse cursor from where you originally clicked in the viewer the faster you will pan and the closer you get to that original click point, the slower you will pan.