I suppose that might be the case if there is sufficient background noise, but there is no mention of background noise from the OP. There are plenty of us who use random exitation for modal analysis for example - it is much quicker and is useful for papering over friction-type non-linearities and getting a "linearised" response.
However, your comment about dwell times does make me think of a reason why you might get a difference between the position of the apparent peaks in a swept sine and the peaks in a random test. If the sweep rate is too fast and the damping very light, then you could get a situation where the resonance doesn't have enough time to build up to it's maximum response. In that case, the peaks in the swept sine test could appear in slightly the wrong frequency. The peaks in the random test will always be in the right place. Of course all this assumes linearity as others have already said.
M
--
Dr Michael F Platten