To backup what Skeltron is noting, often as a designer you are interested in the deflection after something is built, some of the loads are built in and do not contribute the effects that might be perceived at the SLS. For example for a roof the deflection the users will see or feel due to wind is simply the wind load in isolation, how much it sagged under its own self weight is somewhat irrelevant as you are talking about relative deflection being noted vs absolute deflections as far as what is perceived by the observer. Same can apply to concrete slabs, for example if a composite slab is screeded level, then the user might only really sees the additional deflection due to applied SDL and LL. Therefore you need to understand the effect you are looking at for SLS and derive an appropriate load case or loading scenario for each effect you might be looking at, for example:-
1) is it a 'visual' thing like a column swaying over under wind or earthquake
2) is it a 'felt' thing like vibration of a floor
.
3) is it a prevention of damage or loss of serviceability thing (for example too much deflection might cause a serviceability issue with respect to watertightness, too wide a cracks in reinforced concrete might cause a durability issue).
Similarly the limits to apply to each effect may also differ, for example different deflection limits apply often to different loading, for example short or long term deflection limits.
AISC design guide 3 has a number of tables such as outlined below that shows some appropriate limits and also the associated loading to consider. Many standards and codes around the world have similar guidance which is equally applicable to other regions given all buildings share similar construction forms and techniques.
Edit, due to the lack of loadfactors over 1.0, the SLS load should always be less than the ULS load.