at some point you are no longer providing a leveling pad, you are filling a void. for filling voids, any sort of grout or concrete that you can get into the void will work. If it is a large void than use even larger aggregate. However, for leveling structural elements, gravel is not generally used. Up to about an inch or two thick leveling pad, I would stick with sand. Note that for leveling structural elements, I always specify non-shrink grout.