Ideal sleeve would have relative permeability close to 1.0 (non-magnetic, and low conductivity. In my opinion Aluminium might not be a good idea as it has high electric conductivity. If you have high slot-frequency, permeance harmonics due to slot openings will induce high eddy currents on it.
I guess the best compromise is to use stainless steel, as their conductivity is usually rather low, and they are usually non-magnetic.
I have seen also some manufacturers using some kind of hard plastic tube around rotor (with some small servomotors). From magnetic properties point of view, that is a good choice, but if there are some losses in the rotor, it will also insulate the rotor thermally. In worst case, magnets would heat-up and eventually demagnetize. Sleeves that have some conductivity, also act as a "magnetic mirror" for air gap harmonics, prventing them from penetrating the rotor. Non-conducting sleeve lets all the air gap harmonics to pass the rotor and cause extra losses there.