Step 1 is to determine the maximum allowable temperature of the belt.
Step 2 is to calculate how much heat you actually need to remove.
Practically speaking, your options are air and water.
It is possible that using either more water, or colder water will do the job. Remove all the heat from the conveyor as a sensible heat gain only in the water - no phase change, no steam.
Or, do your cooling within a controlled enclosure that the belt passes through. Exhaust the steamy air outside. Use air knifes on the entry and exit.