On which voltage do you operate? Do your system mainly use cables or also overhead conductors?
Earth fault protection using the residual connection:
Use an earth-fault relay with overcurrent-relays. (If you have overcurrent protection, you can easily add earth fault protection) An earth fault relay responds to the residual current in the system, which exists if there is a phase to earth fault. Normally the setting is between 10% and 40% of full load current. Also, like DiscoP noted, especially on electromechanical relays, the relay can be less sensitive on lower settings, due to the burden it puts on the CTs during fault conditions. (For electromechanical relays, the lower the setting, the higher the impedance of the relay) Another factor is the possible predominance of a third harmonic component in the residual current even under quiescent conditions. Third harmonics appear as zero sequence currents and could cause mal-operation of the relay. If the setting is to low, you can find a lot of nuisance trips.
Sensitive earth fault protection:
Used mainly on systems with high-impedance neutral earthing devices. The load current demands high ratio CTs, while the neutral current has been limited to a low value for safety reasons. Normal earth fault protection using the residual connection would be unable to detect earth faults, sensitive earth fault protection should then be considered. Also, on overhead rural distribution systems it is possible for high resistance earth faults to occur, especially if a conductor breaks and falls on very dry ground. This presents a hazard to human live and livestock if left undetected for any lengh of time. The setting of this relays can be very low, down to 1% and even lower. Use of core-balance CTs many times essential. The problem sometimes is to take all three phases through the window of the CT, it is especially a problem on bigger size cables.