There seems to more than one definition of "51N" by the manufacturer's and users of relays, but in general a 51N would be a function of calculated residual current while a 51G would be a function of measured residual current. The measured residual current can be either the hardwire summation of the phase CTs or it can be the result of a single CT around all conductors.
If you have a three wire, solidly grounded, system the 51N is probably sensitive enough detect your ground faults. The 51G can be more sensitive since it can use a CT with a lower ratio. If you have 400:5 phase CTs, the 51N is based on that ratio, but if you use a 50:5 core balance CT, you have 8 times the sensitivity.
Which one is right? It depends. Either will work and it is up to you to evaluate the pluses and minuses of each to determine the best course. Many relays allow you to do both.