It is a matter of matching the turbocharger to the engine, considering the entire operating range, and the parts of the operating range that are most critical. What you end up with is an overall turbocharger efficiency characteristic as a function of engine speed and load. The backpressure imposed by the turbocharging system is an inverse function of the overall turbocharging efficiency. A wastegate, when open, as the name implies, reduces turbocharging efficiency.
A relatively small turbocharger will tend to offer high turbocharging efficiency at lower speeds and loads, and the efficiency will drop off at higher speeds and loads, especially if there is a wastegate that opens up at higher speeds and loads. Conversely, a relatively large turbocharger, because of its inability to provide significant boost at low speeds and loads, will offer low turbocharging efficiency at these lower speeds and loads, but at higher speeds and loads, its efficiency will excel, especially if matched without a wastegate.
But remember, matching is critical. A large compressor does you no good if the engine breathing is such that it is running into surge. On the turbine side it is also important to have the right combination of A/R and trim, for the application, to realize the highest potential efficiency. And of course the turbine needs to be well matched to the compressor as well, especially with respect to diameter, lest the turbine efficiency be penalized by a too low tip speed ratio.
"Schiefgehen wird, was schiefgehen kann" - das Murphygesetz