Retaining walls are one of the most common structures under construction today. Some consultants (one or two man operations) can make a decent living doing nothing but retaining walls. All contractors know the typical drill regarding backfilling requirements, and if they don't they are not qualified to be building retaining walls. I am sure that in your notes you explicitly call out backfilling requirements regarding strength of material, time duration after grouting etc. Hopefully you did, and the contractor's bid should be based on the plans and specifications. Also in your notes, I hope you noted that shoring, bracing, etc. is the contractor's responsibility. As such, write a letter to the owner and the contractor explicitly stating your recommendations but also make it clear that you are not stopping work and that they may proceed at their own risk. However, if they choose to do so you will not sign off on the final inspection for the project.