Did your clay layer have shrinkage cracks? It is surprising how much shrinkage cracks affect water situations in clay. The soil scientists usually refer to these situations as "structure". Even a site with lots of worm holes can give these kind of situations. This also explains why spray irrigation as a way to dispose of waste water on some clay areas can work quite well, almost like dumping the water on sand.
I suspect that you merely were providing a relief for water in those cracks, if that was the case.
This does not seem to be an artesian situation, since you don't have a water flow of importance once the clay has been removed.
I don't know how many times I've seen bore holes with plenty of water, but once an excavation is done, there is little if any. In none of these situations was there any artesian water pressure, but the overburden was somewhat porous and saturated. It can fool one when he decides to make plans for lots of ground water pumping in an excavation and it turns out to be no problem.
While not the same, when excavating peat deposits, located below the water table, the low permeability of the peat makes excavation quite easy, but of course, later that hole fills up with water.