One very big word of warning for you, if you have achieved over 100% compaction, the amount of compaction is only relative to the compactive effort, BUT the fact that the material is at such a low moisture, i.e. 4% compared to 9.7% OMC, then it is highly likely that you have excessive air voids.
Around a compaction curve for a material, there are three zones which are likely to cause failure of the compacted material. Too wet (i.e. bottom right of the compaction graph) and a cohesive soil can suffer long-term settlemt due to consolidation of the material and a loss of water. Too dry and this can result in overcompaction leading to swelling (cohesive soils with very low air voids, upper left hand corner of graph), or too much air causing collapse failure due to inundation of water (bottom left of curve. It sounds very likely that the wrong compactive effort was used to identify the maximum dry density, as you have achieved a 'relatively' high compaction at completly the wrong moisture. If when you re-test the material (with an increased compactive effort), please be sure to assess the previous results in light of the possibility of collapse failure.
I would also suggest that you detemine the particle density (specific gravity in old money), and calcualte the air voids of the previous tests and plot the air void line on the compaction curves.
Geotechnical fill, using cohesive soils, for under a building should be between 100% of the standard proctor to 100% of the modified proctor with 0 to 5% air voids, based on current guidance in the UK.