You have to determine the in-place density by one of several field testing procedures. In the US, the more common method is the nuclear density test done in accordance with ASTM D6938, "Standard Test Method for In-Place Density and Water Content of Soil and Soil-Aggregate by Nuclear Methods (Shallow Depth)". This is a test that is done in the field after compaction is done.
The value of the in place density is then compared to a test run in the laboratory on a sample of soil from the site. This test is to determine the maximum dry density of the soil and its optimum moisture content. There are a couple of common methods for this test in the lab, most commonly the "Modified Proctor" version done in accordance with ASTM D1557. Once the maximum dry density is determined in the lab and the in place density is determined in the field, the two are compared by dividing the in place density by the maximum dry density to determine the percent compaction.
Basic construction materials testing.