A gravel surface is not stable and in my opinion, does not meet the intent of the ADA. While ADA accommodates many forms of disabilities, the most commonly considered accessibility is that for wheelchairs. Wheelchairs cannot negotiate gravel easily. Further, walking on a gravel surface displaces the gravel and often displaces it beyond the 1/2" surface irregularity requirement.
I would not use gravel on an accessible route.
As jgailla notes, the AHJ would have local interpretation; but if you want the design to be fully defensible without local interpretation (sometimes erroneous interpretation), use a stable material for the accessible route.
Remember, the ADA is civil rights legislation. It's dimensional and design considerations have generally been codified by inclusion in building codes; however, claims under the ADA are not building code issues...they are accessible rights issues, with interpretation often exceeding the bounds of simple building code requirements.