i think CESSNA1 was talking about making your own dowels, out of bar stock. i wonder about the strength of such a dowel, it obviously can't be as strong as the BP bolt.
can you enlarge the holes to encompass the offset holes + some, install a plug (suggest shrink fit), then drill out at the correct location ? possibly you can't because of edge distance issues.
i don't think you count on friction between the plates for carrying much load, count on the torsion being reacted by the bolts in shear, so you need a good fit (between the bolts and the holes).
it sounds like you have one plate drilled off, and are planning what to do if (when) you get a mismatch witht he 2nd plate on ass'y. this off, this should have been considered by the original designer, as the holes are not match crilled there will always be some tolerance for mis-match; so what does the drawing say ?
the two plates are oriented to each other somehow in the ass'y. can you align the plates in the ass'y, clamp them together, then drill out on the bench ? if you can do this, next time (if there is a next time) only drill a pilot hole in the first plate, this way you'll get match drilled holes (when you open them up later).