Just a couple of questions.
What voltage is this single phase motor?
I am assuming this is at least a 240Vac motor?
A 5HP, 120Vac draws about 50A while a 240Vac motor draws about 1/2 that at 25A. A 5HP 3 phase motor draws about 15A at 240Vac. What I am getting at is that for a given HP rating the single phase contactor will always be bigger. The size increase is to accomadate more current and larger wires. So when you go to connect a single phase load to a 3 phase contactor you must be sure the contactor is rated for the load. A 5HP 3 phase conatctor may only be good for say 2HP for the same single phase voltage rating. Another problem is that you may not be able to match an overload unit to the contactor you have. The current rating being beyond the range of your contactor. Will your #10, or worse yet, your #6 wire even fit in a terminal on your contactor?
Let's say all the ratings are ok.
If your contactors are 4 pole and interlocked then you are ok.
Since Line B is always connected to ter.4 then you only have to switch the remaining leads. I sketched a drawing but i don't know how best convey it to you. It can be done quite simply.
You have a pencil? Draw 8 contacts on a page left to right. Label contacts left to right, 1 to 8. Top of contact is A, bottom of contact is B. Contacts 1 to 4 are "rev" starter, contacts 5 to 8 are "fwd" starter. This is strictly for the motor wiring not the control. I assume you can handle that?
Connect/JUMPER as follows;
1A TO 2A,
3A TO 4A TO 7A TO 8A TO LINE A,
5A TO 6A,
1B TO 7B TO MOTOR TER.8,
2B TO 6B TO MOTOR TER.J,
3B TO 5B TO MOTOR TER.5,
4B TO 8B TO MOTOR TER.1,
LINE B TO MOTOR TER.4.
Happy motoring.
skiier