Since the stress concentration is at the interface of the round to square section, and assuming that the square section is somewhat larger than the round section then it may be that you could use round to round as your model. Don't know if this reference will help, but I found it in my Advanced Mechanics of Materials by Seely and Smith.
Jacobsen,L.S., "Torsional-stress Concentrations in Shafts of Circular Cross Sections and Variable Diameter," Transactions of the American Society of Mecanical Engineers, Vol 47,1925, p619.