It could be done. Depends on the length of the cantilever, loading on the deck, capacity of the intended screws. Usually though for knee brace connections they use direct bearing via mortise and tenon type joints to deal with the shear load. The simpson product is nice too because you then have a tested capacity.

" a 6x6 deck support post supporting a 3-2x10 beam, can you use structural screws to fasten a 6x6 knee brace to support the end of a cantilevered beam,..."

What arrangement of the screws are you envisionating ?

https://encrypted-tbn0.gstatic.com/images?q=tbn:AN...

or knee braces to the sides of the laminated beam or post. or the knee brace 'notched' into the beam with a vert 2x6 added to the post to pick up the compression. Have to be careful about the horizontal load added to the post.

Rather than think climate change and the corona virus as science, think of it as the wrath of God. Feel any better?

-Dik

I haven't seen structural screws used for that long so I was wondering if their (usually) smaller diameter generally made them less desirable due to an intuitive tendency to cause failure due to splitting the wood. I see contractors using them often. So, I hadn't settled on a pattern, but I'd come up with something appropriate. The full shear (vertical) load at the post connection point would be around 500 lb.

I planned on using one knee brace on each side of the post, so this would reduce the resultant horizontal force (though it would still be unbalanced).

I've used side members that attach to the side face of the beam each side and the side of the post. Then block it to take the compression. Kind of an ugly build-up but can do the trick if required. The single knee brace idea works in theory, but the connection from brace to post is really not that great. Toe screws? Not great load transfer.