I don’t think anybody assumes that men are ALWAYS physically stronger than women in athletic performance, only usually. It’s statistical. There are several reasons why sports are divided between men and women:

1) Men are USUALLY stronger than women physical
2) Aesthetics - many people prefer watching women vs men or vice versa
3) Socially - Women like to have their own space and vice versa.

So any discussion of transgender issues should consider multiple issues.