This problem is more complicated than this, and if we ignore those complications we will be as guilty of irrationalism as anyone on the right. There are questions that need to be answered which you are ignoring here.

1) Are transwomen biologically male? If so, to let them compete with women is to say that there never should have been distinctions between men’s and women’s sports in the first place. This means we should abolish women’s sports altogether, thus making it impossible for transwomen to compete in them. I discuss this more here. https://teedrockwell.medium.com/the-inescapable-dilemma-of-trans-women-in-sports-b7e659cc58a

2) Even if transwomen are biologically female, they still don’t possess all of the biological characteristics of Ciswomen. Are any of the male biological characteristics possessed by Transwomen the same characteristics that make the Women’s sports category necessary? Some sources say yes. That is a question that can only be answered by scientific research, not by dogmatic assertion.

I am inclined to say that we should grasp for the first horn of this dilemma, and eliminate the distinction between men’s and women’s sports. We could however replace it with another set of distinctions which don’t specifically mention gender, but would still cover the inequities the old system was designed to rectify. There are already Ciswomen who were not allowed to compete as women because they have certain kind of masculine bone structure. You don’t have to say they’re really men to let them compete against men. However, If women like these are going to be in the category competing against men, why refer to the categories as “Men”? We could have lots of different categories, based on the science of what body types give advantage in what sport, and make no mention of gender in those categories. We already have weight class distinctions in some sports. Fencing has competitors group by age as well as gender. We could replace the gender distinction all together with distinctions of this sort carried to a higher level of refinement. This would eliminate the humiliation that can come whenever any committee makes a judgment about letting any woman, either Trans or Cis, compete as a woman.