The objection, it would seem, is that Tory and his ilk never seem to ask why women tend to not ask for a higher salary with "active negotiating posture", which is a ball that is on the Tory's side of the court. Not to mention that systemic economic and cultural issues (aside from the idea of passivity vs aggressiveness of women in the workplace) dwarf Tory's reasoning.
This!!!
Citing a fact and interpreting the meaning from that fact are two different things. Tory correctly cites studies that associate the gender pay gap with the different ways that men and women tend to negotiate. However...
1. That's not the only factor that causes the gender pay gap. Fixating on negotiation to the exclusion of other factors should make you question Tory's motives.
2. Rather than using this fact as an opportunity to blame women for their own problems (as Tory seems to do), someone with more empathy and a more critical mind might ask how exactly salary negotiation works. For instance, rather than advising women to pick up golf, a critical thinker might question why golfing (a sport dominated by white men) has any impact on salary decisions.
Tory did make one very good point. He said that his law firm used a formalized process for deciding salaries. So even if men complained more about their salaries, there was no impact on how much they actually got paid. Research has shown that formalized systems of compensation are much more equitable than the subjective decisions of employers and managers.
All of this is about a fact that Tory got right, but interpreted in a questionable way. However Tory also mentioned that the different negotiating styles had to do with the "DNA" of men and women (for some reason, no one picked up on this). This, as far as I know, is a complete falsehood. But it serves the purpose of dismissing the issue: it's just human nature that men will be more aggressive and achieve more than women and there's nothing that can be done about it.
Take away conclusion: Tory comes across as someone who is dismissive of gender inequality and unwilling to critically examine his own privileges as a wealthy, white man. When confronted with the reality of a gender pay gap, his reaction is to essentially shrug and say "don't look at me, I'm not sexist!"