It's not just about Quebec City-Toronto, it's also about Quebec City-Ottawa, Trois-Rivières-Dorval, Laval-Toronto, etc.
There's also the problem that there is no transit service between Quebec City and its airport (a $40 cab/$25 uber is the only option).