I think it's mostly political. They wanted to get as far deep into both Quebec and Ontario to ensure political support from both Provinces and avoid any backlash from Quebec.
I suspect that halfway through the construction of phase 2 to Quebec City and Toronto, they'll start looking at the next phase to Southern Ontario.
Question is, what do we do next? Calgary-Edmonton is the most logical Phase 4, though that could very well be done by another party by then. Anything beyond that would be extremely expensive for little gain.
I would hope that, once sane people are in Washington once again one day, we can all dream, we could see a California/Oregon/Washington/Vancouver HSR.