I really don’t think this is such a big deal. Four tracks is still quite a lot of capacity, and it’s definitely possible to run a very good amount of service - see Berlin’s Stadtbahn for example, which deals with S-Bahn, regional trains, and intercity on four tracks. If modern signalling is installed, and the main limiting factor for frequency becomes train braking distance or dwell times (like with ATC or ETCS), 20-30 tph on each pair of tracks can be achieved, and that is much more than enough for LSE, Stouffville, and VIA.
Six tracks will probably be a good idea eventually, but that’s a long ways out and by that time Union would probably start having capacity problems too, in which case a downtown GO tunnel would probably be needed, and building the portal like 500m to 1.5 km further out is really not a big deal, basically just run the TBMs a bit further. Without stations a GO tunnel is cheaper than a metro tunnel. If the GO tunnel ends up being along Gerrard/College or Dundas, then even without OL the tunnel portal would more or less be at Pape anyways.