I'm pretty sure the reason is that although the frequency along Yonge Street is fairly reasonable all day, the frequency of connecting routes is not. At rush hour, it is actually fairly practical to get around using YRT, but off-peak that ceases to be the case if any transfers are involved.
People are often complaining about how YRT is spending all its effort in the two intensification corridors, but I think that they will cause an increase in demand (and therefore frequency) in the cross routes as well. In Toronto, buse routes across the network have fairly reasonable frequencies all the time, and I'm fairly sure that people going to and from the subway make up a large part of the demand which supports that.
This is a key reason why Brampton Transit is growing ridership (and service) - the frequency of many of the cross-routes to the 3 Zum routes, themselves improvements and extensions of existing routes - have been bumped up to 30 minute minimums. The Route 4 isn't bad (but stuck with 27 minute, not 30 minute schedules), but there's no excuse why the 85 doesn't adhere to clockface, 30 minute 7 day a week schedules; same with the McCowan, Markham Road, Kennedy Road, 2 Denison, Davis Drive, Bathurst, Keele and Jane Street routes. If YRT was bold to say that 30 minute service, 18-19 hours a day (20 min minimum weekday afternoons, early evenings and day Saturday) was to be the core and minimum service level on these routes, ridership would grow and meet latent demand for the cross routes.
In Brampton, routes 1, 2, 3, 4, 5, 7, 11, 14, 15, 18, 29, 30, 51, 52, 53, 54 (plus 501/A) meet the TTC minimum standard for 30 minimum service, 7 days a week, 18 hours a day. Routes 23, 50, 502 and 511 also come close to this standard. For YRT, only routes 4, Blue, Orange and Purple meet this standard (a few come close, though the 85 and 88 end at 8:30 Sundays, and the 55 Davis has 40 minute Sunday/late evening service), and it's a larger system with bigger ridership and a bigger population.
