Possibly people are different and some like to be able to access downtown from time to time instead of being in it all the time. Some people like trees and quieter areas. There are plenty of places to visit and to eat in the area. To each there own man, some people like suburbs and some like busy areas like manhatten, personally I do find Yonge and egg to be a nice area full of character and a good buffer between the fast pace of downtown and the slow sleepy suburbs. Ask me 10 years ago and I probably would have agreed with you.
Crosstown will get you to little jamaica, restos around eg and bathurst/allen, science centre, and new aga khan museum as well as stuff on Bayview and Mt. Pleasant.