Bloor Walk makes me think of additional criteria that would broaden a discussion of "ugly" and I think make it more useful. I find Bloor Walk less objectionable than, say, the RoCP buildings, simply because it is smaller and much less prominent. I think if you really wanted to measure not just ugliness, but a building's impact on the city considered from an aesthetic point of view, you'd need some kind of equation to figure it out.
Perhaps something like:
([Overall Attractiveness] X [Size of Building] X [Prominence of Location] +
[Approachability at ground level] X [Foot traffic nearby])
X [Maintenance] = Beauty Quotient.
If we create scales for [Overall Ugliness] and [Unapproachability] in say, -10 (most ugly) to 10 (beautiful), and allow for multiples of say, .1 to 2 for the other factors, (and allowing for, say, -.9 to 2 in the Maintenance category) we might end up with something like this:
TD Centre
((9 X 1.8 X 1.5) + (7 X 2)) X 2 = 60.4
(ie., a beautiful, large, building in a sensitive location, quite approachable with tons of foot traffic nearby and well maintained, is a great asset for the city).
Bloor Walk
((-5 X 1 X .2) + (5 X .4)) X 1 = 2
(ie., a fairly ugly building, but neutral for approachability, in a not very prominent location with little foot traffic and too new for maintenance to be an issue, is really pretty much a moot point for the city).
2 Bloor East
((-4 X 1.6 X 2) + (-8 X 2)) X 1 = -28.8
(ie., a thoughtless 70's building, large and in a location that is truly very prominent, almost completely unapproachable yet with tons of foot traffic nearby, maintainance not an issue, is a terrible lumpen mass that disrespects our city).
This doesn't replace aesthetic judgements, just tries to broaden them so that the impact on the city is taken into account. It's a long, mathematic way of saying "At least Bloor Walk isn't right on the waterfront".
Or perhaps I'm just insane!