In 2011, the NDP got 54%, the Liberals 26% and Conservatives 14% in polls 22 to 52. And these include the more Yorkville-ish east Annex but don't include Harbord Village, Seaton Village, etc.
The NDP outpolled the Liberals in the Annex over all three elections, only 2008 was close (the "Green Shift" factor pulled votes to the Liberals and Greens). The Tories are closer to 10% than 20% there, as they have little appeal among the intelligentsia.
The question isn't whether the NDP "only" averaged 40% or whether it was more like 50% over three elections - after all, how does that compare to the NDP's share of the popular vote in the City of Toronto or in Ontario?