Other than Morningside Heights, which is almost more apart of the Upper West Side than anythiing else, I would say Upper Manhattan is the least appealling section of Manhattan. It has the highest concentration of housing projects, a significantly poor population, and there are many drug addicts as well. And truthfully, I think Harlem has such a negative image associated with it, since it used to be one of the most troubled neighbourhoods in New York, and still has many issues, that people would make an effort to avoid it. I can't think of one potential tourist attraction other than The Apollo Theatre.
And when I mentioned New York City, it had nothing to do with the connection the OP made. I was discussing what I didn't like about Toronto. And one of those things would be the fact that Toronto tends to look run-down (even in areas that are not sketchy), whereas New York obviously that's not the case (at least in Manhattan).
And I never said Toronto should be ashamed of their run down neighbourhoods. What I said was, how perplexing it is that an area considered to be trendy and fashionable (such as Queen West) looks run down.