While I'm not in the area to often, I used to bike down the new Sherborne bike lanes all the time, I echo the comments above, I've seen a lot of weird shit, but never felt threatened personally. There are homeless that ask for money of course, but actually even this is rare I find a lot of them just stick to their selves.
The worst areas are the sidewalks by the shelters because they end up being used as gathering places.
Example of weird shit:
- Once while biking, someone started yelling to what I believe he thought was a cop (wasn't), "oh there's someone being rapped in that alley you should go down" ... actually I could clearly see into the alley and nothing was going on at all, and he kept changing what he was yelling like "oh now there's a kid being ..." anyway he kept going on and on and yelling loudly. The other folks in the area just walked by ignoring it all, it seemed as if they were used to this stuff.
With all that said, I do know a lot of folks who live in this stretch (and it probably is the worst greater downtown area, not poorest mind you, just worst for this sort of stuff), and there was actually one scary indecent, it involved a female walking across Queen near Parliament, a bunch of folks (not homeless) started walking behind her and kept offering her drugs, and then they started to offer "err, other things", and they started to get more aggressive (didn't touch her though) this went on for a while and eventually she started to yell (telling them to leave basically) and they did (as there were other folks), though she got on the first cab she could find.
To be fair this was fairly late at night and the only time an incident like this has happened to the folks I know in the area.
Now you will see things between others (typically drug users - i.e. these aren't random acts, these folks know each other) there are fights at times, though I personally haven't seen this, more so just a lot of yelling and threatening - if it gets aggressive you should call the police (but not get involved).
I've always wished this area cleans up though ... honestly Queen on this stretch has some really nice old buildings, a bit further west a lot of offices in converted spaces. The area has a very nice feel (King east'ish actually, more so then Queen west).