Do you really think so? Animosity towards Tory is focused on his bad policy-making; people's anger at Ford was about his complete disrespect of the office. As foolish as Ford's policies were, no one called for his resignation until he was caught on video smoking crack. Big difference.
No one is going to seriously call for Tory's resignation unless he violates a moral expectation (e.g., backroom dealing). I would say the bigger risk for Tory is coming to be viewed as a relic from a bygone era with no vision, and as a weak and ineffectual leader. (I don't think thats true, but Tory sure doesn't do himself any favours).