This debate is so polemic and pointless. With any proposal, regardless of scale and content, there will always be issues to improve and weaknesses. There's never such a thing as 'perfect.'
So, yes, obviously there would be issues with a very large project like M-G. The public and the planning department ought naught step out of the way once Gehry's name is mentioned. Where possible these deficiencies should be addressed and, if impossible to remedy, balanced against benefits.
Yet so much of the criticism of this particular project has been outright wrong, which makes it impossible to either address or weigh against other benefits. Not all criticism, but most. It's frustrating to see the Chief Planner pouring cold water all over something which is an opportunity if nothing else rather than try to fix what we can.
Infrastructure wise, it's ridiculous to act as if 'downtown's full' or there's simply no way existing infrastructure can accommodate M-G. It's especially dishonest for the City's Chief Planner to make the argument that it's an unreasonable burden. The City's official plan calls for the area's population to nearly double over the next two decades (roughly ~70k new residents, plus more jobs)! This isn't some kind of unexpected or unreasonable project; it represents a small fraction of development the City has been encouraging and planning for for years now.
Transit wouldn't be impacted terribly. Either residents who would have moved into M-G will move into other surrounding developments (no difference..), or will move further away to a different area and make longer commutes, putting more pressure on infrastructure than would otherwise have been the case.
'Height' doesn't matter, in and of itself. Given tower design, the differences between typical point towers in the area and M-G's towers would be indistinguishable from street level. The podium would be the major impact on street life, which all the evidence suggests is being built to contemporary urban design principles and will frame King St very nicely. Bandying about some warmed over Jane Jacobsian notion of 'human scale' is ridiculous since the street-presence of this project isn't an ~80 story glass wall but 6 storeys, which is highly typical of successful urban streetscapes.
The only semi-legitimate argument is one of heritage preservation. Naturally the City has an interest in heritage preservation. Only the most dogmatic and impractical would take the position that heritage is sacrosanct and cannot be altered. City's aren't museums to the past. It's perfectly normal to destroy heritage structures for any number of reasons. The issue isn't black and white; accepting that heritage status isn't sacrosanct isn't the same as saying it's worthless or we should replace the Acropolis with a Cinnabon. Not everything is Penn Station.
Any judgement here are inherently subjective, but on balance it seems clear that M-G represents a sizable improvement over the existing structures in many ways. The six story podium will better frame King Street. The architecture is clearly unique whereas the existing warehouses are routine. The new structures will provide better retail, commercial and institutional space. The new structures will be more 'mixed use' than the current ones. In other contexts this won't always be the case. There's no need to be dogmatic about the issue.