The argument has been around for a long time - tearing down the gardiner. And from the majority of reasoning I have heard for doing this - is that it 'blocks views to the lake'.
While I agree it does do this, what is the point of removing it for this purpose when the Lakeshore East rail line...