Not sure I agree.
First, you aren't simply replacing a like-sized building with another like-sized building for aesthetic purposes only. You are increasing density in an area that can support it. At the very least, the argument for intensification likely counters the argument for not moving...