Well, we can get into heady debates here about the relation between taste and knowledge, but I don't think we need to distract ourselves that way. The point, as I see it, is pretty simple: when I go to a waterfront beach-like area, I'd prefer not to be sitting beside a factory.
If you want to dress up your satisfaction with that situation by calling this factory a heritage property--sheerly in virtue of its age, not necessarily how it beautifies the area, which I take to be a major part of what constitutes a heritage property--that's fine. I have no claims on how the concept of a 'heritage property' is to be used.
And if we take good taste to be a fondness for dirty, smelly factories in developing areas, I proudly take my self to have extremely poor taste