Marrying animals is different because only humans are "persons", and marriage is an agreement between two "persons". Therefore, really, there is no real legal reason for incest to be prohibited other than for cultural reasons, which obviously isn't enough.
Polygamy is a bit different because, by definition, it is marriage between more than two people. But, if two people are allowed to marry, why not three, as long as they all are willing and in a capacity to enter into the marriage conscious of the ramifications?