I had a lengthy response written here, but I'll reduce it to this.
1) Its not arbitrary.
2) We disagree on the value of at least one setback. (I'm open to requiring only one, subject to outcome tests on wind/shadow etc)
Any restriction of any kind impedes a developer in some fashion. Why bother with a building code? Why not let residential go in next to an aluminum smelter? Why require accessibility?
Because these things have a value to people.
Yes, trade-offs should be considered and costed and I'm all for reasonable flexibility. But suggesting zero setbacks to me isn't on, and isn't desirable.
Again, we very much disagree on this point, for reasons outlined above.
I'm very interested in the other examples you mention here. While we disagree, in some measure on setbacks, we may agree on some others.