Fair question. I believe that we already are funding Canadian amateur athletes to the point they can have quasi-professional lifestyles, i.e. they can train more or less 100% of the time. I also don't have a problem with funding acting troupes in the same manner. However, I don't think we should be funding David Mirvish to build a theatre nor should we be funding Cirque de Soleil to build out their new show.
But the other thing that is Olympics-specific is the egregiousness of the waste, and the way the waste is almost 100% borne by governments. I don't know how much Toronto/Ontario/Canada may have helped fund the Four Seasons Opera house, but I hope it was minimal. However, it will be used for decades to host COC and other events.
In the case of Olympic stadia, even with the most desperate attempts to paint lipstick on the various pigs, we would have been be left, after wasting BILLIONS of dollars on security and logistics, with a billion-dollar track and field stadium, a second new aquatic centre with more seats, and -- I'm assuming -- a second round-and-round bike track with more seats, when no one in North America track cycles.
So -- I don't have a problem with funding the Canadian basketball team to go to FIBA, or an acting troupe to tour Europe. I have an issue, specifically, with hosting very large, corrupt, billion dollar spectacles, and I can't think of an arts festival that is funded by any of our governments to the tune of millions of dollars, much less billions.
TL; DR -- Olympics, specifically, suck money like a black hole.