The bus terminals add about 100 million dollars per station, depending on the size. Sure, not every station on the Yonge Extension needs them, but every station on the Scarborough Subway certainly does. I have shown this before: stations with bus terminals within the suburbs generate far more ridership than those with inconvenient transfers. The terminals are needed because they support the surface networks in the suburbs, thereby generating very high ridership at stations like Kennedy, Kipling, Finch, Islington, Warden, etc. They anchor the station's surrounding transit network, and allow for both grid and hub/spoke travel patterns. Remove those, and people have far less incentive to take the subway to work.
You're comparing a downtown station — whose ridership is defined by employment density — to suburban stations. Remove the Pioneer Village or Finch West bus terminal and I guarantee the majority of people would just continue to ride the bus to Finch Station.
The TTC network is defined by integration, remove that key aspect and far fewer people would take transit.
Besides, terminals are needed to provide looping for buses, washrooms for drivers, waiting areas for passengers, and centralized passenger connections — Not everyone that gets off at Finch is actually taking the subway.