The number of people driving into the core might be overestimated but it's still high enough to have an effect on congestion. I don't really see much same-day delivery being done at rush hour.
If we can't manage to restrict King to transit and police/emergency services, we should at least do everything else to improve the flow of traffic: strictly enforce the diamond lane (with cameras), no parking/stopping (including taxis), no left turns. Parking garages along King might be a problem, but we could have some kind of exemption system, e.g., you can drive on King between these hours but ONLY for garage entry/exit access, if you have the right sticker or whatever.