I think it's a great idea. I'm not sure how liberty village users would all of a sudden not take public transit to work ( are they going to walk?). If you also implement it between Roncy/River, pinches, the once ways would naturally merge into the two way streets after.
Lastly, if implemented with car bans from the streetcar tracks during rush hour, it would substantially speed up the existing commutes. You could have service more often (one eastbound/west bound train as opposed to 2 eastbound/wesbound on both king and queen). It would effectively be a downtown relief line surface route, with no left turn obstacles, heck, if you want to put a curb and make it dedicated 'LRT'. ALL major US cities have one ways in their core, there are plenty of patios on the major avenues in NYC. The whole notion that 'one ways' destroy street vitality is such a false dichotomy. If you manage the speeds, it's perfectly possible. Philly is entirely one ways, but one of the most 'urban' 2nd tier cities in all of North America.
The current commute from liberty village to queen east is probably about 45 minutes at best. If you can shave it down to 25 minutes, I'm pretty sure riders would be willing to walk 400 meters (for faster, more frequent service)