Correct. Non-Native is self-explanatory; but invasive means spreads aggressively, displacing native species in the process.
You can, in fact, have a native invasive. Something that generally belongs here, but by its nature is a very aggressive spreader which takes over an area at the expense of the existing species present.
A good example would be river grape. A species that belongs in our area; but can, in the right conditions kill the majority of vegetation over several acres of land.
But with a native invasive, nuisance aside, that's a natural function of the eco-system and one that self-corrects over time.
A variety of reasons may come into play.
A species may seem to have an ideal climate here; but may be missing rhizomes in the soil from back home; it may receive different levels of humidity, wind or extremes of temperature.
Ph levels in soil can affect what grows; so does whether the plant is an aggressive spreader in the first place (ie. how it does against competition)
Finally there's a question of whether there are natural controls on the species here (does anything eat it, bugs, fungus, animals, birds); does any species assist in spreading its seed? ; and are there any diseases here to which it may be subject.
This link shows you some of the threats that can control Japanese Maple:
japanesemaplelovers.com
Japanese Maples also tend to fail if exposed to temperatures much below - 18C
So they have some limits as you go north or in areas exposed to high wind.
They also don't like their feet wet; they require well-drained soils.
But the limitations of other species will vary.
As they do for the Japanese Maple which appears to be mildly invasive the further south you go.