One of the greatest appeals of Orion was that the bigger you made it, the better it worked. While chemical rockets scale badly - with big ones much harder to build than small ones - Orion was just the opposite. That meant that large spacecraft, capable of long missions, were not merely possible, but actually easier, for a variety of reasons, than small ones. Bigger spaceships meant more mass for absorbing radiation and shock, more room to store fuel, and so on.
Could Orion ever come back? The answer is yes. The Test Ban Treaty is a real obstacle to any future deployment of Orion. However, it binds only a few nations, and many nations (like India and China) that are both nuclear-capable and interested in outer space have never signed it. For an up-and-coming country looking to seize the high ground in space in a hurry, Orion could have considerable appeal. And, of course, even the United States could withdraw from the Treaty, on three months' notice, under the Treaty's own terms.
Orion's scientists weren't worried about fallout. Orion would have produced some, but the amount would have been tiny compared to what was being released already from above-ground tests, and there was hope that additional work would have produced even cleaner bombs designed specifically for propulsion. Today, people are much more nervous about radiation and under current political conditions a ground-launched Orion is a non-starter, at least in Western countries. But not everyone cares as much about radiation, and indeed the countries that worry about it the least are those most likely to find Orion appealing as a way to attain space supremacy over more established space powers in a hurry. What's "Orion" in Chinese?
Some have suggested that the 1967 Outer Space Treaty, which forbids placing "nuclear weapons or any other kinds of weapons of mass destruction" in outer space, would also be a barrier to Orion, but I don't think so. A nuclear "bomb" used for space travel, arguably, isn't a "weapon." It's a tool - just as the Atlas rockets that launched the Mercury astronauts were, because of their use, different from the otherwise identical Atlas missiles aimed at the Soviet Union. (When asked about the difference, Kennedy responded: "attitude.")
My opinion: People get confused about weapons and radiation. These are understandable fears. However, nuclear weapons are not the only ones that could kill most people on the planet and destroy civilization. All out use of biological and chemical weapons could do it too. Conventional explosives used to create firebombing killed 100,000+ in Tokyo back in WWII. It was done in a couple of days. The capability exists to firebomb every population center, wreck the medical infrastructure and then release some flu. So if chemical bombs and weapons can be just as deadly then why do we use them all over our society and not use nuclear power and systems in some key areas where they can make things far better ? The dangers and bad uses still exist. We need to grab the upside and positive uses while not using it for the bad uses. We can use nuclear power and systems for space without increasing proliferation or increasing risks for nuclear war. In the meantime by not doing it we allow other problems to get worse. Also, by depending upon coal and chemicals more people continue to do die now.
In the future, nanotechnology will also just make it easier to kill everyone. There will be more potential downside and more potential upside. We have to go for the upside and control the downside. We should not repeat the same current choice with nuclear which is to limit the upside and without helping with the downside.