"his danger [i.e. misuse by the majority] can never be eradicated and the fact that the only case in federal history when the power to expel was exercised is a case when, we think, the power was demonstrably misused is a compelling argument for its abolition." . . . .