There are lots of potentially fatal things your immune system protects you from (which is why AIDS won???t kill you.) The fact that one thing you mentioned ??? smallpox ??? doesn???t infect everyone and then mostly only kills a one percent to a third of those infected suggests that immune systems across a human population can cope with it.