I believe that while it is true that only the conciousness can travel through time, which means that they can only travel into a place and time in which they have already been, there is a possibility that they have a way to travel in a more free way, to other places they have not yet occupied, but are more like ghosts than actual people, and only certain people have the ability to see them.