As far as I can work out, everyone that vanished on that day came back thinking they were from somewhere else. And the people that vanished might be all the people who decided to take Catherine up on his 'world-visiting' thing, but I can't confirm that.