One of the pictures that Bill sees while she is reviewing the history of Earth on the spaceship is a self portrait of Vincent van Gogh. It's the same portrait that Vincent offered the Eleventh Doctor as a parting gift in Vincent and the Doctor (2010). This portrait is not a true replica, but rather it's slightly altered to resemble Tony Curran, who played Vincent.
The tiny robots called Vardies are named after a professor at Memorial University in St. John's, Newfoundland, Canada. Andrew Vardy teaches computer science and electrical engineering, and his area of research is swarm robotics. Several years ago he worked with the episode's writer, Frank Cottrell Boyce, on a short story based on some of Vardy's ideas.
The Vardy city scenes were filmed at the City of Arts and Sciences in Valencia, Spain.
The spaceship's name is "Erehwon", which is almost the title of a novel by Samuel Butler, 'Erewhon', about a dysfunctional Utopia. It's also "Nowhere" spelled backwards.
The Doctor says "I'm happy, hope you're happy too." This is also a line from David Bowie's 'Ashes to Ashes'. Peter Capaldi is a big Bowie fan.