Carleton probably inherited his literary ambition from his father, General James H. Carleton, who wrote "The Battle of Buena Vista" (1848), based on his experience in the Mexican War. Earlier in life, James had wanted to be a novelist and sought the advice of Charles Dickens whose letter in reply discouraged him but was nevertheless preserved.