Speed was born in Birmingham, England in 1841. He had one brother and one sister, but little else about his family or early life is known. He was in his early twenties when he was tried in Oxfordshire in 1863 for setting fire to a haystack. Homeless and begging for food, he had committed arson in order to get arrested and spend some time in a warm cell. He was sentenced to seven years of convict transportation to Australia. Speed was conditionally released in 1869 and was allowed to live outside of the prison walls and undertake employment, provided he did not commit any further offences. He found work as a general servant in Western Australia and was finally granted his certificate of freedom two years later. He went on to help build bridges across the vast Swan River, and spent the rest of his working life at various companies around the state. He was never re-convicted of any offence and went on to live a perfectly ordinary and law-abiding life, only coming to the attention of the papers a few months before his death.