Horses evolved in North America over 55 million years ago, with the ancestor, Eohippus, being a small, multi-toed animal. 
Domestication began around 4000 BCE in the steppes of Kazakhstan, with humans initially using horses for work, food, and possibly milk. 
Early domesticated horses were primarily used for pulling carts and wagons, with riding becoming more common later. 

Horse Skull in Reins print