It was commonly thought that witches rode goats. The goat being a symbol of evil and gluttony due to its similarity to Baphomet and Pan. Somewhere along the way secret societies such as the Freemasons began using the riding of the goat as part of the initiation process.
From the SUPR NATR zine purchase here!