Turns out it's because they're trying to align with the Earth's magnetic field, according to new research.
"Dogs preferred to excrete with the body being aligned along the north-south axis under calm MF [magnetic field] conditions," according to the findings by researchers in the Czech Republic and Germany. "This directional behavior was abolished under Unstable MF." The results were published in the journal Frontiers in Zoology.
To test their theory, scientists had 70 dogs, from 37 breeds, observed over a two-year span. The orientation of the dogs during defecation was cataloged 1,893 times and during urination 5,582 times.
The report said the planet's magnetic field is "calm" for only about 20 percent of the time during the daytime. "Our findings might provide an explanation why many magnetoreception experiments were hardly replicable and why directional values of records in diverse observations are frequently compromised by scatter," the researchers said.
That's what I thought!