The regulations say this - "It's unlawful to discharge, carry, or have in possession a firearm containing live ammunition in its breach or in its magazine attached to a firearm, in or on a railway car, motor vehicle, sleigh, aircraft
or other conveyance."
I think you are right in saying that a horse is not a motorized vehicle. However, since a horse is used to move a person, or materials from one point to another, it would be considered a "means of transport." Merriam Webster dictionary defines conveyance as "a means of transport." So that implies that a horse is a type
conveyance, therefore even a loaded magazine attached to the firearm would be considered unlawful.