Horizontal

direction that is parallel to the horizon and perpendicular to the vertical

Horizontal is a direction perpendicular to the vertical. That is, it goes in all directions except up and down. The term comes from the horizon.