A line is vertical if and only if it is parallel to the path taken by a body released from rest in a constant gravitational field.
Hence the adjective vertically, meaning in a vertical line.
A plane is vertical if and only if it contains a vertical line.
The term the vertical is frequently seen, particularly in the context of specifying the spatial orientation of a given (straight) line: the angle it makes to the vertical.
It needs to be borne in mind that the terms vertical and horizontal are meaningful only in the context of an environment with a gravitational field, and therefore one where up and down have a meaning.
In the context of the presentation of mathematical graphics on, for example, a piece of paper arranged horizontally, the vertical is that direction towards and away from the reader. Thus such a direction would be actually vertical if that paper were held in a vertical plane oriented for normal reading.
Computer screens are, in normal usage and to a first approximation, vertical planes already, so on such a medium, the terms are literal.
It is of course noted that it is difficult at this level to define vertical and horizontal without the definitions at some point becoming circular. An appeal to an intuitive understanding is therefore necessary to a certain extent.