POSIX terminal interface

The POSIX terminal interface is the generalized abstraction, comprising both an Application Programming Interface for programs, and a set of behavioural expectations for users of a terminal, as defined by the POSIX standard and the Single Unix Specification. It is a historical development from the terminal interfaces of BSD version 4 and Seventh Edition Unix.

General underlying concepts

Hardware

A multiplicity of I/O devices are regarded as "terminals" in Unix systems.[1][2] These include:

serial devices connected by a serial port such as printers/teleprinters, teletypewriters, modems supporting remote terminals via dial-up access, and directly-connected local terminals[1][3][4][5]
display adapter and keyboard hardware directly incorporated into the system unit, taken together to form a local "console", which may be presented to users and to programs as a single CRT terminal or as multiple virtual terminals[1]
software terminal emulators, such as the xterm, Konsole, GNOME Terminal, and Terminal programs, and network servers such as the rlogin daemon and the SSH daemon, which make use of pseudoterminals

Intelligence: terminals are dumb, not intelligent

Unlike its mainframe and minicomputer contemporaries, the original Unix system was developed solely for dumb terminals, and that remains the case today.[6] A terminal is a character-oriented device, comprising streams of characters received from and sent to the device.[6][7] Although the streams of characters are structured, incorporating control characters, escape codes, and special characters, the I/O protocol is not structured as would be the I/O protocol of smart, or intelligent, terminals. There are no field format specifications. There's no block transmission of entire screens (input forms) of input data.

By contrast mainframes and minicomputers in closed architectures commonly use Block-oriented terminals.

Capabilities: terminfo, termcap, curses, et al.

The "capabilities" of a terminal comprise various dumb terminal features that are above and beyond what is available from a pure teletypewriter, which programs can make use of. They (mainly) comprise escape codes that can be sent to or received from the terminal. The escape codes sent to the terminal perform various functions that a CRT terminal (or software terminal emulator) is capable of that a teletypewriter is not, such as moving the terminal's cursor to positions on the screen, clearing and scrolling all or parts of the screen, turning on and off attached printer devices, programmable function keys, changing display colours and attributes (such as reverse video), and setting display title strings. The escape codes received from the terminal signify things such as function key, arrow key, and other special keystrokes (home key, end key, help key, PgUp key, PgDn key, insert key, delete key, and so forth).[8][9]

These capabilities are encoded in databases that are configured by a system administrator and accessed from programs via the terminfo library (which supersedes the older termcap library), upon which in turn are built libraries such as the curses and ncurses libraries. Application programs use the terminal capabilities to provide textual user interfaces with windows, dialogue boxes, buttons, labels, input fields, menus, and so forth.[10][11]

Controlling environment variables: `TERM` et al.

The particular set of capabilities for the terminal that a (terminal-aware) program's input and output uses is obtained from the database rather than hardwired into programs and libraries, and is controlled by the TERM environment variable (and, optionally for the termcap and terminfo libraries, the TERMCAP and TERMINFO environment variables, respectively).[10] This variable is set by whatever terminal monitor program spawns the programs that then use that terminal for its input and output, or sometimes explicitly. For example:

The getty program (or equivalent) sets the TERM environment variable according to a system database (variously inittab or the configuration files for the ttymon or launchd programs) defining what local terminals are attached to what serial ports and what terminal types are provided by local virtual terminals or the local system console.
A dial-up user on a remote terminal is not using the type of terminal that the system commonly expects on that dial-up line, and so manually sets the TERM environment variable immediately after login to the correct type. (More usually, the terminal type set by the getty program for the dial-up line, that the system administrator has determined to be used most often by dial-up users with remote terminals, matches the one used by the dial-up user and that user has no need to override the terminal type.)
The SSH server daemon (or equivalent such as the rlogin daemon) sets the TERM environment variable to the same terminal type as the SSH client.[12]
The software terminal emulator, using a pseudoterminal, sets the TERM environment variable to specify the type of terminal that it is emulating. Emulated terminals often do not exactly match real terminal hardware, and terminal emulators have type names dedicated for their use. The xterm program (by default) sets xterm as the terminal type, for example.[13] The GNU Screen program sets screen as the terminal type.

Job control

Terminals provide job control facilities. Interactively, the user at the terminal can send control characters that suspend the currently running job, reverting to the interactive job control shell that spawned the job, and can run commands that place jobs in the "background" or that switch another, background, job into the foreground (unsuspending it if necessary).[14][15]

Line disciplines

Strictly speaking, in Unices a terminal device comprises the underlying tty device driver, responsible for the physical control of the device hardware via I/O instructions and handling device interrupt requests for character input and output, and the line discipline. A line discipline is independent of the actual device hardware, and the same line discipline can be used for a terminal concentrator device responsible for multiple controlling terminals as for a pseudoterminal. In fact, the line discipline (or, in the case of BSD, AIX, and other systems, line disciplines) are the same across all terminal devices. It is the line discipline that is responsible for local echo, line editing, processing of input modes, processing of output modes, and character mapping. All these things are independent of the actual hardware, dealing as they do in the simple abstractions provided by tty device drivers: transmit a character, receive a character, set various hardware states.[16][17]

In Seventh Edition Unix, BSD systems and derivatives including macOS, and Linux, each terminal device can be switched amongst multiple line disciplines.[18] In the AT&T STREAMS system, line disciplines are STREAMS modules that may be pushed onto and popped off a STREAMS I/O stack.[19]

History

The POSIX terminal interface is derived from the terminal interfaces of various Unix systems.

Early Unices: Seventh Edition Unix

The terminal interface provided by Unix 32V and Seventh Edition Unix, and also presented by BSD version 4 as the old terminal driver, was a simple one, largely geared towards teletypewriters as terminals. Input was entered a line at a time, with the terminal driver in the operating system (and not the terminals themselves) providing simple line editing capabilities. A buffer was maintained by the kernel in which editing took place. Applications reading terminal input would receive the contents of the buffer only when the return key was pressed on the terminal to end line editing. The @ key sent from the terminal to the system would erase ("kill") the entire current contents of the editing buffer, and would be normally displayed as an '@' symbol followed by a newline sequence to move the print position to a fresh blank line. The # key sent from the terminal to the system would erase the last character from the end of the editing buffer, and would be normally displayed as an '#' symbol, which users would have to recognize as denoting a "rubout" of the preceding character (teletypewriters not being physically capable of erasing characters once they have been printed on the paper).[20][21][22][23][18]

From a programming point of view, a terminal device had transmit and receive baud rates, "erase" and "kill" characters (that performed line editing, as explained), "interrupt" and "quit" characters (generating signals to all of the processes for which the terminal was a controlling terminal), "start" and "stop" characters (used for modem flow control), an "end of file" character (acting like a carriage return except discarded from the buffer by the read() system call and therefore potentially causing a zero-length result to be returned) and various basic mode flags determining whether local echo was emulated by the kernel's terminal driver, whether modem flow control was enabled, the lengths of various output delays, mapping for the carriage return character, and the three input modes.[24]

The three input modes were:

line mode (also called "cooked" mode)

In line mode the line discipline performs all line editing functions and recognizes the "interrupt" and "quit" control characters and transforms them into signals sent to processes. Applications programs reading from the terminal receive entire lines, after line editing has been completed by the user pressing return.[21][25]

cbreak mode

cbreak mode is one of two character-at-a-time modes. (Stephen R. Bourne jokingly referred to it (Bourne 1983, p. 288) as a "half-cooked" and therefore "rare" mode.) The line discipline performs no line editing, and the control sequences for line editing functions are treated as normal character input. Applications programs reading from the terminal receive characters immediately, as soon as they are available in the input queue to be read. However, the "interrupt" and "quit" control characters, as well as modem flow control characters, are still handled specially and stripped from the input stream.[26][27]

raw mode: raw mode is the other of the two character-at-a-time modes. The line discipline performs no line editing, and the control sequences for both line editing functions and the various special characters ("interrupt", "quit", and flow control) are treated as normal character input. Applications programs reading from the terminal receive characters immediately, and receive the entire character stream unaltered, just as it came from the terminal device itself.[28][26][27]

The programmatic interface for querying and modifying all of these modes and control characters was the ioctl() system call. (This replaced the stty() and gtty() system calls of Sixth Edition Unix.)[29][30] Although the "erase" and "kill" characters were modifiable from their defaults of # and @, for many years they were the pre-set defaults in the terminal device drivers, and on many Unix systems, which only altered terminal device settings as part of the login process, in system login scripts that ran after the user had entered username and password, any mistakes at the login and password prompts had to be corrected using the historical editing key characters inherited from teletypewriter terminals.[23]

BSD: the advent of job control

With the BSD Unices came job control, and a new terminal driver with extended capabilities.[18] These extensions comprised additional (again programmatically modifiable) special characters:

The "suspend" and "delayed suspend" characters (by default Control+Z and Control+Y — ASCII SUB and EM) caused the generation of a new SIGTSTP signal to processes in the terminal's controlling process group.[27]
The "word erase", "literal next", and "reprint" characters (by default Control+W, Control+V, and Control+R — ASCII ETB, SYN, and DC2) performed additional line editing functions. "word erase" erased the last word at the end of the line editing buffer. "literal next" allowed any special character to be entered into the line editing buffer (a function available, somewhat inconveniently, in Seventh Edition Unix via the backslash character). "reprint" caused the line discipline to reprint the current contents of the line editing buffer on a new line (useful for when another, background, process had generated output that had intermingled with line editing).[27]

The programmatic interface for querying and modifying all of these extra modes and control characters was still the ioctl() system call, which its creators (Leffler et al. 1989, p. 262) described as a "rather cluttered interface". All of the original Seventh Edition Unix functionality was retained, and the new functionality was added via additional ioctl() operation codes, resulting in a programmatic interface that had clearly grown, and that presented some duplication of functionality.[31]

System III and System V

System III introduced a new programming interface that combined Seventh Edition's separate ioctl() operations to get and set flags and to get and set control characters into calls that used a termio structure to hold both flags and control characters and that could get them in a single operation and set them in another single operation. It also split some of the flags from the Seventh Edition interface into multiple separate flags, and added some additional capabilities, although it did not support job control or the cooked-mode enhancements of 4BSD.[32] For example, it replaced the "cooked", "cbreak", and "raw" modes of Seventh Edition with different abstractions. The recognition of signal-generating characters is independent of input mode, and there are only the two input modes: canonical and non-canonical. (This allows a terminal input mode not present in Seventh Edition and BSD: canonical mode with signal generation disabled.)

System III's successors, including System V, used the same interface.

POSIX: Consolidation and abstraction

One of the major problems that the POSIX standard addressed with its definition of a general terminal interface was the plethora of programmatic interfaces. Although by the time of the standard the behaviour of terminals was fairly uniform from system to system, most Unices having adopted the notions of line disciplines and the BSD job control capabilities, the programmatic interface to terminals via the ioctl() system call was a mess. Different Unices supplied different ioctl() operations, with different (symbolic) names, and different flags. Portable source code had to contain a significant amount of conditional compilation to accommodate the differences across software platforms, even though they were all notionally Unix.[33]

The POSIX standard replaces the ioctl() system entirely, with a set of library functions (which, of course, may be implemented under the covers via platform-specific ioctl() operations) with standardized names and parameters. The termio data structure of System V Unix was used as a template for the POSIX termios data structure, whose fields were largely unchanged except that they now used alias data types for specifying the fields, allowing them to be easily ported across multiple processor architectures by implementors, rather than explicitly requiring the unsigned short and char data types of the C and C++ programming languages (which might be inconvenient sizes on some processor architectures).[33][34]

POSIX also introduced support for job control, with the termios structure containing suspend and delayed-suspend characters in addition to the control characters supported by System III and System V. It did not add any of the cooked-mode extensions from BSD, although SunOS 4.x, System V Release 4, Solaris, HP-UX, AIX, newer BSDs, macOS, and Linux have implemented them as extensions to termios.

What the standard defines

Controlling terminals and process groups

Each process in the system has either a single controlling terminal, or no controlling terminal at all. A process inherits its controlling terminal from its parent, and the only operations upon a process are acquiring a controlling terminal, by a process that has no controlling terminal, and relinquishing it, by a process that has a controlling terminal.[33]

No portable way of acquiring a controlling terminal is defined, the method being implementation defined. The standard defines the O_NOCTTY flag for the open() system call, which is the way of preventing what is otherwise the conventional way of acquiring a controlling terminal (a process with no controlling terminal open()s a terminal device file that isn't already the controlling terminal for some other process, without specifying the O_NOCTTY flag[35]) but leaves its conventional semantics optional.

Each process also is a member of a process group. Each terminal device records a process group that is termed its foreground process group. The process groups control terminal access and signal delivery. Signals generated at the terminal are sent to all processes that are members of the terminal's foreground process group. read() and write() I/O operations on a terminal by a process that is not a member of the terminal's foreground process group will and may optionally (respectively) cause signals (SIGTTIN and SIGTTOU respectively) to be sent to the invoking process. Various terminal-mode-altering library functions have the same behaviour as write(), except that they always generate the signals, even if that functionality is turned off for write() itself.[36][37]

The `termios` data structure

The data structure used by all of the terminal library calls is the termios structure,[38] whose C and C++ programming language definition is as follows:[34]

struct termios {
    tcflag_t c_iflag ;  // Input modes
    tcflag_t c_oflag ;  // Output modes
    tcflag_t c_cflag ;  // Control modes
    tcflag_t c_lflag ;  // Local modes
    cc_t c_cc[NCCS] ;   // Control characters
} ;

The order of the fields within the termios structure is not defined, and implementations are allowed to add non-standard fields.[34] Indeed, implementations have to add non-standard fields for recording input and output baud rates. These are recorded in the structure, in an implementation-defined form, and accessed via accessor functions, rather than by direct manipulation of the field values, as is the case for the standardized structure fields.[39]

The data type aliases tcflag_t and cc_t, as well as the symbolic constant NCCS and a whole load of symbolic constants for the various mode flags, control character names, and baud rates, are all defined in a standard header termios.h. (This is not to be confused with the similarly named header termio.h from System III and System V, which defines a similar termio structure and a lot of similarly named symbolic constants. This interface is specific to System III and System V, and code that uses it will not necessarily be portable to other systems.)[40]

The structure's fields are (in summary, for details see the main article):

c_iflag: input mode flags for controlling input parity, input newline translation, modem flow control, 8-bit cleanliness, and response to a (serial port's) "break" condition[34]
c_oflag: output mode flags for controlling implementation-defined output postprocessing, output newline translation, and output delays after various control characters have been sent[41][27]
c_cflag: terminal hardware control flags for controlling the actual terminal device rather than the line discipline: the number of bits in a character, parity type, hangup control, and serial line flow control[42]
c_lflag: local control flags for controlling the line discipline rather than the terminal hardware: canonical mode, echo modes, signal-generation character recognition and handling, and enabling the generation of the SIGTTOU signal by the write() system call[39]

The library functions are (in summary, for details see the main article):

tcgetattr(): query a terminal device's current attribute settings into a termios structure[43]
tcsetattr(): set a terminal device's current attribute settings from a termios structure, optionally waiting for queued output to drain and flushing queued input[43]
cfgetispeed(): query the input baud rate from the implementation-defined fields in a termios structure[44]
cfgetospeed(): query the output baud rate from the implementation-defined fields in a termios structure[44]
cfsetispeed(): set the input baud rate in the implementation-defined fields in a termios structure[44]
cfsetospeed(): set the output baud rate in the implementation-defined fields in a termios structure[44]
tcsendbreak(): send a modem "break" signal on a serial device terminal[45]
tcdrain(): wait for queued output to drain[45]
tcflush(): discard queued input[45]
tcflow(): change flow control[45]
tcgetpgrp(): query the terminal's foreground process group[46]
tcsetpgrp(): set the terminal's foreground process group[46]

Special characters

Programmatically modifiable special characters[47][44]
Field	meaning	Retrieved by `read()`	Notes
`c_cc[VEOF]`	end of file	No	Only processed by canonical mode line editing
`c_cc[VEOL]`	end of line	Yes	Only processed by canonical mode line editing
`c_cc[VERASE]`	"erase"	No	Only processed by canonical mode line editing
`c_cc[VKILL]`	"kill"	No	Only processed by canonical mode line editing
`c_cc[VINTR]`	"interrupt"	No	Signal-generation character independent of input mode
`c_cc[VQUIT]`	"quit"	No	Signal-generation character independent of input mode
`c_cc[VSUSP]`	"suspend"	No	Signal-generation character independent of input mode
`c_cc[VSTOP]`	"stop"	No	Modem flow control character independent of input mode
`c_cc[VSTART]`	"start"	No	Modem flow control character independent of input mode

The c_cc[] array member of the termios data structure specifies all of the (programmatically modifiable) special characters. The indexes into the array are symbolic constants, one for each special character type, as in the table at right. (Two further entries in the array are relevant to non-canonical mode input processing and are discussed below.)[43]

Non-programmatically modifiable special characters are linefeed (ASCII LF) and carriage return (ASCII CR).[47]

Input processing

Input processing determines the behaviour of the read() system call on a terminal device and the line editing and signal-generation characteristics of the line discipline. Unlike the case of Seventh Edition Unix and BSD version 4, and like the case of System III and System V, line editing operates in one of just two modes: canonical mode and non-canonical mode. The basic difference between them is when, from the point of view of the blocking/non-blocking requirements of the read() system call (specified with the O_NONBLOCK flag on the file descriptor via open() or fcntl()), data "are available for reading".[48]

Canonical mode processing

In canonical mode, data are accumulated in a line editing buffer, and do not become "available for reading" until line editing has been terminated by the user (at the terminal) sending a line delimiter character. Line delimiter characters are special characters, and they are end of file, end of line, and linefeed (ASCII LF). The former two are settable programmatically, whilst the latter is fixed. The latter two are included in the line editing buffer, whilst the former one is not.[49]

More strictly, zero or more lines are accumulated in the line editing buffer, separated by line delimiters (which may or may not be discarded once read() comes around to reading them), and line editing operates upon the part of the line editing buffer that follows the last (if any) line delimiter in the buffer. So, for example, the "erase" character (whatever that has been programmed to be) will erase the last character in the line buffer only up to (but not including) a preceding line delimiter.[49]

Non-canonical mode processing

In non-canonical mode, data are accumulated in a buffer (which may or may not be the line editing buffer — some implementations having separate "processed input" and "raw input" queues) and become "available for reading" according to the values of two input control parameters, the c_cc[MIN] and c_cc[TIME] members of the termios data structure. Both are unsigned quantities (because cc_t is required to be an alias for an unsigned type). The former specifies a minimum number of characters, and the latter specifies a timeout in tenths of a second.[50] There are four possibilities:

c_cc[TIME] and c_cc[MIN] are both zero: In this case, the data in the buffer are "available for reading" immediately, and read() returns immediately with whatever data are in the buffer (potentially returning zero if there are zero data available).[51]
c_cc[TIME] is non-zero and c_cc[MIN] is zero: In this case, the data in the buffer are "available for reading" after the specified timeout has elapsed, the timer being triggered by the start of the read() system call, or if a single character is received. In other words, read() waits for a maximum specified total time, and may return zero data, and returns any data as soon as they are received.[51]
c_cc[TIME] is zero and c_cc[MIN] is non-zero: In this case, the data in the buffer are "available for reading" after the specified number of characters have been received in the buffer. In other words, read() waits for a minimum amount of data (which may be larger than what the caller is prepared to read in the system call), will not return zero data, and may wait indefinitely.[51]
c_cc[TIME] and c_cc[MIN] are both non-zero: In this case, the data in the buffer are "available for reading" after the specified number of characters have been received in the buffer or the timeout has expired since the last character was entered. There is no timeout for the very first character. In other words, read() waits for a minimum amount of data (which may be larger than what the caller is prepared to read in the system call), will not return zero data, may wait indefinitely, but won't wait longer than the specified timeout if at least one character is in the buffer to be read.[51]

Output processing

Output processing is largely unchanged from its System III/System V roots. Output mode control flags determine various options:

Carriage returns may be inserted before each linefeed character, to translate Unix newline semantics to the ASCII semantics that many terminals expect.[27][22]
Terminals may be given time to exercise various control codes that would (on a teletypewriter or similar) result in physical movements of the carriage that may take significant (from the computer's point of view) amounts of time, such as backspaces, horizontal tabs, carriage returns, form feeds, and line feeds.[27][52]

Notes

Christian 1988, p. 11.
Bourne 1983, p. 6.
Coffin 1991, p. 820.
Coffin 1991, p. 23–24.
Leffler et al. 1989, p. 259.
Coffin 1991, p. 24.
Leffler et al. 1989, p. 37–38.
Afzal 2008, p. 419.
Frisch 2002, p. 770.
Coffin 1991, p. 115.
Coffin 1991, p. 372.
Coffin 1991, p. 779.
Coffin 1991, p. 751–752.
Leffler et al. 1989, p. 265.
Leffler et al. 1989, p. 103.
Leffler et al. 1989, p. 38.
Leffler et al. 1989, p. 260–261.
Leffler et al. 1989, p. 262.
Christian 1988, p. 395.
Bourne 1983, p. 8.
Bourne 1983, p. 130–131.
Bourne 1983, p. 287.
Christian 1988, p. 26.
Bourne 1983, p. 132–133.
Leffler et al. 1989, p. 259–260.
Bourne 1983, p. 288.
Leffler et al. 1989, p. 260.
Bourne 1983, p. 132.
Bourne 1983, p. 133.
Christian 1988, p. 393.
Leffler et al. 1989, p. 262–263.
"System III tty(4) man page source". Retrieved October 5, 2012.
Zlotnick 1991, p. 157.
Zlotnick 1991, p. 163.
Bourne 1983, p. 130.
Zlotnick 1991, p. 158.
Zlotnick 1991, p. 173–174.
Zlotnick 1991, p. 162.
Zlotnick 1991, p. 166.
Zlotnick 1991, p. 162–163.
Zlotnick 1991, p. 164.
Zlotnick 1991, p. 165.
Zlotnick 1991, p. 167.
Zlotnick 1991, p. 169.
Zlotnick 1991, p. 172.
Zlotnick 1991, p. 174.
Zlotnick 1991, p. 159.
Zlotnick 1991, p. 160.
Zlotnick 1991, p. 160–161.
Zlotnick 1991, p. 161.
Zlotnick 1991, p. 161–162.
Bourne 1983, p. 287–288.

Sources

Afzal, Amir (2008). UNIX unbounded: a beginning approach (5th ed.). Prentice Hall. ISBN 978-0-13-119449-6.CS1 maint: ref=harv (link)
Bourne, Stephen R. (1983). The UNIX system. International computer science series. Addison-Wesley. ISBN 978-0-201-13791-0.CS1 maint: ref=harv (link)
Christian, Kaare (1988). The UNIX Operating System (2nd ed.). John Wiley & Sons. ISBN 978-0-471-84781-6.CS1 maint: ref=harv (link)
Coffin, Stephen (1991). UNIX system V release 4: the complete reference. Osborne McGraw-Hill. ISBN 978-0-07-881653-6.CS1 maint: ref=harv (link)
Frisch, Æleen (2002). Essential system administration. A nutshell handbook (3rd ed.). O'Reilly Media, Inc. ISBN 978-0-596-00343-2.CS1 maint: ref=harv (link)
Leffler, Samuel J.; McKusick, Marshall Kirk; Karels, Michael J.; Quarterman, John S. (1989). "Terminal Handling". The Design and implementation of the 4.3BSD UNIX operating system. Addison-Wesley series in computer science. Addison-Wesley. ISBN 978-0-201-06196-3.CS1 maint: ref=harv (link)
Zlotnick, Fred (1991). "Controlling Terminal Devices". The POSIX.1 standard: a programmer's guide. Benjamin/Cummings Pub. Co. ISBN 978-0-8053-9605-8.CS1 maint: ref=harv (link)