diff options
author | Jeff Quast <contact@jeffquast.com> | 2015-04-13 20:47:54 -0700 |
---|---|---|
committer | Jeff Quast <contact@jeffquast.com> | 2015-04-13 20:47:54 -0700 |
commit | 22d74eb1bc0004c5f079eb43850c45bec3b58c7c (patch) | |
tree | 6ff7b097eb7281224867300483908d586732df07 /docs | |
parent | a5582ca20073fd2f6f3ced320e5a923f151ce2df (diff) | |
parent | aa94c2fd1c4d70559b931ff068adad8715fc6972 (diff) | |
download | blessings-22d74eb1bc0004c5f079eb43850c45bec3b58c7c.tar.gz |
Merge pull request #94 from erikrose/blessed-integration-docs-further-reading-and-growing-pains
(5) Blessed integration docs further reading and growing pains
Diffstat (limited to 'docs')
-rw-r--r-- | docs/further.rst | 106 | ||||
-rw-r--r-- | docs/index.rst | 2 | ||||
-rw-r--r-- | docs/pains.rst | 380 |
3 files changed, 488 insertions, 0 deletions
diff --git a/docs/further.rst b/docs/further.rst new file mode 100644 index 0000000..4847814 --- /dev/null +++ b/docs/further.rst @@ -0,0 +1,106 @@ +Further Reading +=============== + +As a developer's API, blessings is often bundled with frameworks and toolsets +that dive deeper into Terminal I/O programming than :class:`~.Terminal` offers. +Here are some recommended readings to help you along: + +- `terminfo(5) + <http://www.openbsd.org/cgi-bin/man.cgi?query=terminfo&apropos=0&sektion=5>`_ + manpage of your preferred posix-like operating system. The capabilities + available as attributes of :class:`~.Terminal` are directly mapped to those + listed in the column **Cap-name**. + +- `termios(4) + <http://www.openbsd.org/cgi-bin/man.cgi?query=termios&apropos=0&sektion=4>`_ + of your preferred posix-like operating system. + +- `The TTY demystified + <http://www.linusakesson.net/programming/tty/index.php>`_ by + Linus Åkesson. + +- `A Brief Introduction to Termios + <https://blog.nelhage.com/2009/12/a-brief-introduction-to-termios/>`_ by + Nelson Elhage. + +- Richard Steven's `Advance Unix Programming + <http://www.amazon.com/exec/obidos/ISBN=0201563177/wrichardstevensA/>`_ + ("AUP") provides two very good chapters, "Terminal I/O" and + "Pseudo Terminals". + +- GNU's `The Termcap Manual + <https://www.gnu.org/software/termutils/manual/termcap-1.3/html_mono/termcap.html>`_ + by Richard M. Stallman. + +- `Chapter 4 <http://compsci.hunter.cuny.edu/~sweiss/course_materials/unix_lecture_notes/chapter_04.pdf>`_ + of CUNY's course material for *Introduction to System Programming*, by + `Stewart Weiss <http://compsci.hunter.cuny.edu/~sweiss/>`_ + +- `Chapter 11 + <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html>`_ + of the IEEE Open Group Base Specifications Issue 7, "General Terminal + Interface" + +- The GNU C Library documentation, section `Low-Level Terminal Interface + <http://www.gnu.org/software/libc/manual/html_mono/libc.html#toc-Low_002dLevel-Terminal-Interface-1>`_ + +- The source code of many popular terminal emulators. If there is ever any + question of "the meaning of a terminal capability", or whether or not your + preferred terminal emulator actually handles them, read the source! + + These are often written in the C language, and directly map the + "Control Sequence Inducers" (CSI, literally ``\x1b[`` for most modern + terminal types) emitted by most terminal capabilities to an action in a + series of ``case`` switch statements. + + - Many modern libraries are now based `https://github.com/GNOME/vte + <libvte>`_ (or just 'vte'): Gnome Terminal, sakura, Terminator, Lilyterm, + ROXTerm, evilvte, Termit, Termite, Tilda, tinyterm, lxterminal. + - `Thomas E. Dickey <http://invisible-island.net/>` has been maintaining + `xterm <http://invisible-island.net/xterm/xterm.html>`_, as well as a + primary maintainer of many related packages such as `ncurses + <http://invisible-island.net/ncurses/ncurses.html>`_ for quite a long + while. There is often speculation and misinformation among developers of + terminal emulators and programs that interact with them. Thomas Dickey's + analysis is always thorough and complete. + - xterm, urxvt, SyncTerm, and EtherTerm. + - There are far too many to name, Chose one you like! Thomas Dickey + + +- The source code of the tty(4), pty(4), and the given "console driver" for + any posix-like operating system. If you search thoroughly enough, you will + eventually discover a terminal sequence decoder, usually a ``case`` switch + that translates ``\x1b[0m`` into a "reset color" action towards the video + driver. Though ``tty.c`` is linked here (the only kernel file common among + them), it is probably not the most interesting, but it can get you started: + + - `FreeBSD <https://github.com/freebsd/freebsd/blob/master/sys/kern/tty.c>`_ + - `OpenBSD <http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/sys/kern/tty.c?content-type=text/plain>`_ + - `Illumos (Solaris) <https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/io/tty_common.c>` + - `Minix <https://github.com/minix3/minix/blob/master/minix/drivers/tty/tty/tty.c>`_ + - `Linux <https://github.com/torvalds/linux/blob/master/drivers/tty/n_tty.c>`_ + + The TTY driver is a great introduction to Kernel and Systems programming, + because familiar components may be discovered and experimented with. It is + available on all operating systems (except windows), and because of its + critical nature, examples of efficient file I/O, character buffers (often + implemented as "ring buffers") and even fine-grained kernel locking can be + found. + +- `termcap & terminfo (O'Reilly Nutshell) + <http://www.amazon.com/termcap-terminfo-OReilly-Nutshell-Linda/dp/0937175226>`_ + by Linda Mui, Tim O'Reilly, and John Strang. + +- Note that System-V systems, also known as `Unix98 + <http://en.wikipedia.org/wiki/Single_UNIX_Specification>`_ (SunOS, HP-UX, + AIX and others) use a `Streams <http://en.wikipedia.org/wiki/STREAMS>`_ + interface. On these systems, the `ioctl(2) + <http://pubs.opengroup.org/onlinepubs/009695399/functions/ioctl.html>`_ + interface provides the ``PUSH`` and ``POP`` parameters to communicate with + a Streams device driver, which differs significantly from Linux and BSD. + + Many of these systems provide compatible interfaces for Linux, but they may + not always be as complete as the counterpart they emulate, most especially + in regards to managing pseudo-terminals. + + diff --git a/docs/index.rst b/docs/index.rst index eabb7d7..986f483 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -11,6 +11,8 @@ Contents: intro overview api + further + pains history ======= diff --git a/docs/pains.rst b/docs/pains.rst new file mode 100644 index 0000000..5d95d1e --- /dev/null +++ b/docs/pains.rst @@ -0,0 +1,380 @@ +Growing Pains +============= + +When constructing rich console applications, there are a surprisingly great +number of portability issues. Though Blessings provides an abstraction for +the full terminal capability database to ensure compatibility across many +types of terminals, there are still many portability issues that the +terminal :attr:`~.kind` alone cannot abstract for you. + +This section intends to raise awareness and guide you through these kinds of +issues. + + +8 and 16 colors +--------------- + +Where 8 and 16 colors are used, they should be assumed to be the +`CGA Color Palette`_. Though there is no terminal standard that proclaims +that the CGA colors are used, their values are the best approximations +across all common hardware terminals and terminal emulators. + +A recent phenomenon of users is to customize their base 16 colors to provide +(often, more "washed out") color schemes. Furthermore, we are only recently +getting LCD displays of colorspaces that achieve close approximation to the +original video terminals. Some find these values uncomfortably intense: in +their original CRT form, their contrast and brightness was lowered by hardware +dials, whereas today's LCD's typically display well only near full intensity. + +Though one can not *trust*, much less *query* through Terminal I/O routines +the colorspace of the remote terminal, **we can**: + +- Trust that a close approximation of the `CGA Color Palette`_ for the base + 16 colors will be displayed for **most** users. + +- Trust that users who have made the choice to adjust their palette have made + the choice to do so, and are able to re-adjust such palettes as necessary + to accommodate different programs (such as through the use of "Themes"). + +.. note:: + + It has become popular to use dynamic system-wide color palette adjustments + in software such as `f.lux`_, which adjust the system-wide "Color Profile" + of the entire graphics display depending on the time of day. One might + assume that ``term.blue("text")`` may be **completely** invisible to such + users during the night! + + +Where is brown, purple, or grey? +-------------------------------- + +There are **only 8 color names** on even a 16-color terminal: The second +eight colors are "high intensity" versions of the first in direct series. The +colors *brown*, *purple*, and *grey* are not named in the first series, though +they are available! + +- **brown**: *yellow is brown*, only high-intensity yellow + (``bright_yellow``) is yellow! + +- **purple**: *magenta is purple*. In earlier, 4-bit color spaces, there + were only black, cyan, magenta, and white of low and high intensity, such + as found on a common home computer of the time such as the `ZX Spectrum`_. + + Additional "colors" were only possible through dithering. The color names + cyan and magenta on later graphics adapters are carried over from its + predecessors. Although the color cyan remained true in RGB value on + 16-color to its predecessor, magenta shifted farther towards blue from red + becoming purple (as true red was introduced as one of the base 8 colors). + +- **grey**: there are actually **three shades of grey** (or American spelling, + 'gray'), though the color attribute named 'grey' does not exist! + + In ascending order of intensity, the shades of grey are: + + - ``bold_black``: in lieu of the uselessness of an "intense black", this is + color is instead mapped to "dark grey". + - ``white``: white is actually mild compared to the true color 'white': this + is more officially mapped to "common grey", and is often the default + foreground color. + - ``bright_white``: is pure white (``#ffffff``). + + +white-on-black +~~~~~~~~~~~~~~ + +The default foreground and background should be assumed as *white-on-black*. + +For quite some time, the families of terminals produced by DEC, IBM, and +Tektronix dominated the computing world with the default color scheme of +*green-on-black* and less commonly *amber-on-black* monochrome displays: +The inverse was a non-default configuration. The IBM 3270 clients exclusively +used *green-on-black* in both hardware and software emulators, and is likely +a driving factor of the default *white-on-black* appearance of the first IBM +Personal Computer. + +The less common *black-on-white* "ink paper" style of emulators is a valid +concern for those designing terminal interfaces. The color scheme of +*black-on-white* directly conflicts with the intention of `bold is bright`_, +where ``term.bright_red('ATTENTION!')`` may become difficult to read, +as it appears as *pink on white*! + + +History of ink-paper inspired black-on-white +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Early home computers with color video adapters, such as the Commodore 64 +provided *white-on-blue* as their basic video terminal configuration. One can +only assume such appearances were provided to demonstrate their color +capabilities over competitors (such as the Apple ][). + +More common, X11's xterm and the software HyperTerm bundle with MS Windows +provided an "ink on paper" *black-on-white* appearance as their default +configuration. Two popular emulators continue to supply *black-on-white* by +default to this day: Xorg's xterm and Apple's Terminal.app. + +.. note:: Windows no longer supplies a terminal emulator: the "command prompt" + as we know it now uses the MSVCRT API routines to interact and does not + make use of terminal sequences, even ignoring those sequences that MS-DOS + family of systems previously interpreted through the ANSI.SYS driver, + though it continues to default to *white-on-black*. + + +Bold is bright +-------------- + +**Where Bold is used, it should be assumed to be *Bright***. + +Due to the influence of early graphics adapters providing a set of 8 +"low-intensity" and 8 "high intensity" versions of the first, the term +"bold" for terminals sequences is synonymous with "high intensity" in +almost all circumstances. + + +History of bold as "wide stroke" +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In typography, the true translation of "bold" is that a font should be +displayed *with emphasis*. In classical terms, this would be achieved by +pen be re-writing over the same letters. On a teletype or printer, this was +similarly achieved by writing a character, backspacing, then repeating the +same character in a form called **overstriking**. + +To bold a character, ``C``, one would emit the sequence ``C^HC`` where +``^H`` is backspace (0x08). To underline ``C``, one would would emit +``C^H_``. + +**Video terminals do not support overstriking**. Though the mdoc format for +manual pages continue to emit overstriking sequences for bold and underline, +translators such as mandoc will instead emit an appropriate terminal sequence. + +Many characters previously displayable by combining using overstriking of +ASCII characters on teletypes, such as: ±, ≠, or ⩝ were delegated to a +`code page`_ or lost entirely until the introduction of multibyte encodings. + +Much like the "ink paper" introduction in windowing systems for terminal +emulators, "wide stroke" bold was introduced only much later when combined +with operating systems that provided font routines such as TrueType. + + +Enforcing white-on-black +~~~~~~~~~~~~~~~~~~~~~~~~ + +In conclusion, *white-on-black* should be considered the default. If there is +a need to **enforce** *white-on-black* for terminal clients suspected to be +defaulted as *black-on-white*, one would want to trust that a combination of +``term.home + term.white_on_black + term.clear`` should repaint the entire +emulator's window with the desired effect. + +However, this cannot be trusted to **all** terminal emulators to perform +correctly! Depending on your audience, you may instead ensure that the +entire screen (including whitespace) is painted using the ``on_black`` +mnemonic. + +Beware of customized color schemes +---------------------------------- + +A recent phenomenon is for users to customize these first 16 colors of their +preferred emulator to colors of their own liking. Though this has always been +possible with ``~/.XResources``, the introduction of PuTTy and iTerm2 to +interactively adjustment these colors have made this much more common. + +This may cause your audience to see your intended interface in a wildly +different form. Your intended presentation may appear mildly unreadable. + +Users are certainly free to customize their colors however they like, but it +should be known that displaying ``term.black_on_red("DANGER!")`` may appear +as "grey on pastel red" to your audience, reducing the intended effect of +intensity. + + +256 colors can avoid customization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first instinct of a user who aliases ls(1) to ``ls -G`` or ``colorls``, +when faced with the particularly low intensity of the default ``blue`` +attribute is **to adjust their terminal emulator's color scheme of the base +16 colors**. + +This is not necessary: the environment variable ``LSCOLORS`` may be redefined +to map an alternative color for blue, or to use ``bright_blue`` in its place. + +Furthermore, all common terminal text editors such as emacs or vim may be +configured with "colorschemes" to make use of the 256-color support found in +most modern emulators. Many readable shades of blue are available, and many +programs that emit such colors can be configured to emit a higher or lower +intensity variant from the full 256 color space through program configuration. + + +Monochrome and reverse +---------------------- + +Note that ``reverse`` takes the current foreground and background colors and +reverses them. In contrast, the compound formatter ``black_on_red`` would +fail to set the background *or* foreground color on a monochrome display, +resulting in the same stylization as ``normal`` -- it would not appear any +different! + +If your userbase consists of monochrome terminals, you may wish to provide +"lightbars" and other such effects using the compound formatter +``red_reverse``. In the literal sense of "set foreground color to red, then +swap foreground and background", this produces a similar effect on +**both** color and monochrome displays. + +For text, very few ``{color}_on_{color}`` formatters are visible with the +base 16 colors, so you should generally wish for ``black_on_{color}`` +anyway. By using ``{color}_reverse`` you may be portable with monochrome +displays as well. + + +Multibyte Encodings and Code pages +---------------------------------- + +A terminal that supports both multibyte encodings (UTF-8) and legacy 8-bit +code pages (ISO 2022) may instruct the terminal to switch between both +modes using the following sequences: + + - ``\x1b%@``: Select default character set, that is ISO 8859-1 (ISO 2022). + - ``\x1b%G``: Select UTF-8 character set (ISO 2022). + +When a terminal is in ISO 2022 mode, you may use a sequence +to request a terminal to change its `code page`_. It begins by ``\x1b(``, +followed by an ASCII character representing a code page selection. For +example ``\x1b(U`` on the VGA Linux console switches to the `IBM CP437`_ +`code page`_, allowing MS-DOS artwork to be displayed in its natural +8-bit byte encoding. A list of standard codes and the expected code page +may be found on Thomas E. Dickey's xterm control sequences section on +sequences following the `Control-Sequence-Inducer`_. + +Detecting multibyte +~~~~~~~~~~~~~~~~~~~ + +One can be assured that the connecting client is capable of representing +UTF-8 and other multibyte character encodings by the Environment variable +``LANG``. If this is not possible, there is an alternative method: + + - Emit Report Cursor Position (CPR), ``\x1b[6n`` and store response. + - Emit a multibyte UTF-8 character, such as ⦰ (``\x29\xb0``). + - Emit Report Cursor Position (CPR), ``\x1b[6n`` and store response. + - Determine the difference of the *(y, x)* location of the response. + If it is *1*, then the client decoded the two UTF-8 bytes as a + single character, and can be considered capable. If it is *2*, + the client is using a `code page`_ and is incapable of decoding + a UTF-8 bytestream. + +Note that both SSH and Telnet protocols provide means for forwarding +the ``LANG`` environment variable. However, some transports such as +a link by serial cable is incapable of forwarding Environment variables. + +Detecting screen size +~~~~~~~~~~~~~~~~~~~~~ + +While we're on the subject, there are times when :attr:`height` and +:attr:`width` are not accurate -- when a transport does not provide the means +to propagate the COLUMNS and ROWS Environment values, or propagate the +SIGWINCH signals, such as through a serial link. + +The same means described above for multibyte encoding detection may be used to +detect the remote client's window size: + + - Move cursor to row 999, 999. + - Emit Report Cursor Position (CPR), ``\x1b[6n`` and store response. + - The return value is the window dimensions of the client. + +This is the method used by the program ``resize`` provided in the Xorg +distribution, and its source may be viewed as file `resize.c`_. + +Alt or meta sends Escape +------------------------ + +Programs using GNU readline such as bash continue to provide default mappings +such as *ALT+u/l* to uppercase/lowercase word after cursor. This is achieved +by the configuration option, altSendsEscape or `metaSendsEscape +<http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Alt-and-Meta-Keys>`_ + +The default for most terminals, however, is that the meta key is bound by +the operating system (such as *META+f* for find), and that alt is used for +inserting international keys, where the combination *ALT+u, a* is used +to insert the character ``ä``. + +The ability to detect alt or meta key combinations is achieved by two means: + + - The alt or meta key sets the 8th bit "high", so that *ALT+z* becomes + the value of ``'z'`` + 128: ``ú`` + - The alt or meta key prefaces the combining character with escape, so + that *ALT+z* becomes *Escape + z*: ``\x1bz`` + +It is therefore a recommendation to **avoid alt or meta keys entirely** in +applications, and instead prefer the ctrl-key combinations, so as to avoid +instructing your users to configure their terminal emulators to communicate +such sequences. + + +Backspace sends delete +---------------------- + +Typically, backspace is ``^H`` (8, or 0x08) and delete is ^? (127, or 0x7f). + +On some systems however, the key for backspace is actually labeled and +transmitted as "delete", though its function in the operating system behaves +just as backspace. It is highly recommend to accept **both** ``KEY_DELETE`` +and ``KEY_BACKSPACE`` as having the same meaning except when implementing full +screen editors. + +And only then, to provide the choice to map delete as true delete as a +non-default configuration option. + + +The misnomer of ANSI +-------------------- + +When people say 'ANSI Sequence', they are discussing: + +- Standard `ECMA-48`_: Control Functions for Coded Character Sets + +- `ANSI X3.64 <http://sydney.edu.au/engineering/it/~tapted/ansi.html>`_ from + 1981, when the `American National Standards Institute + <http://www.ansi.org/>`_ adopted the `ECMA-48`_ as standard, which was later + withdrawn in 1997 (so in this sense it is *not* an ANSI standard). + +- The `ANSI.SYS`_ driver provided in MS-DOS and + clones. The popularity of the IBM Personal Computer and MS-DOS of the era, + and its ability to display colored text further populated the idea that such + text "is ANSI". + +- The `IBM CP437`_ `code page`_ (which provided "block art" characters) paired + with `ECMA-48`_ sequences supported by the MS-DOS ANSI.SYS driver to create + artwork, known as `ANSI art <http://sixteencolors.net/>`_. + + This is purely an American misnomer, because early IBM PC and clones in the + European nations did not ship with the `IBM CP437`_ `code page`_ by default. + + Many people now mistake the difference between "ASCII art" and "ANSI art" to + be whether or not block art and other characters from the `IBM CP437`_ + code page are used. Where even such "ASCII art" may contain `ECMA-48`_ + color codes! + +- The ANSI terminal database entry and its many descendants in the + `terminfo database + <http://invisible-island.net/ncurses/terminfo.src.html>`_. This is mostly + due to terminals compatible with SCO UNIX, which was the successor of + Microsoft's Xenix, which brought some semblance of the Microsoft DOS + `ANSI.SYS`_ driver capabilities. + +- `Select Graphics Rendition (SGR) <http://vt100.net/docs/vt510-rm/SGR>`_ + on vt100 clones, which include many of the common sequences in `ECMA-48`_. + +- Any sequence started by the `Control-Sequence-Inducer`_ is often + mistakenly termed as an "ANSI Escape Sequence" though not appearing in + `ECMA-48`_ or interpreted by the `ANSI.SYS`_ driver. The adjoining phrase + "Escape Sequence" is so termed because it follows the ASCII character + for the escape key (ESC, ``\x1b``). + +.. _code page: http://en.wikipedia.org/wiki/Code_page +.. _IBM CP437: http://en.wikipedia.org/wiki/Code_page_437 +.. _CGA Color Palette: http://en.wikipedia.org/wiki/Color_Graphics_Adapter#With_an_RGBI_monitor +.. _f.lux: https://justgetflux.com/ +.. _ZX Spectrum: http://en.wikipedia.org/wiki/List_of_8-bit_computer_hardware_palettes#ZX_Spectrum +.. _Control-Sequence-Inducer: http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Controls-beginning-with-ESC +.. _resize.c: http://www.opensource.apple.com/source/X11apps/X11apps-13/xterm/xterm-207/resize.c +.. _ANSI.SYS: http://www.kegel.com/nansi/ +.. _ECMA-48: http://www.ecma-international.org/publications/standards/Ecma-048.htm |