summaryrefslogtreecommitdiff
path: root/pith/IO.ReadMe
blob: 0a0286e8251e46f5d0d616120c1c58026a3c2520 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
Here's an attempt at a high-level description of the I/O flow and the
character set conversions for Alpine from the UNIX point of view. Pico and
Pine were developed separately so there is a big difference in the way
that Alpine handles stuff and the way the pico composer handles stuff.

INPUT

There's a low-level function called input_ready() that does a select
or a poll to see if an octet is ready to be read.  Alpine's read_char
and pico's GetKey call this:

   read_char                     Alpine
     check_for_timeout
       input_ready
or
   GetKey                        Pico
     ReadyForKey
       input_ready

Once they've decided an octet is ready to be read they use various versions
of ttgetc (ttgetc, simple_ttgetc, pine_simple_ttgetc) which all boil down
to a read of one octet from the keyboard. That incoming stream of
characters is a stream of multi-byte characters. For example, it might
be ASCII, UTF-8, ISO-8859-1, or EUC-JP. This would usually be configured
using the LANG or LC_CTYPE environment variables, or possibly using
the Alpine Keyboard-Character-Set or Display-Character-Set options.
The pith routine mbtow is used to convert this stream of bytes into
UCS-4 characters in the routine kbseq.

   read_char
     kbseq          (uses ttgetc to accumulate bytes)
       mbtow
or
   GetKey
     kbseq          ( " )
       mbtow

So read_char and GetKey both return UCS-4 characters from the keyboard.
They actually return a superset of a UCS-4 character (typedef UCS) that
is 32 bits wide. The superset comes about from the handling of
escape sequences and error conditions. Besides the 21 bits of UCS-4
characters other values that might be returned are values like KEY_UP,
KEY_RESIZE, F1-F12, PF1-PF12, NO_OP_COMMAND, NO_OP_IDLE, BADESC, KEY_JUNK,
and NODATA. The way that read_char and GetKey work are slightly different
from each other. GetKey has some special bits (CTRL, FUNC, MENU) that
it might OR together with a character and some of the return values
are possible in only one or the other function.

Pico is internally UCS-4. A CELL contains one character.
Alpine, on the other hand, uses UTF-8 internally. The input from read_char
is read by read_command, radio_buttons, and optionally_enter and is
then converted to UTF-8 before it is used internally in Alpine.

OUTPUT

All Alpine output to the display funnels through Writechar. The input to
Writechar is a stream of UTF-8 characters which are then converted to
the display's multi-byte character stream using c-client's utf8_get
to convert it to UCS-4 and then wtomb to convert to the multi-byte
representation.
The similar function in pico is ttputc (also known as t_putchar) which
takes an incoming stream of UCS-4 characters and converts it to multi-byte
characters for the display.




CHARACTER SET CONVERSIONS

Here is a simplified version of the complicated character set conversions
going on.

    ----------                          |--read_char
   | Keyboard |-ttgetc->-mbtow->-kbseq->|              (UCS-4)
    ----------         (to UCS-4)       |--GetKey


              |--optionally_enter               ----------------
   read_char->|--read_command    ------------> | Alpine internal|
              |--radio_buttons    (to UTF-8)   |     UTF-8      |
                                                ----------------
or
              |--pico
              |--LineEdit                       ---------- 
      GetKey->|--mlyesno         ------------> | Composer |
              |--mlreplyd                      |  UCS-4   |
              |--FileBrowse                     ---------- 


    ----------------                               ---------- 
   | Alpine internal| <-------pico_readc--------- | Composer |
   |    UTF-8       | --------pico_writec-------> |  UCS-4   |
    ----------------                               ---------- 


    ----------------                                   --------- 
   | Alpine internal| -Writechar->-utf8_get->-wtomb-> | Display |
   |    UTF-8       |                                 |  (LANG) |
    ----------------                                   --------- 

    ----------                                   ---------
   | Composer | ---pputc-->ttputc->-----wtomb-> | Display |
   |  UCS-4   |                                 |  (LANG) |
    ----------                                   ---------


    ----------------                                    -----------
   | Alpine internal| <-decode_text--------<-c-client- | MailStore |
   |    UTF-8       |   get_body_part_text             |           |
    ----------------                                    -----------


    ----------------                               ------------
   | Alpine internal| <-------------------------- | FileSystem |
   |    UTF-8       | --------------------------> |            |
    ----------------  see flags READ_FROM_LOCALE   ------------
                        and WRITE_TO_LOCALE

    ----------                                   ------------
   | Composer | <--------ffgetline------------- | FileSystem |
   |  UCS-4   | ---------ffputline------------> |            |
    ----------                                   ------------