1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
|
Here's an attempt at a high-level description of the I/O flow and the
character set conversions for Alpine from the UNIX point of view. Pico and
Pine were developed separately so there is a big difference in the way
that Alpine handles stuff and the way the pico composer handles stuff.
INPUT
There's a low-level function called input_ready() that does a select
or a poll to see if an octet is ready to be read. Alpine's read_char
and pico's GetKey call this:
read_char Alpine
check_for_timeout
input_ready
or
GetKey Pico
ReadyForKey
input_ready
Once they've decided an octet is ready to be read they use various versions
of ttgetc (ttgetc, simple_ttgetc, pine_simple_ttgetc) which all boil down
to a read of one octet from the keyboard. That incoming stream of
characters is a stream of multi-byte characters. For example, it might
be ASCII, UTF-8, ISO-8859-1, or EUC-JP. This would usually be configured
using the LANG or LC_CTYPE environment variables, or possibly using
the Alpine Keyboard-Character-Set or Display-Character-Set options.
The pith routine mbtow is used to convert this stream of bytes into
UCS-4 characters in the routine kbseq.
read_char
kbseq (uses ttgetc to accumulate bytes)
mbtow
or
GetKey
kbseq ( " )
mbtow
So read_char and GetKey both return UCS-4 characters from the keyboard.
They actually return a superset of a UCS-4 character (typedef UCS) that
is 32 bits wide. The superset comes about from the handling of
escape sequences and error conditions. Besides the 21 bits of UCS-4
characters other values that might be returned are values like KEY_UP,
KEY_RESIZE, F1-F12, PF1-PF12, NO_OP_COMMAND, NO_OP_IDLE, BADESC, KEY_JUNK,
and NODATA. The way that read_char and GetKey work are slightly differnt
from each other. GetKey has some special bits (CTRL, FUNC, MENU) that
it might OR together with a character and some of the return values
are possible in only one or the other function.
Pico is internally UCS-4. A CELL contains one character.
Alpine, on the other hand, uses UTF-8 internally. The input from read_char
is read by read_command, radio_buttons, and optionally_enter and is
then converted to UTF-8 before it is used internally in Alpine.
OUTPUT
All Alpine output to the display funnels through Writechar. The input to
Writechar is a stream of UTF-8 characters which are then converted to
the display's multi-byte character stream using c-client's utf8_get
to convert it to UCS-4 and then wtomb to convert to the multi-byte
representation.
The similar function in pico is ttputc (also known as t_putchar) which
takes an incoming stream of UCS-4 characters and converts it to multi-byte
characters for the display.
CHARACTER SET CONVERSIONS
Here is a simplified version of the complicated character set conversions
going on.
---------- |--read_char
| Keyboard |-ttgetc->-mbtow->-kbseq->| (UCS-4)
---------- (to UCS-4) |--GetKey
|--optionally_enter ----------------
read_char->|--read_command ------------> | Alpine internal|
|--radio_buttons (to UTF-8) | UTF-8 |
----------------
or
|--pico
|--LineEdit ----------
GetKey->|--mlyesno ------------> | Composer |
|--mlreplyd | UCS-4 |
|--FileBrowse ----------
---------------- ----------
| Alpine internal| <-------pico_readc--------- | Composer |
| UTF-8 | --------pico_writec-------> | UCS-4 |
---------------- ----------
---------------- ---------
| Alpine internal| -Writechar->-utf8_get->-wtomb-> | Display |
| UTF-8 | | (LANG) |
---------------- ---------
---------- ---------
| Composer | ---pputc-->ttputc->-----wtomb-> | Display |
| UCS-4 | | (LANG) |
---------- ---------
---------------- -----------
| Alpine internal| <-decode_text--------<-c-client- | MailStore |
| UTF-8 | get_body_part_text | |
---------------- -----------
---------------- ------------
| Alpine internal| <-------------------------- | FileSystem |
| UTF-8 | --------------------------> | |
---------------- see flags READ_FROM_LOCALE ------------
and WRITE_TO_LOCALE
---------- ------------
| Composer | <--------ffgetline------------- | FileSystem |
| UCS-4 | ---------ffputline------------> | |
---------- ------------
|
