Tabular Support for Simple Tables

=================================

Some definitions first:

* NO table support

  What it says. :)  Table related tags are treated like other

  completely unrecognized tags.

  Only listed for completeness, this does not describe Lynx.

* MINIMAL table support

  Table related tags are recognized, and are used to separate

  the contents of different cells (by at least a space) and rows

  (by a line break) visibly from each other.

* LYNX minimal table support (LMTS)

  The minimal table support as implemented by Lynx up to this point,

  also includes the way ALIGN attributes are handled on TABLE, TR

  and other specific tweaks (e.g. handle TABLE within PRE specially).

  LMTS formatting is briefly described in the Lynx User Guide, see

  the section "Lynx and HTML Tables" there. (The Users Guide has not

  yet been updated for tabular support.)

* TABULAR support for tables

  Support for tables that really arranges table cells in tabular form.

* Tabular Rendering for SIMPLE Tables (TRST)

  Tabular support for some tables that are 'simple' enough; what this

  code change provides.

One basic idea behind providing TRST is that correct tabular support

for all tables is complex, doesn't fit well into the overwhelmingly

one-pass way in which Lynx does things, and may in the end not give

pleasant results anyway for pages that (ab-)use more complex table

structures for display formatting purposes (especially in view of Lynx

limitations such as fixed character cell size and lack of horizontal

scrolling; see also emacs w3 mode).  Full table support within Lynx

hasn't happened so far, and continues to seem unlikely to happen in the

near future.

The other basic idea is the observation that for simple tables, as

used mostly for data that are really tabular in nature, LMTS rendering

can be transformed into TRST rendering, after parsing the TABLE element,

by two simple transformations applied line by line:

- Insert spaces in the right places.

- Shift the line as a whole.

And that's exactly what TRST does.  An implementation based on the

simple observation above is relatively straightforward, for simple

tables.  On encountering the start of a TABLE element, Lynx generates

output as usual for LMTS.  But it also keeps track of cell positions

and lengths in parallel.  If all goes well, that additional information

is used to fix up the already formatted output lines when the TABLE

ends.  If not all goes well, the table was not 'simple' enough, the

additional processing is canceled.  One advantage is that we always

have a 'safe' fallback to well-understood traditional LMTS formatting:

TRST won't make more complex tables look worse than before.

What are 'simple' tables?  A table is simple enough if each of its TR

rows translates into at most one display line in LMTS formatting (excluding

leading and trailing line breaks), and the width required by each row

(before as well as after fixup) does not exceed the available screen size.

Note that this excludes all tables where some of the cells are marked up as

block elements ('paragraphs').  Tables that include nested TABLE elements

are always specifically excluded, but the inner tables may be subject to

TRST handling.  Also excluded are some constructs that indicate that markup

was already optimized for Lynx (or other browsers with no or minimal table

support): TABLE in PRE, use of TAB.

The description so far isn't completely accurate.  In many cases, tables are

not simple enough according to the last paragraph, but parts of each TR row

can still benefit from some TRST treatment.  Some partial treatment is done

for some tables in this grey zone, which may or may not help to a better

display, depending on how the table is used.  This is an area where tweaks

in the future are most expected, and where the code's behavior is currently

not well defined.

One possible approach:

- The table is 'simple' according to all criteria set out in the previous

  paragraph, except that some cells at the beginning and/or end of TR rows

  may contain block elements (or other markup that results in formatting

  like separate paragraphs).

- There is at most one range of (non-empty) table cells in each row whose

  contents is not paragraph-formatted, and who are rendered on one line

  together by LMTS, separate from the paragraph-formatted cells.  Let's

  call these cells the 'core' of a row.

Fixups are then only applied to the text lines showing the 'core' cells.

The paragraph-formatted cells are effectively pulled out before/after

their row (no horizontal space is allocated to them for the purpose of

determining column widths for core line formatting).

This is expected to be most useful for tables that are mostly

simple tabular data cells, but with the occasional longer

text thrown in.  For example, a table with intended rendering:

      --------------------------------------------------------

     |  date  |   item no.   |  price  |       remarks        |

     |--------|--------------|---------|----------------------|

     | date-1 | item #1      |   $0.00 |                      |

     |--------|--------------|---------|----------------------|

     | date-2 | item #2      | $101.99 | A longer annotation  |

     |        |              |         | marked up as a block |

     |        |              |         | of text.             |

     |--------|--------------|---------|----------------------|

     | date-3 | long item #3 |  $99.00 |                      |

      -------------------------------------------------------- 

It may now be shown by Lynx as

    .................................................

      date    item no.    price  remarks

     date-1 item #1        $0.00

     date-2 item #2      $101.99

     A longer annotation marked up as a block of

     text.

     date-3 long item #3  $99.00

    .................................................

As can be seen, this is still quite far from the intended rendering,

but it is better than without any tabular support.

Whether the code does something sensible with "grey area" tables is up

for testing.  Most of the typical tables in typical Web pages aren't

used in a way that can benefit from the TRST approach.  Parts of such

tables may still end up getting shifted left or right by the TRST code

when that doesn't improve anything, but I haven't seen it make things

really worse so far (with the current code).

TRST and Partial Display

------------------------

[ Partial display mode is the feature which allows viewing and scrolling

of pages while they are loaded, without having to wait for a complete

transfer. ]  During partial display rendering, table lines can sometimes

be shown in the original formatting, i.e. with horizontal fixups not yet

applied.  This is more likely for longer tables, and depends on the state

in which partial display 'catches' the TRST code.  Sometimes the display

may flicker: first the preliminary rendering of table lines is shown, then

after loading is finished it is replaced by the fixed-up version.  I do

not consider this a serious problem: if you have partial display mode

enabled, presumably you want to be able to see as much data as possible,

and scroll up and down through it, as early as possible.  In fact, the

approach taken keeps Lynx free from a problem that may graphical browsers

have: they often cannot render a table at all until it is received in full.

------------------------------------------------------------------------

To summarize:

 - TRST is a solution that works in many cases where lack of tabular support

   was most annoying.

 - TRST doesn't implement a full table model, and it is extremely unlikely

   that it will ever be the basis for that.  Keep on exploring external

   solutions, or perhaps waiting for (better: working on) a more fundamental

   redesign of Lynx's rendering engine.

Klaus Weide - kweide@enteract.com 1999-10-13