Blob Blame History Raw



0- INTRODUCTION.

This document describes actual and proposed changes to the djvu
format since the release of the DjVu3 specification by Lizardtech in
november 2005.



1- AMBIGUOUS PARTS IN THE SPECIFICATION

1.1- BACKGROUND REDUCTION RATIO

The relation between the size (W,H) of the foreground image and the
size (w,h) of the background image is described in page 33 of the
specification.  The requirement ceil(W/w)=ceil(H/h) does not
completely capture the fact that the same reduction factor should be
applied to both dimensions.  The requirements should be read as:

<<There is a single integer reduction factor 1 <= k <= 12 such 
  that w=ceil(W/k) and h=ceil(H/k).  Should there be several
  such integers, the smallest one shall be considered
  to rescale the background image to the foreground resolution.>>

1.2- ENCODING OF URLS IN ANNOTATIONS

Page 16 omits to specify that URLs in annotations should be percent encoded. 
However, consistent with the encoding of strings in the annotation chunk, 
UTF-8 characters in URL strings will be interpreted properly as if
they were percent encoded.

1.3- FLAGS IN DJVUINFO CHUNKS

Page 24 specifies that {1,6,2,5} are the only four allowed 
values of the "flags" field in the INFO chunk.
To maximize compatibility with earlier versions of the standard,
values different from {1,6,2,5} should be ignored
and interpreted as 1 : rightside up orientation.



2- ESCAPE SEQUENCES IN ANNOTATION CHUNK STRINGS.

The treatment of escape sequence in annotation chunk strings has
historically been slightly different in Lizardtech DjVu and
DjVuLibre.  We are expecting that the DjVuLibre solution will
eventually become the standard.

Lizardtech DjVu uses the "old" rule described in section 8.3.4.2.
The sequence of characters BACKSLASH DOUBLEQUOTE represents a
DOUBLEQUOTE character without terminating the string.  There are no
other escape sequences. All other utf8 characters are written
directly. The main drawback of this approach is the inability to
write a string containing the sequence of characters BACKSLASH
DOUBLEQUOTE since there is no way to escape the first BACKSLASH
character.

DjVuLibre has introduced a more flexible scheme a few years ago.
Annotation strings are similar to strings in the C language.
Character sequences starting with a backslash have special meaning.
A BACKSLASH followed by "a", "b", "t", "n", "v", "f", "r", or "\"
stands for the ascii character BEL, BS, HT, LF, VT, FF, CR,
BACKSLASH or DOUBLEQUOTE.  A BACKSLASH followed by one to three
digits stands for the byte whose octal code is expressed by the
digits.  All other backslash sequences are illegal.  Non printable
ascii characters must be escaped. Multibyte characters should either
be entered directly, or represented using octal sequences.

DjVuLibre minimizes the compatibility problems by searching illegal
escape sequences in the annotation chunk.  If any illegal sequence
is found, the Lizardtech rule is used instead of the DjVuLibre rule.
It is expected that Lizardtech will at some point adopt the improved
DjVuLibre rules. We will then be able to state that all DjVu files
with version greater than some constant use the new convention.



3- PAGE TITLES.

Each page in a DjVu document is identified by three strings named
the ID, the NAME, and the TITLE. The semantic distinction between
these three strings is no longer very clear. DjVu libraries
does not work consistently when these strings are different.  See
the comment in the table, section 8.3.2.2 of the specification.

Recent versions of DjVuLibre still require that the ID and NAME
string are equal. The TITLE string however can be different and
should be used to display friendly page names. The djvused program
now features a command 'set-page-title' to install a TITLE different
from the ID string. The djview program then displays and recognizes
these page titles in lieu of the sequential page numbers.

Things get complicated when some page titles are purely numerical.
For instance, the page titled "4" might not be the fourth page of 
the document, and there might be many pages titled "4".
The djview4 viewer uses the following rules:

3.1- DJVU CGI ARGUMENT "PAGE=".

The djvu cgi argument "page=X" is interpreted as follows:
- It first searches a page whose ID matches X.
- Otherwise, if X has the form +N or -N where N is a number,
  this indicates a displacement relative to the current page.
- Otherwise, it searches a page with TITLE X starting
  from the current page and wrapping around.
- Otherwise, if X is numerical and in range, this is the page number.
- Otherwise, it searches a page whose NAME matches X.
 
3.2- PAGE REFERENCES IN MAPAREA AND OUTLINE LINKS

Page references in hyperlink annotations always have the form "#X".
These references are interpretes as follows.
- It first searches a page whose ID matches X.
- Otherwise, if X has the form +N or -N where N is a number,
  this indicates a displacement relative to the page containing the link.
- Otherwise, it searches a page with TITLE X starting
  from the current page and wrapping around.
- Otherwise, if X is numerical and in range, this is the page number.
- Otherwise, it searches a page whose NAME matches X.



4- METADATA

4.1- DJVULIBRE METADATA

DjVuLibre has introduced metadata annotations a few years ago.
Metadata entries for each page are represent by key/value pairs
located in a metadata directive in the annotation chunk.
Metadata entries for the document are represented similarly
using the methods described in the next section.

The metadata directive has the form

  (metadata ... (key "value") ... )

Each entry is identified by a symbol <key> 
representing the nature of the metadata entry.
The string <"value"> represents
the value associated with the corresponding key.

Several sets of keys are noteworthy.

* Keys borrowed from the BibTex bibliography system. 
  These key names are always expressed 
  in lowercase, such as 'year', 'booktitle', 'editor', 
  'author', etc.  

* Keys borrowed from the PDF DocInfo.
  These key names start with an uppercase letter:
  'Title', 'Author', 'Subject', 'Keywords', 'Creator', 
  'Producer', 'Trapped', 'CreationDate', and 'ModDate'.
  The values associated with the last two keys
  should be dates expressed according to RFC 3339.

4.2- XMP METADATA

The XMP specification describes a general purpose RDF/XML format for
metadata. Just like DjVuLibre metadata, XMP metadata is embedded in
an annotation chunk at the page or document level using the following
annotation directive

   (xmp "<rdf:RDF xmlns:rdf=... [escaped XMP here] ...</rdf:RDF>")

The sole argument of the xmp directive is the serialized XMP data
without the "xpacket" wrapper. The "x:xmpmeta" element may also be
dropped. Only elements from "rdf:RDF" inwards are needed.
Since the XMP data is represented as a string, doublequotes and
backslashes must be escaped.  Other characters may be escaped as well
(see section 2 above).

The full XMP specification is available from Adobe:

   http://www.adobe.com/devnet/xmp/

To maximize interoperability with current viewers, it is recommended
that XMP manipulation programs keep the DjVuLibre metadata in sync.
This is facilitated by synchronizing the PDF DocInfo keys with XMP
properties as follows:

   DocInfo key    XMP property
   ------------   ---------------
   Title          dc:title
   Author         dc:creator
   Subject        dc:description
   Keywords       pdf:Keywords
   Producer       pdf:Producer
   Trapped        pdf:Trapped
   Creator        xmp:CreatorTool
   CreationDate   xmp:CreateDate
   ModDate        xmp:ModifyDate

4.3- DOCUMENT ANNOTATIONS AND METADATA

The above schemes provide ways to specify metadata for each page.
But it is often useful to provide metadata that applies to the whole
document. Document wide metadata are represented using one or
several metadata directives in the shared annotations chunk.

This scheme has a potential drawback. Since the shared annotations
is included by all pages, the document wide metadata also appears as
page metadata for all pages. This might not be adequate for some
uses. As a workaround, the djview4 viewer only displays 
page metadata that differ from the document metadata.
A more definitive answer would be the definition of a document
annotation chunk located after the DIRM chunk and before any
component file. This space is already used by the NAVM chunk.  
This is being considered.



5- CGI STYLE OPTIONS IN MAPAREA AND OUTLINE LINKS

Outline and maparea annotation are
UTF-8 encoded strings that can be 
interpreted as page specification (see section 3.1)
or as percent-encoded URLs (see section 1.3.)

In addition, strings starting with a question mark '?' are 
interpreted as CGI style options separated by the 
ampersand character '&'.  These options are ignored
when the maparea link target is another window.
Otherwise these options are passed verbatim to the viewer.
This can cause portability problems because different djvu 
viewers support different sets of CGI style options.



6- THE "SECURE DJVU" FORMAT.

Recently Lizardtech introduced an incompatible "Secure DJVU" format.
This format encrypt djvu data in the hope of controlling
whether users can use, copy or print djvu documents.
A recent specification describes the container format
but does not provide enough information to decode the content.
Providing such an information would obviously provide
a means to avoid the usage restrictions. In fact there is no durable 
way to enforce such constraints besides "security through obscurity".

The current djvulibre library simply emits an 
error message when encountering such files.

Some observations regarding these files:

- They use the same IFF85 structure as djvu files.
  Chunk "SINF" contains a scrambled version of the decryption 
  key and describes which actions are authorized or denied.
  Chunks "CELX" encapsulate the regular DjVu chunks.  
  
- Each CELX chunk starts with four bytes for the original chunk 
  name, and four bytes for the original chunk length.
  This is followed encrypted data, composed of enough
  blocks of 8 bytes to covert the initial chunk length.

- Lizardtech claims encryption is 32 bit blowfish.
  It appears to be in fact composed of 64 bits block
  as you would expect with blowfish.