<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="GENERATOR" content="Microsoft FrontPage 6.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<title>Boost Tokenizer Overview</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink=
"#FF0000">
<p><img src="../../boost.png" alt="C++ Boost" width="277" height=
"86"><br></p>
<h1 align="center">Table Of Contents</h1>
<p align="left"> </p>
<h2 align="left"><a href="introduc.htm">Introduction</a></h2>
<h2 align="left">Containers and Iterators</h2>
<ul>
<li>
<h3 align="left"><a href="tokenizer.htm">tokenizer</a></h3>
</li>
<li>
<h3 align="left"><a href="token_iterator.htm">token iterator</a></h3>
</li>
</ul>
<h2><a href="tokenizerfunction.htm">TokenizerFunction Concept</a></h2>
<h2>TokenizerFunction Models</h2>
<ul>
<li>
<h3><a href="char_separator.htm">char_separator</a></h3>
</li>
<li>
<h3><a href=
"escaped_list_separator.htm">escaped_list_separator</a></h3>
</li>
<li>
<h3><a href="offset_separator.htm">offset_separator</a></h3>
</li>
<li><font color="red">Deprecated:</font> <a href=
"char_delimiters_separator.htm">char_delimiters_separator</a></li>
</ul>
<h2> </h2>
<h2>Acknowledgements</h2>
<p>I wish to thank the members of the boost mailing list, whose comments,
compliments, and criticisms during both the development and formal review
helped make the Tokenizer library what it is. I especially wish to thank
Aleksey Gurtovoy for the idea of using a pair of iterators to specify the
input, instead of a string. I also wish to thank Jeremy Siek for his idea
of providing a container interface for the token iterators and for
simplifying the template parameters for the TokenizerFunctions. He and
Daryle Walker also emphasized the need to separate interface and
implementation. Gary Powell sparked the idea of using the isspace and
ispunct as the defaults for char_delimiters_separator. Jeff Garland
provided ideas on how to change to order of the template parameters in
order to make tokenizer easier to declare. Thanks to Douglas Gregor who
served as review manager and provided many insights both on the boost list
and in e-mail on how to polish up the implementation and presentation of
Tokenizer. Finally, thanks to Beman Dawes who integrated the final version
into the boost distribution.</p>
<hr>
<p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src=
"../../doc/images/valid-html401.png" alt="Valid HTML 4.01 Transitional"
height="31" width="88"></a></p>
<p>Revised
<!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25
December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p>
<p><i>Copyright © 2000 Jeremy Siek<br>
Copyright © 2001 John R. Bandela</i></p>
<p><i>Distributed under the Boost Software License, Version 1.0. (See
accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or
copy at <a href=
"http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p>
</body>
</html>