Blame ChangeLog

Packit 709fb3
2017-07-02  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	version 3.1
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2017-07-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: avoid false failures when run in qemu user mode
Packit 709fb3
	* tests/filename-lineno.pl: Derive the program name that grep
Packit 709fb3
	will use in diagnostics, based on a suggestion from Assaf Gordon.
Packit 709fb3
	* tests/in-eq-out-infloop: Similar: accept an arbitrary "command_name: "
Packit 709fb3
	prefix on checked diagnostics, rather than requiring "grep: ".
Packit 709fb3
	* tests/reversed-range-endpoints: Likewise.
Packit 709fb3
	* tests/write-error-msg: Likewise.
Packit 709fb3
	Reported by Bruno Haible in http://bugs.gnu.org/27532
Packit 709fb3
Packit 709fb3
2017-06-25  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
	* gnulib: Update to latest for these portability fixes:
Packit 709fb3
	- stat: port to xlc 12.01
Packit 709fb3
	- xalloc-oversized: port to icc
Packit 709fb3
Packit 709fb3
	doc: fix another typo
Packit 709fb3
	* doc/grep.texi (File and Directory Selection): Fix typo: s/afer/after/
Packit 709fb3
Packit 709fb3
2017-06-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	doc: stop calling --perl-regexp (-P) "highly" experimental
Packit 709fb3
	Use wording that is less likely to make readers think that
Packit 709fb3
	support for -P may be removed.
Packit 709fb3
	* doc/grep.in.1: s/highly experimental/experimental/
Packit 709fb3
	* doc/grep.texi: Likewise.
Packit 709fb3
	Suggested by Evan Sheahan.
Packit 709fb3
Packit 709fb3
2017-06-21  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	doc: correct typo
Packit 709fb3
	* doc/grep.texi (Performance): s/suprisingly/surprisingly/
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2017-06-21  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -m no longer cuts off trailing context
Packit 709fb3
	Problem reported by Markus Jochim (Bug#26254).
Packit 709fb3
	* NEWS, doc/grep.texi (General Output Control): Document this.
Packit 709fb3
	* src/grep.c (prpending): Selected lines no longer cut off context.
Packit 709fb3
	(usage): Say "selected" instead of "matching", where appropriate.
Packit 709fb3
	* tests/foad1, tests/max-count-vs-context, tests/yesno:
Packit 709fb3
	Adjust to match new behavior.
Packit 709fb3
Packit 709fb3
2017-05-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Document grep performance
Packit 709fb3
	* doc/grep.texi (Performance): New section.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2017-05-21  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: make the announcement template Cc the devel- list
Packit 709fb3
	* cfg.mk (announcement_Cc_): Define.
Packit 709fb3
Packit 709fb3
	gnulib: update to latest; and update tests/init.sh
Packit 709fb3
Packit 709fb3
	maint: accommodate GCC7's -Werror=duplicated-branches
Packit 709fb3
	* src/system.h (IGNORE_DUPLICATE_BRANCH_WARNING): Define.
Packit 709fb3
	* src/grep.c (grepfile): Use it.
Packit 709fb3
	* src/kwset.c (bmexec, acexec): Use it.
Packit 709fb3
Packit 709fb3
	maint: update to work with GCC7's -Werror=implicit-fallthrough=
Packit 709fb3
	* src/system.h (FALLTHROUGH): Define.
Packit 709fb3
	* src/grep.c (context_length_arg): Use new FALLTHROUGH macro in place
Packit 709fb3
	of comments
Packit 709fb3
	(fgrep_to_grep_pattern, try_fgrep_pattern, main): Likewise.
Packit 709fb3
Packit 709fb3
2017-05-13  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest and adapt src/kwset.c
Packit 709fb3
	* gnulib: Update to latest.
Packit 709fb3
	* src/kwset.c: Include "verify.h" for use of assume.
Packit 709fb3
Packit 709fb3
2017-03-22  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest for dfa [0-9] performance improvement
Packit 709fb3
	This pulls in the following change that is very relevant to grep:
Packit 709fb3
Packit 709fb3
	    commit 6afba02d7869d39ed7f61981045ddbdcb2814101
Packit 709fb3
	    Author: Paul Eggert <eggert@cs.ucla.edu>
Packit 709fb3
	    dfa: make [0-9] faster in non-C locales
Packit 709fb3
Packit 709fb3
	* gnulib: Update to latest.
Packit 709fb3
	* NEWS (Improvements): Describe the effect on grep.
Packit 709fb3
Packit 709fb3
2017-03-05  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	build: use $(builddir), not $(srcdir)
Packit 709fb3
	* cfg.mk (PATH): Use $(builddir), so this also takes effect
Packit 709fb3
	in a non-srcdir build.  Also, switch ${PATH} syntax to $(PATH).
Packit 709fb3
Packit 709fb3
2017-03-05  Juan Manuel Guerrero  <juan.guerrero@gmx.de>
Packit 709fb3
Packit 709fb3
	build: use $(PATH_SEPARATOR), not ":" to augment PATH
Packit 709fb3
	* cfg.mk (PATH): Use $(PATH_SEPARATOR), for those systems that
Packit 709fb3
	use something other than ":".
Packit 709fb3
	* THANKS.in: Remove name, to avoid syntax-check failure due to
Packit 709fb3
	the duplicate, now that there is this commit.
Packit 709fb3
Packit 709fb3
2017-02-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: fix distcheck failure: remove stale dosbuf.c reference
Packit 709fb3
	* src/Makefile.am (EXTRA_DIST): Do not attempt to distribute
Packit 709fb3
	the recently deleted file, dosbuf.c.
Packit 709fb3
Packit 709fb3
	maint: fix new syntax-check errors
Packit 709fb3
	* po/POTFILES.in: Add lib/xbinary-io.c.
Packit 709fb3
	* cfg.mk (FILTER_LONG_LINES): Add TODO to the list of exempt files.
Packit 709fb3
Packit 709fb3
2017-02-16  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Fix up recent -U patches
Packit 709fb3
	Inspired by a suggestion by Eric Blake (Bug#25707#17).
Packit 709fb3
	* bootstrap.conf (gnulib_modules): Add xbinary-io,
Packit 709fb3
	and remove binary-io and xfreopen.
Packit 709fb3
	* doc/grep.texi (Other Options):
Packit 709fb3
	Fix typo and reword to be a bit more general.
Packit 709fb3
	* src/grep.c: Include xbinary-io.h instead of xfreopen.h.
Packit 709fb3
	(grepfile): Open with O_BINARY if binary.
Packit 709fb3
	(grepdesc): No need for set_binary_mode now.
Packit 709fb3
	(grep_command_line_arg, main): Set stdin to binary mode if binary.
Packit 709fb3
	(main): Avoid unnecessary test of stdin == NULL.
Packit 709fb3
	Use xsetmode instead of xfreopen.
Packit 709fb3
	* src/system.h: Do not include binary-io.h.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
	Simplify -U on MS-Windows by removing guesswork
Packit 709fb3
	Suggested by Eric Blake (Bug#25707#11).
Packit 709fb3
	* NEWS, doc/grep.texi: Document this.
Packit 709fb3
	* src/dosbuf.c: Remove.
Packit 709fb3
	* bootstrap.conf (gnulib_modules): Add xfreopen.
Packit 709fb3
	* src/grep.c: Include xfreopen.h, not dosbuf.c.
Packit 709fb3
	(fillbuf, print_line_head): Do not undossify input.
Packit 709fb3
	(binary): New static var.
Packit 709fb3
	(grepdesc): Apply BINARY to input file.
Packit 709fb3
	(usage): Remove -u help.
Packit 709fb3
	(main): Set BINARY if -U, and apply it to stdout.  Do nothing if -u.
Packit 709fb3
	With -f, apply BINARY to input file.
Packit 709fb3
Packit 709fb3
2017-02-16  Eric Blake  <eblake@redhat.com>
Packit 709fb3
Packit 709fb3
	grep: don't forcefully strip carriage returns
Packit 709fb3
	Commit 5c92a54 made the mistaken assumption that using fopen("rt")
Packit 709fb3
	on platforms where O_TEXT is non-zero makes sense.  However, POSIX
Packit 709fb3
	already requires fopen("r") to open a file in text mode, vs.
Packit 709fb3
	fopen("rb") when binary mode is wanted, and at least on Cygwin,
Packit 709fb3
	where it is possible to control whether a mount point is binary
Packit 709fb3
	or text by default (using just "r"), the use of fopen("rt") actively
Packit 709fb3
	breaks assumptions on a binary mount by silently corrupting any
Packit 709fb3
	carriage returns that are supposed to be preserved.
Packit 709fb3
Packit 709fb3
	* src/grep.c (main): Never use fopen("rt") (Bug#25707).
Packit 709fb3
Packit 709fb3
2017-02-13  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Update TODO and doc
Packit 709fb3
	* TODO: Bring up-to-date and fix formatting glitches.
Packit 709fb3
	* doc/grep.in.1, doc/grep.texi: Fix minor glitches.
Packit 709fb3
	The above patches should address the same problems that recent
Packit 709fb3
	Debian doc patches address, albeit in a different way.
Packit 709fb3
Packit 709fb3
2017-02-12  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: clarify default input (Bug#25651)
Packit 709fb3
	* doc/grep.in.1:
Packit 709fb3
	* src/grep.c (usage): Clarify default input when -r.
Packit 709fb3
	* src/grep.c (usage): Do not bother documenting egrep and fgrep;
Packit 709fb3
	the manual is enough.
Packit 709fb3
Packit 709fb3
2017-02-09  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 3.0
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2017-02-08  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: do not mishandle \. in multiple patterns
Packit 709fb3
	Problem reported by Lars Wendler (Bug#25655).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (try_fgrep_pattern): Fix typo that prevented
Packit 709fb3
	keys from being properly updated.
Packit 709fb3
	* tests/foad1: Test for the bug.
Packit 709fb3
Packit 709fb3
2017-02-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Do not assume PCRE 8.20 or later
Packit 709fb3
	Problem reported by Zube (Bug#25647)
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/pcresearch.c (struct pcre.com.jit_stack):
Packit 709fb3
	Declare only if PCRE_STUDY_JIT_COMPILE.
Packit 709fb3
Packit 709fb3
2017-02-06  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.28
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2017-02-02  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2017-02-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: tune to avoid memchr2 sometimes
Packit 709fb3
	Problem noted by Norihiro Tanaka in:
Packit 709fb3
	http://lists.gnu.org/archive/html/grep-devel/2017-01/msg00027.html
Packit 709fb3
	Although not enough to restore all the previous performance in the
Packit 709fb3
	case he noted, it helps significantly.
Packit 709fb3
	* src/kwset.c (memchr_kwset): Bring back small_heuristic,
Packit 709fb3
	in a somewhat different form.
Packit 709fb3
Packit 709fb3
2017-01-29  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2017-01-23  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: simplify recent kwset change
Packit 709fb3
	* src/kwset.c (acexec_trans): Simplify.
Packit 709fb3
Packit 709fb3
2017-01-23  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: really add the new test name
Packit 709fb3
	* tests/Makefile.am (TESTS): Add fgrep-longest.
Packit 709fb3
Packit 709fb3
2017-01-21  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep -Fo could report a match that is not the longest
Packit 709fb3
	* src/kwset.c (acexec): Fix it.
Packit 709fb3
	* tests/fgrep-longest: New test.
Packit 709fb3
	* tests/Makefile.am: Add the test.
Packit 709fb3
	* NEWS: Mention it.
Packit 709fb3
Packit 709fb3
2017-01-18  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: speed up Aho-Corasick when at most 2 bytes
Packit 709fb3
	When using Aho-Corasick and all matched strings either begin with
Packit 709fb3
	the same byte, or begin with one of at most two bytes, use memchr2
Packit 709fb3
	to search for these matching bytes and apply the Aho-Corasick
Packit 709fb3
	algorithm only when a memchr2 match is found.  On my platform,
Packit 709fb3
	this speeds up 'grep -F -e aa -e ba in' by a factor of 7, where
Packit 709fb3
	the file 'in' was created by 'seq -f %040.0f 10000000 >in'.
Packit 709fb3
	* src/kwset.c (struct kwset.gc1): Now int, not char.
Packit 709fb3
	If negative, there is no single terminal byte.  All uses changed.
Packit 709fb3
	(struct kwset.gc1help): Now int, not char.
Packit 709fb3
	If negative, memchr2 cannot be used.
Packit 709fb3
	(kwsprep): Set up gc1 and gc1help from kwset->next, with
Packit 709fb3
	the new (slightly changed) interpretation.
Packit 709fb3
	(memchr_kwset): Use memchr2 if possible.
Packit 709fb3
	Adjust to match new meaning of gc1, gc1help.
Packit 709fb3
	(memoff2_kwset): Remove; no longer needed.
Packit 709fb3
	(acexec_trans): Use memchr_kwset when possible, for speed.
Packit 709fb3
	It now supersedes memoff2_kwset.
Packit 709fb3
Packit 709fb3
	grep: remove Commentz-Walter code
Packit 709fb3
	This code was not being used, and complicated maintenance.
Packit 709fb3
	We can bring it back from the repository if it turns out
Packit 709fb3
	to be useful later.
Packit 709fb3
	* src/kwset.c (struct kwset.reverse): Remove.  All uses of
Packit 709fb3
	FOO->reverse replaced by (FOO->kwsexec == bmexec).
Packit 709fb3
	(kwsalloc): Remove 'reverse' arg, as callers outside this
Packit 709fb3
	module do not care about algorithm choice.  All callers changed.
Packit 709fb3
	(kwsprep): When deciding whether to use Boyer-Moore, do not worry
Packit 709fb3
	about being called twice on the same kwset, as that is not allowed.
Packit 709fb3
	(cwexec): Remove; it was never called.  All uses removed.
Packit 709fb3
Packit 709fb3
2017-01-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: avoid new syntax-check failures
Packit 709fb3
	* src/kwset.c (struct kwset): Split a line longer than 80.
Packit 709fb3
	* bootstrap: Update from gnulib. This fixes a new syntax-check
Packit 709fb3
	failure due to its use of "time stamp".
Packit 709fb3
Packit 709fb3
2017-01-17  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	* NEWS: Fix typo.
Packit 709fb3
Packit 709fb3
	* src/kwset.c: Fix comment typo.
Packit 709fb3
Packit 709fb3
	Improve -i performance in typical UTF-8 searches
Packit 709fb3
	Currently ‘grep -i i’ is slow in a UTF-8 locale, because ‘i’ in
Packit 709fb3
	the pattern matches the two-byte character 'ı' (U+0131, LATIN
Packit 709fb3
	SMALL LETTER DOTLESS I) in data, and kwset handles only
Packit 709fb3
	single-byte character translations, so grep falls back on a slower
Packit 709fb3
	DFA-based search for all searches.  Improve -i performance in the
Packit 709fb3
	typical case by using kwset when data are free of troublesome
Packit 709fb3
	characters like 'ı', falling back on the DFA only when data
Packit 709fb3
	contain troublesome characters.
Packit 709fb3
	* src/dfasearch.c (GEAcompile):
Packit 709fb3
	* src/grep.c (compile_fp_t):
Packit 709fb3
	* src/kwsearch.c (Fcompile):
Packit 709fb3
	* src/pcresearch.c (Pcompile):
Packit 709fb3
	Pattern arg is now char *, not char const *, since Fcompile
Packit 709fb3
	now reallocates it sometimes.
Packit 709fb3
	* src/grep.c (all_single_byte_after_folding): Remove.
Packit 709fb3
	All callers removed.
Packit 709fb3
	(fgrep_icase_charlen): New function.
Packit 709fb3
	(fgrep_icase_available, try_fgrep_pattern):
Packit 709fb3
	Use it, for more-generous semantics.
Packit 709fb3
	(fgrep_to_grep_pattern): Now extern.
Packit 709fb3
	(main): Do not free keys, since Fexecute may use them.
Packit 709fb3
	* src/kwsearch.c (struct kwsearch): New struct.
Packit 709fb3
	(Fcompile): Return it.  If -i, be more generous about patterns.
Packit 709fb3
	(Fexecute): Use it.  Fall back on DFA when the data contain
Packit 709fb3
	troublesome characters; this should be rare in practice.
Packit 709fb3
	* src/kwset.c, src/kwset.h (kwswords): New function.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2017-01-15  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: prefer ptrdiff_t to size_t
Packit 709fb3
	The code already cannot handle objects with size greater than
Packit 709fb3
	SIZE_MAX / 2, so be more honest about it and use ptrdiff_t instead
Packit 709fb3
	of size_t.  ptrdiff_t arithmetic is signed, which allows for more
Packit 709fb3
	checking via -fsanitize=undefined.  It also makes the code a tad
Packit 709fb3
	smaller on x86-64, since it can test for < 0 rather than for ==
Packit 709fb3
	SIZE_MAX.
Packit 709fb3
	* src/dfasearch.c (struct dfa_comp.kwset_exact_matches):
Packit 709fb3
	(kwsmusts, EGexecute):
Packit 709fb3
	* src/kwsearch.c (Fcompile, Fexecute):
Packit 709fb3
	* src/kwset.c (struct kwset.kwsexec, kwsincr, memchr_kwset)
Packit 709fb3
	(memoff2_kwset, bmexec_trans, bmexec, cwexec, acexec_trans)
Packit 709fb3
	(acexec, kwsexec):
Packit 709fb3
	* src/kwset.h (struct kwsmatch.index, .offset, .size):
Packit 709fb3
	Prefer ptrdiff_t to size_t where either will do.
Packit 709fb3
Packit 709fb3
2017-01-11  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: improve comments, mostly in kwset
Packit 709fb3
	Remove kwset.h comments that are obsolete and seemingly not
Packit 709fb3
	maintained anyway; people can look in kwset.c instead.
Packit 709fb3
	Update comments to reflect current behavior better.
Packit 709fb3
	Cite Faro & Lecroq 2013.  Use GNU style for end-of-sentence.
Packit 709fb3
Packit 709fb3
2017-01-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: update gnulib and copyright dates for 2017
Packit 709fb3
	* gnulib: Update to latest.
Packit 709fb3
	* all files: Run "make update-copyright".
Packit 709fb3
Packit 709fb3
2016-12-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: speed up -x with many patterns
Packit 709fb3
	* src/kwsearch.c (Fcompile): Improve buffer allocation overhead
Packit 709fb3
	with -x and multiple patterns.  In the common case where '\n' is
Packit 709fb3
	the end-of-line byte, avoid copying other than the first and last
Packit 709fb3
	patterns.
Packit 709fb3
Packit 709fb3
2016-12-31  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest, fixing a parallel getopt test failure
Packit 709fb3
Packit 709fb3
2016-12-29  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	maint: space before paren
Packit 709fb3
Packit 709fb3
	grep: int cleanup in kwset.c
Packit 709fb3
	This should affect only theoretical bugs with very large inputs.
Packit 709fb3
	On my platform, this patch shrinks the grep text by 136 bytes.
Packit 709fb3
	* src/kwset.c: Include intprops.h, for INT_MULTIPLY_WRAPV.
Packit 709fb3
	(struct trie, struct kwset, kwsalloc, kwsincr, treedelta, kwsprep)
Packit 709fb3
	(bm_delta2_search, bmexec_trans, cwexec): Prefer ptrdiff_t to int
Packit 709fb3
	when counts can exceed INT_MAX in large inputs, at least in theory.
Packit 709fb3
	(hasevery): Use bool for booleans.
Packit 709fb3
	(bmexec_trans): Avoid undefined behavior on integer overflow.
Packit 709fb3
Packit 709fb3
2016-12-27  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: improve performance with multiple patterns
Packit 709fb3
	* src/grep.c (main): Avoid fgrep-to-grep conversion for word matching
Packit 709fb3
	with multiple patterns in single byte locales.
Packit 709fb3
Packit 709fb3
2016-12-27  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	* NEWS: Fix typo.
Packit 709fb3
Packit 709fb3
	grep: fix bug with '... | grep pat >> /dev/null'
Packit 709fb3
	Problem reported by Benno Fünfstück (Bug#25283).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (drain_input) [SPLICE_F_MOVE]:
Packit 709fb3
	Don't assume /dev/null is always acceptable output to splice.
Packit 709fb3
	* tests/grep-dev-null-out: Test for the bug.
Packit 709fb3
Packit 709fb3
2016-12-26  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: minor performance tweak for pure functions
Packit 709fb3
	* src/search.h (wordchars_size, wordchar_next, wordchar_prev):
Packit 709fb3
	Declare to be pure.
Packit 709fb3
Packit 709fb3
2016-12-25  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	grep: move localeinfo to grep.c
Packit 709fb3
	It's not really dfasearch-specific, and grep.c initializes it, so it
Packit 709fb3
	seems like the most appropriate "owner".
Packit 709fb3
Packit 709fb3
	* src/dfasearch.c (localeinfo): Remove.
Packit 709fb3
	* src/grep.c (localeinfo): Add.
Packit 709fb3
	* src/search.h (localeinfo): Move to new commented section.
Packit 709fb3
Packit 709fb3
2016-12-25  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	pcresearch: thread safety
Packit 709fb3
	* src/pcresearch.c (pcre_comp): New struct to hold previously-global
Packit 709fb3
	state.
Packit 709fb3
	(jit_exec): Operate on a pcre_comp parameter instead of global state.
Packit 709fb3
	(Pcompile): Allocate and return a pcre_comp instead of setting global
Packit 709fb3
	variables.
Packit 709fb3
	(Pexecute): Operate on a pcre_comp parameter instead of global state.
Packit 709fb3
Packit 709fb3
	kwsearch: thread safety
Packit 709fb3
	* src/kwsearch.c (Fcompile): Return a kwset_t instead of setting a
Packit 709fb3
	global variable.
Packit 709fb3
	(Fexecute): Use a passed-in kwset_t instead of a global variable.
Packit 709fb3
	(kwset): Remove global variable.
Packit 709fb3
Packit 709fb3
	dfasearch: thread safety
Packit 709fb3
	* src/dfasearch.c (struct dfa_comp): New struct to hold
Packit 709fb3
	previously-global variables.
Packit 709fb3
	(dfawarn): Remove static variable.
Packit 709fb3
	(kwsmusts): Operate on a dfa_comp parameter instead of global
Packit 709fb3
	variables.
Packit 709fb3
	(GEAcompile): Allocate and return a dfa_comp struct instead of setting
Packit 709fb3
	global variables.
Packit 709fb3
	(EGexecute): Operate on a dfa_comp parameter instead of global
Packit 709fb3
	variables.
Packit 709fb3
	* src/searchutils.c (kwsinit): Replace a static array with a
Packit 709fb3
	dynamically-allocated one.
Packit 709fb3
Packit 709fb3
2016-12-25  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	grep: prepare search backends for thread-safety
Packit 709fb3
	To facilitate removing mutable global state from search backends,
Packit 709fb3
	compile() functions will return an opaque pointer to backend-specific
Packit 709fb3
	data, which must then be passed back into the corresponding execute()
Packit 709fb3
	function.  This is merely a preparatory step changing function
Packit 709fb3
	signatures and call sites, so the pointers passed & returned are
Packit 709fb3
	dummies for now and not (yet) actually used.
Packit 709fb3
Packit 709fb3
	* src/grep.c (compile_fp_t): Now returns an opaque pointer (the
Packit 709fb3
	compiled pattern).
Packit 709fb3
	(execute_fp_t): Now passed the pointer returned by a compile_fp_t.
Packit 709fb3
	All call sites updated accordingly.
Packit 709fb3
	(compiled_pattern): New static variable.
Packit 709fb3
	* src/dfasearch.c (GEAcompile): Return a void pointer (dummy NULL).
Packit 709fb3
	(EGexecute): Receive a void pointer argument (unused).
Packit 709fb3
	* src/kwsearch.c (Fcompile): Return a void pointer (dummy NULL).
Packit 709fb3
	(Fexecute): Receive a void pointer argument (unused).
Packit 709fb3
	* src/pcresearch.c (Pcompile): Return a void pointer (dummy NULL).
Packit 709fb3
	(Pexecute): Receive a void pointer argument (unused).
Packit 709fb3
	* src/search.h: Update compile/execute function prototypes.
Packit 709fb3
Packit 709fb3
2016-12-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: fix "syntax-check" failure
Packit 709fb3
	* src/grep.c (SEP_STR_GROUP): Declare "static".
Packit 709fb3
Packit 709fb3
2016-12-23  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: fix comment in searchutils.c
Packit 709fb3
Packit 709fb3
	grep: improve word checking with UTF-8
Packit 709fb3
	* src/searchutils.c: Do not include <verify.h>.
Packit 709fb3
	(word_start): Remove, replacing with ...
Packit 709fb3
	(sbwordchar): New static var.  All uses changed.
Packit 709fb3
	(wordchar_prev): Return size_t, not bool, as this generates
Packit 709fb3
	slightly better code.  Go back faster if UTF-8.
Packit 709fb3
Packit 709fb3
	grep: standardize on localeinfo.multibyte
Packit 709fb3
	* src/dfasearch.c (EGexecute):
Packit 709fb3
	* src/grep.c (main):
Packit 709fb3
	* src/kwsearch.c (Fexecute):
Packit 709fb3
	* src/pcresearch.c (Pcompile):
Packit 709fb3
	Prefer localeinfo.multibyte to (MB_CUR_MAX > 1).
Packit 709fb3
Packit 709fb3
	grep: speed up -wf in C locale
Packit 709fb3
	Problem reported by Norihiro Tanaka (Bug#22357#100).
Packit 709fb3
	This patch improves the performance on that benchmark on my
Packit 709fb3
	platform so that grep is now only about 2x slower than grep 2.26,
Packit 709fb3
	which means it is considerably faster than grep 2.25 and earlier.
Packit 709fb3
	* src/kwsearch.c (Fexecute):
Packit 709fb3
	Use wordchars_size to boost performance for this case.
Packit 709fb3
	* src/search.h, src/searchutils.c (wordchars_size): New function.
Packit 709fb3
Packit 709fb3
	grep: specialize word-finding functions
Packit 709fb3
	This improves performance a bit.
Packit 709fb3
	* src/dfasearch.c, src/kwsearch.c (wordchar):
Packit 709fb3
	Remove; now in searchutils.c.
Packit 709fb3
	* src/grep.c (main): Call wordinit if -w.
Packit 709fb3
	* src/search.h: Adjust.
Packit 709fb3
	* src/searchutils.c: Include verify.h.
Packit 709fb3
	(word_start): New static var.
Packit 709fb3
	(wordchar): Move here from dfasearch.c and kwsearch.c.
Packit 709fb3
	(wordinit, wordchars_count, wordchar_next, wordchar_prev):
Packit 709fb3
	New functions.
Packit 709fb3
	(mb_prev_wc, mb_next_wc): Remove.
Packit 709fb3
	All callers changed to use the new functions instead.
Packit 709fb3
Packit 709fb3
	grep: simplify Fexecute
Packit 709fb3
	* src/kwsearch.c (Fexecute): Avoid the need for a 'try' local or
Packit 709fb3
	for a 'goto success'.  Update mb_start to reflect newline found.
Packit 709fb3
Packit 709fb3
	grep: remove C label
Packit 709fb3
	* src/kwsearch.c (Fexecute): Remove label.
Packit 709fb3
Packit 709fb3
	maint: rewrite to avoid some macros
Packit 709fb3
	These days, the dangerous powers of C macros are not needed if
Packit 709fb3
	constants or functions will do just as well.
Packit 709fb3
	* src/grep.c (SEP_CHAR_SELECTED, SEP_CHAR_REJECTED, SEP_STR_GROUP)
Packit 709fb3
	(INITIAL_BUFSIZE):
Packit 709fb3
	* src/kwset.c (DEPTH_SIZE):
Packit 709fb3
	Now constants, not macros.
Packit 709fb3
	* src/kwset.c (link): Remove macro. Instead, rename local vars
Packit 709fb3
	from 'link' to 'cur'.
Packit 709fb3
	(malloc) [GREP]: Remove macro.  All uses of malloc changed to xmalloc.
Packit 709fb3
	Omit double-inclusion of xalloc.h.  Do not depend on 'GREP'.
Packit 709fb3
	(U): Now a function, not a macro.
Packit 709fb3
	* src/kwset.c, src/searchutils.c (NCHAR): Move this macro to ...
Packit 709fb3
	* src/system.h: ... here, and make it a constant.
Packit 709fb3
Packit 709fb3
2016-12-20  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: fix performance with multiple patterns
Packit 709fb3
	Problem reported by Jaroslav Skarvada (Bug#22357).
Packit 709fb3
	* NEWS: Document this and other recent performance fixes.
Packit 709fb3
	* src/grep.c (E_MATCHER_INDEX): New constant.
Packit 709fb3
	(all_single_byte_after_folding):
Packit 709fb3
	New function, split out from fgrep_icase_available.
Packit 709fb3
	(fgrep_icase_available): Use it.
Packit 709fb3
	(try_fgrep_pattern): New function, which also uses it.
Packit 709fb3
	(main): With two or more patterns, use try_fgrep_pattern to fix
Packit 709fb3
	performance regression.  The number "two" here is just a heuristic.
Packit 709fb3
Packit 709fb3
	grep: simplify matcher configuration
Packit 709fb3
	* src/grep.c (matcher, compile): Remove static vars.
Packit 709fb3
	(compile_fp_t): Now takes a 3rd syntax argument.
Packit 709fb3
	(Gcomppile, Ecompile, Acompile, GAcompile, PAcompile): Remove.
Packit 709fb3
	(struct matcher): Now nameless, since it is used only once.
Packit 709fb3
	Make 'name' a bit shorter.  New member 'syntax'.
Packit 709fb3
	(matchers): Initialize it, and change removed functions to GEAcompile.
Packit 709fb3
	(F_MATCHER_INDEX, G_MATCHER_INDEX): New constants.
Packit 709fb3
	(setmatcher): New arg MATCHER, and return new matcher index.
Packit 709fb3
	Avoid unnecessary call to strcmp.
Packit 709fb3
	(main): Keep matcher as a local int, not a global pointer.
Packit 709fb3
	* src/kwsearch.c (Fcompile):
Packit 709fb3
	* src/pcresearch.c (Pcompile): Ignore the 3rd syntax argument.
Packit 709fb3
Packit 709fb3
	grep: simplify line counting in patterns
Packit 709fb3
	* src/grep.c (n_patterns): Rename from patfile_lineno,
Packit 709fb3
	as it is now origin-zero.  Now size_t, not uintmax_t.
Packit 709fb3
	(count_nl_bytes, fl_add): Simplify to just buffer and size.
Packit 709fb3
	All callers changed.
Packit 709fb3
Packit 709fb3
2016-12-19  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-12-18  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-12-13  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: use just-built grep in more places
Packit 709fb3
	* cfg.mk (PATH): Prepend $(srcdir)/src, so that we use the just-
Packit 709fb3
	built grep also when running commands like those of "make distcheck".
Packit 709fb3
	This would have avoided the recently-luckily-noticed infloop bug.
Packit 709fb3
	Tested by running this in a just-built directory:
Packit 709fb3
	f=src/grep; printf '%s\n' '#!/bin/sh' 'sleep 9h' > $f; chmod a+x $f
Packit 709fb3
	and then verifying that nearly every "make syntax-check" rule hangs.
Packit 709fb3
Packit 709fb3
	maint: tell "syntax-check" not to worry about the NEWS update
Packit 709fb3
	Whenever we change "old" NEWS, we have to update this checksum.
Packit 709fb3
	Otherwise, a "make syntax-check" test that guards against a class
Packit 709fb3
	of logical merge conflicts will fail.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Update this hash to accommodate the
Packit 709fb3
	recent clarification of a 2.27 NEWS entry.
Packit 709fb3
Packit 709fb3
2016-12-13  Arnold D. Robbins  <arnold@skeeve.com>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
	* src/dfasearch.c (GEAcompile): Remove use of flag, RE_ICASE covers it.
Packit 709fb3
Packit 709fb3
2016-12-12  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: work around proc lseek glitch
Packit 709fb3
	Problem reported by Andreas Schwab (Bug#25180).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (finalize_input): Ignore EINVAL lseek failures.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add proc.
Packit 709fb3
	* tests/proc: New file.
Packit 709fb3
Packit 709fb3
2016-12-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: simplify finalize_input
Packit 709fb3
	* src/grep.c (finalize_input): Simplify without changing behavior.
Packit 709fb3
	It's still a bit of a rat's-nest, but it's a cozier rat's-nest.
Packit 709fb3
Packit 709fb3
	maint: clarify early-exit news for 2.27
Packit 709fb3
	* NEWS: Mention early-exit options to avoid confusion.  See:
Packit 709fb3
	http://lists.gnu.org/archive/html/grep-devel/2016-12/msg00007.html
Packit 709fb3
Packit 709fb3
2016-12-06  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.27
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2016-11-29  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep: fix DFA-induced infloop
Packit 709fb3
	* gnulib: Update to latest, for the DFA infloop fix.
Packit 709fb3
	* tests/dfa-infloop: New test, to trigger an infinite loop
Packit 709fb3
	in the DFA matcher.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2016-11-28  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: use "returns_ N env VAR=val ..."
Packit 709fb3
	rather than "VAR=val returns_ N ..."
Packit 709fb3
	Some shells do not propagate envvar settings through our use
Packit 709fb3
	of the "returns_" function, so set any envvar via use of "env".
Packit 709fb3
	This was an issue at least on Ubuntu and Debian-based systems,
Packit 709fb3
	presumably due to their common use of "dash" as /bin/sh.
Packit 709fb3
	Reported by Assaf Gordon.
Packit 709fb3
	* tests/char-class-multibyte: As above.
Packit 709fb3
	* tests/euc-mb: Likewise.
Packit 709fb3
	* tests/false-match-mb-non-utf8: Likewise.
Packit 709fb3
	* tests/pcre-infloop: Likewise.
Packit 709fb3
	* tests/pcre-jitstack: Likewise.
Packit 709fb3
	* tests/sjis-mb: Likewise.
Packit 709fb3
	* tests/warn-char-classes: Likewise.
Packit 709fb3
Packit 709fb3
2016-11-28  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: revert check for unibyte French range bug
Packit 709fb3
	The test wasn't portable, as it assumed that rational ranges
Packit 709fb3
	were not in effect.  Problem reported by Eric Blake (Bug#25048#8).
Packit 709fb3
	There doesn't seem to be a portable way to do the test, so omit it.
Packit 709fb3
	* tests/init.cfg, tests/unibyte-bracket-expr:
Packit 709fb3
	Revert previous change.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-11-27  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep: avoid false matches in non-UTF8 multibyte locales
Packit 709fb3
	* gnulib: Update to latest, for the dfa.c fix.
Packit 709fb3
	* NEWS (Bug fixes): Mention it.
Packit 709fb3
	* tests/false-match-mb-non-utf8: New file, with tests for this.
Packit 709fb3
	Based on tests from Stephane Chazelas.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	Introduced by commit v2.18-54-g3ef4c8e, a change that made grep use
Packit 709fb3
	its DFA matcher more aggressively.  The malfunction arises only with
Packit 709fb3
	the DFA matcher, not with regex.
Packit 709fb3
	Reported by Stephane Chazelas in https://bugs.gnu.org/24975
Packit 709fb3
Packit 709fb3
2016-11-20  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: check for unibyte French range bug
Packit 709fb3
	Problem reported by Stephane Chazelas (Bug#24973).
Packit 709fb3
	This bug was fixed in Gnulib.
Packit 709fb3
	* NEWS: Document the fix.
Packit 709fb3
	* tests/init.cfg (require_ru_RU_koi8_r): Remove.
Packit 709fb3
	* tests/unibyte-bracket-expr: Add a test for the bug.
Packit 709fb3
	Call get-mb-cur-max directly instead of bothering with
Packit 709fb3
	require_ru_RU_koi8_r.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-11-19  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: further -P performance fix
Packit 709fb3
	Problem reported by Stephane Chazelas in:
Packit 709fb3
	http://bugs.gnu.org/22655#103
Packit 709fb3
	* src/pcresearch.c (Pexecute): Set the subject to the start of
Packit 709fb3
	each line as it is found.
Packit 709fb3
Packit 709fb3
	grep: -P no longer uses PCRE_MULTILINE
Packit 709fb3
	This reverts commit f6603c4e1e04dbb87a7232c4b44acc6afdf65fef,
Packit 709fb3
	as the extra performance is not worth the trouble for PCRE users.
Packit 709fb3
	Problem reported by Stephane Chazelas in:
Packit 709fb3
	http://bugs.gnu.org/22655#103
Packit 709fb3
	* NEWS: Document this and the next patch.
Packit 709fb3
	* src/dfasearch.c (EGexecute):
Packit 709fb3
	* src/grep.c (execute_fp_t):
Packit 709fb3
	* src/kwsearch.c (Fexecute):
Packit 709fb3
	* src/pcresearch.c (Pexecute):
Packit 709fb3
	First arg is now a const pointer again.
Packit 709fb3
	* src/grep.c (buf_has_encoding_errors): Now static.
Packit 709fb3
	* src/grep.h (buf_has_encoding_errors): Remove decl.
Packit 709fb3
	* src/search.h: Adjust decls.
Packit 709fb3
	* src/pcresearch.c (reflags): Remove.  All uses removed.
Packit 709fb3
	(Pcompile, Pexecute): Do not use PCRE_MULTILINE.
Packit 709fb3
Packit 709fb3
2016-11-19  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	doc: fix a doubled "the"
Packit 709fb3
	* doc/grep.texi (--perl-regexp): s/the\nthe/the/
Packit 709fb3
Packit 709fb3
2016-11-19  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: fix -zxP bug
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/pcresearch.c (Pcompile): Search a line at a time if -x is
Packit 709fb3
	used, since -x uses ^ and $.
Packit 709fb3
	* tests/pcre: Test this.
Packit 709fb3
Packit 709fb3
	grep: simplify by using PRIuMAX
Packit 709fb3
	* configure.ac (HAVE_PRINTF_C99_SIZES): Remove; no longer needed.
Packit 709fb3
	* src/grep.c (print_offset): Simplify (Bug#24451).
Packit 709fb3
Packit 709fb3
	grep: -T now adjusts number widths for worst case
Packit 709fb3
	* NEWS, doc/grep.texi (Output Line Prefix Control):
Packit 709fb3
	Document this (Bug#24451).
Packit 709fb3
	* src/grep.c (offset_width): New static var.
Packit 709fb3
	(print_offset): Use it instead of arg.  All callers changed.
Packit 709fb3
	(grep): Set it.
Packit 709fb3
	* tests/initial-tab: Test this.
Packit 709fb3
Packit 709fb3
	grep: -T no longer outputs BS
Packit 709fb3
	* NEWS: Document this (Bug#24451).
Packit 709fb3
	* src/grep.c (print_line_head): Do not attempt to backspace output.
Packit 709fb3
	* tests/initial-tab: New test.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
	grep: document -oz better
Packit 709fb3
	* doc/grep.texi (General Output Control, Usage): Tweak (Bug#24961).
Packit 709fb3
Packit 709fb3
	grep: fix performance typo with -P
Packit 709fb3
	Reported by Zev Weiss in: http://bugs.gnu.org/22655#88
Packit 709fb3
	* src/pcresearch.c (Pcompile): Initialize reflags.
Packit 709fb3
Packit 709fb3
	tests: use "returns_" rather than "$?"
Packit 709fb3
	* tests/grep-dev-null-out: Use "returns_ 124" rather than testing
Packit 709fb3
	$? = 124.
Packit 709fb3
Packit 709fb3
	grep -f /dev/null -L PAT FILE outputs FILE
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (main): Do not exit right away with -L.
Packit 709fb3
	* tests/skip-read: Test for the fix.
Packit 709fb3
Packit 709fb3
	grep: tune -f /dev/null
Packit 709fb3
	* src/grep.c (main): Do the -f /dev/null early-exit checks before
Packit 709fb3
	more-expensive tests that involve syscalls.
Packit 709fb3
Packit 709fb3
	grep: treat -f /dev/null like -m0
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (main): With -f /dev/null, don't bother to read the
Packit 709fb3
	input.  This is what FreeBSD grep does.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add skip-read.
Packit 709fb3
	* tests/skip-read: New file.
Packit 709fb3
Packit 709fb3
	grep: avoid O(N**2) buffer reallocation
Packit 709fb3
	* src/grep.c (main): Use x2realloc to avoid O(N**2) performance as
Packit 709fb3
	pattern buffers grow.
Packit 709fb3
Packit 709fb3
	grep: avoid unnecessary gettext call
Packit 709fb3
	Translate "(standard input)" lazily.
Packit 709fb3
	* src/grep.c (input_filename): New function.
Packit 709fb3
	(suppressible_error): Remove 1st arg, since it is always
Packit 709fb3
	input_filename ().  All callers changed.
Packit 709fb3
	(suppressible_error, print_filename, grep, grepdesc): Use it.
Packit 709fb3
	(grep_command_line_arg): Set filename to NULL if standard
Packit 709fb3
	input has no label.  Often, this avoids all calls to gettext,
Packit 709fb3
	which can be a win as the first call can be expensive.
Packit 709fb3
Packit 709fb3
	grep: drain the input pipe faster
Packit 709fb3
	* src/grep.c (dev_null_output): Now static.
Packit 709fb3
	(drain_input): New function, using 'splice' if that makes sense.
Packit 709fb3
	(finalize_input): Use it.
Packit 709fb3
	(main): Omit now-unnecessary initialization.
Packit 709fb3
Packit 709fb3
	grep: scale back /dev/null speedup
Packit 709fb3
	The performance improvement when output is /dev/null (commit
Packit 709fb3
	af6af288eac28951b5eee1eaaf373e22b2193b7b dated 2016-05-01)
Packit 709fb3
	breaks scripts that run "PROGRAM | grep PATTERN >/dev/null"
Packit 709fb3
	where PROGRAM dies when writing into a broken pipe.
Packit 709fb3
	Suppress the improvement if standard input is not seekable.
Packit 709fb3
	Problem reported by Gary Johnson (Bug#24941).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (seek_failed): New static var.
Packit 709fb3
	(seek_data_failed): Move decl earlier, to be next to seek_failed.
Packit 709fb3
	(file_must_have_nulls): Skip useless syscalls if seek_failed.
Packit 709fb3
	Lessen source-code nesting.
Packit 709fb3
	(reset): Set seek_failed and seek_data_failed.
Packit 709fb3
	Try lseek even on non-regular files.
Packit 709fb3
	(grep): New arg INEOF.  All callers changed.
Packit 709fb3
	Do not clear seek_data_failed here, since 'reset' now does this.
Packit 709fb3
	(finalize_input): New static function.
Packit 709fb3
	(grepdesc): Use it.
Packit 709fb3
	(main): Do not exit on first match merely because output is
Packit 709fb3
	/dev/null.
Packit 709fb3
	* tests/grep-dev-null-out: Adjust to new behavior.
Packit 709fb3
Packit 709fb3
	grep: improve diagnostic on lseek failure
Packit 709fb3
	* src/grep.c (reset): Mention the file name in the (unlikely)
Packit 709fb3
	chance of an lseek failure.
Packit 709fb3
Packit 709fb3
	grep: avoid unnecessary isatty calls
Packit 709fb3
	This fixes an inefficiency that was mistakenly introduced a while
Packit 709fb3
	back, when the macro SET_BINARY became defined on all platforms.
Packit 709fb3
	* src/grep.c (grepdesc, main): Do not unecessarily call isatty on
Packit 709fb3
	POSIXish platforms.
Packit 709fb3
Packit 709fb3
	grep: -Pz no longer rejects ^, $
Packit 709fb3
	Problem reported by Stephane Chazelas (Bug#22655).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* doc/grep.texi (grep Programs): Warn about -Pz.
Packit 709fb3
	* src/pcresearch.c (reflags): New static var.
Packit 709fb3
	(multibyte_locale): Remove static var; now local to Pcompile.
Packit 709fb3
	(Pcompile): Check for (? and (* too.  Set reflags instead of
Packit 709fb3
	dying when problematic operators are found.
Packit 709fb3
	(Pexecute): Use reflags to decide whether searches should
Packit 709fb3
	be multiline.
Packit 709fb3
	* tests/pcre: Test new behavior.
Packit 709fb3
Packit 709fb3
2016-11-14  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: use "returns_" rather than explicit comparison with "$?"
Packit 709fb3
	* tests/sjis-mb (encode): Rearrange to emit desired input into
Packit 709fb3
	a file, rather than piping directly into grep.  That permits
Packit 709fb3
	the use of returns_ 1 to verify timeout's exit status.
Packit 709fb3
	* tests/euc-mb: Use "returns_ 1" rather than testing $? = 1
Packit 709fb3
	* tests/char-class-multibyte: Likewise.
Packit 709fb3
	* tests/dfa-heap-overrun: Likewise.
Packit 709fb3
	* tests/encoding-error: Likewise.
Packit 709fb3
	* tests/fedora: Likewise.
Packit 709fb3
	* tests/grep-dev-null: Likewise.
Packit 709fb3
	* tests/init.cfg (envvar_check_fail): Likewise.
Packit 709fb3
	* tests/kwset-abuse: Likewise.
Packit 709fb3
	* tests/mb-non-UTF8-overrun: Likewise.
Packit 709fb3
	* tests/multibyte-white-space: Likewise.
Packit 709fb3
	* tests/pcre-infloop: Likewise.
Packit 709fb3
	* tests/surrogate-pair: Likewise.
Packit 709fb3
	* tests/warn-char-classes: Likewise.
Packit 709fb3
	Do the same for other values:
Packit 709fb3
	* tests/backref-multibyte-slow: Likewise.
Packit 709fb3
	* tests/euc-mb: Likewise.
Packit 709fb3
	* tests/pcre-abort: Likewise.
Packit 709fb3
	* tests/pcre-jitstack: Likewise.
Packit 709fb3
	* tests/repetition-overflow: Likewise.
Packit 709fb3
	* tests/reversed-range-endpoints: Likewise.
Packit 709fb3
	* tests/warn-char-classes: Likewise.
Packit 709fb3
Packit 709fb3
2016-10-26  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	doc: grep builds on HP-UX once again
Packit 709fb3
	* NEWS (Bug fixes): Mention the HP-UX fix.
Packit 709fb3
Packit 709fb3
	gnulib: update to latest, for getprogname HPUX port
Packit 709fb3
Packit 709fb3
2016-10-22  Mark Veltzer  <mark.veltzer@gmail.com>
Packit 709fb3
Packit 709fb3
	ignore coverage generated files
Packit 709fb3
Packit 709fb3
	ignore ar-lib in build-aux
Packit 709fb3
Packit 709fb3
2016-10-20  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	grep: use 'j' intmax_t printf length modifier if supported
Packit 709fb3
	* configure.ac: Use gl_PRINTF_SIZES_C99 to test printf and
Packit 709fb3
	(conditionally) define HAVE_PRINTF_C99_SIZES.
Packit 709fb3
	* src/grep.c (print_offset): Use printf("%j...") for printing
Packit 709fb3
	[u]intmax_t if HAVE_PRINTF_C99_SIZES is defined; otherwise continue
Packit 709fb3
	using the existing hand-rolled loop.
Packit 709fb3
Packit 709fb3
2016-10-15  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	build: distribute new file, die.h, so "make distcheck" passes
Packit 709fb3
	* src/Makefile.am (grep_SOURCES): Add die.h.
Packit 709fb3
	Also, sort these file names.
Packit 709fb3
Packit 709fb3
2016-10-10  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-10-09  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: die.h: add the "#define ..." part of double inclusion guard
Packit 709fb3
	* src/die.h (DIE_H): Define to 1.
Packit 709fb3
Packit 709fb3
2016-10-04  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: don't assume stdbool.h before die call
Packit 709fb3
	* src/die.h: Include stdbool.h, since 'die' uses 'false'
Packit 709fb3
Packit 709fb3
	grep: die more systematically
Packit 709fb3
	* src/die.h: New file.
Packit 709fb3
	* src/dfasearch.c, src/grep.c, src/pcresearch.c: Include die.h.
Packit 709fb3
	* src/dfasearch.c (dfaerror):
Packit 709fb3
	* src/grep.c (context_length_arg, add_count, prline, setmatcher, main):
Packit 709fb3
	* src/pcresearch.c (jit_exec, Pcompile, Pexecute):
Packit 709fb3
	Use 'die' instead of 'error' when exiting.
Packit 709fb3
	* src/pcresearch.c: Do not include verify.h.
Packit 709fb3
	(die): Remove; now in die.h.
Packit 709fb3
	* src/search.h: Do not include error.h here, since this file does
Packit 709fb3
	not use anything defined in error.h.  Instead, dfasearch.c, which
Packit 709fb3
	uses error.h's symbols, now includes error.h directly.
Packit 709fb3
Packit 709fb3
2016-10-02  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.26
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2016-10-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest; for getprogname fix
Packit 709fb3
Packit 709fb3
2016-10-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests/grep-dir: port to Solaris 10
Packit 709fb3
	* tests/grep-dir: Port to Solaris 10 'cat', which
Packit 709fb3
	exits with status 0 even after 'read' fails from a directory.
Packit 709fb3
Packit 709fb3
2016-09-28  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	build: placate GCC 7's -Wimplicit-fallthrough
Packit 709fb3
	* src/pcresearch.c (die): New macro.
Packit 709fb3
	(Pexecute): Use it in place of offending uses of error,
Packit 709fb3
	to placate GCC 7's -Wimplicit-fallthrough.
Packit 709fb3
	Include verify.h.  Since this is grep's first explicit use of this
Packit 709fb3
	gnulib module, ...
Packit 709fb3
	* bootstrap.conf (gnulib_modules): Add verify.
Packit 709fb3
Packit 709fb3
	gnulib: update to latest; for ...
Packit 709fb3
	This includes the following:
Packit 709fb3
	 - a getprogname-vs-openbsd-5.1 portability fix
Packit 709fb3
	 - "fallthru" comment-adding changes for dfa and unistr/u8-uctomb-aux.c
Packit 709fb3
	 - another getprograme fix to avoid breaking newer glibc
Packit 709fb3
Packit 709fb3
2016-09-27  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: reword .git old-GCC warning
Packit 709fb3
	* configure.ac (gl_gcc_warnings): Reword diagnostic.
Packit 709fb3
	Suggested by Assaf Gordon in:
Packit 709fb3
	http://lists.gnu.org/archive/html/grep-devel/2016-09/msg00024.html
Packit 709fb3
Packit 709fb3
	build: port .git builds to newer GCC
Packit 709fb3
	* configure.ac (gl_gcc_warnings): Omit duplicate copy of 'main'.
Packit 709fb3
	Problem reported by Assaf Gordon in:
Packit 709fb3
	http://lists.gnu.org/archive/html/grep-devel/2016-09/msg00024.html
Packit 709fb3
Packit 709fb3
	build: port .git builds to older GCC
Packit 709fb3
	Problem reported by Dagobert Michelsen in:
Packit 709fb3
	http://lists.gnu.org/archive/html/grep-devel/2016-09/msg00018.html
Packit 709fb3
	* configure.ac (gl_gcc_warnings): Default to false if .git
Packit 709fb3
	exists but GCC is too old.
Packit 709fb3
Packit 709fb3
2016-09-27  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests/long-pattern-perf: avoid false-failure due to cache speed
Packit 709fb3
	* tests/long-pattern-perf: This test would fail semi-consistently
Packit 709fb3
	on some systems, probably because the smaller regexp fit well
Packit 709fb3
	within cache, yet the larger one did not.  In that case, there
Packit 709fb3
	was a relative speed difference greater than 20x and the test
Packit 709fb3
	would fail.  Quadruple the sizes, to make that less likely.
Packit 709fb3
	Also, construct the 10x larger regexp directly from the smaller,
Packit 709fb3
	rather than relying on seq with endpoints to induce that
Packit 709fb3
	approximate size ratio.  Reported by Bruce Dubbs in
Packit 709fb3
	https://lists.gnu.org/archive/html/grep-devel/2016-09/msg00013.html
Packit 709fb3
Packit 709fb3
2016-09-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	build: avoid "./configure && make dist" missing-dep. failure
Packit 709fb3
	* Makefile.am (run-syntax-check): Depend on "all", to avoid a
Packit 709fb3
	parallel build failure due to a missing dependency. Reported by
Packit 709fb3
	Paul Eggert in https://bugs.gnu.org/24256#50
Packit 709fb3
Packit 709fb3
2016-09-24  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-09-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests/fmbtest: avoid false-failure due to reliance on MB-correct sed
Packit 709fb3
	* tests/fmbtest: Several of these tests would mistakenly fail due to
Packit 709fb3
	postprocessing with a combination of sed and locale support that failed
Packit 709fb3
	to handle some multibyte characters in the cs_CZ.UTF-8 locale.  Instead
Packit 709fb3
	of relying on sed's multibyte support or anything locale-related to
Packit 709fb3
	perform this simple filtering, just use this: tr -cs '0-9' '[ *]'
Packit 709fb3
	Also, rather than exporting LC_ALL, just set it for each command.
Packit 709fb3
	Reported by Nelson H. F. Beebe.
Packit 709fb3
	https://bugs.gnu.org/24534
Packit 709fb3
Packit 709fb3
	tests: revamp multibyte-white-space test to be more permissive
Packit 709fb3
	This test elicits too many failures.  Whether a system has accurate
Packit 709fb3
	unicode "whitespace" attributes should not influence whether grep's
Packit 709fb3
	test suite passes.  In many cases, now you will see a warning that
Packit 709fb3
	some multibyte characters do not pass whitespace-related tests, but
Packit 709fb3
	this test no longer fails.  However, if you run this test on a modern
Packit 709fb3
	enough system, it does require that \s and \S do work properly with
Packit 709fb3
	most of the listed characters.
Packit 709fb3
	* tests/multibyte-white-space: Confirm that Fedora 24's locale
Packit 709fb3
	tables still declare those four Unicode code points *not* whitespace.
Packit 709fb3
	Honor a new column telling how to handle failure.  Provide more
Packit 709fb3
	information in each diagnostic.
Packit 709fb3
	Reported by Nelson H. F. Beebe.
Packit 709fb3
	https://bugs.gnu.org/24530
Packit 709fb3
Packit 709fb3
	tests: avoid erroneous failure of pcre-jitstack test
Packit 709fb3
	On some systems (*BSD), 'ulimit -s unlimited' would fail, yet the
Packit 709fb3
	test for that mistakenly masked the failure, so the following grep
Packit 709fb3
	command ended up failing with a segfault.
Packit 709fb3
	* tests/pcre-jitstack: Don't mask the ulimit failure.
Packit 709fb3
	Reported privately by Nelson H. F. Beebe.
Packit 709fb3
	https://bugs.gnu.org/24524
Packit 709fb3
Packit 709fb3
2016-09-23  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep: avoid unwarranted "input file 'F' is also the output" on *BSD
Packit 709fb3
	On *BSD systems, any command like "echo y | grep x", where grep reads
Packit 709fb3
	from a pipe and writes to standard output, would mistakenly emit this:
Packit 709fb3
	  grep: input file '(standard input)' is also the output
Packit 709fb3
	* src/grep.c (grepdesc): Ensure that the file descriptor we're
Packit 709fb3
	reading is a regular one before using SAME_INODE to test whether
Packit 709fb3
	it is the same as the descriptor open on standard output.
Packit 709fb3
	Nelson Beebe reported privately that the foad1 tests failed on many
Packit 709fb3
	BSD systems.  Exposed by commit v2.25-2-gaf6af28.
Packit 709fb3
	https://bugs.gnu.org/24522
Packit 709fb3
Packit 709fb3
	tests: avoid backref-multibyte-slow false failure
Packit 709fb3
	* tests/backref-multibyte-slow (max_seconds): If we calculate
Packit 709fb3
	a max duration of 1 second, use 5.  Otherwise, on high-latency
Packit 709fb3
	systems, it would be way too easy for the duration of the final
Packit 709fb3
	test run to exceed that limit.  Reported by Nelson H. F. Beebe.
Packit 709fb3
	http://bugs.gnu.org/24516
Packit 709fb3
Packit 709fb3
2016-09-22  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest; for getprogname-vs-AIX fix
Packit 709fb3
Packit 709fb3
2016-09-18  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: add news entry for fix to bug#24233
Packit 709fb3
	* NEWS (Bug fixes): Add an entry describing bug#24233.
Packit 709fb3
	The bug was fixed by commit v2.25-77-gad468bb, by chance.
Packit 709fb3
Packit 709fb3
2016-09-15  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2016-09-10  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	dfa: reflect move of grep's DFA code to gnulib
Packit 709fb3
	Now that the core DFA code and tests reside in gnulib,
Packit 709fb3
	remove the copies here and use what gnulib provides.
Packit 709fb3
	* bootstrap.conf: Use the dfa module.
Packit 709fb3
	* cfg.mk: Remove settings involving files that have moved.
Packit 709fb3
	(_gl_TS_unmarked_extern_functions): Add dfaerror and dfawarn.
Packit 709fb3
	It is wrong/ugly to have to define these global symbols to use
Packit 709fb3
	the dfa module, but we'll adjust that separately.
Packit 709fb3
	* po/POTFILES.in: Apply s/src/lib/ to src/dfa.c.
Packit 709fb3
	* src/Makefile.am: Remove mention of dfa.[ch] and localeinfo.[ch].
Packit 709fb3
	* tests/Makefile.am: Remove mention of the tests that we have
Packit 709fb3
	moved to the gnulib module.
Packit 709fb3
	* src/dfa.c: Remove file.
Packit 709fb3
	* src/dfa.h: Likewise.
Packit 709fb3
	* src/localeinfo.c: Likewise.
Packit 709fb3
	* src/localeinfo.h: Likewise.
Packit 709fb3
	* tests/dfa-match: Likewise.
Packit 709fb3
	* tests/dfa-match-aux.c: Likewise.
Packit 709fb3
	* tests/invalid-char-class: Likewise.
Packit 709fb3
Packit 709fb3
	gnulib: update to latest, for new dfa module
Packit 709fb3
Packit 709fb3
2016-09-08  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: encoding errors suppress just their line
Packit 709fb3
	From a suggestion by Marcello Perathoner (Bug#22838).
Packit 709fb3
	* NEWS, doc/grep.texi (File and Directory Selection): Document this.
Packit 709fb3
	* src/grep.c (print_line_head): Do not suppress later output lines
Packit 709fb3
	merely because an earlier output line would have had an encoding error.
Packit 709fb3
	* tests/encoding-error: Test for the new behavior.
Packit 709fb3
Packit 709fb3
2016-09-08  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest, for getprogname fixes
Packit 709fb3
Packit 709fb3
2016-09-08  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: additional change new option for anchored searches
Packit 709fb3
	* src/dfa.c (dfaexec_main): Do it.
Packit 709fb3
Packit 709fb3
2016-09-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: define "context lines"
Packit 709fb3
	Reported by Igor Bogomazov via Santiago Ruano Rincón (Bug#24024).
Packit 709fb3
	* doc/grep.texi (Context Line Control): Define "context lines".
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-09-05  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: switch from gnulib's progname to getprogname module
Packit 709fb3
	* gnulib: Update to latest, for its new getprogname module.
Packit 709fb3
	* bootstrap.conf (avoided_gnulib_modules): Include the getprogname
Packit 709fb3
	module rather than the now-obsolescent progname.
Packit 709fb3
	* src/grep.c: Include "getprogname.h" rather than "progname.h"
Packit 709fb3
	and remove any use of set_program_name.
Packit 709fb3
	* tests/dfa-match-aux.c (main): Likewise.
Packit 709fb3
	* tests/get-mb-cur-max.c (main): Likewise.
Packit 709fb3
	* src/grep.c (usage, main): Use getprogname() in place of program_name.
Packit 709fb3
Packit 709fb3
2016-09-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: minor cleanup of previous change
Packit 709fb3
	* src/dfa.c (dfaexec_main): Omit redundant code and reindent.
Packit 709fb3
Packit 709fb3
2016-09-02  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: additional change new option for anchored searches
Packit 709fb3
	* src/dfa.c (dfaexec_main): Do it.
Packit 709fb3
Packit 709fb3
	dfa: use single-byte algorithm even in non-UTF-8
Packit 709fb3
	* src/dfa.c (dfaexec_main): Do it.  (This was inadvertently
Packit 709fb3
	omitted in a recent patch.)
Packit 709fb3
Packit 709fb3
2016-09-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: merge xalloc.h changes from Gawk
Packit 709fb3
	* src/dfa.h (_GL_ATTRIBUTE_MALLOC): Define here, as other
Packit 709fb3
	Gnulib .h files do.  This is more consistent with Gawk.
Packit 709fb3
	* src/dfa.c: Include xalloc.h, since dfa.h no longer does so.
Packit 709fb3
	Include localeinfo.h later; we don't care about order, but Gawk does.
Packit 709fb3
Packit 709fb3
2016-09-02  Arnold Robbins  <arnold@skeeve.com>
Packit 709fb3
Packit 709fb3
	dfa: port to C90
Packit 709fb3
	* src/dfa.c (dfamust): Avoid declarations after statement (Bug#21486).
Packit 709fb3
Packit 709fb3
2016-09-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: new option for anchored searches
Packit 709fb3
	This follows up on a suggestion by Norihiro Tanaka (Bug#24262).
Packit 709fb3
	* src/dfa.c (struct regex_syntax): New member 'anchor'.
Packit 709fb3
	(char_context): Use it.
Packit 709fb3
	(dfasyntax): Change signature to specify it, along with the old
Packit 709fb3
	FOLD and EOL args, as a single DFAOPTS arg.  All uses changed.
Packit 709fb3
	* src/dfa.h (DFA_ANCHOR, DFA_CASE_FOLD, DFA_EOL_NUL): New constants
Packit 709fb3
	for dfasyntax new last arg.
Packit 709fb3
Packit 709fb3
2016-09-02  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: simplify and optimize at initial state in execution
Packit 709fb3
	* src/dfa.c (skip_remains_mb): Remove argument *pwc.  Update calller.
Packit 709fb3
	(dfaexec_main): Simplify and optimize at initial state (Bug#24261).
Packit 709fb3
Packit 709fb3
	dfa: simplify to find state index for state 0
Packit 709fb3
	* src/dfa.c (dfastate): Simplify to find state index for state 0.
Packit 709fb3
Packit 709fb3
2016-09-01  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	tests: add a new test for SJIS locale
Packit 709fb3
	* tests/sjis-mb: Add a new test.  It fails in grep-2.25 or prior.
Packit 709fb3
Packit 709fb3
2016-09-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: update NEWS
Packit 709fb3
	* NEWS: Describe previous change.
Packit 709fb3
Packit 709fb3
	grep: use regex fastmap unless -i
Packit 709fb3
	This builds on a suggestion by Norihiro Tanaka (Bug#24009).
Packit 709fb3
	* src/dfasearch.c (GEAcompile): Use a fastmap unless -i.
Packit 709fb3
	This improves performance 20x for me using the first benchmark
Packit 709fb3
	given in Bug#24009.
Packit 709fb3
Packit 709fb3
	grep: improve dfasearch storage management
Packit 709fb3
	This patch is mostly refactoring, with a bit of performance tweaking.
Packit 709fb3
	It is done in preparation for a fix for Bug#24009.
Packit 709fb3
	* src/dfasearch.c (patterns): Now of type struct re_pattern_buffer *
Packit 709fb3
	instead of an anonymous struct pointer, since there is no longer
Packit 709fb3
	any need to keep regs here.  All uses changed.
Packit 709fb3
	(GEAcompile): Use patlim instead of a hard-to-follow "total".
Packit 709fb3
	Use x2nrealloc to avoid potential O(N**2) reallocation algorithm.
Packit 709fb3
	Initialize just the pattern members that need clearing.
Packit 709fb3
	(EGexecute): Put regs into a static variable, as this code did
Packit 709fb3
	before 2001-02-18, as there is no need to have a separate set of
Packit 709fb3
	regs for each pattern.  Explain the "Q@#%!#" comment better.
Packit 709fb3
Packit 709fb3
2016-09-01  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: remove separation by context in transition in non-UTF8 multibyte locales
Packit 709fb3
	* src/dfa.c (struct dfa): Remove member curr_dependent.  All uses
Packit 709fb3
	removed.
Packit 709fb3
Packit 709fb3
2016-09-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: document previous change
Packit 709fb3
	* NEWS: Adjust to match previous change.
Packit 709fb3
Packit 709fb3
2016-09-01  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: avoid invalid character matching period
Packit 709fb3
	* dfa.c (transit_state): Avoid invalid character matching period.
Packit 709fb3
Packit 709fb3
	dfa: use single-byte algorithm even in non-UTF-8
Packit 709fb3
	Even in non-UTF8 locales, if the current input character
Packit 709fb3
	is single byte, we can use CSET to match ANYCHAR.
Packit 709fb3
	* src/dfa.c (struct dfa): New member canychar.
Packit 709fb3
	Cache index of CSET for ANYCHAR.
Packit 709fb3
	(lex): Make CSET for ANYCHAR.
Packit 709fb3
	(state_index): Simplify.
Packit 709fb3
	(dfastate): Consider CSET for ANYCHAR.
Packit 709fb3
	(transit_state_singlebyte, transit_state): Remove handling for eolbyte,
Packit 709fb3
	as we assume that eolbyte does not appear at current position.
Packit 709fb3
	(dfaexec_main): Use algorithm for single byte character to any single
Packit 709fb3
	byte character in input text always.
Packit 709fb3
	(dfasyntax): Initialize canychar.
Packit 709fb3
Packit 709fb3
2016-09-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: avoid code duplication with -iF
Packit 709fb3
	This follows up on the -iF performance improvement (Bug#23752).
Packit 709fb3
	* NEWS: Simplify description of -iF improvement.
Packit 709fb3
	* src/dfa.c: Do not include wctype.h.
Packit 709fb3
	(lonesome_lower, case_folded_counterparts): Move to localeinfo.c.
Packit 709fb3
	(CASE_FOLDED_BUFSIZE): Move to localeinfo.h.
Packit 709fb3
	* src/grep.c: Do not include wctype.h.
Packit 709fb3
	(lonesome_lower): Remove.
Packit 709fb3
	(fgrep_icase_available): Use case_folded_counterparts instead.
Packit 709fb3
	Do not call it for the same character twice.
Packit 709fb3
	Return false on wcrtomb failures (which should never happen).
Packit 709fb3
	(fgrep_to_grep_pattern, main): Simplify.  Let fgrep_to_grep’s
Packit 709fb3
	caller fiddle with the global variables.
Packit 709fb3
	* src/localeinfo.c: Include <wctype.h>
Packit 709fb3
	(lonesome_lower, case_folded_counterparts):
Packit 709fb3
	Move here from src/dfa.c.  Return int, not unsigned int.
Packit 709fb3
	Verify that CASE_FOLDED_BUFSIZE is big enough.
Packit 709fb3
	* src/localeinfo.h (CASE_FOLDED_BUFSIZE): Now 32, so that
Packit 709fb3
	we don’t expose lonesome_lower’s size.
Packit 709fb3
	* src/searchutils.c (kwsinit): Return new kwset instead of
Packit 709fb3
	storing it via a pointer.  All callers changed.  Simplify a bit.
Packit 709fb3
Packit 709fb3
2016-09-01  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: speed up -iF in multibyte locales
Packit 709fb3
	In a multibyte locale, if a pattern is composed of only single byte
Packit 709fb3
	characters and their all counterparts are also single byte characters
Packit 709fb3
	and the pattern does not have invalid sequences, grep -iF uses the
Packit 709fb3
	fgrep matcher, the same as in a single byte locale (Bug#23752).
Packit 709fb3
	* NEWS: Mention it.
Packit 709fb3
	* src/grep.c (lonesome_lower): New constant.
Packit 709fb3
	(fgrep_icase_available): New function.
Packit 709fb3
	(fgrep_to_grep_pattern): Simplify it.
Packit 709fb3
	(main): Use them.
Packit 709fb3
	* src/searchutils.c (kwsinit): New arg MB_TRANS; all uses changed.
Packit 709fb3
	Try fgrep matcher for case insensitive matching by grep -F in multibyte
Packit 709fb3
	locale.
Packit 709fb3
Packit 709fb3
2016-08-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-08-31  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: avoid new 'make syntax-check' failure
Packit 709fb3
	* src/dfa.c (using_simple_locale): Prefer STREQ(a,b) over
Packit 709fb3
	strcmp(a,b) == 0.
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2016-08-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: make dfa.c fully thread-safe
Packit 709fb3
	This follows up on Zev Weiss’s recent patches to make the DFA code
Packit 709fb3
	thread-safe (Bug#24249).  It removes the remaining static
Packit 709fb3
	variables used by dfa.c.  These variables are locale-dependent, so
Packit 709fb3
	they would cause problems in multithreaded code where different
Packit 709fb3
	threads are in different locales (e.g., via uselocale).  I
Packit 709fb3
	abstracted most of the variables into a new localeinfo module.
Packit 709fb3
	* src/Makefile.am (grep_SOURCES): Add localeinfo.c.
Packit 709fb3
	(noinst_HEADERS): Add localeinfo.h.
Packit 709fb3
	* src/dfa.c: Include localeinfo.h.
Packit 709fb3
	(struct dfa): Remove multibyte member, as it is now part of
Packit 709fb3
	localeinfo.  New members simple_locale and localeinfo.
Packit 709fb3
	Put locale-related members at the end.
Packit 709fb3
	(mbrtowc_cache): Remove; now part of dfa->localeinfo.
Packit 709fb3
	(charclass_index): Rename back from dfa_charclass_index,
Packit 709fb3
	since it's private.
Packit 709fb3
	(unibyte_word_constituent): New arg DFA; use its sbctowc member.
Packit 709fb3
	(using_utf8, dfa_using_utf8, init_mbrtowc_cache, check_utf8):
Packit 709fb3
	Remove; now done by localeinfo members.  All uses changed.
Packit 709fb3
	(dfasyntax): New localeinfo arg.  Move to end to avoid forward decls.
Packit 709fb3
	Initialize the entire DFA.
Packit 709fb3
	(unibyte_c, check_unibyte_c): Remove; now in simple_locale member.
Packit 709fb3
	(using_simple_locale): Now takes bool instead of DFA.
Packit 709fb3
	Do the locale check here, rather than in the caller,
Packit 709fb3
	as the result is now cached in dfa->simple_locale.
Packit 709fb3
	(dfaalloc): Just allocate the DFA.  dfasyntax now initializes it.
Packit 709fb3
	* src/dfa.h: Add forward decl of struct localeinfo.
Packit 709fb3
	Adjust to new dfa.c API.
Packit 709fb3
	* src/dfasearch.c (localeinfo): New var, replacing former static
Packit 709fb3
	vars like mbrtowc_cache.
Packit 709fb3
	* src/localeinfo.c, src/localeinfo.h: New files.
Packit 709fb3
	* src/search.h: Include localeinfo.h.
Packit 709fb3
	(localeinfo): New decl.
Packit 709fb3
	* src/searchutils.c (mbclen_cache, build_mbclen_cache):
Packit 709fb3
	Remove.  All uses changed to localeinfo.
Packit 709fb3
	* tests/Makefile.am (dfa_match_aux_LDADD): Add localeinfo.o.
Packit 709fb3
	* tests/dfa-match-aux.c: Include localeinfo.h.
Packit 709fb3
	(main): Adjust to changes in DFA API.
Packit 709fb3
Packit 709fb3
2016-08-28  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
	This should fix Bug#24323 reported by Dennis Clarke, where grep
Packit 709fb3
	does not build on Solaris 10 when compiled with Solaris Studio 12.4.
Packit 709fb3
Packit 709fb3
2016-08-23  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: minor thread-safety cleanups
Packit 709fb3
	* src/dfa.c (struct lexer_state): Rename lexptr to ptr and lexleft
Packit 709fb3
	to left, for brevity.  All uses changed.
Packit 709fb3
	(struct dfa): Rename lexstate to lex and parsestate to parse,
Packit 709fb3
	for brevity.  All uses changed.
Packit 709fb3
	(using_simple_locale): Simplify boolean expression.
Packit 709fb3
	(FETCH_WC): Parenthesize uses of dfa macro arg.
Packit 709fb3
	(FETCH_WC, parse_bracket_exp, addtok_mb): Prefer suffix operators
Packit 709fb3
	on structure members when possible, for clarity.
Packit 709fb3
	(parse_bracket_exp): Check for buffer exhaustion before
Packit 709fb3
	dereferencing buffer pointer.
Packit 709fb3
	(struct lexptr): New type.
Packit 709fb3
	(push_lex_state, pop_lex_state): Use it.  Change from macros
Packit 709fb3
	PUSH_LEX_STATE and POP_LEX_STATE to static functions, and add
Packit 709fb3
	parameters to make them proper C functions.  All uses changed.
Packit 709fb3
	(lex): Simplify tests for \) and \|.  Avoid some string
Packit 709fb3
	duplication by using &"^..."[boolean].
Packit 709fb3
	(dfaalloc): Use xzalloc, not xcalloc with 1.
Packit 709fb3
Packit 709fb3
2016-08-21  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: minor tweaks of initial buffer alloc
Packit 709fb3
	* src/grep.c (main): Allocate input buffer only when about
Packit 709fb3
	to do I/O.  Avoid int overflow on systems with 2 GiB pages.
Packit 709fb3
	Fix size_t overflow check.
Packit 709fb3
Packit 709fb3
2016-08-20  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	dfa: constify some function parameters
Packit 709fb3
	* src/dfa.c (char_context): Mark dfa parameter const.
Packit 709fb3
	(charclass_context): Likewise.
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: initialize mbrtowc_cache in dfa_init
Packit 709fb3
	* src/dfa.c (dfasyntax): Remove initialization of mbrtowc_cache.
Packit 709fb3
	(init_mbrtowc_cache): New function.
Packit 709fb3
	(dfa_init): Call it.
Packit 709fb3
	http://bugs.gnu.org/24259
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: eliminate static local variables
Packit 709fb3
	* src/dfa.c: Replace utf8 and unibyte_c static local variables with
Packit 709fb3
	static globals initialized by a new function dfa_init() which must be
Packit 709fb3
	called before any other dfa*() functions.
Packit 709fb3
	(dfa_using_utf8): Rename using_utf8() to dfa_using_utf8() for
Packit 709fb3
	consistency with other exported functions.
Packit 709fb3
	* src/dfa.h (dfa_using_utf8): Rename using_utf8() to dfa_using_utf8();
Packit 709fb3
	also add _GL_ATTRIBUTE_PURE.
Packit 709fb3
	(dfa_init): New function.
Packit 709fb3
	* src/grep.c (main), tests/dfa-match-aux.c (main): Call dfa_init().
Packit 709fb3
	* src/dfasearch.c (EGexecute): Replace using_utf8 with dfa_using_utf8.
Packit 709fb3
	* src/kwsearch.c (Fexecute): Likewise.
Packit 709fb3
	* src/pcresearch.c (Pcompile): Likewise.
Packit 709fb3
	http://bugs.gnu.org/24259
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: move regex syntax configuration into struct dfa
Packit 709fb3
	* src/dfa.c: move global variables holding regex syntax configuration
Packit 709fb3
	into a new struct (`struct regex_syntax') and add an instance of it to
Packit 709fb3
	struct dfa.  All references to the globals are replaced with
Packit 709fb3
	references to the dfa struct's new member.  As a side effect, a
Packit 709fb3
	`struct dfa' must be allocated with dfaalloc() and passed to
Packit 709fb3
	dfasyntax().
Packit 709fb3
	* src/dfa.h (dfasyntax): Add new struct dfa* parameter.
Packit 709fb3
	* src/dfasearch.c (GEAcompile): Allocate `dfa' earlier and pass it to
Packit 709fb3
	dfasyntax().
Packit 709fb3
	* tests/dfa-match-aux.c (main): Pass `dfa' to dfasyntax().
Packit 709fb3
	http://bugs.gnu.org/24259
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: move parser state into struct dfa
Packit 709fb3
	* src/dfa.c: move global variables holding parser state (`tok' and
Packit 709fb3
	`depth') into a new struct (`struct parser_state') and add an instance
Packit 709fb3
	of it to struct dfa.  All references to the globals are replaced by
Packit 709fb3
	references to the dfa struct's new member.
Packit 709fb3
	http://bugs.gnu.org/24259
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: move lexer state into struct dfa
Packit 709fb3
	* src/dfa.c: move global variables holding lexer state into a new
Packit 709fb3
	struct (`struct lexer_state') and add an instance of this struct to
Packit 709fb3
	struct dfa.  All references to the globals are replaced with
Packit 709fb3
	references to the dfa struct's new member.
Packit 709fb3
	http://bugs.gnu.org/24259
Packit 709fb3
Packit 709fb3
2016-08-19  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	dfa: thread-safety: remove dfa.c's "dfa" global
Packit 709fb3
	Remove the global dfa struct.  Instead, add a struct dfa pointer
Packit 709fb3
	parameter to each function that had been using the global.
Packit 709fb3
	* src/dfa.c (dfa): Remove file-scoped global.
Packit 709fb3
	(charclass_index): Remove now-unnecessary function.
Packit 709fb3
	(using_simple_locale): Add a dfa parameter and update all callers.
Packit 709fb3
	(FETCH_WC, parse_bracket_exp, lex, addtok_mb, addtok): Likewise.
Packit 709fb3
	(addtok_wc, add_utf8_anychar, atom, nsubtoks, copytoks): Likewise.
Packit 709fb3
	(closure, branch, regexp): Likewise.
Packit 709fb3
	(dfaparse): No longer set the global.
Packit 709fb3
	http://bugs.gnu.org/24260
Packit 709fb3
Packit 709fb3
2016-08-18  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: tune list_files conversion to enum
Packit 709fb3
	* src/grep.c (grepdesc): Use a slightly more-efficient way to test
Packit 709fb3
	list_files.
Packit 709fb3
Packit 709fb3
	grep: prefer bitwise to short-circuit when shorter
Packit 709fb3
	* src/grep.c (skip_devices, initialize_unibyte_mask, fillbuf, main)
Packit 709fb3
	* src/kwsearch.c (Fexecute): Prefer bitwise to short-circuit ops
Packit 709fb3
	when they are logically equivalent and the bitwise ops generate
Packit 709fb3
	shorter code on GCC 6.1 x86-64.
Packit 709fb3
	* src/grep.c (get_nondigit_option, parse_grep_colors):
Packit 709fb3
	Use c_isdigit instead of spelling it out with a short-circuit op.
Packit 709fb3
Packit 709fb3
2016-08-17  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: use 64-bit when ulong is at least that wide
Packit 709fb3
	* src/dfa.c (charclass_word): Now unsigned long instead of unsigned.
Packit 709fb3
	(CHARCLASS_WORD_BITS): Now 64 on 64-bit platforms.
Packit 709fb3
	(CHARCLASS_PAIR, CHARCLASS_INIT): New macros.
Packit 709fb3
	(CHARCLASS_WORD_MASK): Now a static const, since it no longer
Packit 709fb3
	needs to be a macro.
Packit 709fb3
	(equal): Open-code rather than calling memcmp.
Packit 709fb3
	(add_utf8_anychar): Use CHARCLASS_INIT.
Packit 709fb3
Packit 709fb3
	dfa: avoid uninitialized constants
Packit 709fb3
	Some compilers warn about 'static int const x;' on the grounds
Packit 709fb3
	that X should have an initializer.  Instead of worrying about
Packit 709fb3
	this, rewrite to avoid this sort of thing.
Packit 709fb3
	* src/dfa.c (emptyset): New function.
Packit 709fb3
	(parse_bracket_exp): Use it instead of 'equal' and a zero constant.
Packit 709fb3
	* src/dfasearch.c (struct patterns): Remove tag 'patterns'.
Packit 709fb3
	(patterns0): Remove zero constant.
Packit 709fb3
	(GEAcompile): Use memset instead of the zero constant.
Packit 709fb3
Packit 709fb3
2016-08-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: avoid new "make syntax-check" failure
Packit 709fb3
	* src/dfa.c: Adjust comment not to go past column 80.
Packit 709fb3
Packit 709fb3
	tests: pcre-jitstack: avoid false failure without base64 -d support
Packit 709fb3
	* tests/pcre-jitstack: Try harder to find a base64 decoder:
Packit 709fb3
	try 'base64 -d', 'base64 -D', 'openssl base64 -d' and perl's
Packit 709fb3
	MIME::Base64 decode_base64.  The old code would fail at least on
Packit 709fb3
	OS X, for which base64 expects -D or --decode.
Packit 709fb3
	Reported by Jack Howarth in http://bugs.gnu.org/24243.
Packit 709fb3
Packit 709fb3
2016-08-16  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: minor refactoring and doc fixes
Packit 709fb3
	* NEWS: Improve description of recent change.
Packit 709fb3
	* src/dfa.c: Improve commentary.  Indent new code (and some
Packit 709fb3
	long-existing howlers) more in GNU style.
Packit 709fb3
	(dfa_state): Reorder members to make struct smaller on x86.
Packit 709fb3
	mb_trindex member is now state_num, not size_t, so that -1 is more
Packit 709fb3
	natural; all uses changed.
Packit 709fb3
	(struct dfa): Similarly for mb_trcount member.
Packit 709fb3
	(state_index): Compute values for new state components before
Packit 709fb3
	allocating the state, to make the code easier to understand.
Packit 709fb3
	(state_index, dfastate): Prefer A & ~B to other forms like (A & B)
Packit 709fb3
	!= A.
Packit 709fb3
	(dfastate, build_state, transit_state): In new code, prefer i++ to
Packit 709fb3
	++i in for-loop control.
Packit 709fb3
	(build_state, transit_state): In new code, prefer < to >.
Packit 709fb3
	(transit_state): Add to *PP in one assignment, rather than in a
Packit 709fb3
	loop.  Prefer !x to x == NULL.  Use xmalloc instead of xnmalloc,
Packit 709fb3
	since the size is a constant.  Do the size calculation as a signed
Packit 709fb3
	integer constant expression, so that the compiler diagnoses any
Packit 709fb3
	overflow.
Packit 709fb3
	(transit_state, free_mbdata): Tune by looping from -1 to N - 1,
Packit 709fb3
	rather than from 0 to N - 1 with a separate instance for -1.
Packit 709fb3
	(dfaexec_main): Rewrite to avoid side effects in if-part.
Packit 709fb3
	(free_mbdata): Simplify.
Packit 709fb3
Packit 709fb3
	dfa: port to C90
Packit 709fb3
	* src/dfa.c (transit_state, dfa_supported, dfamust):
Packit 709fb3
	Don't use declarations after statements.
Packit 709fb3
	If I recall correctly, gawk still wants to port to C90.
Packit 709fb3
Packit 709fb3
	dfa: fix context newline confusion
Packit 709fb3
	* src/dfa.c (transit_state): Fix "... & ~0" that was evidently
Packit 709fb3
	intended to be "... & ~1".  Do index calculation in a simpler way,
Packit 709fb3
	that uses just addition (Bug#21486).
Packit 709fb3
Packit 709fb3
2016-08-16  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: improve leading "." with non-UTF8 multibyte
Packit 709fb3
	In non-UTF8 multibyte locales, matching the dot expression is very
Packit 709fb3
	slow, as the next state is calculated on demand.  This change caches
Packit 709fb3
	the result for the typical case (Bug#21486).
Packit 709fb3
Packit 709fb3
	Compare the run times of this command before and after this change,
Packit 709fb3
	on a i5-4570 CPU @ 3.20GHz using rawhide (~fedora 22) and compiled
Packit 709fb3
	with gcc 5.1.1 20150618:
Packit 709fb3
	yes "$(printf 'a%38db\n' 0)" | head -1000000 >in
Packit 709fb3
	env LC_ALL=ja_JP.eucJP time -p \
Packit 709fb3
	src/grep .......................................... in
Packit 709fb3
	  Before: 19.10
Packit 709fb3
	  After :  0.55
Packit 709fb3
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/dfa.c: (struct dfa_state): New members curr_dependent, mb_trindex.
Packit 709fb3
	(MAX_TRCOUNT): New constant.
Packit 709fb3
	(struct dfa): New members mb_trans, mb_trcount.
Packit 709fb3
	(state_index): Initialize new members of struct dfa_state and calculate
Packit 709fb3
	dependency on context of next character for positions for dot.
Packit 709fb3
	(dfastate): Calculate follows positions for dot if enabled.
Packit 709fb3
	(realloc_trans_if_necessary): Allocate transition tables.
Packit 709fb3
	(build_state): Use new constant and reset transition tables.
Packit 709fb3
	(transit_state): Use cache for transition from a state with the dot
Packit 709fb3
	expression.
Packit 709fb3
	(free_mbdata): Deallocate transition tables.
Packit 709fb3
Packit 709fb3
2016-08-06  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: standardize on 10-second timeouts to avoid rare false failure
Packit 709fb3
	In a parallel test run, it is not unusual to exceed a timeout of
Packit 709fb3
	1-3 seconds.  Increase several from 3 or fewer to 10 seconds.
Packit 709fb3
	* tests/skip-device: Increase timeout from 2 to 10 seconds.
Packit 709fb3
	* tests/grep-dev-null-out: Likewise, but s/1/10/.
Packit 709fb3
	* tests/pcre-invalid-utf8-input: Likewise, but s/3/10/.
Packit 709fb3
	* tests/dfa-match: Likewise.
Packit 709fb3
	* tests/pcre-invalid-utf8-infloop: Likewise.
Packit 709fb3
	* tests/pcre-infloop: Likewise.
Packit 709fb3
	* tests/max-count-overread: Likewise.
Packit 709fb3
	* tests/invalid-multibyte-infloop: Likewise.
Packit 709fb3
	Prompted by http://bugs.gnu.org/24159.
Packit 709fb3
Packit 709fb3
	tests/backref-multibyte-slow:: avoid false positive
Packit 709fb3
	* tests/backref-multibyte-slow: When redirecting the "fast" LC_ALL=C
Packit 709fb3
	run's output to /dev/null, we got an artificially low timing (of 0),
Packit 709fb3
	due to grep's own stdout-vs-/dev/null optimization.  With an initial
Packit 709fb3
	timing of 0 on that first run, the derived timeout for the UTF-8 run
Packit 709fb3
	(which redirects to a file) would be a mere 1 second.  The fix: also
Packit 709fb3
	redirect that first run's output to a file, not to /dev/null.
Packit 709fb3
Packit 709fb3
2016-08-05  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: minor fix for whether dfa is "fast"
Packit 709fb3
	* src/dfa.c (dfaoptimize): When a UTF-8 optimization succeeds for
Packit 709fb3
	a DFA (it can use single-byte code paths), record that by setting
Packit 709fb3
	its ->fast flag.
Packit 709fb3
Packit 709fb3
2016-07-25  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep: print "filename:lineno:" in invalid-regex diagnostic
Packit 709fb3
	Determining the file name and line number is a little tricky because
Packit 709fb3
	of the way the regular expressions are all concatenated onto a newline-
Packit 709fb3
	separated list.  By the time grep would compile regular expressions,
Packit 709fb3
	the <filename,lineno> origin of each regexp was no longer available.
Packit 709fb3
	This patch adds a list of filename,first_lineno pairs, one per input
Packit 709fb3
	source, by which we can then map the ordinal regexp number to a
Packit 709fb3
	filename,lineno pair for the diagnostic.
Packit 709fb3
Packit 709fb3
	* src/dfasearch.c (GEAcompile): When diagnosing an invalid regexp
Packit 709fb3
	specified via -f FILE, include the "FILENAME:LINENO: " prefix.
Packit 709fb3
	Also, when there are two or more lines with compilation failures,
Packit 709fb3
	diagnose all of them, rather than stopping after the first.
Packit 709fb3
	* src/grep.h (pattern_file_name): Declare it.
Packit 709fb3
	* src/grep.c: (struct FL_pair): Define type.
Packit 709fb3
	(fl_pair, n_fl_pair_slots, n_pattern_files, patfile_lineno):
Packit 709fb3
	Define globals.
Packit 709fb3
	(fl_add, pattern_file_name): Define functions.
Packit 709fb3
	(main): Call fl_add for each type of the following: -e argument,
Packit 709fb3
	-f argument, command-line-specified (without -e) regexp.
Packit 709fb3
	* tests/filename-lineno.pl: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	* NEWS (Improvements): Mention this.
Packit 709fb3
	Initially reported by Gunnar Wolf in https://bugs.debian.org/525214
Packit 709fb3
	Forwarded to grep's bug list by Santiago Ruano Rincón as
Packit 709fb3
	http://debbugs.gnu.org/23965
Packit 709fb3
Packit 709fb3
2016-07-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: add coreutils' perl-driven test framework
Packit 709fb3
	* configure.ac: Set the AM_CONDITIONAL variable, HAVE_PERL.
Packit 709fb3
	* tests/Coreutils.pm: New file.
Packit 709fb3
	* tests/CuSkip.pm: New file.
Packit 709fb3
	* tests/CuTmpdir.pm: New file.
Packit 709fb3
	* tests/no-perl: New file.
Packit 709fb3
	* tests/Makefile.am: Set up to use .pl tests:
Packit 709fb3
	(TEST_EXTENSIONS, TESTSUITE_PERL, TESTSUITE_PERL_OPTIONS): Define.
Packit 709fb3
	(SH_LOG_COMPILER, PL_LOG_COMPILER): Define.
Packit 709fb3
	(EXTRA_DIST): Add the four new file names.
Packit 709fb3
Packit 709fb3
	doc: omit an excess word in HACKING
Packit 709fb3
Packit 709fb3
2016-07-21  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: always match single line only with DFA superset
Packit 709fb3
	\n cannot occur inside a multibyte character.  So an input always
Packit 709fb3
	matches single line only with DFA superset.
Packit 709fb3
Packit 709fb3
	* src/dfasearch.c (EGexecute): Simplify it with above.
Packit 709fb3
Packit 709fb3
2016-07-15  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: fix whitespace problems
Packit 709fb3
	* src/dfa.c: Use GNU style for pointer decls.
Packit 709fb3
Packit 709fb3
2016-07-15  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	maint: modernize HACKING a bit
Packit 709fb3
	* HACKING: Remove some ancient history to simplify maintenance.
Packit 709fb3
Packit 709fb3
2016-07-14  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: minor style changes for -F crash fix
Packit 709fb3
	* src/kwset.c (memoff2_kwset): Use ?: instead of if-else.
Packit 709fb3
Packit 709fb3
2016-07-14  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: fix -F crash when alternating duplicates
Packit 709fb3
	grep -F crashes with a pattern like 0\n0.
Packit 709fb3
	This bug was introduced in 966f6586fbce3081ce6e5e2f9b55301b0ec3d2b4.
Packit 709fb3
Packit 709fb3
	* src/kwset.c (memoff2_kwset): If two characters are the same,
Packit 709fb3
	use memchr instead of memchr2.
Packit 709fb3
	* tests/two-chars: New test.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2016-07-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: fix comments to match code better
Packit 709fb3
	* src/dfa.c: Fix comments.
Packit 709fb3
Packit 709fb3
2016-07-06  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: don't treat null bytes specially
Packit 709fb3
	* src/dfa.c (transit_state): Do not treat null byte specially
Packit 709fb3
	when eolbyte == '\n'.
Packit 709fb3
Packit 709fb3
2016-07-06  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: don't distingish letter in non-POSIX locales
Packit 709fb3
	For non-POSIX locales, dfa does not support word delimiter
Packit 709fb3
	support, so remove distinction between letters and non-letters.
Packit 709fb3
	* src/dfa.c (struct dfa): Remove members initstate_letter,
Packit 709fb3
	initstate_others.  All uses removed.  New member initstate_notbol.
Packit 709fb3
	(dfaanalyze, dfaexec_main): Replace old members with new member.
Packit 709fb3
	(wchar_context): Remove.  Update callers.
Packit 709fb3
Packit 709fb3
2016-07-06  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: minor cleanups for non-POSIX simplification
Packit 709fb3
	* src/dfa.c (transit_state_singlebyte): Remove unnecessary 'const'
Packit 709fb3
	from arg; we usually don't bother with 'const' on locals.
Packit 709fb3
	(transit_state_singlebyte): Omit '!= NULL' in boolean context.
Packit 709fb3
	Use assert rather than abort.
Packit 709fb3
Packit 709fb3
2016-07-06  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: simplify for non-POSIX locales
Packit 709fb3
	Simplify the dfa code, since it no longer supports ranges,
Packit 709fb3
	collating elements, and equivalent classes in non-POSIX locales.
Packit 709fb3
	* src/dfa.c (struct dfa): Remove mb_match_lens.
Packit 709fb3
	(enum status_transit_state, match_anychar)
Packit 709fb3
	(check_matching_with_multibyte_ops, transit_state_consume_1char):
Packit 709fb3
	(State_transition): Remove.
Packit 709fb3
	(transit_state_singlebyte): Accepts pointer-to-pointer position,
Packit 709fb3
	instead of pointer, and no longer accept pointer to next state.
Packit 709fb3
	Return next state instead of status_transit_state.  All callers
Packit 709fb3
	changed.
Packit 709fb3
	(transit_state_singlebyte, transit_state): Simplify.
Packit 709fb3
	(dfaexec_main): Now transit_state is called only when next character
Packit 709fb3
	matches with ANYCHAR.
Packit 709fb3
Packit 709fb3
2016-06-14  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: propagate more changes from grep.texi
Packit 709fb3
	Problem reported by Björn Voigt in: http://bugs.gnu.org/23763#27
Packit 709fb3
	* doc/grep.in.1: Fix more inconsistencies with grep.texi.
Packit 709fb3
Packit 709fb3
2016-06-13  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: remove obsolete MS-DOS mention
Packit 709fb3
	* doc/grep.in.1: Remove obsolete discussion of MS-DOS heuristics.
Packit 709fb3
	Problem reported by Björn Voigt in: http://bugs.gnu.org/23763
Packit 709fb3
Packit 709fb3
2016-06-09  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	grep: do pagesize initialization and buffer allocation earlier
Packit 709fb3
	* src/grep.c (reset, main): We're going to need pagesize and buffer
Packit 709fb3
	initialized anyway, so we might as well do so unconditionally early on
Packit 709fb3
	rather than checking on every call to reset().
Packit 709fb3
	http://bugs.gnu.org/23717
Packit 709fb3
Packit 709fb3
	grep: remove unnecessary dirdesc variable.
Packit 709fb3
	* src/grep.c (grepdirent): Remove dirdesc variable and just use
Packit 709fb3
	fts_cwd_fd directly, since the fts_options test was guaranteed to
Packit 709fb3
	succeed (and fts_cwd_fd was already being used directly in fstatat()
Packit 709fb3
	anyway).  http://bugs.gnu.org/23716
Packit 709fb3
Packit 709fb3
	grep: convert list_files to an enum
Packit 709fb3
	* src/grep.c: Make list_files a tristate enum instead of an int.
Packit 709fb3
	http://bugs.gnu.org/23715
Packit 709fb3
Packit 709fb3
	grep: correct a stale comment and remove dead code
Packit 709fb3
	* src/grep.c (grepdesc): The `grep()' function no longer has
Packit 709fb3
	special-case negative return values, since it no longer handles
Packit 709fb3
	directories, so don't bother checking for them.
Packit 709fb3
	http://bugs.gnu.org/23714
Packit 709fb3
Packit 709fb3
	maint: replace bitwise with logical OR
Packit 709fb3
	* src/grep.c (main): replace bitwise ORs with logical ORs where it
Packit 709fb3
	makes sense (when dealing with boolean conditions as opposed to
Packit 709fb3
	bitmasks).  http://bugs.gnu.org/23713
Packit 709fb3
Packit 709fb3
	maint: mark a couple of static variables const
Packit 709fb3
	* src/dfa.c (parse_bracket_exp): mark zeroclass const.
Packit 709fb3
	* src/dfasearch.c: mark patterns0 const.
Packit 709fb3
	http://bugs.gnu.org/23712
Packit 709fb3
Packit 709fb3
2016-06-03  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: fix similar bug in exit status test
Packit 709fb3
	* tests/grep-dir (status_range): New shell function.
Packit 709fb3
	Use it to fix bug where $? was not saved properly.
Packit 709fb3
Packit 709fb3
2016-06-03  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	tests: fix bug in exit status test
Packit 709fb3
	When checking $? against multiple values, save its value in another
Packit 709fb3
	variable and check that so as to avoid tests beyond the first seeing a
Packit 709fb3
	$? clobbered by earlier ones.
Packit 709fb3
Packit 709fb3
	* tests/status: save $? in a temporary variable before testing it.
Packit 709fb3
Packit 709fb3
2016-06-02  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: more simplification of dfaexec_main
Packit 709fb3
	* src/dfa.c (dfaexec_main): Failure at an acceptable position and demand
Packit 709fb3
	to build state is unlikely.  So go next loop without checking them after
Packit 709fb3
	a newline.  This commit induces no semantic change.
Packit 709fb3
Packit 709fb3
2016-06-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	maint: correct attribution
Packit 709fb3
	* build-aux/git-log-fix: Fix attribution of primary Aho-Corasick patch
Packit 709fb3
Packit 709fb3
2016-06-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: simplify -F Aho-Corasick a bit
Packit 709fb3
	This removes some tuning that complicates the code without providing
Packit 709fb3
	performance benefits that I could measure (GCC 6.1, x86-64).
Packit 709fb3
	(acexec_trans): Do not hand-unroll.  Unduplicate the code for a
Packit 709fb3
	transition step.
Packit 709fb3
Packit 709fb3
	* src/kwset.c (struct kwset.kwsexec, bmexec, acexec_trans, acexec)
Packit 709fb3
Packit 709fb3
2016-06-02  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: minor cleanups for -F Aho-Corasick
Packit 709fb3
	* NEWS: Don't claim 7x, as the value seems to be system-dependent.
Packit 709fb3
	* src/kwset.c (struct kwset.kwsexec, bmexec, acexec_trans, acexec):
Packit 709fb3
	* src/kwset.c, src/kwset.h (kwsalloc, kwsexec):
Packit 709fb3
	Don't put 'const' into the declaration when that is irrelevant to
Packit 709fb3
	the API.  More generally, don't bother with 'const' when it's only
Packit 709fb3
	a local so it is reasonably obvious to a reader that it is 'const'
Packit 709fb3
	anyway.  It would be overkill to add 'const' to all locals that
Packit 709fb3
	never change.
Packit 709fb3
	* src/kwset.c (U): Avoid unnecessary parens.
Packit 709fb3
	(treefails, memoff2_kwset, bmexec_trans, bmexec, cwexec, acexec_trans):
Packit 709fb3
	Prefer SIZE_MAX to (size_t) -1.
Packit 709fb3
	(bmexec_trans, cwexec, acexec_trans):
Packit 709fb3
	Remove attributes for static functions that no longer seem needed.
Packit 709fb3
	(memoff2_kwset): Rename from memchr2_kwset, since it returns
Packit 709fb3
	an offset, not a pointer.  All uses changed.
Packit 709fb3
	(cwexec, acexec_trans) [lint]: Remove initialization that is no
Packit 709fb3
	longer needed; at least, GCC 6.1 x86-64 does not need it.
Packit 709fb3
	(acexec_trans): Clarify code by using nesting rather than 'continue'.
Packit 709fb3
Packit 709fb3
2016-06-02  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: use memchr2 for two patterns of a character
Packit 709fb3
	* src/kwset.c (memchr2_kwset): Add a new function.  grep uses memchr2 to
Packit 709fb3
	search just two letters.
Packit 709fb3
	(cwexec, acexec_trans): Use it.
Packit 709fb3
Packit 709fb3
	grep: -F multiword longest match not always needed
Packit 709fb3
	Searching multiple fixed words, grep immediately returns without longest
Packit 709fb3
	match if not needed.  Without this change, grep tries longest match for
Packit 709fb3
	multiple words even if not needed.
Packit 709fb3
	* src/kwset.c (kwsexec, acexec, cwexec, bmexec): Add a bool argument
Packit 709fb3
	for whether longest match is needed.  All callers changed.
Packit 709fb3
	* src/kwset.h (kwsexec): Update prototype.
Packit 709fb3
Packit 709fb3
2016-06-02  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: use Aho-Corasick algorithm to search multiple fixed words
Packit 709fb3
	Searching multiple fixed words, grep used the Commentz-Walter
Packit 709fb3
	algorithm, but this was O(m*n) and was very slow in the worst case.
Packit 709fb3
	For example:
Packit 709fb3
Packit 709fb3
	  - input: yes `printf %040d` | head -10000000
Packit 709fb3
	  - word1: x0000000000000000000
Packit 709fb3
	  - word2: x
Packit 709fb3
Packit 709fb3
	This change instead uses the Aho-Corasick algorithm to search multiple
Packit 709fb3
	fixed words.  It uses a high-quality trie-building function that is
Packit 709fb3
	already defined for Commentz-Walter in kwset.c.
Packit 709fb3
Packit 709fb3
	I see 7x speed-up even for a typical case on Fedora 21 with a 3.2GHz i5
Packit 709fb3
	by this change.  Using best-of-5 trials for the benchmark:
Packit 709fb3
Packit 709fb3
	    find /usr/share/doc/ -type f |
Packit 709fb3
	    LC_ALL=C time -p xargs.sh src/grep -Ff /usr/share/dict/linux.words >/dev/null
Packit 709fb3
Packit 709fb3
	The results were:
Packit 709fb3
Packit 709fb3
	    real 11.37  user 11.03  sys 0.24  [without the change]
Packit 709fb3
	    real 1.49   user 1.31   sys 0.15  [with the change]
Packit 709fb3
Packit 709fb3
	* src/kwset.c (struct kwset): Add a new member 'mode'.
Packit 709fb3
	(kwsalloc): Use it.
Packit 709fb3
	All callers are changed.
Packit 709fb3
	(kwsincr): Using Aho-Corasick algorithm, build tries in normal order.
Packit 709fb3
	(acexec_trans, acexec): Add a new function.
Packit 709fb3
	(kwsexec): Use it.
Packit 709fb3
	* src/kwset.h (kwsalloc): Update a prototype.
Packit 709fb3
	* NEWS (Improvements): Mention it.
Packit 709fb3
Packit 709fb3
2016-05-13  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: do not let a LANGUAGE envvar setting perturb tests
Packit 709fb3
	E.g., running "LANGUAGE=eo make check" would provoke a failure
Packit 709fb3
	of the encoding-error test, on systems that mistakenly let that
Packit 709fb3
	envvar trump the setting of LC_ALL.
Packit 709fb3
	* tests/envvar-check: New file, copied from coreutils.
Packit 709fb3
	* tests/Makefile.am (EXTRA_DIST): Add it.
Packit 709fb3
	(TESTS_ENVIRONMENT): Source it.
Packit 709fb3
	Also select TMPDIR as we do for coreutils tests.
Packit 709fb3
	Reported by Benno Schulenberg in http://bugs.gnu.org/23527.
Packit 709fb3
Packit 709fb3
2016-05-02  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: avoid NEWS syntax-check failure
Packit 709fb3
	* NEWS: Move the mention of the /dev/null speed-up from the
Packit 709fb3
	block for 2.25 into the current, in-preparation block.
Packit 709fb3
Packit 709fb3
2016-05-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: prefer bool for boolean
Packit 709fb3
	* src/dfa.c (syntax_bits_set, dfasyntax, using_utf8, FETCH_WC)
Packit 709fb3
	(POP_LEX_STATE, State_transition):
Packit 709fb3
	* src/dfa.h (using_utf_8):
Packit 709fb3
	Use bool for boolean.
Packit 709fb3
Packit 709fb3
2016-05-01  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: stop exporting internal functions
Packit 709fb3
	* src/dfa.c, src/dfa.h (dfaparse, dfaanalyze, dfastate, dfainit):
Packit 709fb3
	Now static.
Packit 709fb3
Packit 709fb3
	dfa: prefer bool at DFA interfaces
Packit 709fb3
	* src/dfa.c (struct dfa, dfasyntax, dfaanalyze, dfaexec_main)
Packit 709fb3
	(dfaexec_mb, dfaexec_sb, dfaexec_noop, dfaexec, dfacomp):
Packit 709fb3
	* src/dfa.h (dfasyntax, dfacomp, dfaexec, dfaanalyze):
Packit 709fb3
	* src/dfasearch.c (EGexecute):
Packit 709fb3
	Use bool for boolean.
Packit 709fb3
Packit 709fb3
2016-05-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: speed up checking for character boundary
Packit 709fb3
	This should help performance with gawk; not so much with grep.
Packit 709fb3
	Suggested by Norihiro Tanaka in: http://bugs.gnu.org/18777
Packit 709fb3
	* src/dfa.c (never_trail): New static var.
Packit 709fb3
	(dfasyntax): Initialize it.
Packit 709fb3
	(skip_remains_mb): Use it to speed up a common case in Gawk.
Packit 709fb3
Packit 709fb3
	grep: /dev/null output speedup
Packit 709fb3
	This sped up 'seq 10000000000 | grep . >/dev/null' by a factor of
Packit 709fb3
	380,000 on my platform (Fedora 23, x86-64, AMD Phenom II X4 910e,
Packit 709fb3
	en_US.UTF-8 locale).
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (grepbuf): exit_on_match no longer implies that -q
Packit 709fb3
	was specified, so when a match is found, exit with exit_failure if
Packit 709fb3
	an error was also found.
Packit 709fb3
	(grepdesc): Omit unnecessary S_ISREG and st_ino checks.
Packit 709fb3
	out_stat.st_ino is zero if stdout is not a regular file,
Packit 709fb3
	and this cannot possibly equal st->st_ino.
Packit 709fb3
	(main): Omit duplicate initialization of exit_failure.  Do not
Packit 709fb3
	bother with isatty unless -q is not used and stdout is a character
Packit 709fb3
	special file and --color=auto and TERM says colorization is
Packit 709fb3
	possible.  Most importantly, set exit_on_match if the output is
Packit 709fb3
	/dev/null.
Packit 709fb3
	* tests/grep-dev-null-out: New test.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	* tests/status: Do not require grep to actually read all the input
Packit 709fb3
	files when the output is /dev/null and a matching line has been
Packit 709fb3
	found.
Packit 709fb3
Packit 709fb3
2016-04-21  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.25
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2016-04-19  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: remove dependency on btowc
Packit 709fb3
	MirOS BSD btowc is a macro that (when GCC is being used) hardcodes
Packit 709fb3
	btowc (0x80) == WEOF regardless of locale, which contradicts
Packit 709fb3
	future POSIX in the C locale.  Instead of bothering to develop a
Packit 709fb3
	Gnulib workaround for the btowc incompatibility, use mbrtowc,
Packit 709fb3
	which we are using elsewhere and fixing anyway, and are caching so
Packit 709fb3
	it is fast here.  Problem reported by Nelson H. F. Beebe via Jim
Packit 709fb3
	Meyering in: http://bugs.gnu.org/23269#14
Packit 709fb3
	* bootstrap.conf (gnulib_modules): Remove btowc.
Packit 709fb3
	* src/dfa.c (struct dfa): Remove mbrtowc_cache member, replacing with ...
Packit 709fb3
	(mbrtowc_cache): ... this new static var.  All uses changed.
Packit 709fb3
	(dfambcache): Remove; now done by setsyntax.  Call removed.
Packit 709fb3
	(is_valid_unibyte_character): Remove.
Packit 709fb3
	(IS_WORD_CONSTITUENT): Remove this macro, replacing it with ...
Packit 709fb3
	(unibyte_word_constituent): ... this new function.  It uses
Packit 709fb3
	mbrtowc_cache rather than btowc.
Packit 709fb3
	(dfasyntax): Initialize mbrtowc_cache before using it.
Packit 709fb3
Packit 709fb3
2016-04-10  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: minor doc tweaks inspired by Debian
Packit 709fb3
	Problem reported by Santiago Ruano Rincón in: http://bugs.gnu.org/22911
Packit 709fb3
	* doc/grep.in.1:
Packit 709fb3
	* doc/grep.texi (Matching Control, grep Programs)
Packit 709fb3
	(Regular Expressions):
Packit 709fb3
	Document -e, -f, and PCRE more carefully.
Packit 709fb3
Packit 709fb3
2016-04-10  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: remove unused mbtoupper function
Packit 709fb3
	* src/searchutils.c (mbtoupper): Remove now-unused function.
Packit 709fb3
	Also remove inclusion of <assert.h>, since this change removed
Packit 709fb3
	the final use of assert.
Packit 709fb3
	* src/search.h (mbtoupper): Remove declaration.
Packit 709fb3
Packit 709fb3
2016-04-10  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: in C locale, all bytes are valid characters
Packit 709fb3
	This works around glibc bug 19932:
Packit 709fb3
	https://sourceware.org/bugzilla/show_bug.cgi?id=19932
Packit 709fb3
	The actual bug fix was the update to the current version of Gnulib.
Packit 709fb3
	grep problem reported by Björn Jacke in: http://bugs.gnu.org/23234
Packit 709fb3
	* NEWS: Mention this.
Packit 709fb3
	* doc/grep.texi (File and Directory Selection): Crossref to LC_*
Packit 709fb3
	section.  Suggest why -a or LC_ALL=C might be useful.
Packit 709fb3
	(Environment Variables): Mention 'locale -a'.
Packit 709fb3
	Say that LC_CTYPE also specifies encoding, and that every
Packit 709fb3
	byte is a valid character in the C or POSIX locale.
Packit 709fb3
	* tests/c-locale: New test.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2016-04-05  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Give another example of binary file processing
Packit 709fb3
	Problem reported by Shlomi Fish
Packit 709fb3
	* doc/grep.texi (File and Directory Selection):
Packit 709fb3
	Document that 'q$' might match 'q' followed by a NUL
Packit 709fb3
	if --binary-files=binary is in effect.
Packit 709fb3
Packit 709fb3
2016-04-03  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: test egrep/fgrep help only if our grep
Packit 709fb3
	Problem reported by Christian Weisgerber in: http://bugs.gnu.org/23146
Packit 709fb3
	* tests/Makefile.am (TESTS_ENVIRONMENT):
Packit 709fb3
	Test egrep and fgrep only if they use our grep.
Packit 709fb3
Packit 709fb3
2016-03-29  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: remove spurious test of egrep
Packit 709fb3
	* tests/reversed-range-endpoints: Do not test egrep here.
Packit 709fb3
	There is already a test of grep -E.
Packit 709fb3
	Prompted by http://bugs.gnu.org/23146
Packit 709fb3
Packit 709fb3
2016-03-23  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -Pz no longer misdiagnoses [^a]
Packit 709fb3
	Problem reported by Michael Jess.
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/pcresearch.c (Pcompile): Do not diagnose [^ when [ is unescaped.
Packit 709fb3
	* tests/pcre: Test for the bug.
Packit 709fb3
Packit 709fb3
2016-03-22  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: move new 'Improvements' blurb into proper section
Packit 709fb3
	* NEWS (Improvements): Move this new section from within the block
Packit 709fb3
	for the already-released 2.24 into the proper "next-release" block.
Packit 709fb3
	Also, retain the 2-blank-line separator between blocks.
Packit 709fb3
Packit 709fb3
2016-03-18  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: avoid spurious "binary file ... matches" in generated THANKS
Packit 709fb3
	* Makefile.am (THANKS): Don't apply grep to a stream containing
Packit 709fb3
	NUL bytes.  Sync this rule from the one in coreutils: it was missing
Packit 709fb3
	some improvements.
Packit 709fb3
	Reported by Bailes Magio in http://bugs.gnu.org/22899
Packit 709fb3
Packit 709fb3
2016-03-18  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -oz now outputs null bytes, not newlines
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* doc/grep.texi (Other Options): Clarify that -z affects output
Packit 709fb3
	as well as input data.
Packit 709fb3
	* src/grep.c (print_line_middle): Output eolbyte, not newline, if -o.
Packit 709fb3
	* tests/null-byte: Test -o too.
Packit 709fb3
	* tests/pcre-context: Adjust test to match new behavior.
Packit 709fb3
Packit 709fb3
2016-03-17  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: use errno consistently in write diagnostics
Packit 709fb3
	Feature request and initial version reported by Assaf Gordon in:
Packit 709fb3
	http://bugs.gnu.org/23031
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c: Include <stdarg.h>.
Packit 709fb3
	(stdout_errno): New static var.
Packit 709fb3
	(write_error_seen): Remove; superseded by stdout_errno.
Packit 709fb3
	All uses changed.
Packit 709fb3
	(putchar_errno, fputs_errno, printf_errno, fwrite_errno)
Packit 709fb3
	(fflush_errno): New static functions.
Packit 709fb3
	(print_filename, print_sep, print_offset, print_line_head)
Packit 709fb3
	(print_line_middle, print_line_tail, prline, prtext, grep)
Packit 709fb3
	(grepdesc): Use them.
Packit 709fb3
	* tests/write-error-msg: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2016-03-10  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.24
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2016-02-28  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: add dist-check.mk
Packit 709fb3
	This file augments "make distcheck" rules.
Packit 709fb3
	* dist-check.mk: New file, from coreutils via gzip.
Packit 709fb3
	* Makefile.am (EXTRA_DIST): Add it.
Packit 709fb3
	* cfg.mk: Include it.
Packit 709fb3
Packit 709fb3
2016-02-21  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -Pz is incompatible with ^ and $
Packit 709fb3
	Problem reported by Sergei Trofimovich in: http://bugs.gnu.org/22655
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/pcresearch.c (Pcompile): Warn with -Pz and anchors.
Packit 709fb3
	* tests/pcre: Test new behavior.
Packit 709fb3
Packit 709fb3
2016-02-21  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: test cleanup
Packit 709fb3
	* tests/z-anchor-newline: Remove test artifact that would write
Packit 709fb3
	to /t/x.
Packit 709fb3
Packit 709fb3
2016-02-20  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep -z: avoid erroneous match with regexp anchor and \n in text
Packit 709fb3
	* src/dfasearch.c (EGexecute): Clear the newline_anchor bit when
Packit 709fb3
	eolbyte is not '\n'.
Packit 709fb3
	* tests/z-anchor-newline: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	* NEWS (Bug fixes): Describe it.
Packit 709fb3
	Originally reported by Ulrich Mueller in
Packit 709fb3
	https://bugs.gentoo.org/show_bug.cgi?id=574662
Packit 709fb3
	Reported to us by Sergei Trofimovich as http://debbugs.gnu.org/22655
Packit 709fb3
Packit 709fb3
	tests: convert "cmd && fail=1" to "returns_ 1 cmd || fail=1"
Packit 709fb3
	The latter is robust, while the former can silently ignore
Packit 709fb3
	failure due to signals.
Packit 709fb3
	* cfg.mk (sc_prohibit_and_fail_1): New rule, copied from coreutils.
Packit 709fb3
	* tests/long-pattern-perf: Perform the above substitution.
Packit 709fb3
	* tests/mb-non-UTF8-performance: Likewise.
Packit 709fb3
	* tests/help-version: Merge from coreutils.
Packit 709fb3
Packit 709fb3
2016-02-09  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: add a check-very-expensive target
Packit 709fb3
	* Makefile.am (check-very-expensive): New convenience rule,
Packit 709fb3
	currently merely equivalent to check-expensive.
Packit 709fb3
Packit 709fb3
2016-02-04  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.23
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2016-02-02  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
	Update for this "make distcheck"-fixing change:
Packit 709fb3
	> verify-tests: also remove stray test-verify.Tpo
Packit 709fb3
Packit 709fb3
2016-02-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests/null-byte: test another code path
Packit 709fb3
	* tests/null-byte: Also exercise the case in which there is
Packit 709fb3
	a match in the block along with the NUL byte.
Packit 709fb3
Packit 709fb3
2016-01-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Omit excess "Binary file ... matches"
Packit 709fb3
	Problem reported in: http://bugs.gnu.org/22461
Packit 709fb3
	* src/grep.c (grep): Don't report "Binary file ... matches"
Packit 709fb3
	merely because the file contained both matches and binary data.
Packit 709fb3
	Insist that the binary data contained a match.
Packit 709fb3
	* tests/null-byte: Add a test for this.
Packit 709fb3
Packit 709fb3
2016-01-28  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2016-01-23  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
	maint: fix typo in NEWS: s/a/an/
Packit 709fb3
Packit 709fb3
2016-01-15  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -x now supersedes -w more consistently
Packit 709fb3
	* NEWS, doc/grep.texi (Matching Control): Mention this.
Packit 709fb3
	* src/dfasearch.c (EGexecute):
Packit 709fb3
	* src/pcresearch.c (Pcompile):
Packit 709fb3
	Don't get confused by -w if -x is also present.
Packit 709fb3
	* src/pcresearch.c (Pcompile): Remove misleading comment about
Packit 709fb3
	non-UTF-8 multibyte locales, as PCRE doesn't support them.
Packit 709fb3
	Calculate buffer sizes more carefully; the old method
Packit 709fb3
	allocated a buffer slightly too big, seemingly due to luck.
Packit 709fb3
	* tests/backref-word, tests/pcre: Add tests for this bug.
Packit 709fb3
Packit 709fb3
	tests: omit update-copyright-tests
Packit 709fb3
	This test does not check how 'grep' itself operates, so it is
Packit 709fb3
	out of place for grep's 'make check'.  Problem reported by Sam Razavi in:
Packit 709fb3
	http://bugs.gnu.org/22376
Packit 709fb3
	* bootstrap.conf (avoided_gnulib_modules): Add update-copyright-tests.
Packit 709fb3
Packit 709fb3
2016-01-11  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: do use "yes" but via an AWK replacement
Packit 709fb3
	Also, use sed Nq in place of head -N
Packit 709fb3
	* tests/init.cfg (yes): Define.
Packit 709fb3
	Thanks to Paul Eggert for this definition.
Packit 709fb3
	* tests/max-count-overread: Revert to using "yes".
Packit 709fb3
	* tests/mb-non-UTF8-performance: Likewise, and use
Packit 709fb3
	"sed Nq" in place of head -N.
Packit 709fb3
Packit 709fb3
2016-01-11  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	* tests/pcre-count: Don't assume the page size is 32kB.
Packit 709fb3
Packit 709fb3
2016-01-08  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: port to other POSIXish platforms
Packit 709fb3
	I tested this on Solaris 10 and AIX 7.1.
Packit 709fb3
	* tests/max-count-overread:
Packit 709fb3
	* tests/mb-non-UTF8-performance:
Packit 709fb3
	Don't assume 'yes' exists, as 'yes' is not in POSIX.
Packit 709fb3
	* tests/mb-non-UTF8-performance:
Packit 709fb3
	Don't rely on 'head -1000', as that option syntax is not POSIX.
Packit 709fb3
	* tests/pcre-count: Don't rely on "printf '\x0'".
Packit 709fb3
	* tests/unibyte-binary: Don't assume \200 is an encoding error
Packit 709fb3
	in every unibyte locale.
Packit 709fb3
Packit 709fb3
2016-01-08  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: fix encoding-error test failure to use of printf '\xHH'
Packit 709fb3
	* tests/encoding-error: Don't rely on printf having support for \xHH
Packit 709fb3
	hexadecimal.  That is not portable.  Use \OOO octal, instead.
Packit 709fb3
Packit 709fb3
	maint: fix typo in NEWS: s/a/an/
Packit 709fb3
Packit 709fb3
2016-01-07  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	mb-non-UTF8-performance: avoid FP test failure on fast hardware
Packit 709fb3
	* tests/mb-non-UTF8-performance: Don't use a fixed size.
Packit 709fb3
	Otherwise, on a fast system, the fixed-size unibyte test
Packit 709fb3
	would complete in a nominal 0 ms, which might well be
Packit 709fb3
	smaller than 1/30 of the multibyte duration, provoking
Packit 709fb3
	a false positive test failure.  Instead, increase the
Packit 709fb3
	size of the input until we obtain a unibyte duration of
Packit 709fb3
	at least 10ms.
Packit 709fb3
Packit 709fb3
2016-01-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: mention unibyte encoding fix
Packit 709fb3
	* NEWS: Document recent fix for encoding errors in unibyte locales.
Packit 709fb3
Packit 709fb3
	grep: improve unibyte -P performance
Packit 709fb3
	This is a followon to the recent changes prompted by Bug#20526.
Packit 709fb3
	In <http://bugs.gnu.org/bug=20526#86> Norihiro Tanaka pointed out
Packit 709fb3
	that grep mistakenly assumed that unibyte locales cannot have
Packit 709fb3
	encoding errors.  Here, the mistake hurt performance significantly.
Packit 709fb3
	On Fedora 23 x86-64 in the C locale, this patch improved grep's
Packit 709fb3
	performance by a factor of 7 when run as "grep -P 'z.*a'" on the
Packit 709fb3
	output of "yes $(printf '\200\n') | head -n 1000000000".
Packit 709fb3
	* src/pcresearch.c (multibyte_locale) [HAVE_LIBPCRE]: New static var.
Packit 709fb3
	(Pcompile): Set it.
Packit 709fb3
	(Pexecute): Use it to avoid the need to call
Packit 709fb3
	buf_has_encoding_errors in unibyte locales.
Packit 709fb3
Packit 709fb3
2016-01-06  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Improve on fix for Bug#22181
Packit 709fb3
	* src/pcresearch.c (Pexecute): Update subject when skipping past
Packit 709fb3
	easily-determined encoding errors, as this is faster than letting
Packit 709fb3
	pcre_exec skip them.  On my platform this improves performance
Packit 709fb3
	4.7x on a benchmark created via "yes $(printf '\200\200\200\200
Packit 709fb3
	\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200x\n')
Packit 709fb3
	| head -n 1000000 >j; grep -oP y j" in a UTF-8 locale.  Rework
Packit 709fb3
	code that deals with PCRE_ERROR_BADUTF8 return, to avoid an
Packit 709fb3
	incorrect (albeit currently harmless) 'bol = false' assignment.
Packit 709fb3
Packit 709fb3
	grep: restore -P optimization (followup fix)
Packit 709fb3
	* src/search.h (EGexecute, Fexecute, Pexecute):
Packit 709fb3
	Change decls to match new implementations.
Packit 709fb3
	I forgot to add this file to the previous commit.
Packit 709fb3
Packit 709fb3
	grep: restore -P PCRE_NO_UTF8_CHECK optimization
Packit 709fb3
	On my platform in the en_US.utf8 locale, this makes 'grep -P "z.*a" k'
Packit 709fb3
	220x faster, where k is created by the shell command:
Packit 709fb3
	yes 'abcdefg hijklmn opqrstu vwxyz' | head -n 10000000 >k
Packit 709fb3
	* src/dfasearch.c (EGexecute):
Packit 709fb3
	* src/grep.c (execute_fp_t):
Packit 709fb3
	* src/kwsearch.c (Fexecute):
Packit 709fb3
	* src/pcresearch.c (Pexecute):
Packit 709fb3
	First arg is now char *, not char const *, since Pexecute now
Packit 709fb3
	temporarily modifies this argument.
Packit 709fb3
	* src/grep.c, src/grep.h (buf_has_encoding_errors): Now extern.
Packit 709fb3
	* src/pcresearch.c (Pexecute): Use it.  If the input is free of
Packit 709fb3
	encoding errors, use a multiline search and the PCRE_NO_UTF8_CHECK
Packit 709fb3
	option, as this is typically way faster.  This restores an
Packit 709fb3
	optimization that was removed with the recent changes for binary
Packit 709fb3
	file detection.
Packit 709fb3
Packit 709fb3
2016-01-05  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Fix calculation of unibyte_mask
Packit 709fb3
	* src/grep.c (initialize_unibyte_mask): The old method worked for
Packit 709fb3
	UTF-8 and other typical encodings, but did not work for weird
Packit 709fb3
	encodings, e.g., one where all bytes other than 0x7f and 0x80 are
Packit 709fb3
	unibyte characters.
Packit 709fb3
Packit 709fb3
2016-01-01  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: fix bug with with invalid unibyte sequence
Packit 709fb3
	This was introduced by the recent binary-data-detection changes.
Packit 709fb3
	Problem reported by Norihiro Tanaka in: http://bugs.gnu.org/20526#86
Packit 709fb3
	* src/grep.c (HIBYTE, easy_encoding, init_easy_encoding): Remove,
Packit 709fb3
	replacing with ...
Packit 709fb3
	(uword_max, unibyte_mask, initialize_unibyte_mask): ... this new
Packit 709fb3
	constant, static var, and function.  All uses changed.  The
Packit 709fb3
	unibyte_mask var generalizes the old local var hibyte_mask, which
Packit 709fb3
	worked only for encodings where every byte with 0x80 turned off is
Packit 709fb3
	a single-byte character.
Packit 709fb3
	(buf_has_encoding_errors): Return false immediately if
Packit 709fb3
	unibyte_mask is zero, not whether the current encoding is unibyte.
Packit 709fb3
	The old test was incorrect in unibyte locales in which some bytes
Packit 709fb3
	were encoding errors.
Packit 709fb3
	* tests/pcre-z: Require UTF-8 locale, since the grep -z . test now
Packit 709fb3
	needs this.  Use printf \0 rather than tr.  Port the 'grep -z .'
Packit 709fb3
	test to platforms where the C locale says '\200' is an encoding
Packit 709fb3
	error.  Use cmp rather than compare, as the file is binary and
Packit 709fb3
	so non-GNU diff might not work.
Packit 709fb3
	* tests/unibyte-binary: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2016-01-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: update copyright year, bootstrap, init.sh
Packit 709fb3
	Run "make update-copyright" and then...
Packit 709fb3
Packit 709fb3
	* gnulib: Update to latest.
Packit 709fb3
	* tests/init.sh: Update from gnulib.
Packit 709fb3
	* bootstrap: Likewise.
Packit 709fb3
Packit 709fb3
2015-12-31  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: clarify text vs binary match output
Packit 709fb3
	* NEWS:
Packit 709fb3
	* doc/grep.texi (File and Directory Selection):
Packit 709fb3
	Make it clearer that grep can now output matching text before
Packit 709fb3
	reporting a binary match.  Problem reported by Norihiro Tanaka in:
Packit 709fb3
	http://bugs.gnu.org/20526#83
Packit 709fb3
Packit 709fb3
	doc: minor clarifications
Packit 709fb3
	* doc/grep.in.1, doc/grep.texi: Minor clarifications suggested by
Packit 709fb3
	Debian documentation patches.  Problem reported by Santiago Ruano
Packit 709fb3
	Rincón in: http://bugs.gnu.org/18651
Packit 709fb3
Packit 709fb3
	grep: fix -l --line-buffer bug
Packit 709fb3
	Problem reported by Louis Sautier in: http://bugs.gnu.org/18750
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (grep, grepdesc): If --line-buffered, flush
Packit 709fb3
	stdout after outputting newline (or null byte, if applicable).
Packit 709fb3
Packit 709fb3
2015-12-30  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: remove duplicate init
Packit 709fb3
	* src/grep.c (print_line_middle): Remove duplicate initialization.
Packit 709fb3
Packit 709fb3
	grep: report line-buffered write error right away
Packit 709fb3
	* src/grep.c (prline): When line buffered, if there is a write
Packit 709fb3
	error, report it immediately rather than waiting until the next
Packit 709fb3
	line of output.
Packit 709fb3
Packit 709fb3
	grep: -c should keep counting after binary data
Packit 709fb3
	Problem and fix reported by Jaroslav Škarvada, and test case
Packit 709fb3
	reported by Norihiro Tanaka, in: http://bugs.gnu.org/22028
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (grep): Don't stop counting merely because nulls seen.
Packit 709fb3
	* tests/pcre-count: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
	dfa: port to tinycc
Packit 709fb3
	* src/dfa.c (add_utf8_anychar): Put 'const' after type.
Packit 709fb3
	Problem reported by Aharon Robbins in:
Packit 709fb3
	http://bugs.gnu.org/22260
Packit 709fb3
Packit 709fb3
	grep: be less picky about encoding errors
Packit 709fb3
	This fixes a longstanding problem introduced in grep 2.21,
Packit 709fb3
	which is overly picky about binary files.
Packit 709fb3
	* NEWS:
Packit 709fb3
	* doc/grep.texi (File and Directory Selection): Document this.
Packit 709fb3
	* src/grep.c (input_textbin, textbin_is_binary, buffer_textbin)
Packit 709fb3
	(file_textbin):
Packit 709fb3
	Remove.  All uses removed.
Packit 709fb3
	(encoding_error_output): New static var.
Packit 709fb3
	(buf_has_encoding_errors, buf_has_nulls, file_must_have_nulls):
Packit 709fb3
	New functions, which reuse bits
Packit 709fb3
	and pieces of the removed functions.
Packit 709fb3
	(lastout, print_line_head, print_line_middle, print_line_tail, prline)
Packit 709fb3
	(prpending, prtext, grepbuf):
Packit 709fb3
	Avoid use of const, now that we have
Packit 709fb3
	functions that require modifying a sentinel.
Packit 709fb3
	(print_line_head): New arg LEN.  All uses changed.
Packit 709fb3
	(print_line_head, print_line_tail):
Packit 709fb3
	Return indicator whether the output line was printed.
Packit 709fb3
	All uses changed.
Packit 709fb3
	(print_line_middle): Exit early on encoding error.
Packit 709fb3
	(grep): Use new method for determining whether file is binary.
Packit 709fb3
	* src/grep.h (enum textbin, TEXTBIN_BINARY, TEXTBIN_UNKNOWN)
Packit 709fb3
	(TEXTBIN_TEXT, input_textbin): Remove decls.  All uses removed.
Packit 709fb3
	* src/pcresearch.c (Pexecute): Remove multiline optimization,
Packit 709fb3
	since the main program no longer checks for encoding errors on input.
Packit 709fb3
	* tests/encoding-error: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2015-12-29  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: correct (make sorted) order of test file names
Packit 709fb3
	* tests/Makefile.am (TESTS): Insert new test name in sorted order.
Packit 709fb3
Packit 709fb3
2015-12-28  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: --exclude matches trailing parts of args
Packit 709fb3
	Problem reported by Vincent Lefevre in:
Packit 709fb3
	http://bugs.gnu.org/22144
Packit 709fb3
	* NEWS:
Packit 709fb3
	* doc/grep.texi (File and Directory Selection): Document this.
Packit 709fb3
	* src/grep.c (excluded_patterns, excluded_directory_patterns):
Packit 709fb3
	Now 2-element arrays, with one element for subfiles and another
Packit 709fb3
	for command-line args.  All uses changed.  This implements the change.
Packit 709fb3
	(exclude_options): New function.
Packit 709fb3
	* tests/include-exclude: Test the change.
Packit 709fb3
Packit 709fb3
2015-12-18  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep -oP: don't infloop when processing invalid UTF8 preceding a match
Packit 709fb3
	* src/pcresearch.c (Pexecute): When advancing SUBJECT past an
Packit 709fb3
	encoding error, don't blindly set P to that new value, since we
Packit 709fb3
	will soon compute SEARCH_OFFSET = P - SUBJECT, and mistakenly
Packit 709fb3
	making that difference too small would allow us to match some
Packit 709fb3
	previously-processed text, resulting in an infinite loop.
Packit 709fb3
	* NEWS (Bug fixes): Mention it.
Packit 709fb3
	* THANKS.in: Add Christian's name and email address.
Packit 709fb3
	* tests/pcre-invalid-utf8-infloop: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	Reported by Christian Boltz in http://debbugs.gnu.org/22181
Packit 709fb3
	Introduced by commit, v2.21-37-g14f8e48.
Packit 709fb3
Packit 709fb3
2015-11-04  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: mark performance-related tests as expensive
Packit 709fb3
	These performance-related tests are slightly failure prone due to
Packit 709fb3
	varying system load during the two runs.
Packit 709fb3
	Marking these tests as "expensive" makes it so they are no longer run
Packit 709fb3
	via "make check". You can still run them via make "check-expensive".
Packit 709fb3
	This makes them less likely to be run by regular users.
Packit 709fb3
	* tests/long-pattern-perf: Use expensive_.
Packit 709fb3
	* tests/mb-non-UTF8-performance: Likewise.
Packit 709fb3
	Reported by Jaroslav Skarvada in http://debbugs.gnu.org/21826
Packit 709fb3
	and by Andreas Schwab in http://debbugs.gnu.org/21812.
Packit 709fb3
Packit 709fb3
2015-11-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.22
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
	tests: pcre-jitstack: upon failure, retry with no stack size limit
Packit 709fb3
	* tests/pcre-jitstack: Don't let an example that provokes inordinate
Packit 709fb3
	stack space use cause a test failure.  Thanks to reports from and
Packit 709fb3
	analysis by Bruce Dubbs; see http://debbugs.gnu.org/21755
Packit 709fb3
Packit 709fb3
2015-10-27  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: update THANKS.in
Packit 709fb3
	* THANKS.in: Add name+email of those who found and reported
Packit 709fb3
	the bug that made grep -E '^x|x$' match any "x".
Packit 709fb3
Packit 709fb3
2015-10-25  Zev Weiss  <zev@bewilderbeest.net>
Packit 709fb3
Packit 709fb3
	dfa: plug a memory leak in dfamust
Packit 709fb3
	* src/dfa.c (dfamust): Ensure MP is freed, by refraining
Packit 709fb3
	from returning early when, at "done:" *RESULT is NULL.
Packit 709fb3
Packit 709fb3
2015-10-25  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
	* gnulib: Pull in one more portability fix:
Packit 709fb3
	  stdalign: port to Sun C 5.9
Packit 709fb3
Packit 709fb3
2015-10-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest, for portability fixes
Packit 709fb3
	* gnulib: Pull in changes like these:
Packit 709fb3
	  fts: port to C11 alignof
Packit 709fb3
	  stdalign: work around pre-4.9 GCC x86 bug
Packit 709fb3
Packit 709fb3
	maint: NEWS: correct/amend
Packit 709fb3
	* NEWS: Move the long-regexp-performance-improvement from
Packit 709fb3
	"Bug fixes" to "Improvements."  Say more and include an example.
Packit 709fb3
	The -Fw degradation was introduced in commit v2.18-125-g94555dd
Packit 709fb3
Packit 709fb3
	tests: avoid spurious failure on OpenBSD 5.8
Packit 709fb3
	* tests/fedora: Don't rely on "diff - FILE" reading from stdin.
Packit 709fb3
	Reported privately by Nelson Beebe.
Packit 709fb3
Packit 709fb3
2015-10-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest; also bootstrap and tests/init.sh
Packit 709fb3
	* bootstrap: Update from gnulib.
Packit 709fb3
	* tests/init.sh: Likewise.
Packit 709fb3
	* gnulib: Update submodule to latest.
Packit 709fb3
Packit 709fb3
	build: avoid spurious bootstrap failure involving pkg.m4
Packit 709fb3
	Running ./bootstrap could fail mistakenly at the very end in
Packit 709fb3
	its attempt to obtain a copy of pkg.m4.  It would search only
Packit 709fb3
	$(aclocal --print-ac-dir) and some other directories, but not
Packit 709fb3
	those listed in $(aclocal --print-ac-dir)/dirlist.
Packit 709fb3
	* bootstrap.conf (bootstrap_post_import_hook): Also search the
Packit 709fb3
	directories named in $(aclocal --print-ac-dir)/dirlist when that
Packit 709fb3
	file exists with nonzero size.
Packit 709fb3
Packit 709fb3
2015-10-16  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	maint: add news item
Packit 709fb3
	* NEWS: Document grep -Fw speedup.
Packit 709fb3
Packit 709fb3
	grep: simplify previous change
Packit 709fb3
	* src/grep.c (main): Simplify recently-changed grep -Fw test.
Packit 709fb3
Packit 709fb3
2015-10-16  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: use grep matcher for grep -Fw when unibyte
Packit 709fb3
	In single byte locales with grep -Fw, prefer the grep matcher to the
Packit 709fb3
	kwset matcher, as the former uses KWset and a DFA, whereas the latter
Packit 709fb3
	calls kwsexec many times until it matches a word.
Packit 709fb3
	* src/grep.c (main): Change pattern for fgrep into grep for grep -Fw in
Packit 709fb3
	single byte locales.
Packit 709fb3
Packit 709fb3
2015-10-16  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: use memchr/memrchar
Packit 709fb3
	* src/kwsearch.c (Fexecute): Prefer memchr and memrchr to doing it
Packit 709fb3
	by hand.
Packit 709fb3
Packit 709fb3
2015-10-16  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: improve performance of grep -Fw
Packit 709fb3
	* src/kwsearch.c (Fexecute): grep -Fw examined whether the previous
Packit 709fb3
	character is a word character after matching from the head of the
Packit 709fb3
	buffer.  It is extremely slow.  Now, if grep found a potential match,
Packit 709fb3
	it looks for the previous newline, and examines from there.
Packit 709fb3
Packit 709fb3
2015-10-13  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: use single quote rather than UTF-8 multi-byte version
Packit 709fb3
	* tests/backref-alt: Translate unnecessary non-ASCII in comment.
Packit 709fb3
Packit 709fb3
2015-10-13  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: make the executable a bit smaller
Packit 709fb3
	* src/dfa.c (dfamust): Hoist MB_CUR_MAX calculation out of loops.
Packit 709fb3
Packit 709fb3
2015-10-13  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: fix bug in alternate of sub-patterns that differ only in constraints
Packit 709fb3
	Fix a bug where a line incorrectly matches alternates of sub-patterns
Packit 709fb3
	that differ only in the constraints, e.g., the ERE '^a|a$'.
Packit 709fb3
	Reported by Greg Boyd in: http://debbugs.gnu.org/21670
Packit 709fb3
	* src/dfa.c (dfamust): For a pattern with constraints, check that it is
Packit 709fb3
	matched including the constraints, to judge whether it is exact.
Packit 709fb3
Packit 709fb3
	dfa: fix off-by-one error
Packit 709fb3
	* src/dfa.c (dfamust): Fix off-by-one error in computing 'must' length,
Packit 709fb3
	which caused the 'must' to be too short.  See:
Packit 709fb3
	http://bugs.gnu.org/21670#28
Packit 709fb3
Packit 709fb3
2015-10-12  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	doc: NEWS: mention a bug fix
Packit 709fb3
	* NEWS (Bug fixes): Describe it.
Packit 709fb3
	This bug was introduced by commit v2.18-85-g2c94326
Packit 709fb3
	and fixed by commit v2.21-51-g256a4b4.
Packit 709fb3
Packit 709fb3
2015-10-11  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: add test case for Bug#21670
Packit 709fb3
	* tests/options: Add test #4 to catch Bug#21670.
Packit 709fb3
	Also, do not overescape # in shell strings.
Packit 709fb3
Packit 709fb3
2015-09-19  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Add test for pop_fail_stack bug
Packit 709fb3
	Problem reported by Hanno Böck in: http://bugs.gnu.org/21513
Packit 709fb3
	If you use --with-included-regex the bug fix is in gnulib, here:
Packit 709fb3
	http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=5513b40999149090987a0341c018d05d3eea1272
Packit 709fb3
	If you use glibc, the bug fix has not been installed yet.
Packit 709fb3
	* tests/Makefile.am (XFAIL_TESTS): Add backref-alt if system matcher.
Packit 709fb3
	(TESTS): Add backref-alt.
Packit 709fb3
	* tests/backref-alt: New file.
Packit 709fb3
	* tests/triple-backref: Remove unused var.
Packit 709fb3
	Don't skip if tested with glibc, as Makefile.am now handles this.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
2015-08-19  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: avoid use of uninitialized variable
Packit 709fb3
	EGexecute would use "backref" uninitialized.
Packit 709fb3
	While that could have no bearing on correctness, it could
Packit 709fb3
	impact performance, via an unnecessary use of regexp.
Packit 709fb3
	* src/dfasearch.c (EGexecute): Initialize backref.
Packit 709fb3
	Reported as http://debbugs.gnu.org/21273
Packit 709fb3
	Introduced by commit v2.21-55-gea0ebaa.
Packit 709fb3
Packit 709fb3
2015-08-12  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep: remove fgrep code for case insensitive match
Packit 709fb3
	The fgrep matcher is no longer called in case insensitive matching,
Packit 709fb3
	so remove the code to support it.
Packit 709fb3
	* src/kwsearch.c (mb_case_map_apply): Remove function.
Packit 709fb3
	(Fexecute): Remove now-unused code.
Packit 709fb3
Packit 709fb3
2015-08-12  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	dfa: optimize [x-x]
Packit 709fb3
	* src/dfa.c (parse_bracket_exp): Treat [x-x] as if it were [x].
Packit 709fb3
	This also pacifies GCC, which otherwise complains about wc2
Packit 709fb3
	being set but not used.
Packit 709fb3
Packit 709fb3
2015-08-12  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: remove unused multibyte support
Packit 709fb3
	Now regex should be used for range, collating element, equivalent class
Packit 709fb3
	in non POSIX locales.  So remove code to support these features.
Packit 709fb3
	* dfa.c (struct mb_char_classes): Remove members ch_classes,
Packit 709fb3
	nch_classes, ranges, nranges, equivs, nequivs, coll_elems, ncoll_elems.
Packit 709fb3
	All uses removed.
Packit 709fb3
	(match_mb_charset): Remove function.
Packit 709fb3
Packit 709fb3
2015-08-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: mb-non-UTF8-performance: use new function
Packit 709fb3
	* tests/mb-non-UTF8-performance: Rewrite to use
Packit 709fb3
	the user-time measuring function in init.cfg.
Packit 709fb3
Packit 709fb3
	tests: long-pattern-perf: measure user time, not elapsed
Packit 709fb3
	Measuring user time makes this test less prone to false
Packit 709fb3
	positive failure, and also lets us use a tighter bound.
Packit 709fb3
	* tests/long-pattern-perf: Measure elapsed user time rather than
Packit 709fb3
	wall-clock time, to permit a tighter bound on the ratio of
Packit 709fb3
	N-to-10N timings.  Suggested by Giuseppe Ottaviano.
Packit 709fb3
	Also, use regexps built from mostly 5-digit numbers, so that the 10:1
Packit 709fb3
	ratio applies to lines of "seq" output as well as to total bytes.
Packit 709fb3
Packit 709fb3
	tests: new function to measure elapsed user time
Packit 709fb3
	* tests/init.cfg (user_time_): New function.
Packit 709fb3
Packit 709fb3
2015-07-25  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: remove word delimiter support for multibyte locales
Packit 709fb3
	DFA supports word delimiter expressions, but it does not behave
Packit 709fb3
	correctly for multibyte locales.  Even if it were to be fixed,
Packit 709fb3
	the DFA matcher's performance would be no better than that of regex.
Packit 709fb3
	Thus, this change removes DFA support for word delimiter expressions
Packit 709fb3
	in multibyte locales.
Packit 709fb3
Packit 709fb3
	* src/dfa.c (dfa_supported): Return false also when a pattern uses any
Packit 709fb3
	word delimiter expression in a multibyte locale.
Packit 709fb3
Packit 709fb3
2015-07-25  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: avoid execution for a pattern including an unsupported expression
Packit 709fb3
	If a pattern includes a construct unsupported by the DFA matcher,
Packit 709fb3
	the DFA search would fail in most cases.  Make dfaexec immediately
Packit 709fb3
	return for any such pattern.
Packit 709fb3
Packit 709fb3
	* src/dfa.c (struct dfa_state) [has_backref, has_mbcset]: Remove members
Packit 709fb3
	and all uses.
Packit 709fb3
	(dfaexec_main): Remove 'backref' parameter.  Update callers.
Packit 709fb3
	(dfaexec_noop): New function.
Packit 709fb3
	(dfa_supported): New function.
Packit 709fb3
	(dfassbuild): Remove now-unused code.
Packit 709fb3
	(dfacomp): When a pattern uses a DFA-unsupported construct, do not
Packit 709fb3
	waste time performing any further analysis.
Packit 709fb3
Packit 709fb3
2015-07-19  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: DEBUG: print detail of DFA states
Packit 709fb3
	When compiled with -DDEBUG, grep outputs tokens etc.
Packit 709fb3
	With this change, also print DFA states and transitions.
Packit 709fb3
	This change is very useful when debugging those.
Packit 709fb3
Packit 709fb3
	* src/dfa.c (prtok) [DEBUG]: Change `%c' to `%02x' in printf format.
Packit 709fb3
	(state_index) [DEBUG]: Print detail of new state.
Packit 709fb3
	(dfastate) [DEBUG]: Print detail of DFA states.
Packit 709fb3
	Reported as http://debbugs.gnu.org/18707
Packit 709fb3
Packit 709fb3
2015-07-18  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	tests: sjis-mb: accept two more locales
Packit 709fb3
	* tests/sjis-mb: Accept the ja_JP.SJIS and ja_JP.PCK locales
Packit 709fb3
	as well as ja_JP.SHIFT_JIS, so this test is less likely to
Packit 709fb3
	be skipped unnecessarily.  Reported as http://bugs.gnu.org/18983
Packit 709fb3
Packit 709fb3
2015-07-18  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: add a test for the performance fix
Packit 709fb3
	* tests/long-pattern-perf: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2015-07-18  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: speed up handling of long pattern
Packit 709fb3
	DFA tries to find a long sequence of characters that must appear
Packit 709fb3
	in any matching line.  However, when a pattern is long (length N),
Packit 709fb3
	it is very slow, because it makes O(N^2) strstr calls.
Packit 709fb3
	This change reduces that to O(N) by processing each sequence of
Packit 709fb3
	adjacent "regular" characters as a group.
Packit 709fb3
Packit 709fb3
	Compare the run times of this command before and after this change:
Packit 709fb3
	(on a i7-4770S CPU @ 3.10GHz using rawhide (~fedora 22) and compiled
Packit 709fb3
	with gcc 6.0.0 20150627)
Packit 709fb3
	  : | env time -f %e grep -f <(seq -s '' 9999)
Packit 709fb3
	  Before: 0.85
Packit 709fb3
	   After: 0.02
Packit 709fb3
Packit 709fb3
	* src/dfa.c (dfamust): Process each string of concatenated normal
Packit 709fb3
	characters as a unit.
Packit 709fb3
	* NEWS (Improvement): Mention it.
Packit 709fb3
	Prompted by a bug report and patch by Ivan Yanikov
Packit 709fb3
	in http://bugs.gnu.org/15191#5
Packit 709fb3
Packit 709fb3
2015-07-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: fix mis-applied patch.
Packit 709fb3
	* tests/include-exclude: I applied "|sort" to the wrong creation
Packit 709fb3
	of "out", and didn't push the same patch that I'd tested.
Packit 709fb3
Packit 709fb3
	tests: avoid FS-dependent false-positive failure
Packit 709fb3
	* tests/include-exclude: Sort file name list, so that this test
Packit 709fb3
	is not sensitive to the order in which those names are returned
Packit 709fb3
	via readdir. I noticed the failure on a Fedora 21 system using ext4.
Packit 709fb3
	Also fix a typo: s/framework_failure+/framework_failure_/
Packit 709fb3
Packit 709fb3
2015-07-13  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: fix bug with --exclude-dir and command line
Packit 709fb3
	Reported by Aron Griffis in: http://bugs.gnu.org/21027
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (grepdirent): Don't check whether the file is skipped
Packit 709fb3
	when on the command line, as that's the caller's responsibility.
Packit 709fb3
	(main): Anchor the exclude patterns.
Packit 709fb3
	* tests/include-exclude: Adjust test case to match fixed behavior.
Packit 709fb3
	Add some more test cases.
Packit 709fb3
Packit 709fb3
	tests: fix $? typo in null-byte
Packit 709fb3
	* tests/null-byte: Don't assume $? survives an invocation of 'test'.
Packit 709fb3
Packit 709fb3
2015-07-05  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: dfa: used unsigned types where appropriate
Packit 709fb3
	* src/dfa.c (case_folded_counterparts): Return unsigned int, not int.
Packit 709fb3
	Change type of two locals to unsigned int, to reflect that their
Packit 709fb3
	values are never negative.
Packit 709fb3
	(parse_bracket_exp): Adjust type of result at each use, as well
Packit 709fb3
	as that of related index variables.
Packit 709fb3
Packit 709fb3
2015-07-04  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: build struct dfamust on demand
Packit 709fb3
	If we won't use KWset, do not build a "struct dfamust".
Packit 709fb3
	Now it is built only when needed.
Packit 709fb3
	* src/dfa.c (struct dfa) [musts]: Remove member.
Packit 709fb3
	(dfacomp): Don't build dfamust here.
Packit 709fb3
	(dfamustfree): New function to free a struct dfamust.
Packit 709fb3
	(dfamust): Make it a global function, and make it return a pointer
Packit 709fb3
	to a malloc'd struct dfamust.
Packit 709fb3
	(dfamusts): Remove it.
Packit 709fb3
	* src/dfa.h (struct dfamust) [next]: Remove member.
Packit 709fb3
	In the implementation preceding this patch, there was
Packit 709fb3
	never more than one of these in a given "struct dfa".
Packit 709fb3
	(dfamustfree, dfamust): Add prototypes.
Packit 709fb3
	(dfamusts): Remove prototype.
Packit 709fb3
	(dfaalloc): Declare with _GL_ATTRIBUTE_MALLOC.
Packit 709fb3
	To make that symbol usable there, move the inclusion
Packit 709fb3
	of "xalloc.h" from dfa.c to this file, dfa.h.
Packit 709fb3
	* src/dfasearch.c (kwsmusts): Adapt to use the new interface.
Packit 709fb3
	Update the comments to reflect reality.
Packit 709fb3
	This addresses http://bugs.gnu.org/17715
Packit 709fb3
Packit 709fb3
2015-07-04  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: use recent gnulib syntax bits
Packit 709fb3
	* src/grep.c (Gcompile, Ecompile): Use plain RE_SYNTAX_GREP
Packit 709fb3
	and RE_SYNTAX_EGREP, now that we assume a recent-enough gnulib.
Packit 709fb3
Packit 709fb3
	maint: ignore gendocs_template_min
Packit 709fb3
	* doc/.gitignore: Add '/gendocs_template_min'.
Packit 709fb3
Packit 709fb3
	build: update gnulib submodule to latest
Packit 709fb3
Packit 709fb3
	dfa: '.' and '[^x]' now consistently match newline
Packit 709fb3
	* src/dfa.c (parse_bracket_exp, lex, add_utf8_anychar)
Packit 709fb3
	(match_anychar): RE_DOT_NEWLINE and RE_HAT_LISTS_NOT_NEWLINE
Packit 709fb3
	are about LF, not about eolbyte.  This patch does not affect
Packit 709fb3
	'grep', but may affect other users of dfa.c.
Packit 709fb3
Packit 709fb3
	grep: -z '[^x]' now consistently matches newline
Packit 709fb3
	Problem reported by Norihiro Tanaka in: http://bugs.gnu.org/20974#19
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (Gcompile, Ecompile): Clear RE_HAT_LISTS_NOT_NEWLINE.
Packit 709fb3
	* tests/utf8-bracket: Test this.
Packit 709fb3
Packit 709fb3
2015-07-03  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: -z '.' now consistently matches newline
Packit 709fb3
	Problem reported by Balazs Kezes in: http://bugs.gnu.org/20974
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* tests/utf8-bracket: New file, to test for this bug.
Packit 709fb3
	* src/grep.c (Gcompile, Ecompile): Also specify RE_DOT_NEWLINE.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
	grep: simplify print_line_middle slightly
Packit 709fb3
	* src/grep.c (print_line_middle): Simplify.
Packit 709fb3
Packit 709fb3
	grep: don't mishandle left context in -P
Packit 709fb3
	http://bugs.gnu.org/20957
Packit 709fb3
	* src/pcresearch.c (jit_exec): New arg SEARCH_OFFSET.
Packit 709fb3
	Caller changed.
Packit 709fb3
	(Pexecute): Pass the left context to pcre_exec, so that PCRE
Packit 709fb3
	regular-expression matching can see it.
Packit 709fb3
	* tests/pcre-context: New file, to test for this bug.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2015-06-28  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests/case-fold-backref: factor test
Packit 709fb3
Packit 709fb3
2015-06-26  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: don't hang on command-line fifo if -D skip
Packit 709fb3
	* NEWS: Document this.
Packit 709fb3
	* src/grep.c (skip_devices):
Packit 709fb3
	New function, with code taken from grepdirent.
Packit 709fb3
	(grepdirent): Use it.  Avoid an unnecessary initialization.
Packit 709fb3
	(grepfile): If skipping devices, open files with O_NONBLOCK.
Packit 709fb3
	Throw in O_NOCTTY while we're at it.
Packit 709fb3
	(grepdesc): Skip devices here, too.  Not only does this fix the
Packit 709fb3
	bug, it fixes an unlikely race condition if some other process
Packit 709fb3
	renames a device between fstatat and openat.
Packit 709fb3
	* tests/skip-device: Add a test for this bug.
Packit 709fb3
Packit 709fb3
	grep: minor tweaks
Packit 709fb3
	* src/grep.c (main): Change recently-added static vars to be
Packit 709fb3
	constants, which makes them sharable.  Prefer 'return' to 'exit'
Packit 709fb3
	when returning/exiting from 'main'.  Move decl closer to first use
Packit 709fb3
	and rename local from 'ok' (which was confusing) to 'status'.
Packit 709fb3
	Prefer named constant STDOUT_FILENO to unnamed constant 1.
Packit 709fb3
Packit 709fb3
2015-06-26  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: unify three argv-processing calls
Packit 709fb3
	* src/grep.c (main): Unify three calls to grep_commandline_arg.
Packit 709fb3
Packit 709fb3
	maint: alphabetize anonymous enum member names
Packit 709fb3
Packit 709fb3
2015-05-30  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	test: tighten tests for bracket exprs
Packit 709fb3
	* tests/posix-bracket: Test '[a-a[.-.]--]'.
Packit 709fb3
	Also, test that failures are with status 1
Packit 709fb3
	(nonmatching data), not status 2 (invalid expressions).
Packit 709fb3
Packit 709fb3
2015-04-26  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: update bootstrap from gnulib
Packit 709fb3
	* bootstrap: Update from gnulib.
Packit 709fb3
Packit 709fb3
	maint: reword a diagnostic not to trigger leading capital check
Packit 709fb3
	* src/pcresearch.c: Reword diagnostic to avoid "make syntax-check"
Packit 709fb3
	failure.
Packit 709fb3
Packit 709fb3
	maint: sort test names in tests/Makefile.am and add syntax-check rule
Packit 709fb3
	* cfg.mk (sc_sorted_tests): New rule.
Packit 709fb3
	* tests/Makefile.am (TESTS): Alphabetize.
Packit 709fb3
Packit 709fb3
2015-04-25  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: make find_pred return NULL for an invalid predicate
Packit 709fb3
	This could never happen when invoked via grep, but could have triggered
Packit 709fb3
	a bug if dfa.c's find_pred function were invoked by some other program.
Packit 709fb3
	* src/dfa.c (find_pred): Return NULL for an invalid predicate.
Packit 709fb3
	* tests/invalid-char-class: New file to test for this.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add that new file name to the list.
Packit 709fb3
	This addresses http://debbugs.gnu.org/18631
Packit 709fb3
Packit 709fb3
2015-04-06  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	build: improve pkg-config doc and error handling
Packit 709fb3
	Error-handling improvement suggested by Mike Frysinger in:
Packit 709fb3
	http://bugs.gnu.org/16757#29
Packit 709fb3
	* NEWS: Document pkg-config changes.
Packit 709fb3
	* README-prereq: pkg-config is now a prereq when building from
Packit 709fb3
	repository.
Packit 709fb3
	* m4/pcre.m4 (gl_FUNC_PCRE): Report an error if pcre is explicitly
Packit 709fb3
	requested but not available.  Defer to user-supplied PCRE_CFLAGS
Packit 709fb3
	and PCRE_LIBS.
Packit 709fb3
Packit 709fb3
	build: remove typo and don't bother with /usr/include/pcre
Packit 709fb3
	Problem reported by Holger Bruenjes.
Packit 709fb3
	* m4/pcre.m4: Remove test for /usr/include/libpng (a typo).
Packit 709fb3
	Come to think of it, don't bother worrying about
Packit 709fb3
	/usr/include/pcre, as hosts with that problem can use pkg-config
Packit 709fb3
	or configure with CFLAGS by hand.
Packit 709fb3
Packit 709fb3
	build: use pkg-config (if available) to configure libpcre
Packit 709fb3
	Problem reported by Mike Frysinger in: http://bugs.gnu.org/16757
Packit 709fb3
	* bootstrap.conf (bootstrap_post_import_hook):
Packit 709fb3
	Copy pkg-config's pkg.m4.
Packit 709fb3
	* configure.ac: Invoke PKG_PROG_PKG_CONFIG.
Packit 709fb3
	* m4/pcre.m4 (gl_FUNC_PCRE): Rewrite to use pkg-config if
Packit 709fb3
	available, and to test that pcre_compile can be linked to.
Packit 709fb3
	* src/Makefile.am (AM_CFLAGS): Add PCRE_CFLAGS.
Packit 709fb3
	(grep_LDADD): Add PCRE_LIBS.
Packit 709fb3
	* src/pcresearch.c: Simply include <pcre.h> if HAVE_LIBPCRE,
Packit 709fb3
	since 'configure' arranges for the appropriate -I option now.
Packit 709fb3
Packit 709fb3
2015-03-11  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	grep: output "." file name in diagnostic
Packit 709fb3
	This is bug C as reported by David Grayson in:
Packit 709fb3
	http://bugs.gnu.org/16444#18
Packit 709fb3
	This bug occurs only in obscure circumstances, and I didn't see
Packit 709fb3
	how to write a reasonable test case for it.
Packit 709fb3
	* src/grep.c (filename_prefix_len): Remove, replacing with ...
Packit 709fb3
	(omit_dot_slash): New static var.  All uses of the former replaced
Packit 709fb3
	with uses of the latter.
Packit 709fb3
	(grepdirent): Don't add 2 if the filename is just ".".
Packit 709fb3
Packit 709fb3
	egrep, fgrep: just use what's in PATH
Packit 709fb3
	* src/egrep.sh: Don't monkey with PATH; just use whatever 'grep'
Packit 709fb3
	is in the path.  This is simpler, and lets the user specify
Packit 709fb3
	default options with a script for only grep, with no need for
Packit 709fb3
	egrep and fgrep scripts.
Packit 709fb3
	Fixes: bug#19998
Packit 709fb3
Packit 709fb3
	doc: give a script wrapper example
Packit 709fb3
	* doc/grep.texi (Environment Variables): Give an example of a
Packit 709fb3
	wrapper script, as an alternative to using GREP_OPTIONS.
Packit 709fb3
	Fixes: bug#19998
Packit 709fb3
Packit 709fb3
	doc: clarify how -a matches
Packit 709fb3
	* doc/grep.in.1, doc/grep.texi (File and Directory Selection):
Packit 709fb3
	Give an example of how non-text bytes affect pattern matching in
Packit 709fb3
	binary files.
Packit 709fb3
	Fixes: bug#20080
Packit 709fb3
Packit 709fb3
2015-02-23  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Cover the non-INSTALL case
Packit 709fb3
	* README: Mention what to do if there is no INSTALL file.
Packit 709fb3
	Fixes: bug#19928
Packit 709fb3
Packit 709fb3
2015-02-11  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: use ASAN-poisoning more carefully
Packit 709fb3
	The ASAN-poisoning instituted by commit v2.21-14-g1555185 was
Packit 709fb3
	incomplete, since the poisoned tail of the read buffer could well
Packit 709fb3
	be the target of a legitimate follow-on read.  To accommodate that,
Packit 709fb3
	we must unpoison each such region just before beginning fillbuf's
Packit 709fb3
	read loop.
Packit 709fb3
	* src/grep.c [HAVE_ASAN] (asan_poison): Define.
Packit 709fb3
	(clear_asan_poison): Define.
Packit 709fb3
	(fillbuf): Clear before reading, since we are likely to read
Packit 709fb3
	into memory that was poisoned on the preceding iteration.
Packit 709fb3
	* tests/two-files: New file, to test for this.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
Packit 709fb3
2015-02-10  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	Grow the JIT stack if it becomes exhausted
Packit 709fb3
	Problem reported by Oliver Freyermuth in: http://bugs.gnu.org/19833
Packit 709fb3
	* NEWS: Document the fix.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add pcre-jitstack.
Packit 709fb3
	* tests/pcre-jitstack: New file.
Packit 709fb3
	* src/pcresearch.c (NSUB): Move decl earlier, since it's needed
Packit 709fb3
	earlier now.
Packit 709fb3
	(jit_stack_size) [PCRE_STUDY_JIT_COMPILE]: New static var.
Packit 709fb3
	(jit_exec): New function.
Packit 709fb3
	(Pcompile): Initialize jit_stack_size.
Packit 709fb3
	(Pexecute): Use new jit_exec function.  Report a useful diagnostic
Packit 709fb3
	if the error is PCRE_ERROR_JIT_STACKLIMIT.
Packit 709fb3
Packit 709fb3
2015-02-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: reference CVE-2015-1345 from NEWS
Packit 709fb3
	* NEWS: Mention the CVE that was addressed by v2.21-13-g83a95bd,
Packit 709fb3
	"grep -F: fix a heap buffer (read) overrun".
Packit 709fb3
Packit 709fb3
2015-01-18  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: convert "goto" to "continue" and remove now-spurious label
Packit 709fb3
	* src/kwset.c (bmexec_trans): Using "goto big_advance" here is
Packit 709fb3
	equivalent to using "continue".  Make that change and remove
Packit 709fb3
	the now-unused label.
Packit 709fb3
Packit 709fb3
2015-01-10  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: add support for ASAN memory poisoning
Packit 709fb3
	This lets us reliably detect with ASAN some UMR bugs
Packit 709fb3
	that would otherwise be detectable only some of the time
Packit 709fb3
	with MSAN.  Use __asan_poison_memory_region to mark the unused
Packit 709fb3
	portion of a read buffer as inaccessible.  Then, with ASAN,
Packit 709fb3
	any attempt to access those bytes results in an ASAN abort.
Packit 709fb3
	* src/system.h: Include "ignore-value.h".
Packit 709fb3
	(__has_feature): Define.
Packit 709fb3
	(HAVE_ASAN): Define when address sanitizer is enabled.
Packit 709fb3
	[HAVE_ASAN]: Declare these two __asan_* symbols.
Packit 709fb3
	[!HAVE_ASAN] (__asan_poison_memory_region): Define stub.
Packit 709fb3
	[!HAVE_ASAN] (__asan_unpoison_memory_region): Likewise.
Packit 709fb3
	* src/grep.c: Use __asan_poison_memory_region.
Packit 709fb3
Packit 709fb3
2015-01-09  Yuliy Pisetsky  <ypisetsky@fb.com>
Packit 709fb3
Packit 709fb3
	grep -F: fix a heap buffer (read) overrun
Packit 709fb3
	grep's read buffer is often filled to its full size, except when
Packit 709fb3
	reading the final buffer of a file.  In that case, the number of
Packit 709fb3
	bytes read may be far less than the size of the buffer.  However, for
Packit 709fb3
	certain unusual pattern/text combinations, grep -F would mistakenly
Packit 709fb3
	examine bytes in that uninitialized region of memory when searching
Packit 709fb3
	for a match.  With carefully chosen inputs, one can cause grep -F to
Packit 709fb3
	read beyond the end of that buffer altogether.  This problem arose via
Packit 709fb3
	commit v2.18-90-g73893ff with the introduction of a more efficient
Packit 709fb3
	heuristic using what is now the memchr_kwset function. The use of
Packit 709fb3
	that function in bmexec_trans could leave TP much larger than EP,
Packit 709fb3
	and the subsequent call to bm_delta2_search would mistakenly access
Packit 709fb3
	beyond end of the main input read buffer.
Packit 709fb3
Packit 709fb3
	* src/kwset.c (bmexec_trans): When TP reaches or exceeds EP,
Packit 709fb3
	do not call bm_delta2_search.
Packit 709fb3
	* tests/kwset-abuse: New file.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	* THANKS.in: Update.
Packit 709fb3
	* NEWS (Bug fixes): Mention it.
Packit 709fb3
Packit 709fb3
	Prior to this patch, this command would trigger a UMR:
Packit 709fb3
Packit 709fb3
	  printf %0360db 0 | valgrind src/grep -F $(printf %019dXb 0)
Packit 709fb3
Packit 709fb3
	  Use of uninitialised value of size 8
Packit 709fb3
	     at 0x4142BE: bmexec_trans (kwset.c:657)
Packit 709fb3
	     by 0x4143CA: bmexec (kwset.c:678)
Packit 709fb3
	     by 0x414973: kwsexec (kwset.c:848)
Packit 709fb3
	     by 0x414DC4: Fexecute (kwsearch.c:128)
Packit 709fb3
	     by 0x404E2E: grepbuf (grep.c:1238)
Packit 709fb3
	     by 0x4054BF: grep (grep.c:1417)
Packit 709fb3
	     by 0x405CEB: grepdesc (grep.c:1645)
Packit 709fb3
	     by 0x405EC1: grep_command_line_arg (grep.c:1692)
Packit 709fb3
	     by 0x4077D4: main (grep.c:2570)
Packit 709fb3
Packit 709fb3
	See the accompanying test for how to trigger the heap buffer overrun.
Packit 709fb3
Packit 709fb3
	Thanks to Nima Aghdaii for testing and finding numerous
Packit 709fb3
	ways to break early iterations of this patch.
Packit 709fb3
Packit 709fb3
2015-01-08  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	grep: avoid false-positive UMR
Packit 709fb3
	For some inputs, valgrind would report an uninitialized
Packit 709fb3
	memory read error, but it was harmless.
Packit 709fb3
	* src/grep.c (fillbuf): Initialize those trailing bytes.
Packit 709fb3
Packit 709fb3
2015-01-01  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
	maint: update copyright year ranges to include 2015
Packit 709fb3
	Run "make update-copyright".  Also, ...
Packit 709fb3
	* grep.texi: Update manually, converting each "--" to "-".
Packit 709fb3
Packit 709fb3
2014-12-15  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	doc: document binary-data heuristic better
Packit 709fb3
	Problem reported by Martin Hoch in: http://bugs.gnu.org/19388
Packit 709fb3
	* doc/grep.texi (File and Directory Selection):
Packit 709fb3
	Document what non-text bytes are.
Packit 709fb3
	(Usage): Fix cross reference.
Packit 709fb3
Packit 709fb3
2014-12-12  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: fix a new "make syntax-check" failure
Packit 709fb3
	* tests/dfa-match-aux.c: s/can not/cannot/
Packit 709fb3
Packit 709fb3
2014-12-12  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	build: avoid build failure with --enable-gcc-warnings and no PCRE
Packit 709fb3
	* src/pcresearch.c [HAVE_LIBPCRE] (empty_match): Guard the declaration
Packit 709fb3
	of this PCRE-only variable.
Packit 709fb3
Packit 709fb3
2014-12-07  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: port fmbtest to CentOS 6 and earlier
Packit 709fb3
	* tests/fmbtest: Port to platforms where the 'sed' pattern
Packit 709fb3
	'[^0-9]' does not match every non-digit character.  Problem
Packit 709fb3
	reported by Norihiro Tanaka in: http://bugs.gnu.org/19293
Packit 709fb3
Packit 709fb3
2014-12-06  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: simplify dfaexec
Packit 709fb3
	* src/dfa.c (dfaexec): Simplify by rearrangement of IF conditions.
Packit 709fb3
	This commit induces no semantic change, and reverts part of commit
Packit 709fb3
	v2.5.4-144-gbafa134.
Packit 709fb3
Packit 709fb3
2014-12-06  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: avoid invalid match or infinite loop in unused matching mode
Packit 709fb3
	Neither grep nor gawk uses this DFA code in its matching mode,
Packit 709fb3
	since each always calls dfacomp with a nonzero final argument.
Packit 709fb3
	However, when used in that mode, it had bug:
Packit 709fb3
	After failing to match in matching mode, it should return NULL,
Packit 709fb3
	but instead would either report a false match or enter an
Packit 709fb3
	infinite loop.
Packit 709fb3
Packit 709fb3
	* src/dfa.c (dfaexec_main): After failing to match in matching mode
Packit 709fb3
	return NULL, rather than transitioning to the next state.
Packit 709fb3
	* tests/dfa-match: Add a new test.
Packit 709fb3
	* tests/dfa-match-aux.c: Add a new program to exercise this
Packit 709fb3
	otherwise-unused part of dfa.c.
Packit 709fb3
	* tests/Makefile.am: Add a rule to build new test.
Packit 709fb3
	(check_PROGRAMS): Add dfa-match-aux.
Packit 709fb3
	(AM_CPPFLAGS): Add -I$(top_srcdir)/src.
Packit 709fb3
	(TESTS): Add dfa-match.
Packit 709fb3
	* cfg.mk (exclude_file_name_regexp--sc_bindtextdomain):
Packit 709fb3
	(exclude_file_name_regexp--sc_prohibit_atoi_atof):
Packit 709fb3
	Exempt the new test file from some syntax-check rules.
Packit 709fb3
Packit 709fb3
2014-12-04  Santiago Ruano Rincón  <santiago@debian.org>
Packit 709fb3
Packit 709fb3
	doc: document grep-2.11 change in behavior of -r, --recursive
Packit 709fb3
	* doc/grep.texi (--recursive, -r): Mention the new behavior
Packit 709fb3
	of recursively searching "." when there is no FILE argument.
Packit 709fb3
	* doc/grep.in.1: Likewise.
Packit 709fb3
	That change first appeared in grep-2.11, released on 2012-03-02.
Packit 709fb3
Packit 709fb3
2014-11-24  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: correct for four Author: name misspellings
Packit 709fb3
	* .mailmap: Correct for misspelling in Norihiro Tanaka's last name
Packit 709fb3
	as listed in four commit Author: fields: s/Norihirio/Norihiro/
Packit 709fb3
Packit 709fb3
2014-11-23  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	maint: post-release administrivia
Packit 709fb3
	* NEWS: Add header line for next release.
Packit 709fb3
	* .prev-version: Record previous version.
Packit 709fb3
	* cfg.mk (old_NEWS_hash): Auto-update.
Packit 709fb3
Packit 709fb3
	version 2.21
Packit 709fb3
	* NEWS: Record release date.
Packit 709fb3
Packit 709fb3
2014-11-21  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: sjis-mb: remove now-obsolete and failing sub-tests
Packit 709fb3
	* tests/sjis-mb: Commit v2.18-123-geb3292b changed how grep
Packit 709fb3
	handles patterns with encoding errors.  These SJIS tests are
Packit 709fb3
	skipped so often that we didn't notice until now that there were
Packit 709fb3
	two tests of that changed behavior, and that on any system with
Packit 709fb3
	the ja_JP.SHIFT_JIS locale, they would always fail.  Remove those
Packit 709fb3
	two tests, since this functionality is well tested separately,
Packit 709fb3
	via tests/prefix-of-multibyte.
Packit 709fb3
Packit 709fb3
2014-11-20  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep -F could erroneously fail to match in non-UTF8 multibyte locales
Packit 709fb3
	This fixes a bug that can strike only when using a non-UTF8 multibyte
Packit 709fb3
	locale like ja_JP.SHIFT_JIS.
Packit 709fb3
Packit 709fb3
	Consider this example: it would mistakenly fail to match before
Packit 709fb3
	this patch:
Packit 709fb3
Packit 709fb3
	  printf '\203AA\n'|LC_ALL=ja_JP.SHIFT_JIS src/grep -F A
Packit 709fb3
Packit 709fb3
	When searching for a single byte that happens to be the latter
Packit 709fb3
	byte of a multibyte character, and the target byte also follows
Packit 709fb3
	that multibyte character, grep -F would advance an internal pointer
Packit 709fb3
	by one byte too many, thus missing the target byte.  A test case
Packit 709fb3
	for this bug is already included in tests/sjis-mb.
Packit 709fb3
Packit 709fb3
	* src/kwsearch.c (Fexecute): Skip one byte less, after matched middle of a
Packit 709fb3
	multi-byte character.  Introduced by commit v2.18-119-gfb7d538.
Packit 709fb3
Packit 709fb3
2014-11-17  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: big-match: disable OOM-provoking subtest
Packit 709fb3
	* tests/big-match: Our application of this regexp '^.*x\(\)\1'
Packit 709fb3
	to a file containing a single matching line of length 2GiB+2
Packit 709fb3
	would cause inordinate memory consumption (over 100GB) via
Packit 709fb3
	regexec.c, but no leak.  That would cause disruption on most
Packit 709fb3
	systems, so remove this subtest.  Reported by Assaf Gordon.
Packit 709fb3
Packit 709fb3
2014-11-16  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	dfa: avoid undefined behavior
Packit 709fb3
	* src/dfa.c (dfassbuild): Don't call memcpy with a second
Packit 709fb3
	argument of NULL, even when the size (3rd argument) is 0.
Packit 709fb3
Packit 709fb3
2014-11-14  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	gnulib: update to latest
Packit 709fb3
Packit 709fb3
2014-11-14  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	grep -F -x -o PAT would print an extra newline for each match
Packit 709fb3
	* src/kwsearch.c (Fexecute): Correctly compute the length of a match
Packit 709fb3
	by subtracting 2 (not 1) when match_lines is set.  With -x, we augment
Packit 709fb3
	the "line" by both prepending and appending an EOLBYTE to the search
Packit 709fb3
	pattern.  Here, we must correct for that.  However, to compensate,
Packit 709fb3
	when we are using -x (--line-regexp) and start_ptr is NULL, we have
Packit 709fb3
	to add 1 to the length so that we still print the trailing EOLBYTE.
Packit 709fb3
	Introduced by commit v2.18-85-g2c94326.
Packit 709fb3
	* tests/match-lines: Add a new test.
Packit 709fb3
	* tests/Makefile.am (TESTS): Add it.
Packit 709fb3
	* NEWS (Bug fixes): Mention it.
Packit 709fb3
Packit 709fb3
2014-11-11  Paul Eggert  <eggert@cs.ucla.edu>
Packit 709fb3
Packit 709fb3
	tests: port to Darwin
Packit 709fb3
	The 'sed' command 's/.//' does not delete all bytes in the C locale.
Packit 709fb3
	Problem reported by Nelson H. F. Beebe.
Packit 709fb3
	* tests/fmbtest: Don't assume that sed treats bytes with the
Packit 709fb3
	top bit set as valid characters in the C locale, as this is not
Packit 709fb3
	true for Darwin.  Use the cs_CZ.UTF-8 locale instead, and
Packit 709fb3
	simplify the sed script.
Packit 709fb3
Packit 709fb3
	tests: fix recently-introduced stray output
Packit 709fb3
	* tests/init.cfg (require_pcre_): Remove stray debugging output.
Packit 709fb3
Packit 709fb3
	build: port to GCC 4.6.4 + glibc 2.5
Packit 709fb3
	On platforms this old, building with _FORTIFY_SOURCE equal to 2
Packit 709fb3
	results in duplicate definitions of standard library functions.
Packit 709fb3
	Problem reported by Nelson H. F. Beebe.
Packit 709fb3
	* configure.ac (_FORTIFY_SOURCE): Sort after GNULIB_PORTCHECK.
Packit 709fb3
	By default, do not enable this unless GNULIB_PORTCHECK is defined.
Packit 709fb3
	This better matches the original intent, which as I recall was to
Packit 709fb3
	enable these extra checks only with --enable-gcc-warnings.
Packit 709fb3
Packit 709fb3
	tests: port to libpcre sans UTF-8 support
Packit 709fb3
	Problem reported by Nelson H. F. Beebe.
Packit 709fb3
	* tests/pcre-infloop, tests/pcre-invalid-utf8-input, tests/pcre-utf8:
Packit 709fb3
	Skip the test unless PCRE works in an en_US.UTF-8 locale.
Packit 709fb3
Packit 709fb3
2014-11-09  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: do not fail when the zh_CN.UTF-8 locale is not installed
Packit 709fb3
	* tests/word-multibyte: This test would fail on a system with
Packit 709fb3
	no zh_CN.UTF-8 locale. Use it only if it is installed.
Packit 709fb3
Packit 709fb3
	tests: avoid hex_printf_ portability problems
Packit 709fb3
	* tests/init.cfg (hex_printf_): Spell out a-f and A-F, for
Packit 709fb3
	non-C locales, ensure that the input to sed is newline-terminated,
Packit 709fb3
	and quote the final octal format string.
Packit 709fb3
	Suggestions from Paul Eggert.
Packit 709fb3
Packit 709fb3
2014-11-08  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: avoid a multibyte tr portability problem
Packit 709fb3
	* tests/init.cfg (tr): New wrapper function.
Packit 709fb3
	See comments for details.  Reported by Norihiro Tanaka
Packit 709fb3
	in http://debbugs.gnu.org/18991
Packit 709fb3
Packit 709fb3
	maint: remove spurious LC_ALL setting from one test
Packit 709fb3
	* tests/word-multibyte: Remove unnecessary setting of LC_ALL.
Packit 709fb3
Packit 709fb3
	tests: fix typo in previous change
Packit 709fb3
	* tests/init.cfg (hex_printf_): Fix typo s/A-f/A-F/.
Packit 709fb3
	For the record, I introduced that error, not Norihiro.
Packit 709fb3
Packit 709fb3
2014-11-08  Norihiro Tanaka  <noritnk@kcn.ne.jp>
Packit 709fb3
Packit 709fb3
	tests: avoid awk+printf+\xHH portability trap
Packit 709fb3
	* tests/init.cfg (hex_printf_): Rewrite in terms of printf and sed.
Packit 709fb3
	Using awk's printf with \xHH in the format string was not portable
Packit 709fb3
	to the awk of Solaris 10, AIX 7 or HP-UX 11.23, as reported in
Packit 709fb3
	http://debbugs.gnu.org/18987.
Packit 709fb3
	* tests/word-multibyte: Use printf rather than hex_printf_,
Packit 709fb3
	and give the character we're printing a name: e_acute (rather
Packit 709fb3
	than A-grave), since that is used in other tests.
Packit 709fb3
	a trailing \n in the format string, adjust by removing it, and
Packit 709fb3
	instead invoking echo.
Packit 709fb3
	* tests/multibyte-white-space: Simply remove each trailing \n.
Packit 709fb3
	They were not needed.
Packit 709fb3
Packit 709fb3
2014-11-07  Jim Meyering  <meyering@fb.com>
Packit 709fb3
Packit 709fb3
	tests: avoid printf+\xHH portability trap
Packit 709fb3
	* tests/word-multibyte: Using the bourne shell's printf function
Packit 709fb3
	with strings like "\xHH\xHH" happens to work for most interactive