CVS log for mandoc/mansearch.c

[BACK] Up to [cvsweb.bsd.lv] / mandoc

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.82 / (download) - annotate - [select for diffs], Mon Jul 1 22:56:24 2019 UTC (13 months, 1 week ago) by schwarze
Branch: MAIN
CVS Tags: HEAD
Changes since 1.81: +2 -2 lines
Diff to previous 1.81 (colored)

delete trailing whitespace and space-tab sequences; no code change;
patch from Michal Nowak <mnowak at startmail dot com>
who found these with git pbchk in the illumos tree

Revision 1.81 / (download) - annotate - [select for diffs], Tue Apr 30 18:51:57 2019 UTC (15 months, 1 week ago) by schwarze
Branch: MAIN
Changes since 1.80: +7 -3 lines
Diff to previous 1.80 (colored)

In man(1) mode, i.e. when asking for a single manual page by name,
prefer file name matches over .Dt/.TH matches over first NAME matches
over later NAME matches, but do not change the ordering for apropos(1)
nor for man -a.

This reverts main.c rev. 1.310 and mansearch.h rev. 1.29
and includes a partial revert of mansearch.c rev. 1.79.

Regression reported by Lorenzo Beretta <loreb at github>
as part of https://github.com/void-linux/void-packages/issues/9868 .

Revision 1.80 / (download) - annotate - [select for diffs], Thu Dec 13 11:55:46 2018 UTC (19 months, 3 weeks ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_5
Changes since 1.79: +1 -2 lines
Diff to previous 1.79 (colored)

Cleanup, no functional change:
Split the top level parser interface out of the utility header
mandoc.h, into a new header mandoc_parse.h, for use in the main
program and in the main parser only.
Move enum mandoc_os into roff.h because struct roff_man is the
place where it is stored.
This allows removal of mandoc.h from seven files in low-level
parsers and in formatters.

Revision 1.79 / (download) - annotate - [select for diffs], Thu Nov 22 12:01:46 2018 UTC (20 months, 2 weeks ago) by schwarze
Branch: MAIN
Changes since 1.78: +3 -7 lines
Diff to previous 1.78 (colored)

In apropos(1) output, stop sorting .Nm search results by name
priorities (bits).  The obscure feature wasn't documented and merely
confused people - for example Edward Tomasz Napierala <trasz at
FreeBSD>, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227408.

Smaller patch provided by Yuri Pankov <yuripv at FreeBSD>, but i'm
also retiring the now unused "bits" member from struct manpage.
Simplification is good.

Revision 1.78 / (download) - annotate - [select for diffs], Mon Nov 19 19:27:37 2018 UTC (20 months, 3 weeks ago) by schwarze
Branch: MAIN
Changes since 1.77: +5 -4 lines
Diff to previous 1.77 (colored)

Improve POSIX compliance by making case-insensitive extended
regular expressions the default in man(1) -k searches, also matching
what the man-db package used by many Linux distributions does.

Originally requested by Wolfram Schneider <wosch at FreeBSD>
via Yuri Pankov <yuripv at FreeBSD>.

Feedback and OK cheloha@, and no objections when shown on tech@.

Thanks to cheloha@ for pointing out that POSIX requires this behaviour
and for the suggestion to explicitly say that *extended* regular
expressions are used here.

While here, unify spelling of case-[in]sensitive, fix a typo,
update the EXAMPLES, and add a STANDARDS section.

Revision 1.77 / (download) - annotate - [select for diffs], Tue Aug 22 17:50:11 2017 UTC (2 years, 11 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_4
Changes since 1.76: +11 -1 lines
Diff to previous 1.76 (colored)

When finding a bogus database entry,
print a meaningful warning and skip the entry.
Issue reported by espie@.

Revision 1.76 / (download) - annotate - [select for diffs], Wed Aug 2 13:29:04 2017 UTC (3 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_3
Changes since 1.75: +49 -18 lines
Diff to previous 1.75 (colored)

No longer use names that only occur in the SYNOPSIS section as names
for man(1) lookup.  For OpenBSD base and Xenocara, that functionality
was never intended to be required, and i just fixed the last handful
of offenders using it - not counting the horribly ill-designed
interfaces engine(3) and lh_new(3) which are impossible to properly
document in the first place.

Of course, apropos(1) and whatis(1) continue to use SYNOPSIS .Nm,
.Fn, and .Fo macros, so "man -k ENGINE_get_load_privkey_function"
still works.

This change also gets rid of a few bogus warnings "cross reference
to self" which actually are *not* to self, like in yp(8).

This former functionality was intended to help third-party software
in the ports tree and on non-OpenBSD systems containing manual pages
with incomplete or corrupt NAME sections.  But it turned out it did
more harm than good, and caused more confusion than relief,
specifically for third party manuals and for maintainers of
mandoc-portable on other operating systems.  So kill it.
Problems reported, among others, by Yuri Pankov (illumos).

OK jmc@

Revision 1.75 / (download) - annotate - [select for diffs], Wed Jul 19 14:05:13 2017 UTC (3 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_2
Changes since 1.74: +19 -3 lines
Diff to previous 1.74 (colored)

Prefer arch-dependent over arch-independent pages if the name priority,
the section number, and all names match.  Changes little on installed
systems except the ordering of apropos(1) results, because we install
base and Xenocara manuals in different trees, but fixes lookup of pages
like apm(4) vs. apm(4/amd64) in man.cgi(8).
Issue discovered by martian67 on freenode and reported via tj@.

Revision 1.74 / (download) - annotate - [select for diffs], Sat Jul 1 09:47:30 2017 UTC (3 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.73: +9 -3 lines
Diff to previous 1.73 (colored)

Basic reporting of .Xrs to manual pages that don't exist
in the base system, inspired by mdoclint(1).

We are able to do this because (1) the -mdoc parser, the -Tlint validator,
and the man(1) manual page lookup code are all in the same program
and (2) the mandoc.db(5) database format allows fast lookup.

Feedback from, previous versions tested by, and OK jmc@.

A few features will be added to this in the tree, step by step.

Revision 1.73 / (download) - annotate - [select for diffs], Wed May 17 21:19:32 2017 UTC (3 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.72: +2 -1 lines
Diff to previous 1.72 (colored)

Do not warn if a database file does not exist
because that is simply equivalent to an empty database.
Suggested by ajacoutot@

Revision 1.72 / (download) - annotate - [select for diffs], Tue Apr 18 15:06:49 2017 UTC (3 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.71: +6 -1 lines
Diff to previous 1.71 (colored)

The apropos(1) manual still documents the unary -i operator for
regular expression search terms, but it appears that somewhere
along the way, the implementation got lost, so restore it.
Bug found while investigating other reports from Gonzalo Tornaria.

Revision 1.71 / (download) - annotate - [select for diffs], Tue Apr 18 13:57:56 2017 UTC (3 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.70: +2 -0 lines
Diff to previous 1.70 (colored)

Fix previous: i forgot explicit NUL termination;
noticed by Gonzalo <Tornaria at cmat dot edu dot uy>, thanks!

Revision 1.70 / (download) - annotate - [select for diffs], Mon Apr 17 20:05:08 2017 UTC (3 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.69: +52 -24 lines
Diff to previous 1.69 (colored)

Restore -kO Nd, -kO Nm, -kO sec, and -kO arch to working order.
They got broken in the SQLite removal.
As opposed to the rest of -kO, they are no longer very useful,
but they are certainly not supposed to fail assertions.
Issue reported by Gonzalo Tornaria <tornaria at cmat dot edu dot uy>.

Revision 1.69 / (download) - annotate - [select for diffs], Fri Mar 3 13:11:21 2017 UTC (3 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.68: +1 -0 lines
Diff to previous 1.68 (colored)

A missing initialization could randomly cause regular expression
searches to be case-insensitive that ought to be case sensitive.
Found by jsg@ with scan-build.

Revision 1.68 / (download) - annotate - [select for diffs], Fri Aug 5 23:08:03 2016 UTC (4 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_1
Changes since 1.67: +2 -0 lines
Diff to previous 1.67 (colored)

Really protect <err.h> as already promised in the previous commit message.

Revision 1.67 / (download) - annotate - [select for diffs], Wed Jul 20 00:23:14 2016 UTC (4 years ago) by schwarze
Branch: MAIN
Changes since 1.66: +2 -1 lines
Diff to previous 1.66 (colored)

Linux compatibility: Work around the lack of EFTYPE and protect <err.h>.

Revision 1.66 / (download) - annotate - [select for diffs], Tue Jul 19 21:31:55 2016 UTC (4 years ago) by schwarze
Branch: MAIN
Changes since 1.65: +469 -573 lines
Diff to previous 1.65 (colored)

Remove the dependency on SQLite without loss of functionality.
Stop supporting systems that don't have mmap(3).
Drop the obsolete names_check() now that we deleted MLINKS.

Revision 1.65 / (download) - annotate - [select for diffs], Sat Jul 9 15:24:19 2016 UTC (4 years, 1 month ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_4, VERSION_1_13
Changes since 1.64: +1 -2 lines
Diff to previous 1.64 (colored)

getopt(3) is declared in <unistd.h>, and <getopt.h> is not needed;
from Joerg Sonnenberger via Thomas Klausner, NetBSD.

Revision 1.64 / (download) - annotate - [select for diffs], Fri Jan 8 15:02:54 2016 UTC (4 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.63: +6 -6 lines
Diff to previous 1.63 (colored)

Prefer warn(3) over perror(3) at the few places where it was used.
It is useful to see the program name, and we have err.h compat in place anyway.
Suggested by Christos Zoulas (NetBSD).

Revision 1.63 / (download) - annotate - [select for diffs], Thu Nov 26 07:42:11 2015 UTC (4 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.62: +10 -7 lines
Diff to previous 1.62 (colored)

No point in trying to go on when elementary database operations
like preparing queries or binding variables fail; that won't yield
useful results anyway but may generate huge pointless error messages.
Issue reported by deraadt@.

Revision 1.62 / (download) - annotate - [select for diffs], Sat Nov 7 14:22:29 2015 UTC (4 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.61: +3 -1 lines
Diff to previous 1.61 (colored)

Without HAVE_ERR, don't try to include <err.h>, it probably isn't there.
In that case, the required prototypes are in "config.h".
Patch from Peter Bray <pdb_ml at yahoo dot com dot au>.

Revision 1.61 / (download) - annotate - [select for diffs], Fri Nov 6 21:19:09 2015 UTC (4 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.60: +8 -1 lines
Diff to previous 1.60 (colored)

In ./configure, select a RE syntax for word boundaries supported by libc;
issue reported by Svyatoslav Mishyn, Peter Bray, and Daniel Levai.

Revision 1.60 / (download) - annotate - [select for diffs], Tue Oct 13 15:53:05 2015 UTC (4 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.59: +3 -38 lines
Diff to previous 1.59 (colored)

Reduce the amount of code by moving the three copies of the ohash
callback functions into one common place, preparing for the use of
ohash for some additional purposes.  No functional change.

Revision 1.59 / (download) - annotate - [select for diffs], Sun Oct 11 21:12:55 2015 UTC (4 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.58: +17 -18 lines
Diff to previous 1.58 (colored)

Finally use __progname, err(3) and warn(3).
That's more readable and less error-prone than fumbling around
with argv[0], fprintf(3), strerror(3), perror(3), and exit(3).

It's a bad idea to boycott good interfaces merely because standards
committees ignore them.  Instead, let's provide compatibility modules
for archaic systems (like commercial Solaris) that still don't have
them.  The compat module has an UCB Copyright (c) 1993...

Revision 1.58 / (download) - annotate - [select for diffs], Tue Oct 6 18:32:19 2015 UTC (4 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.57: +24 -24 lines
Diff to previous 1.57 (colored)

modernize style: "return" is not a function

Revision 1.57 / (download) - annotate - [select for diffs], Wed Apr 1 12:48:33 2015 UTC (5 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.56: +34 -36 lines
Diff to previous 1.56 (colored)

Let man(1) and apropos(1) work even when the current directory
is unusable: Only change back to the current directory when the
directory was changed before and the next path is relative.
This is now more similar to what makewhatis(8) does.
Issue reported by espie@.

Revision 1.56 / (download) - annotate - [select for diffs], Fri Mar 27 17:37:25 2015 UTC (5 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.55: +4 -4 lines
Diff to previous 1.55 (colored)

Parse the new man.conf(5) "output" directive.
The next step will be to actually use the parsed data.

Revision 1.55 / (download) - annotate - [select for diffs], Wed Mar 11 13:11:22 2015 UTC (5 years, 5 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_3
Changes since 1.54: +4 -3 lines
Diff to previous 1.54 (colored)

When interpreting the -O argument as a macro name fails,
fall back to showing Nd rather than not showing anything.
Issue reported by jmc@.

Revision 1.54 / (download) - annotate - [select for diffs], Fri Feb 27 16:02:10 2015 UTC (5 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.53: +29 -4 lines
Diff to previous 1.53 (colored)

When man(1) and apropos(1) look for a file man1/foo.1 but it's unavailable,
fall back to glob(man1/foo.*), which is more like what old man(1) did.
Do this both for file names from the database and for fs_lookup().
This is relevant because some ports install files like man1/xset.1x.
Regression reported by patrick keshishian <pkeshish at gmail dot com>.

Revision 1.53 / (download) - annotate - [select for diffs], Tue Jan 20 18:21:18 2015 UTC (5 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.52: +23 -49 lines
Diff to previous 1.52 (colored)

Make the man(1) and apropos(1) options -s and -S much less expensive:
Do not append an SQL clause looking into the large "keys" table.
Instead, filter the result of the SQL query in buildnames() where
equivalent data from the much smaller "mlinks" table is already
available for free.

This is relevant because man(1) uses the equivalent of "-S ${MACHINE}"
by default since main.c rev. 1.216, to make sure that manuals for
the current architecture are shown.  With many ports installed, this
patch can speed up man(1) by a factor of more than a hundred.

Slowness reported by Theo Buehler <theo at math dot ethz dot ch>, thanks!

Revision 1.52 / (download) - annotate - [select for diffs], Sat Dec 6 01:23:24 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_2
Changes since 1.51: +4 -2 lines
Diff to previous 1.51 (colored)

When opening mandoc.db fails, tell the user in which directory.
Improving an unhelpful error message reported by millert@.

Revision 1.51 / (download) - annotate - [select for diffs], Thu Nov 27 01:58:21 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.50: +8 -13 lines
Diff to previous 1.50 (colored)

Make makewhatis(8) understand .so links to .gz pages.
Drop the FORM_GZ annotation in the mpages table; it is conceptually wrong
because it ought to be in the mlinks table: An uncompressed .so link file
can point to a compressed manual page file and vice versa.
Besides, it is no longer needed because mparse_open() handles it all.
Sprinkle some KNF while here.

Revision 1.50 / (download) - annotate - [select for diffs], Tue Nov 18 01:15:21 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.49: +12 -7 lines
Diff to previous 1.49 (colored)

In man(1) mode, prefer file name matches over .Dt name matches over
first .Nm entries over other NAME .Nm entries over SYNOPSIS .Nm entries.
For example, this makes sure "man ypbind" does not return yp(8).
Re-run "makewhatis" to profit from this change.

Revision 1.49 / (download) - annotate - [select for diffs], Tue Nov 11 19:04:55 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.48: +9 -1 lines
Diff to previous 1.48 (colored)

In man(1) mode without -a, stop searching after the first manual tree
that contained at least one match in order to not prefer mdoc(1) from
ports over mdoc(7).  As a bonus, this results in a speedup.

Revision 1.48 / (download) - annotate - [select for diffs], Wed Sep 3 18:09:14 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.47: +10 -5 lines
Diff to previous 1.47 (colored)

If a manual page is installed gzip(1)ed, let makewhatis(8) take
note in mandoc.db(5), such that man(1) -w and apropos(1) -w can
report the correct filename.
This is a prerequisite for letting apropos -a and man support
gzip'ed manuals in the future, which doesn't work yet.

Revision 1.47 / (download) - annotate - [select for diffs], Mon Sep 1 22:45:53 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.46: +2 -1 lines
Diff to previous 1.46 (colored)

In man(1) mode, change to the right directory before starting the parser,
just like traditional man(1) does, such that .so links have a chance to
work.  After this point, we don't need the current directory for anything
else before exit, so we don't need to worry about getting back and we can
safely ignore failure.

Revision 1.46 / (download) - annotate - [select for diffs], Thu Aug 21 20:29:07 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.45: +2 -1 lines
Diff to previous 1.45 (colored)

Bugfix: make whatis(1) case-insensitive again.
The traditional whatis(1) was case-insensitve and it's still documented
that way, but that apparently got broken with or after the switch.

Revision 1.45 / (download) - annotate - [select for diffs], Sun Aug 17 03:24:47 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.44: +29 -28 lines
Diff to previous 1.44 (colored)

Fully integrate apropos(1) into mandoc(1).
Switch the argmode on the progname, including man(1).
Provide -f and -k options to switch the argmode.
Store the argmode inside struct search, generalizing the flags.
Derive the deftype from the argmode when needed instead of storing it.
Store the outkey inside struct search instead of passing it alone.
While here, get rid of the trailing blanks in Makefile.depend.

Revision 1.44 / (download) - annotate - [select for diffs], Sat Aug 16 19:00:01 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.43: +2 -2 lines
Diff to previous 1.43 (colored)

Improve build system and autodetection.
* Make ./configure standalone, that's what people expect.
* Let people write a ./configure.local from scratch, not edit existing files.
* Autodetect wchar, sqlite3, and manpath and act accordingly.
* Autodetect the need for -L/usr/local/lib and -lutil.
* Get rid of config.h.p{re,ost}, let ./configure only write what's needed.
* Let ./configure write a Makefile.local snippet, that's quite flexible.

Revision 1.43 / (download) - annotate - [select for diffs], Sun Aug 10 23:54:41 2014 UTC (6 years ago) by schwarze
Branch: MAIN
Changes since 1.42: +3 -3 lines
Diff to previous 1.42 (colored)

Get rid of HAVE_CONFIG_H, it is always defined; idea from libnbcompat.
Include <sys/types.h> where needed, it does not belong in config.h.
Remove <stdio.h> from config.h; if it is missing somewhere, it should
be added, but i cannot find a *.c file where it is missing.

Revision 1.42 / (download) - annotate - [select for diffs], Sat Aug 9 14:24:53 2014 UTC (6 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_1
Changes since 1.41: +3 -2 lines
Diff to previous 1.41 (colored)

mmap(2) requires MAP_PRIVATE ^ MAP_SHARED for flags;
found by kristaps@ on Mac OS X

Revision 1.41 / (download) - annotate - [select for diffs], Tue Aug 5 14:43:10 2014 UTC (6 years ago) by schwarze
Branch: MAIN
Changes since 1.40: +6 -6 lines
Diff to previous 1.40 (colored)

Absurdly, the return value of sqlite3_column_text()
is "const unsigned char *", which causes warnings with GCC on Linux.
Explicitly cast to "const char *" to avoid this.
Issue noticed by kristaps@.

Revision 1.40 / (download) - annotate - [select for diffs], Tue Aug 5 12:34:08 2014 UTC (6 years ago) by schwarze
Branch: MAIN
Changes since 1.39: +4 -1 lines
Diff to previous 1.39 (colored)

If an old SQLite version doesn't provide SQLITE_DETERMINISTIC,
simply ignore it, as using it is merely an optimization.
Issue noticed by kristaps@.

Revision 1.39 / (download) - annotate - [select for diffs], Thu Jul 24 20:30:45 2014 UTC (6 years ago) by schwarze
Branch: MAIN
Changes since 1.38: +21 -1 lines
Diff to previous 1.38 (colored)

Sort result pages first by section number, then by name.
By moving the sort from cgi.c to mansearch.c, we get two advantages:
Easier access to the data needed for sorting, in particular the section
number, and the apropos(1) command line utility profits as well.

Feature requested by deraadt@.

Revision 1.38 / (download) - annotate - [select for diffs], Sat Jul 12 14:00:25 2014 UTC (6 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.37: +51 -23 lines
Diff to previous 1.37 (colored)

Fix whatis(1) to correctly match words instead of any substrings.
While here, also provide an internal mode (MANSEARCH_MAN) to match
complete names, to be used by man.cgi(8).

Revision 1.37 / (download) - annotate - [select for diffs], Fri Jun 20 02:24:40 2014 UTC (6 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.36: +8 -10 lines
Diff to previous 1.36 (colored)

Merge from OpenBSD - Marc Espie improved the ohash interface:
* rename the halloc callback to calloc, provide overflow protection
* rename the hfree callback to free, drop the useless size argument
* prevent integer overflows in ohash_resize

Revision 1.36 / (download) - annotate - [select for diffs], Wed Apr 23 21:06:41 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.35: +4 -4 lines
Diff to previous 1.35 (colored)

Audit malloc(3)/calloc(3)/realloc(3) usage.
* Change eight reallocs to reallocarray to be safe from overflows.
* Change one malloc to reallocarray to be safe from overflows.
* Change one calloc to reallocarray, no zeroing needed.
* Change the order of arguments of three callocs (aesthetical).

Revision 1.35 / (download) - annotate - [select for diffs], Wed Apr 23 16:34:50 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.34: +7 -5 lines
Diff to previous 1.34 (colored)

improve SQL style: avoid "SELECT *", be explicit in what columns we want;
suggested by espie@.

Revision 1.34 / (download) - annotate - [select for diffs], Sun Apr 20 16:46:05 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.33: +13 -13 lines
Diff to previous 1.33 (colored)

KNF: case (FOO):  ->  case FOO:, remove /* LINTED */ and /* ARGSUSED */,
remove trailing whitespace and blanks before tabs, improve some indenting;
no functional change

Revision 1.33 / (download) - annotate - [select for diffs], Thu Apr 17 19:20:01 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.32: +8 -8 lines
Diff to previous 1.32 (colored)

Garbage collect one pair of needless parentheses in SQL code generation;
note this doesn't affect performance, SQLite generates the same byte code.
While here, make the calls to exprspec() easier to understand.

Revision 1.32 / (download) - annotate - [select for diffs], Wed Apr 16 21:36:18 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.31: +18 -18 lines
Diff to previous 1.31 (colored)

Rename the mpages.id column to mpages.pageid.  There is no good reason
to call this kid by a different name here than in all other tables.

Revision 1.31 / (download) - annotate - [select for diffs], Wed Apr 16 00:33:47 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.30: +5 -3 lines
Diff to previous 1.30 (colored)

Pass the function flags SQLITE_UTF8 (because SQLITE_ANY is deprecated)
and SQLITE_DETERMINISTIC when creating deterministic functions;
best practice measure suggested by espie@ and jeremy@;
as expected by jeremy@, no measurable effect on performance.

Revision 1.30 / (download) - annotate - [select for diffs], Tue Apr 15 23:51:39 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.29: +7 -5 lines
Diff to previous 1.29 (colored)

Oops, sorry, revert previous and commit the correct patch:
At the end of mansearch(), fchdir() back to where we started from.

Revision 1.29 / (download) - annotate - [select for diffs], Tue Apr 15 23:48:51 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.28: +2 -1 lines
Diff to previous 1.28 (colored)

At the end of mansearch(), fchdir() back to where we started from;
this is cleaner and helps to not scatter gmon.out files all over
the place when profiling.

Revision 1.28 / (download) - annotate - [select for diffs], Fri Apr 11 15:46:52 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.27: +49 -1 lines
Diff to previous 1.27 (colored)

Further apropos(1) speed optimization was trickier than anticipated.
Contrary to what i initially thought, almost all time is now spent
inside sqlite3(3) routines, and i found no easy way calling less of them.
However, sqlite(3) spends substantial time in malloc(3), and even more
(twice that) in its immediate malloc wrapper, sqlite3MemMalloc(),
keeping track of all individual malloc chunk sizes.  Typically about
90% of the malloced memory is used for purposes of the pagecache.

By providing an mmap(3) MAP_ANON SQLITE_CONFIG_PAGECACHE, execution
time decreases by 20-25% for simple (Nd and/or Nm) queries, 10-20% for
non-NAME queries, and even apropos(1) resident memory size as reported
by top(1) decreases by 20% for simple and by 60% for non-NAME queries.
The new function, mansearch_setup(), spends no measurable time.

The pagesize chosen is optimal:
* Substantially smaller pages yield no gain at all.
* Larger pages provide no additional benefit and just waste memory.

The chosen number of pages in the cache is a compromise:
* For simple queries, a handful of pages would suffice to get the full
speed effect, at an apropos(1) resident memory size of about 2.0 MB.
* For non-NAME queries, a large pagecache with 2k pages (2.5 MB) might
gain a few more percent in speed, but at the expense of doubling the
apropos(1) resident memory size for *all* queries.
* The chosen number of 256 pages (330 kB) allows nearly full speed gain
for all queries at the price of a 15% resident memory size increase.

Revision 1.27 / (download) - annotate - [select for diffs], Thu Apr 10 02:46:21 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.26: +27 -14 lines
Diff to previous 1.26 (colored)

Next speed optimization step for the new apropos(1).
Split manual names out of the common "keys" table into their
own "names" table.  This reduces standard apropos(1) search
times (i.e. searching for names and descriptions only) by
typically about 70% for the full /usr/share/man database.
(Yes, that multiplies with the previous optimization step,
so both together have reduced search times by a factor of
more than six.  I'm not done yet, expect more to come.)

Even with the minimal databases built with makewhatis(8) -Q,
this step still reduces search times by 15-20%.  For both cases,
database sizes and build times hardly change (+/-2%).

Revision 1.26 / (download) - annotate - [select for diffs], Wed Apr 9 21:50:08 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.25: +43 -16 lines
Diff to previous 1.25 (colored)

After careful gprof(1)ing of the new apropos(1), move the descriptions
back from the keys table to the mpages table:  I found a good way
to still use them in searches, without complication of the code.

On my notebook, this reduces typical apropos(1) search times by about 40%,
it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode
and by 2% in standard mode (less overhead storing pointers to mpages),
and it doesn't measurably change database build times (may even be
going down by a percent or so because less data is being copied
around in ohashes).

Revision 1.25 / (download) - annotate - [select for diffs], Fri Mar 28 19:17:12 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.24: +2 -1 lines
Diff to previous 1.24 (colored)

Properly initialize malloc(3)ed memory.
With this bug fix, partly unitialized memory could sometimes be
returned, sometimes causing crashes by bogus free(3)s in apropos(1).

Revision 1.24 / (download) - annotate - [select for diffs], Sun Mar 23 12:11:18 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.23: +12 -30 lines
Diff to previous 1.23 (colored)

avoid repetitive code for asprintf error handling

Revision 1.23 / (download) - annotate - [select for diffs], Sun Mar 23 11:25:26 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.22: +2 -1 lines
Diff to previous 1.22 (colored)

The files mandoc.c and mandoc.h contained both specialised low-level
functions used for multiple languages (mdoc, man, roff), for example
mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary
functions.  Split the auxiliaries out into their own file and header.
While here, do some #include cleanup.

Revision 1.22 / (download) - annotate - [select for diffs], Mon Mar 17 16:31:44 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.21: +55 -7 lines
Diff to previous 1.21 (colored)

in apropos(1) output, sort names and avoid multiple section numbers

Revision 1.21 / (download) - annotate - [select for diffs], Sun Jan 19 23:09:30 2014 UTC (6 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.20: +3 -1 lines
Diff to previous 1.20 (colored)

Always compare arch case-insensitively.

Revision 1.20 / (download) - annotate - [select for diffs], Sun Jan 19 01:18:34 2014 UTC (6 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.19: +29 -67 lines
Diff to previous 1.19 (colored)

Get rid of the local keys table, use the new mansearch_const.c.
No functional change.

Revision 1.19 / (download) - annotate - [select for diffs], Mon Jan 6 03:52:13 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.18: +19 -10 lines
Diff to previous 1.18 (colored)

Remove the redundant "file" column from the "mlinks" table.
The contents can easily be reconstructed from sec, arch, name, form.
Shrinks the database by another 3% in standard mode and 9% in -Q mode.

Revision 1.18 / (download) - annotate - [select for diffs], Mon Jan 6 03:02:46 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.17: +3 -7 lines
Diff to previous 1.17 (colored)

Drop Nd from the mpages table, it is still in the keys table.
This shrinks the database in standard mode by 3%, in -Q mode by 9%,
without loss of functionality.

Revision 1.17 / (download) - annotate - [select for diffs], Sun Jan 5 04:13:52 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.16: +30 -24 lines
Diff to previous 1.16 (colored)

Remove the obsolete file name column from the mpages table.
This column wasn't helpful because one manpage can have multiple MLINKS.
Use the file name column in the mlinks table, instead.

Revision 1.16 / (download) - annotate - [select for diffs], Sun Jan 5 03:25:51 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.15: +4 -4 lines
Diff to previous 1.15 (colored)

Remove the obsolete sec and arch columns from the mpages table.
They were confusing because a manpage can have MLINKS in different
sections and architectures.

Revision 1.15 / (download) - annotate - [select for diffs], Sun Jan 5 03:06:43 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.14: +52 -22 lines
Diff to previous 1.14 (colored)

Reimplement apropos -s NUM -S ARCH EXPR by internally converting it to
apropos \( EXPR \) -a 'sec~^NUM$' -a 'arch~^(ARCH|any)$'
in preparation for removal of sec and arch from the mpage table.

Almost no functional change except for the following bonus:
This also makes sure that for cross-section and cross-arch MLINKs,
all of the following work:
apropos -s 1 encrypt
apropos -s 8 encrypt
apropos -s 1 makekey
apropos -s 8 makekey

While here, print error messages about invalid regexps to stderr.

Revision 1.14 / (download) - annotate - [select for diffs], Sun Jan 5 00:29:54 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.13: +3 -1 lines
Diff to previous 1.13 (colored)

Put section and architecture info into the keys table,
in preparation for removing them from the mpages table,
aiming for cleaner and more uniform interfaces.
Database growth is below 4%, part of which will be reclaimed.

As a bonus, this allows searches like:
./obj/apropos An=kettenis -a arch=ppc
./obj/apropos An=kettenis -a sec~[^4]

Revision 1.13 / (download) - annotate - [select for diffs], Sat Jan 4 23:43:53 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.12: +92 -56 lines
Diff to previous 1.12 (colored)

New implementation of complex search criteria using \(, \), -a because
the old implementation got lost in the Berkeley to SQLite switch.
Note that this is not just feature creep, but required for upcoming
database format cleanup and simplification.

Revision 1.12 / (download) - annotate - [select for diffs], Tue Dec 31 03:41:14 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.11: +63 -5 lines
Diff to previous 1.11 (colored)

Experimental feature to let apropos(1) show different keys than .Nd.
This really takes us beyond what grep -R /usr/*/man/ can do
because now you can search for pages by *one* criterion and then
display the contents of *another* macro from those pages, like in
$ apropos -O Ox Fa~wchar
to get an impression how long wide character handling is available.

Revision 1.11 / (download) - annotate - [select for diffs], Tue Dec 31 02:42:29 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.10: +41 -31 lines
Diff to previous 1.10 (colored)

Split buildnames() out of mansearch(); the latter function is getting
too long and unwieldy, but will grow more code soon.  No functional change.

Revision 1.10 / (download) - annotate - [select for diffs], Fri Dec 27 18:51:25 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.9: +50 -9 lines
Diff to previous 1.9 (colored)

Change the mansearch() interface to use the mlinks table in the database
and return a list of names with sections, used by apropos(1) for display.
While here, improve uniformity of the interface by allocating the file
name dynamically, just like the names list and the description.

Revision 1.9 / (download) - annotate - [select for diffs], Fri Dec 27 01:16:54 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.8: +3 -3 lines
Diff to previous 1.8 (colored)

Add an additional mlinks table to the database, redundant for now,
both because it contains nothing but a subset of the data of the
existing mpages table and because the relationship of mpage and mlink
entries is still 1:1.  But all that will eventually change.

Revision 1.8 / (download) - annotate - [select for diffs], Sun Oct 20 00:03:05 2013 UTC (6 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.7: +72 -29 lines
Diff to previous 1.7 (colored)

Fix another regression introduced when switching from DB to SQLite:
The ~ operator has to do regular expression search, not globbing.

Revision 1.7 / (download) - annotate - [select for diffs], Sat Oct 19 20:43:13 2013 UTC (6 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.6: +34 -6 lines
Diff to previous 1.6 (colored)

Fix a regression introduced when switching from DB to SQLite:
The = operator has to do substring search, not word search.

Revision 1.6 / (download) - annotate - [select for diffs], Wed Jun 5 02:00:26 2013 UTC (7 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.5: +7 -8 lines
Diff to previous 1.5 (colored)

Some places used PATH_MAX from <limits.h>, some MAXPATHLEN from <sys/param.h>.
Consistently use the PATH_MAX since it is specified by POSIX,
while MAXPATHLEN is not.
In preparation for using this at a few more places.

Revision 1.5 / (download) - annotate - [select for diffs], Sat Jun 9 14:11:16 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN
Changes since 1.4: +25 -17 lines
Diff to previous 1.4 (colored)

Merge whatis.1 into apropos.1 (and remove), add whatis bits to apropos
(via mansearch), and merge mandocdb.h into mansearch.h (and remove).

Revision 1.4 / (download) - annotate - [select for diffs], Sat Jun 9 11:00:13 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN
Changes since 1.3: +5 -1 lines
Diff to previous 1.3 (colored)

Add a compatibility interface for ohash.
This include's espie@'s wholesale src/lib/libc/ohash directory from OpenBSD
into compat_ohash.c (with a single copyright/license notice at the top)
and src/include/ohash.h as compat_ohash.h.
The ohash_int.h part of compat_ohash.c has been changed only in that ohash.h
points to compat_ohash.h.
Added HAVE_OHASH test (test-ohash.c) to Makefile.
In mandocdb.c and mansearch.c, check HAVE_OHASH test for inclusion.

Revision 1.3 / (download) - annotate - [select for diffs], Fri Jun 8 15:06:28 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN
Changes since 1.2: +7 -7 lines
Diff to previous 1.2 (colored)

Turn off sqlite3 synchronous mode when creating a new database.
This makes it run about 5x faster.
While here, wrap some sqlite3 statements in #defines to extract errors.
(Really, the warning/error/etc. macros should be functionified.)

Revision 1.2 / (download) - annotate - [select for diffs], Fri Jun 8 14:14:30 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN
Changes since 1.1: +35 -21 lines
Diff to previous 1.1 (colored)

Allow mansearch to fail if invariancy (cwd) is violated.  Also be more
verbose if sqlite3 errors occur.

Revision 1.1 / (download) - annotate - [select for diffs], Fri Jun 8 10:36:23 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN

Add a new mansearch.h interface, which replaces apropos_db.c
This is a much more minimal interface that stuffs all operations into
a single function.
It uses sqlite3 and ohash.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb