CVS log for mandoc/mansearch.h

[BACK] Up to [] / mandoc

Request diff between arbitrary revisions

Default branch: MAIN

Revision 1.30 / (download) - annotate - [select for diffs], Tue Apr 30 18:51:57 2019 UTC (15 months, 1 week ago) by schwarze
Branch: MAIN
Changes since 1.29: +2 -1 lines
Diff to previous 1.29 (colored)

In man(1) mode, i.e. when asking for a single manual page by name,
prefer file name matches over .Dt/.TH matches over first NAME matches
over later NAME matches, but do not change the ordering for apropos(1)
nor for man -a.

This reverts main.c rev. 1.310 and mansearch.h rev. 1.29
and includes a partial revert of mansearch.c rev. 1.79.

Regression reported by Lorenzo Beretta <loreb at github>
as part of .

Revision 1.29 / (download) - annotate - [select for diffs], Thu Nov 22 12:01:46 2018 UTC (20 months, 2 weeks ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_5
Changes since 1.28: +1 -2 lines
Diff to previous 1.28 (colored)

In apropos(1) output, stop sorting .Nm search results by name
priorities (bits).  The obscure feature wasn't documented and merely
confused people - for example Edward Tomasz Napierala <trasz at
FreeBSD>, see

Smaller patch provided by Yuri Pankov <yuripv at FreeBSD>, but i'm
also retiring the now unused "bits" member from struct manpage.
Simplification is good.

Revision 1.28 / (download) - annotate - [select for diffs], Mon Apr 17 20:05:08 2017 UTC (3 years, 3 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_4, VERSION_1_14_3, VERSION_1_14_2
Changes since 1.27: +5 -2 lines
Diff to previous 1.27 (colored)

Restore -kO Nd, -kO Nm, -kO sec, and -kO arch to working order.
They got broken in the SQLite removal.
As opposed to the rest of -kO, they are no longer very useful,
but they are certainly not supposed to fail assertions.
Issue reported by Gonzalo Tornaria <tornaria at cmat dot edu dot uy>.

Revision 1.27 / (download) - annotate - [select for diffs], Mon Aug 1 12:31:00 2016 UTC (4 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_1
Changes since 1.26: +3 -3 lines
Diff to previous 1.26 (colored)

Adjust MANDOC_DB and bump MANDOCDB_VERSION now that
this was put into production in OpenBSD-current.

Revision 1.26 / (download) - annotate - [select for diffs], Wed Jul 20 00:23:14 2016 UTC (4 years ago) by schwarze
Branch: MAIN
Changes since 1.25: +2 -2 lines
Diff to previous 1.25 (colored)

Linux compatibility: Work around the lack of EFTYPE and protect <err.h>.

Revision 1.25 / (download) - annotate - [select for diffs], Tue Jul 19 21:31:55 2016 UTC (4 years ago) by schwarze
Branch: MAIN
Changes since 1.24: +14 -7 lines
Diff to previous 1.24 (colored)

Remove the dependency on SQLite without loss of functionality.
Stop supporting systems that don't have mmap(3).
Drop the obsolete names_check() now that we deleted MLINKS.

Revision 1.24 / (download) - annotate - [select for diffs], Sat Nov 7 14:01:16 2015 UTC (4 years, 9 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_4, VERSION_1_13
Changes since 1.23: +1 -4 lines
Diff to previous 1.23 (colored)

In private header files, __BEGIN_DECLS and __END_DECLS are pointless.
Because these work slightly differently on different systems,
they are becoming a maintenance burden in the portable version,
so delete them.

Besides, one of the chief design goals of the mandoc toolbox is to
make sure that nothing related to documentation requires C++.
Consequently, linking mandoc against any kind of C++ program would
defeat the purpose and is not supported.
I don't understand why kristaps@ added them in the first place.

Revision 1.23 / (download) - annotate - [select for diffs], Mon Dec 1 08:05:52 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_3, VERSION_1_13_2
Changes since 1.22: +3 -1 lines
Diff to previous 1.22 (colored)

header cleanup:
* add missing forward declarations
* remove needless header inclusions
* some style unification

Revision 1.22 / (download) - annotate - [select for diffs], Mon Dec 1 04:05:32 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.21: +1 -5 lines
Diff to previous 1.21 (colored)

remove unneccessary inclusion protection; patch from deraadt@

Revision 1.21 / (download) - annotate - [select for diffs], Thu Nov 27 01:58:21 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.20: +1 -2 lines
Diff to previous 1.20 (colored)

Make makewhatis(8) understand .so links to .gz pages.
Drop the FORM_GZ annotation in the mpages table; it is conceptually wrong
because it ought to be in the mlinks table: An uncompressed .so link file
can point to a compressed manual page file and vice versa.
Besides, it is no longer needed because mparse_open() handles it all.
Sprinkle some KNF while here.

Revision 1.20 / (download) - annotate - [select for diffs], Tue Nov 18 01:15:21 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.19: +6 -5 lines
Diff to previous 1.19 (colored)

In man(1) mode, prefer file name matches over .Dt name matches over
first .Nm entries over other NAME .Nm entries over SYNOPSIS .Nm entries.
For example, this makes sure "man ypbind" does not return yp(8).
Re-run "makewhatis" to profit from this change.

Revision 1.19 / (download) - annotate - [select for diffs], Tue Nov 11 19:04:55 2014 UTC (5 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.18: +2 -1 lines
Diff to previous 1.18 (colored)

In man(1) mode without -a, stop searching after the first manual tree
that contained at least one match in order to not prefer mdoc(1) from
ports over mdoc(7).  As a bonus, this results in a speedup.

Revision 1.18 / (download) - annotate - [select for diffs], Wed Sep 3 18:09:14 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.17: +6 -1 lines
Diff to previous 1.17 (colored)

If a manual page is installed gzip(1)ed, let makewhatis(8) take
note in mandoc.db(5), such that man(1) -w and apropos(1) -w can
report the correct filename.
This is a prerequisite for letting apropos -a and man support
gzip'ed manuals in the future, which doesn't work yet.

Revision 1.17 / (download) - annotate - [select for diffs], Mon Sep 1 22:45:53 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.16: +2 -1 lines
Diff to previous 1.16 (colored)

In man(1) mode, change to the right directory before starting the parser,
just like traditional man(1) does, such that .so links have a chance to
work.  After this point, we don't need the current directory for anything
else before exit, so we don't need to worry about getting back and we can
safely ignore failure.

Revision 1.16 / (download) - annotate - [select for diffs], Sun Aug 17 03:24:47 2014 UTC (5 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.15: +13 -8 lines
Diff to previous 1.15 (colored)

Fully integrate apropos(1) into mandoc(1).
Switch the argmode on the progname, including man(1).
Provide -f and -k options to switch the argmode.
Store the argmode inside struct search, generalizing the flags.
Derive the deftype from the argmode when needed instead of storing it.
Store the outkey inside struct search instead of passing it alone.
While here, get rid of the trailing blanks in Makefile.depend.

Revision 1.15 / (download) - annotate - [select for diffs], Thu Jul 24 20:30:45 2014 UTC (6 years ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_1
Changes since 1.14: +2 -1 lines
Diff to previous 1.14 (colored)

Sort result pages first by section number, then by name.
By moving the sort from cgi.c to mansearch.c, we get two advantages:
Easier access to the data needed for sorting, in particular the section
number, and the apropos(1) command line utility profits as well.

Feature requested by deraadt@.

Revision 1.14 / (download) - annotate - [select for diffs], Sat Jul 12 14:00:25 2014 UTC (6 years ago) by schwarze
Branch: MAIN
Changes since 1.13: +3 -2 lines
Diff to previous 1.13 (colored)

Fix whatis(1) to correctly match words instead of any substrings.
While here, also provide an internal mode (MANSEARCH_MAN) to match
complete names, to be used by man.cgi(8).

Revision 1.13 / (download) - annotate - [select for diffs], Fri Apr 11 15:46:52 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.12: +2 -1 lines
Diff to previous 1.12 (colored)

Further apropos(1) speed optimization was trickier than anticipated.
Contrary to what i initially thought, almost all time is now spent
inside sqlite3(3) routines, and i found no easy way calling less of them.
However, sqlite(3) spends substantial time in malloc(3), and even more
(twice that) in its immediate malloc wrapper, sqlite3MemMalloc(),
keeping track of all individual malloc chunk sizes.  Typically about
90% of the malloced memory is used for purposes of the pagecache.

By providing an mmap(3) MAP_ANON SQLITE_CONFIG_PAGECACHE, execution
time decreases by 20-25% for simple (Nd and/or Nm) queries, 10-20% for
non-NAME queries, and even apropos(1) resident memory size as reported
by top(1) decreases by 20% for simple and by 60% for non-NAME queries.
The new function, mansearch_setup(), spends no measurable time.

The pagesize chosen is optimal:
* Substantially smaller pages yield no gain at all.
* Larger pages provide no additional benefit and just waste memory.

The chosen number of pages in the cache is a compromise:
* For simple queries, a handful of pages would suffice to get the full
speed effect, at an apropos(1) resident memory size of about 2.0 MB.
* For non-NAME queries, a large pagecache with 2k pages (2.5 MB) might
gain a few more percent in speed, but at the expense of doubling the
apropos(1) resident memory size for *all* queries.
* The chosen number of 256 pages (330 kB) allows nearly full speed gain
for all queries at the price of a 15% resident memory size increase.

Revision 1.12 / (download) - annotate - [select for diffs], Thu Apr 10 02:46:21 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.11: +48 -42 lines
Diff to previous 1.11 (colored)

Next speed optimization step for the new apropos(1).
Split manual names out of the common "keys" table into their
own "names" table.  This reduces standard apropos(1) search
times (i.e. searching for names and descriptions only) by
typically about 70% for the full /usr/share/man database.
(Yes, that multiplies with the previous optimization step,
so both together have reduced search times by a factor of
more than six.  I'm not done yet, expect more to come.)

Even with the minimal databases built with makewhatis(8) -Q,
this step still reduces search times by 15-20%.  For both cases,
database sizes and build times hardly change (+/-2%).

Revision 1.11 / (download) - annotate - [select for diffs], Wed Apr 9 21:50:08 2014 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.10: +40 -40 lines
Diff to previous 1.10 (colored)

After careful gprof(1)ing of the new apropos(1), move the descriptions
back from the keys table to the mpages table:  I found a good way
to still use them in searches, without complication of the code.

On my notebook, this reduces typical apropos(1) search times by about 40%,
it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode
and by 2% in standard mode (less overhead storing pointers to mpages),
and it doesn't measurably change database build times (may even be
going down by a percent or so because less data is being copied
around in ohashes).

Revision 1.10 / (download) - annotate - [select for diffs], Fri Apr 4 15:55:19 2014 UTC (6 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.9: +43 -42 lines
Diff to previous 1.9 (colored)

Remember which names are in the NAME section.
This helps to find missing MLINKS.
Database build times do not change and database growth is minimal
(1.2% with -Q, 0.7% without -Q in /usr/share/man),
so making this optional would be pointless.

Revision 1.9 / (download) - annotate - [select for diffs], Sat Jan 18 08:23:55 2014 UTC (6 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.8: +41 -41 lines
Diff to previous 1.8 (colored)

Sort the macro keys by their real-world frequency to reduce the average
mask size.  No functional change.

This shrinks the standard /usr/share/man database by 7%, now at 10.3x
the size of whatis.db, and with -Q even by 11%, now at 3.0x of whatis.db.
Now i'm out of ideas to easily shrink the size of the database.

Revision 1.8 / (download) - annotate - [select for diffs], Mon Jan 6 03:02:46 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.7: +1 -2 lines
Diff to previous 1.7 (colored)

Drop Nd from the mpages table, it is still in the keys table.
This shrinks the database in standard mode by 3%, in -Q mode by 9%,
without loss of functionality.

Revision 1.7 / (download) - annotate - [select for diffs], Sun Jan 5 00:29:54 2014 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.6: +3 -1 lines
Diff to previous 1.6 (colored)

Put section and architecture info into the keys table,
in preparation for removing them from the mpages table,
aiming for cleaner and more uniform interfaces.
Database growth is below 4%, part of which will be reclaimed.

As a bonus, this allows searches like:
./obj/apropos An=kettenis -a arch=ppc
./obj/apropos An=kettenis -a sec~[^4]

Revision 1.6 / (download) - annotate - [select for diffs], Tue Dec 31 03:41:14 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.5: +4 -1 lines
Diff to previous 1.5 (colored)

Experimental feature to let apropos(1) show different keys than .Nd.
This really takes us beyond what grep -R /usr/*/man/ can do
because now you can search for pages by *one* criterion and then
display the contents of *another* macro from those pages, like in
$ apropos -O Ox Fa~wchar
to get an impression how long wide character handling is available.

Revision 1.5 / (download) - annotate - [select for diffs], Fri Dec 27 18:51:25 2013 UTC (6 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.4: +3 -2 lines
Diff to previous 1.4 (colored)

Change the mansearch() interface to use the mlinks table in the database
and return a list of names with sections, used by apropos(1) for display.
While here, improve uniformity of the interface by allocating the file
name dynamically, just like the names list and the description.

Revision 1.4 / (download) - annotate - [select for diffs], Thu Jun 6 14:27:02 2013 UTC (7 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.3: +2 -2 lines
Diff to previous 1.3 (colored)

No point in inventing yet another file name for the database.
We are not going back to Berkeley DB and don't care about
overwriting any obsolete files that may be lying around.
So, go back from "mandocdb.db" to "mandoc.db".

Revision 1.3 / (download) - annotate - [select for diffs], Wed Jun 5 02:00:26 2013 UTC (7 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.2: +2 -2 lines
Diff to previous 1.2 (colored)

Some places used PATH_MAX from <limits.h>, some MAXPATHLEN from <sys/param.h>.
Consistently use the PATH_MAX since it is specified by POSIX,
while MAXPATHLEN is not.
In preparation for using this at a few more places.

Revision 1.2 / (download) - annotate - [select for diffs], Sat Jun 9 14:11:16 2012 UTC (8 years, 1 month ago) by kristaps
Branch: MAIN
Changes since 1.1: +53 -5 lines
Diff to previous 1.1 (colored)

Merge whatis.1 into apropos.1 (and remove), add whatis bits to apropos
(via mansearch), and merge mandocdb.h into mansearch.h (and remove).

Revision 1.1 / (download) - annotate - [select for diffs], Fri Jun 8 10:36:23 2012 UTC (8 years, 2 months ago) by kristaps
Branch: MAIN

Add a new mansearch.h interface, which replaces apropos_db.c
This is a much more minimal interface that stuffs all operations into
a single function.
It uses sqlite3 and ohash.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.