CVS log for mandoc/preconv.c

[BACK] Up to [cvsweb.bsd.lv] / mandoc

Request diff between arbitrary revisions


Default branch: MAIN
Current tag: MAIN


Revision 1.17 / (download) - annotate - [select for diffs], Thu Dec 13 11:55:47 2018 UTC (5 years, 3 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_6, VERSION_1_14_5, HEAD
Changes since 1.16: +4 -1 lines
Diff to previous 1.16 (unified)

Cleanup, no functional change:
Split the top level parser interface out of the utility header
mandoc.h, into a new header mandoc_parse.h, for use in the main
program and in the main parser only.
Move enum mandoc_os into roff.h because struct roff_man is the
place where it is stored.
This allows removal of mandoc.h from seven files in low-level
parsers and in formatters.

Revision 1.16 / (download) - annotate - [selected], Sat Feb 18 13:43:52 2017 UTC (7 years, 1 month ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_14_4, VERSION_1_14_3, VERSION_1_14_2, VERSION_1_14_1, VERSION_1_13
Changes since 1.15: +7 -7 lines
Diff to previous 1.15 (unified)

preconv_encode() can take a const input buffer;
diff from <christos at NetBSD>

Revision 1.15 / (download) - annotate - [select for diffs], Tue Oct 6 18:32:19 2015 UTC (8 years, 5 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_4
Changes since 1.14: +10 -10 lines
Diff to previous 1.14 (unified) to selected 1.16 (unified)

modernize style: "return" is not a function

Revision 1.14 / (download) - annotate - [select for diffs], Fri Mar 6 09:24:59 2015 UTC (9 years ago) by kristaps
Branch: MAIN
CVS Tags: VERSION_1_13_3
Changes since 1.13: +2 -2 lines
Diff to previous 1.13 (unified) to selected 1.16 (unified)

Allow compilation on Mac OS X, which doesn't have MACHINE defined.
While there, specify some casts to satisfy the compiler warnings.

OK schwarze@

Revision 1.13 / (download) - annotate - [select for diffs], Fri Dec 19 04:58:35 2014 UTC (9 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.12: +58 -75 lines
Diff to previous 1.12 (unified) to selected 1.16 (unified)

Rewrite the low-level UTF-8 parser from scratch.
It accepted invalid byte sequences like 0xc080-c1bf, 0xe08080-e09fbf,
0xeda080-edbfbf, and 0xf0808080-f08fbfbf, produced valid roff Unicode
escape sequences from them, and the algorithm contained strong
defenses against any attempt to fix it.

This cures an assertion failure in the terminal formatter caused
by sneaking in ASCII 0x08 (backspace) by "encoding" it as an (invalid)
multibyte UTF-8 sequence, found by jsg@ with afl.

As a bonus, the new algorithm also reduces the code in the function
by about 20%.

Revision 1.12 / (download) - annotate - [select for diffs], Fri Nov 14 04:24:04 2014 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_2
Changes since 1.11: +5 -25 lines
Diff to previous 1.11 (unified) to selected 1.16 (unified)

Remove needless and harmful byte swapping on big endian architectures.
Problem found and patch provided by Martin Natano at bitrig, thanks!
Tested on macppc by natano@ and on i386, amd64, and sparc64 myself.

While here, sync with OpenBSD by removing some trailing whitespace.

Revision 1.11 / (download) - annotate - [select for diffs], Sat Nov 1 04:08:43 2014 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.10: +12 -11 lines
Diff to previous 1.10 (unified) to selected 1.16 (unified)

Refactor, no functional change: Remove the parse point from struct buf.
Some functions need multiple parse points, some none at all,
and it varies whether any of them need to be passed around.
So better pass them as a separate argument, and only when needed.

Revision 1.10 / (download) - annotate - [select for diffs], Sun Oct 26 18:22:51 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.9: +4 -5 lines
Diff to previous 1.9 (unified) to selected 1.16 (unified)

KNF: indentation and sort variables by size; no functional change

Revision 1.9 / (download) - annotate - [select for diffs], Sat Oct 25 01:03:52 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.8: +59 -368 lines
Diff to previous 1.8 (unified) to selected 1.16 (unified)

integrate preconv(1) into mandoc(1);
enhances functionality and reduces code and docs by more than 300 lines

Revision 1.8 / (download) - annotate - [select for diffs], Sat Aug 16 19:00:01 2014 UTC (9 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.7: +4 -4 lines
Diff to previous 1.7 (unified) to selected 1.16 (unified)

Improve build system and autodetection.
* Make ./configure standalone, that's what people expect.
* Let people write a ./configure.local from scratch, not edit existing files.
* Autodetect wchar, sqlite3, and manpath and act accordingly.
* Autodetect the need for -L/usr/local/lib and -lutil.
* Get rid of config.h.p{re,ost}, let ./configure only write what's needed.
* Let ./configure write a Makefile.local snippet, that's quite flexible.

Revision 1.7 / (download) - annotate - [select for diffs], Sun Aug 10 23:54:41 2014 UTC (9 years, 7 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_12_4, VERSION_1_12
Changes since 1.6: +2 -3 lines
Diff to previous 1.6 (unified) to selected 1.16 (unified)

Get rid of HAVE_CONFIG_H, it is always defined; idea from libnbcompat.
Include <sys/types.h> where needed, it does not belong in config.h.
Remove <stdio.h> from config.h; if it is missing somewhere, it should
be added, but i cannot find a *.c file where it is missing.

Revision 1.6 / (download) - annotate - [select for diffs], Sun Jun 2 03:52:21 2013 UTC (10 years, 9 months ago) by schwarze
Branch: MAIN
CVS Tags: VERSION_1_13_1, VERSION_1_12_3, VERSION_1_12_2
Changes since 1.5: +3 -8 lines
Diff to previous 1.5 (unified) to selected 1.16 (unified)

Sync to OpenBSD, no functional change:
* Add the missing mparse_parse_buffer prototype.
* Drop the useless MAP_FILE constant: It's not specified in POSIX,
  so it's not required, it's the default anyway, and it's 0 anyway.

Revision 1.5 / (download) - annotate - [select for diffs], Sun Jul 24 18:15:14 2011 UTC (12 years, 8 months ago) by kristaps
Branch: MAIN
CVS Tags: VERSION_1_12_1, VERSION_1_12_0, VERSION_1_11_7, VERSION_1_11_6, VERSION_1_11_5
Changes since 1.4: +8 -2 lines
Diff to previous 1.4 (unified) to selected 1.16 (unified)

Scary-looking but otherwise harmless changes allow me to build for Windows.
That is to say, with mingw32.  This amounts to the following:

 (1) break compat.c into compat_strlcpy.c and compat_strlcat.c
 (2) add compat_getsubopt.c (from OpenBSD) and test-getsubopt.c
 (3) add test-strptime.c for HAVE_STRPTIME
 (4) add ifdef bits here and there, where necessary
 (5) remove some harmless unportable stuff (u_char, localtime_r)

I've added the appropriate mdocml.zip target to the Makefile, too.

Revision 1.4 / (download) - annotate - [select for diffs], Thu May 26 21:13:07 2011 UTC (12 years, 10 months ago) by kristaps
Branch: MAIN
CVS Tags: VERSION_1_11_4, VERSION_1_11_3
Changes since 1.3: +6 -6 lines
Diff to previous 1.3 (unified) to selected 1.16 (unified)

Some small lint checks in preconv.  Also add it to the default lint rule.

Revision 1.3 / (download) - annotate - [select for diffs], Thu May 26 14:43:07 2011 UTC (12 years, 10 months ago) by kristaps
Branch: MAIN
Changes since 1.2: +105 -3 lines
Diff to previous 1.2 (unified) to selected 1.16 (unified)

preconv is now on encoding-recognition parity with groff.  This last
commit adds parsing of "File Variables" in the first two lines in order
to grok the encoding.  This completes groff's recognition sequence (-e,
BOM, File variables, -D, default).  I've also cleaned up the manual to
indicate this and for some general readability.

preconv is now compiled by default in the Makefile.

Revision 1.2 / (download) - annotate - [select for diffs], Thu May 26 12:01:14 2011 UTC (12 years, 10 months ago) by kristaps
Branch: MAIN
Changes since 1.1: +117 -13 lines
Diff to previous 1.1 (unified) to selected 1.16 (unified)

Significantly improve preconv.  Allow it to recode UTF-8 characters into
the \[uNNNN] strings (taking into account big-endian archs).  Also allow
it to determine from the BOM whether it's a UTF-8 file.  Also add the
initial manual.  This has been tested over a random selection of UTF-8
documents, as

 % preconv -e utf-8 foo.1 | ./mandoc -Tlocale

where -Tlocale is allowed (-DUSE_WCHAR).

Note that we're still missing the "type" indicator that preconv accepts.

Revision 1.1 / (download) - annotate - [select for diffs], Thu May 26 00:30:11 2011 UTC (12 years, 10 months ago) by kristaps
Branch: MAIN
Diff to selected 1.16 (unified)

It's annoying that we don't have preconv, so throw together a quick
version and let it grow in-tree.  Right now, this only supports the
Latin-1 and US-ASCII encoding.  I'll do UTF-8 next.  It's
call-compatible with GNU's preconv although I don't do fancy stuff like
BOM or header check.  This will come.  I used read.c's file-grokking
code.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb