.\" $Id: regress.pl.1,v 1.1 2017/02/08 03:02:13 schwarze Exp $ .\" .\" Copyright (c) 2017 Ingo Schwarze .\" .\" Permission to use, copy, modify, and distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" .Dd $Mdocdate: February 8 2017 $ .Dt REGRESS.PL 1 .Os .Sh NAME .Nm regress.pl .Nd portable steering script for mandoc regression tests .Sh SYNOPSIS .Nm ./regress.pl .Oo .Ar directory Ns Op Pf : Ar test .Op Ar modifier ... .Oc .Sh DESCRIPTION The .Nm steering script allows running the .Xr mandoc 1 regression suite on arbitrary operating systems, even though the suite was designed for OpenBSD only. .Pp When run without an argument, .Nm runs the complete regression suite. .Pp When run with one argument, that argument can be: .Bl -enum .It A single dot to run the complete suite. .It One of the top level directories, for example .Pa mdoc , to run the test suite for a complete language or feature group. .It A subdirectory, for example .Pa man/IP , to run the tests for a specific macro or an individual feature. .It A subdirectory with a test name appended with a colon, for example .Pa char/unicode : Ns Pa named , to run the tests for one particular input file. .El .Pp Any additional arguments modify the way the tests are run. The default is .Cm all . The following modifiers are available: .Bl -tag -width verbose .It Cm all Run all kinds of subtests. This implies all other modifiers except .Cm verbose and .Cm clean . .It Cm ascii Run subtests for .Fl T Cm ascii output mode. .It Cm clean Remove all output files created by running the tests. .It Cm html Run subtests for .Fl T Cm html output mode. .It Cm lint Run subtests for .Fl T Cm lint warning and error output. .It Cm man Run subtests for .Fl T Cm man output mode. .It Cm utf8 Run subtests for .Fl T Cm utf8 output mode. .It Cm verbose Display approximate indications of what is being done. .El .Pp The amount of summary lines shown can be modified by giving an argument consisting of a single digit: .Bl -tag -width verbose .It Cm 3 Show all summary lines for all directories entered. Even without .Cm verbose , this generates more than hundred lines of output when running the complete regression suite. .It Cm 2 This is the default. It shows the summary lines for the .Ar directory given on the command line and its immediate children. Except for .Pa mdoc , the output usually fits on one screen. .It Cm 1 Only show a single summary line for the whole run. .It Cm 0 Do not show any summary lines. No output means success. Success or failure can also be seen from the exit status. .El .Pp All failed tests are always reported, even when the .Cm 0 modifier is given. .Sh EXIT STATUS .Ex -std .Sh EXAMPLES The recommended invocation for casual users: .Pp .Dl ./regress.pl .Pp Maximum output: .Pp .Dl ./regress.pl \&. verbose .Pp Complete check, but keep the tree clean: .Pp .Dl ./regress.pl \&. all clean .Pp Test all of .Pa mdoc , but don't print the usual 65 lines of output: .Pp .Dl ./regress.pl mdoc 1 .Pp Investigate a specific failure: .Pp .Dl ./regress.pl mdoc/Bd:broken man verbose .Sh HISTORY The .Nm script appeared in release 1.14.1 of the portable .Sy mandoc distribution. .Sh AUTHORS .An Ingo Schwarze Aq Mt schwarze@openbsd.org .Sh CAVEATS This script is not optimized for elegance. Regression suites for other software should not copy the design. .Pp The problem it solves is that the .Sy mandoc regression suite is tightly integrated into the regression testing system of the OpenBSD base system, which requires both OpenBSD .Xr make 1 , working neither with POSIX make nor with GNU make, and which also requires the OpenBSD-specific Makefile fragments in .Pa /usr/share/mk . The workaround of parsing the Makefiles by hand and constructing the required command lines by hand is unavoidably messy; it's the classic no-no of parsing a language with an ad-hoc incomplete parser. But the problem of providing this regression suite for other operating systems stood unsolved for many years, and no cleaner solution was found that could be implemented with reasonable effort. So maybe this is better than nothing. .Pp The top-level Makefiles for running this regression suite on OpenBSD are not included in the portable distribution. They are too OpenBSD-specific to be useful elsewhere, and on OpenBSD itself, the suite ought be run natively from .Pa /usr/src/regress/usr.bin/mandoc and not from the portable distribution. .Pp The .Pa db subdirectory of the regression suite is not included. It uses a Makefile structure that differs vastly from the rest of the suite.