===================================================================
RCS file: /cvs/pta/pta_import.1,v
retrieving revision 1.3
retrieving revision 1.17
diff -u -p -r1.3 -r1.17
--- pta/pta_import.1	2020/11/12 13:08:13	1.3
+++ pta/pta_import.1	2021/06/04 03:09:50	1.17
@@ -1,4 +1,4 @@
-.\" $Id: pta_import.1,v 1.3 2020/11/12 13:08:13 schwarze Exp $
+.\" $Id: pta_import.1,v 1.17 2021/06/04 03:09:50 freda Exp $
 .\"
 .\" Copyright (c) 2020 Freda Bundchen
 .\"
@@ -14,7 +14,7 @@
 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .\"
-.Dd $Mdocdate: November 12 2020 $
+.Dd $Mdocdate: June 4 2021 $
 .Dt PTA_IMPORT 1
 .Os
 .Sh NAME
@@ -22,7 +22,7 @@
 .Nd print pta journal entries from bank CSV file input
 .Sh SYNOPSIS
 .Nm pta_import
-.Fl I Ar bankFormat
+.Fl I Ar accountname
 .Op Ar file
 .Sh DESCRIPTION
 The
@@ -37,209 +37,137 @@ If the
 .Ar file
 argument is omitted, standard input is read instead.
 .Pp
-.Ar bankFormat
-is exactly one of
+The
+.Ar accountname
+argument corresponds to an existing
+.Pa ./import/ Ns Ar accountname Ns Pa .txt
+configuration file.
+.Ss Configuration file format
+Each configuration file contains rules
+used to import transactions from a financial institution.
+Like all
+.Xr pta 1
+input files, it is an
+.Xr ascii 7
+text file using strings of one or more space characters
+as the field delimiter and using the hash character
+.Pq Sq #
+in the first column to mark lines as ignored.
+.Pp
+The rules are as follows:
 .Bl -tag -width Ds
-.It capital_one_credit
-.It chase_credit
-.It optum_hsa
+.It Ic ACCOUNT Ar csv_account_number
+The number in
+.Xr pta-accounts 5
+of the bank or credit card account this CSV file contains entries for.
+This rule is required in every configuration file.
+.It Ic AMOUNT Oo Ar debit_column Oc Ar credit_column
+The column numbers in the CSV file containing amounts to be
+booked on the debit or credits side of the contra account
+.Ar account_number ,
+respectively.
+If
+.Ar debit_column
+is omitted, amounts preceded by a minus sign or enclosed
+in parentheses are booked to the debit side.
+This rule is required in every configuration file.
+.It Ic COSTCENTER Ar costcenter_column
+The column number in the CSV file containing the cost center.
+Numbering starts from 1.
+If this rule is not specified, no cost centers are assigned.
+.It Ic DATE Ar date_column date_format
+The column number in the CSV file containing the date.
+Numbering starts from 1.
+The
+.Ar date_format
+is one of
+.Ic MM/DD/YY ,
+.Ic MM/DD/YYYY ,
+.Ic YYYY-MM-DD ,
+or
+.Ic DD.MM.YY .
+This rule is required in every configuration file.
+.It Ic DESCRIPTION Ar descr_column ...
+The column numbers in the CSV file to be concatenated to form the
+text string for the journal entry.
+Numbering starts from 1.
+This rule is required in every configuration file.
+.It Ic DELIM Ar delimiter
+The character used to separate fields.
+Most financial institutions use commas but some use semicolons.
+This rule is required in every configuration file.
+.It Ic HEADER Ar header
+Skip the first line of the CSV file if it matches the
+.Ar header
+line verbatim, or cause parsing to fail if it does not.
+.It Ic IGNORE Ar regular_expression
+Skip all lines in the CSV file matching the
+.Ar regular_expression .
+.It Ic QUANTITY Ar quantity_column
+The column number in the CSV file containing the number of stock shares.
+Numbering starts from 1.
+If this rule is not specified, no information about the number of traded
+securities is appended to the text string.
+.It Ic QUOTE Ar quote
+Expects that every field starts and ends with the
+.Ar quote
+character.
+.It Ar selector account_number booking_identifier Op Ar subaccount
+Lines in the CSV file that match the
+.Ar selector
+are associated with the contra account
+.Ar account_number ,
+the optional
+.Ar subaccount ,
+and are assigned the
+.Ar booking_identifier .
+The selector is a string consisting of regular expressions separated
+by delimiters defined in the
+.Ic DELIM
+rule.
+These rules appear after all other rule types.
 .El
-.Pp
-and corresponds to an existing
-.Qq import_bankFormat.txt
-file in the current directory.
 .Sh FILES
-.Bl -tag -width Ds
-.It Pa ./import_capital_one_credit.txt
-.It Pa ./import_chase_credit.txt
-.It Pa ./import_optum_hsa.txt
+Example configuration files:
+.Bl -tag -offset indent -width Ds -compact
+.It Pa ./import/bbva_usa.txt
+.It Pa ./import/capital_one_360.txt
+.It Pa ./import/capital_one_credit.txt
+.It Pa ./import/chase_credit.txt
+.It Pa ./import/etrade_ira.txt
+.It Pa ./import/optum_hsa.txt
+.It Pa ./import/sparkasse_camt.txt
+.It Pa ./import/wellsfargo.txt
 .El
 .Pp
-These files contain the header that the financial institution uses, the
-account number from
-.Pa ./accounts.txt
-used by this account, as well as patterns used to map transactions to
-the correct account and booking identifier.
-.Ss Defining the account number
-Each file must contain one line beginning with
-.Qq ACCOUNT
-and is followed by whitespace and then an account number.
-The account number corresponds to the account in
-.Pa ./accounts.txt
-for the imported account.
-.Ss Patterns in the import_bankFormat.txt files
-Each file contains lines of the form
-.Pp
-.Ar selector account_number booking_identifier
+Example CSV files:
+.Bl -tag -offset indent -width Ds -compact
+.It Pa ./csv/bbva_usa.csv
+.It Pa ./csv/capital_one_360.csv
+.It Pa ./csv/capital_one_credit.csv
+.It Pa ./csv/chase_credit.csv
+.It Pa ./csv/etrade_ira.csv
+.It Pa ./csv/optum_hsa.csv
+.It Pa ./csv/wellsfargo.csv
+.El
 .Sh EXIT STATUS
 .Ex -std
 .Sh EXAMPLES
 Add journal entries to a journal file using a CSV file generated by
-a financial institution:
+a financial institution, and a configuration file
+.Pa ./import/my_bank.txt :
 .Pp
-.Dl "$ pta_import -I chase_credit chasedownload.csv >> journal.txt"
-.Ss A sample credit card purchase
-Suppose a line in the
-.Pa chasedownload.csv
-file looks like
+.Dl "$ pta_import -I my_bank downloaded.csv >> journal.txt"
 .Pp
-.Dl "10/26/2020,10/26/2020,example.com*asldfkj,Shopping,Sale,-17.07"
-.Pp
-If the line matching the
-.Qq Shopping
-category in the user's
-.Pa ./import_chase_credit.txt
-file is
-.Pp
-.Dl ,,,Shopping,,, 9996 SHOP
-.Pp
-then the
-.Qq Shopping
-category defined by Chase is mapped to the user's
-.Qq SHOP
-booking identifier, and the user's account for shopping
-expenses is defined as
-.Qq 9996
-in their
-.Pa ./accounts.txt .
-.Pp
-We further assume in this example that the user has edited
-.Pa ./import_chase_credit.txt
-so that the line beginning with
-.Qq ACCOUNT
-contains the liability account they have defined for their Chase card
-in
-.Pa ./accounts.txt
-.Pp
-In this example,
-.Pa ./import_chase_credit.txt
-contains
-.Dl ACCOUNT 3234
-.Pp
-Then
-.Nm
-prints
-.Pp
-.Dl "20201026 SHOP 9996 3234 17.07 example.com*asldfkj"
-.Ss A sample credit card payment
-.Dl "10/27/2020,10/27/2020,Payment Thank You - Web,,Payment,1653.42"
-.Pp
-Suppose the user's
-.Pa ./import_chase_credit.txt
-file contains
-.Pp
-.Dl ,,,,Payment,, 1700 PAY
-.Pp
-which matches Chase's payment category to the user's asset account
-.Qq 1700
-used to pay off the credit card.
-Then
-.Nm
-prints
-.Pp
-.Dl "20201027 PAY 3234 1700 1653.2 Payment Thank You - Web"
-.Ss Account numbers in the import_bankFormat.txt files
-For example, if
-.Pa ./import_capital_one_credit.txt
-contains
-.Pp
-.Dl ACCOUNT 1293
-.Pp
-then there must be a corresponding entry in
-.Pa ./accounts.txt
-such as
-.Dl 1293 L Capital One credit card
-.Pp
-The description
-.Qq Capital One credit card
-may vary, but in this example, the account numbers
-.Qq 1293
-match, and the account type is
-.Qq L
-to indicate that it is a liability account.
-.Ss Patterns in the import_bankFormat.txt files
-Suppose
-.Dl ,,,Automotive,,, 1111 AUTO
-.Pp
-is a sample entry in the
-.Pa ./import_chase_credit.txt
-file which uses headers of the form
-.Pp
-.Dl Transaction Date,Post Date,Description,Category,Type,Amount
-.Pp
-Then this
-.Ar selector
-ignores the transaction date, post date, description, type, and amount.
-It matches the
-.Qq Automotive
-category and maps transactions to account
-.Qq 1111
-defined in
-.Pa ./accounts.txt
-with the
-.Qq Auto
-booking identifier.
-.Ss A complete example
-Suppose
-.Pa ./accounts.txt
-is given by
-.Bd -literal
-1700 A My Bank
-3234 L My Chase Credit Card
-5102 X Physician expenses
-5103 X Dental expenses
-5203 X Shopping
-
-.Ed
-.Pp
-and
-.Pa chasedownload.csv
-is given by
-.Bd -literal
-Transaction Date,Post Date,Description,Category,Type,Amount,Memo
-11/06/2020,11/09/2020,Dr. Joseph Schmoe,Health & Wellness,Sale,-100.00,
-11/06/2020,11/09/2020,Dr. Sally Sue,Health & Wellness,Sale,-183.00,
-11/08/2020,11/08/2020,example.com,Shopping,Sale,-53.83,
-11/04/2020,11/04/2020,Payment Thank You - Web,,Payment,75.43,
-.Ed
-.Pp
-and
-.Pa ./import_chase_credit.txt
-is given by
-.Bd -literal
-# Chase credit card header is
-# Transaction Date,Post Date,Description,Category,Type,Amount
-# The format of this file is
-# selector account_number booking_identifier
-# selector is a comma-separated list of regular expressions which
-# matches corresponding fields in the CSV file
-ACCOUNT 3234
-,,,Automotive,,, 1111 AUTO
-,,,Bills & Utilities,,, 2222 BILLS
-,,,Fees & Adjustments,,, 3333 FEES
-,,,Food & Drink,,, 4444 FOOD
-,,,Gas,,, 5555 GAS
-,,,Gifts & Donations,,, 6666 GIFTS
-,,,Groceries,,, 7777 GROCERIES
-,,Dr. Sally Sue,,,, 5103 DENTAL
-,,,Health & Wellness,,, 5102 PHYSICIAN
-,,,Home,,, 9999 HOME
-,,,Personal,,, 9998 PERSONAL
-,,,Professional Services,,, 9997 PRO
-,,,Shopping,,, 5203 SHOP
-,,,,Payment,, 1700 PAY
-.Ed
-.Pp
-Then
-.Dl pta_import -I chase_credit chasedownload.csv
-outputs
-.Bd -literal
-20201109 PHYSICIAN 5102 3234 100.00 Dr. Joseph Schmoe
-20201109 DENTAL 5103 3234 183.00 Dr. Sally Sue
-20201108 SHOP 5203 3234 53.83 example.com
-20201104 PAY 3234 1700 75.43 Payment Thank You - Web
-.Ed
+The result of applying a configuration file
+of the form
+.Pa ./import/accountname.txt
+to the corresponding CSV file
+.Pa ./csv/accountname.csv
+may be seen in
+.Pa ./regress/import/expected/accountname.out .
 .Sh SEE ALSO
+.Xr pta 1 ,
 .Xr pta-accounts 5 ,
 .Xr pta-journal 5 ,
 .Xr pta-glossary 7
@@ -250,21 +178,6 @@ has the following limitations:
 .Bl -dash
 .It
 Pattern recognition in the
-.Pa import_bankFormat.txt
+.Pa import/accountname.txt
 files is under continued development.
-.It
-The only defined import formats are for Chase credit cards,
-Capital One credit cards, and Optum Health Savings Accounts.
-.It
-Note that the first pattern match wins in
-.Pa ./import_bankFormat.txt .
-In the complete example above, we put the line containing
-.Qq Dr. Sally Sue
-before the line matching the
-.Qq Health & Wellness
-category because we wanted to override the
-.Qq Health & Wellness
-category, and use the
-.Qq Dental expenses
-account instead.
 .El