===================================================================
RCS file: /cvs/pta/pta_import.1,v
retrieving revision 1.5
retrieving revision 1.13
diff -u -p -r1.5 -r1.13
--- pta/pta_import.1	2020/11/14 22:15:56	1.5
+++ pta/pta_import.1	2020/12/03 14:46:44	1.13
@@ -1,4 +1,4 @@
-.\" $Id: pta_import.1,v 1.5 2020/11/14 22:15:56 schwarze Exp $
+.\" $Id: pta_import.1,v 1.13 2020/12/03 14:46:44 freda Exp $
 .\"
 .\" Copyright (c) 2020 Freda Bundchen
 .\"
@@ -14,7 +14,7 @@
 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .\"
-.Dd $Mdocdate: November 14 2020 $
+.Dd $Mdocdate: December 3 2020 $
 .Dt PTA_IMPORT 1
 .Os
 .Sh NAME
@@ -42,21 +42,29 @@ is exactly one of
 .Pp
 .Bl -bullet -offset indent -compact
 .It
+.Cm bbva_usa
+.It
+.Cm capital_one_360
+.It
 .Cm capital_one_credit
 .It
 .Cm chase_credit
 .It
+.Cm etrade_ira
+.It
 .Cm optum_hsa
+.It
+.Cm sparkasse_camt
+.It
+.Cm wellsfargo
 .El
 .Pp
 and corresponds to an existing
-.Pa import_ Ns Ar accountname Ns Pa .txt
-configuration file in the current directory.
+.Pa ./import/ Ns Ar accountname Ns Pa .txt
+configuration file.
 .Ss Configuration file format
-Each file of the form
-.Pa import_ Ns Ar accountname Ns Pa .txt
-in the current directory is a configuration file used to
-import transactions from a financial institution.
+Each configuration file contains rules
+used to import transactions from a financial institution.
 Like all
 .Xr pta 1
 input files, it is an
@@ -66,201 +74,113 @@ as the field delimiter and using the hash character
 .Pq Sq #
 in the first column to mark lines as ignored.
 .Pp
-Each file contains one line beginning with
-.Qq ACCOUNT
-and is followed by whitespace and then an account number.
-.Pp
-Each file contains lines of the form
-.Pp
-.Ar selector account_number booking_identifier
-.Pp
+The rules are as follows:
+.Bl -tag -width Ds
+.It Ic ACCOUNT Ar csv_account_number
+The number in
+.Xr pta-accounts 5
+of the bank or credit card account this CSV file contains entries for.
+This rule is required in every configuration file.
+.It Ic AMOUNT Oo Ar debit_column Oc Ar credit_column
+The column numbers in the CSV file containing amounts to be
+booked on the debit or credits side of the contra account
+.Ar account_number ,
+respectively.
+If
+.Ar debit_column
+is omitted, amounts preceded by a minus sign or enclosed
+in parentheses are booked to the debit side.
+.It Ic COSTCENTER Ar costcenter_column
+The column number in the CSV file containing the cost center.
+Numbering starts from 1.
+If this rule is not specified, no cost centers are assigned.
+.It Ic DATE Ar date_column date_format
+The column number in the CSV file containing the date.
+Numbering starts from 1.
 The
+.Ar date_format
+is one of
+.Ic MM/DD/YY ,
+.Ic MM/DD/YYYY ,
+.Ic YYYY-MM-DD ,
+or
+.Ic DD.MM.YY .
+This rule is required in every configuration file.
+.It Ic DESCRIPTION Ar descr_column ...
+The column numbers in the CSV file to be concatenated to form the
+text string for the journal entry.
+Numbering starts from 1.
+.It Ic DELIM Ar delimiter
+The character used to separate fields.
+This rule is required in every configuration file.
+Most financial institutions use commas but some use semicolons.
+.It Ic HEADER Ar header
+Skip the first line of the CSV file if it matches the
+.Ar header
+line verbatim, or cause parsing to fail if it does not.
+.It Ic IGNORE Ar regular_expression
+Skip all lines in the CSV file matching the
+.Ar regular_expression .
+.It Ic QUANTITY Ar quantity_column
+The column number in the CSV file containing the number of stock shares.
+Numbering starts from 1.
+If this rule is not specified, no information about the number of traded
+securities is appended to the text string.
+.It Ic QUOTE Ar quote
+Expects that every field starts and ends with the
+.Ar quote
+character.
+.It Ar selector account_number booking_identifier
+Lines in the CSV file that match the
 .Ar selector
-is a string consisting of regular expressions
-separated by commas.
-Its fields correspond to header fields in the financial
-institution's CSV file and is used to classify transactions
-that match the search pattern.
-The first selector that
-matches a transaction wins.
-.Pp
-The
+are associated with the contra account
 .Ar account_number
-corresponds to an account number in
-.Pa ./accounts.txt .
-The
-.Ar booking_identifier
-is defined by the user to classify
-the transaction.
+and are assigned the
+.Ar booking_identifier .
+The selector is a string consisting of regular expressions separated
+by delimiters defined in the
+.Ic DELIM
+rule.
+.El
 .Sh FILES
-.Bl -tag -width Ds -compact
-.It Pa ./import_capital_one_credit.txt
-.It Pa ./import_chase_credit.txt
-.It Pa ./import_optum_hsa.txt
+Example configuration files:
+.Bl -tag -offset indent -width Ds -compact
+.It Pa ./import/bbva_usa.txt
+.It Pa ./import/capital_one_360.txt
+.It Pa ./import/capital_one_credit.txt
+.It Pa ./import/chase_credit.txt
+.It Pa ./import/etrade_ira.txt
+.It Pa ./import/optum_hsa.txt
+.It Pa ./import/sparkasse_camt.txt
+.It Pa ./import/wellsfargo.txt
 .El
 .Pp
-These files contain the header that the financial institution uses, the
-account number from
-.Pa ./accounts.txt
-used by this account, as well as patterns used to map transactions to
-the correct account and booking identifier.
+Example CSV files:
+.Bl -tag -offset indent -width Ds -compact
+.It Pa ./csv/bbva_usa.csv
+.It Pa ./csv/capital_one_360.csv
+.It Pa ./csv/capital_one_credit.csv
+.It Pa ./csv/chase_credit.csv
+.It Pa ./csv/etrade_ira.csv
+.It Pa ./csv/optum_hsa.csv
+.It Pa ./csv/wellsfargo.csv
+.El
 .Sh EXIT STATUS
 .Ex -std
 .Sh EXAMPLES
 Add journal entries to a journal file using a CSV file generated by
-a financial institution:
+a financial institution, and a configuration file
+.Pa ./import/my_bank.txt :
 .Pp
-.Dl "$ pta_import -I chase_credit chasedownload.csv >> journal.txt"
-.Ss Account numbers in the configuration file
-The line
+.Dl "$ pta_import -I my_bank downloaded.csv >> journal.txt"
 .Pp
-.Dl ACCOUNT 3235
-.Pp
-in the configuration file corresponds to a line such as
-.Pp
-.Dl 3235 L Capital One credit card
-.Pp
-in
-.Pa ./accounts.txt .
-.Pp
-The description
-.Qq Capital One credit card
-may vary, but in this example, the account numbers
-.Qq 3235
-match, and the account type is
-.Qq L
-to indicate that it is a liability account.
-.Ss Patterns in the configuration file
-Suppose
-.Pp
-.Dl ,,,Automotive,,, 1111 AUTO
-.Pp
-is a sample entry in the
-.Pa ./import_chase_credit.txt
-file which uses headers of the form
-.Pp
-.Dl Transaction Date,Post Date,Description,Category,Type,Amount
-.Pp
-Then this
-.Ar selector
-ignores the transaction date, post date, description, type, and amount.
-It matches the
-.Qq Automotive
-category and maps transactions to account
-.Qq 1111
-defined in
-.Pa ./accounts.txt
-with the
-.Qq Auto
-booking identifier.
-.Ss A sample credit card purchase
-If
-.Pa chasedownload.csv
-file contains
-.Pp
-.Dl "10/26/2020,10/26/2020,example.com*asldfkj,Shopping,Sale,-17.07"
-.Pp
-and
-.Pa ./import_chase_credit.txt
-contains
-.Bd -literal -offset Ds
-ACCOUNT 3234
-,,,Shopping,,, 9996 SHOP
-.Ed
-.Pp
-and
-.Pa ./accounts.txt
-contains
-.Bd -literal -offset Ds
-3234 L Chase credit card
-9996 X shopping
-.Ed
-.Pp
-then the
-.Qq Shopping
-category defined by Chase is mapped to the user's
-.Qq SHOP
-booking identifier, the user's account for shopping
-expenses is defined as
-.Qq 9996
-in their
-.Pa ./accounts.txt ,
-the user's account for the credit card is 3234 in their
-.Pa ./accounts.txt ,
-and
-.Nm
-prints
-.Pp
-.Dl "20201026 SHOP 9996 3234 17.07 example.com*asldfkj"
-.Ss A complete example
-Suppose
-.Pa ./accounts.txt
-is given by
-.Bd -literal
-1700 A My Bank
-3234 L My Chase Credit Card
-5102 X Physician expenses
-5103 X Dental expenses
-5203 X Shopping
-.Ed
-.Pp
-and
-.Pa chasedownload.csv
-is given by
-.Bd -literal
-Transaction Date,Post Date,Description,Category,Type,Amount,Memo
-11/06/2020,11/09/2020,Dr. Joseph Schmoe,Health & Wellness,Sale,-100.00,
-11/06/2020,11/09/2020,Dr. Sally Sue,Health & Wellness,Sale,-183.00,
-11/08/2020,11/08/2020,example.com,Shopping,Sale,-53.83,
-11/04/2020,11/04/2020,Payment Thank You - Web,,Payment,75.43,
-.Ed
-.Pp
-and
-.Pa ./import_chase_credit.txt
-is given by
-.Bd -literal
-# Chase credit card header is
-# Transaction Date,Post Date,Description,Category,Type,Amount
-# The format of this file is
-# selector account_number booking_identifier
-# selector is a comma-separated list of regular expressions which
-# matches corresponding fields in the CSV file
-ACCOUNT 3234
-,,,Automotive,,, 1111 AUTO
-,,,Bills & Utilities,,, 2222 BILLS
-,,,Fees & Adjustments,,, 3333 FEES
-,,,Food & Drink,,, 4444 FOOD
-,,,Gas,,, 5555 GAS
-,,,Gifts & Donations,,, 6666 GIFTS
-,,,Groceries,,, 7777 GROCERIES
-,,Dr. Sally Sue,,,, 5103 DENTAL
-,,,Health & Wellness,,, 5102 PHYSICIAN
-,,,Home,,, 9999 HOME
-,,,Personal,,, 9998 PERSONAL
-,,,Professional Services,,, 9997 PRO
-,,,Shopping,,, 5203 SHOP
-,,,,Payment,, 1700 PAY
-.Ed
-.Pp
-Then
-.Pp
-.Dl pta_import -I chase_credit chasedownload.csv
-.Pp
-outputs
-.Bd -literal
-20201109 PHYSICIAN 5102 3234 100.00 Dr. Joseph Schmoe
-20201109 DENTAL 5103 3234 183.00 Dr. Sally Sue
-20201108 SHOP 5203 3234 53.83 example.com
-20201104 PAY 3234 1700 75.43 Payment Thank You - Web
-.Ed
-.Pp
-Note that the selector for Dr\&. Sally Sue
-in the example configuration file comes before the
-Health & Wellness selector.
-Since the first match wins, transactions for Dr\&. Sally Sue
-can be classified with the DENTAL booking identifier,
-even though the bank's CSV file classifies
-these transactions as Health & Wellness.
+The result of applying a configuration file
+of the form
+.Pa ./import/accountname.txt
+to the corresponding CSV file
+.Pa ./csv/accountname.csv
+may be seen in
+.Pa ./regress/import/expected/accountname.out .
 .Sh SEE ALSO
 .Xr pta-accounts 5 ,
 .Xr pta-journal 5 ,
@@ -272,9 +192,9 @@ has the following limitations:
 .Bl -dash
 .It
 Pattern recognition in the
-.Pa import_accountname.txt
+.Pa import/accountname.txt
 files is under continued development.
 .It
-The only defined import formats are for Chase credit cards,
-Capital One credit cards, and Optum Health Savings Accounts.
+User-defined configuration files are not currently
+supported.
 .El