FILEPP(1)

#define CPP1 cpp(1) #define SH1 sh(1) #define PERL1 perl(1) #define FILEPP1 filepp

NAME
filepp - A generic file preprocessor

SYNOPSIS
filepp [options] filename(s)

DESCRIPTION
FILEPP1 is a generic file preprocessor designed to allow the functionality provided by the C preprocessor CPP1 to be used with any file type. FILEPP1 is designed to be easily customised and extended.

OPTIONS
FILEPP1 accepts the following command line options:

-b  Suppress blank lines originating from include files (this has no effect on the top-level file).

-c  Read input from STDIN instead of a file. Note: if both -c and input files are specified, both are used as inputs in the order given.

-Dmacro
    Predefine macro to have a definition of `1'.

-Dmacro=defn
    Predefine macro to have a definition of defn.

-d  Output debugging information.

-dd Output verbose debugging information. This option shows all normal debugging information, plus the full list of defined macros every time the list changes.

-dl Output light debugging information. This option shows minimal debugging information.

-dprechar
    Prefix all debugging information with char (can be character or string), can be used to make debugging easier to read.

-dpostchar
    Postfix all debugging information with char (can be character or string), this defaults to a newline. If char does not contain a newline, then no newline will be printed after debugging messages. (Newlines can be put in char using the __NEWLINE__ macro.)

-ds Print debugging info on stdout rather than stderr.

-e  Define all environment variables as macros with prefix envchar.

-ec char
    Set envchar (prefix of environment variables defined as macros) to char, defaults to $. (Note: this option only takes effect at the time the environment variables are converted to macros).

-ecn
    Set envchar (prefix of environment variables defined as macros) to nothing (no prefix).

-h  Show summary of options.

-Idir
    Append directory dir to the list of directories searched for include files.

-imacros file
    Reads in macros from file, but discards everything else in the file.

-k  Turn off parsing of all keywords. This is useful if you just want to use the macro expansion facilities of FILEPP1. With this option all keywords found will be ignored, FILEPP1 will just replace any macros specified with the -Dmacro=defn option.

-kc char
    Set keyword prefix character to char (can also be a string). All FILEPP1 keywords are prefixed with the character # by default. This option allows the prefix to be changed to something else.

-lc char
    Set line continuation character to char (can also be a string). When the line continuation character is found with a newline following it, it and the newline are replaced by the line continuation replacement character. Default is \ (CPP1 style).

-lec char
    Set optional keyword line end character to char (can also be a string). This allows extra characters to be placed at the end of a line containing a keyword. The extra characters will be ignored. This is useful if keywords are to be embedded in HTML or C style comments. For example, to embed keywords in an HTML comment the keyword prefix character could be set to <--!# and the optional keyword line end character set to -->. An example keyword would then be: <!--#include "header.h" --> In the case the optional keyword line end characters --> would be ignored.

-lr char
    Set line continuation replacement character to char (can also be a string). Default is a null string (CPP1 style).

-lrn
    Set line continuation replacement character to be a newline.

-m module.pm
    Load module module.pm. module.pm is a PERL1 module which can be used to extend or modify the behaviour of FILEPP1. See section FILEPP MODULES for details of modules included with filepp and FILEPP MODULE API for details on how to write your own modules.

-Mdir
    Append directory dir to the list of directories searched for filepp modules. This list defaults to the directory the filepp modules are installed (if any) plus the default Perl module paths. (Note: this adds the directory to the Perl @INC list.)

-mp char
    Prefix all macros with char. Macros are defined in the normal way, but will only be replaced when found prefixed with char. For example, filepp macros will behave similar to Bourne shell (SH1) variables if char is set to $.

-mpnk
    Turns off macro prefixes within keywords. When using a macro prefix character this option allows macros to be used without the prefix in keyword processing. For example, if the macro prefix is $ then and #if would be written as: #if $MACRO == 1 Using the mpnk option allows the #if to be written as: #if MACRO == 1

-o name
    Write output to name instead of STDOUT. If there is only one input file and it has the same name as the output file, the original input file will be backed-up as name~.

-ov Overwrite mode, causes the output file to overwrite the input file. Useful when modifying a large number of files at once, eg: FILEPP1 -ov -DTHIS=THAT * The original input file(s) will be backed-up as name~.

-ovc IN=OUT
    Similar to overwrite mode, the difference is the output filename is input filename with IN part converted to OUT. For example, to process a set of files all ending with .in and have the output files all ending in .out do: FILEPP1 -ovc .in=.out *.in In this case a file called test.in will be processed and the output file will be test.out. Note: if the input file does not contain IN then the output file will have the same name as the input file and the original input file(s) will be backed-up as name~!

-pb Preserve blank lines. Using this option attempts to keep as many lines in the output file as are in the input file, so all blank lines which normally would not get printed are printed. Useful when comparing intput file with output.

-re Treat keyword and macro prefix characters and line continuation character as Perl regular expressions instead of normal strings.

-s  Run FILEPP1 in safe mode. This turns off the pragma keyword.

-Umacro
    Undefine previously defined macro.

-u  Undefine all currently defined macros, including predefined ones.

-v  Show version of program.

-w  Turn on word boundaries when replacing macros. When word boundaries are on, macros will only be replaced if the macro appears in the text as a word. For example, by default macro would be replaced in both cases of the following text: macro as word, macroNOTaword but only the first occurrence would be replaced with the -w option. With this option enabled FILEPP1 will only replace macros which contain alphanumeric characters. International (non-ASCII) character sets can be supported using Perl's locale handling.

KEYWORDS
FILEPP1 supports the following keywords:

#include <FILE>
    Include a file in the file being processed. This variant is used for "system" include files. It searches for a file named FILE in a list of directories specified by you. Directories are specified with the command option `-I'. FILEPP1 does not predefine any system directories in which to search for files.

#include "FILE"
    Include a file in the file being processed. This variant is used for include files of your own project. It searches for a file named FILE first in the current directory, then in the list of directories specified with the command option `-I'. The current directory is the directory the base input file is in.

#define macro
    Define the macro macro to have a definition of `1'. macro can then be used with the keywords #ifdef and #ifndef.

#define macro defn
    Define the macro macro to have the value defn. macro can then be used with the keywords #ifdef and #ifndef. Also, all instances of macro following the #define statement will be replaced with the string defn. The string defn is taken to be all the characters on the line following macro.

#define macro(arg1, arg2, ...) defn
    Define the macro macro to have the value defn with arguments (arg1, arg2, ...). macro can be used as follows: #define macro(foo) defn with foo in Now when replacing occurs: macro(bar) will become: defn with bar in Macros can have any number of comma separated arguments. Macros can also have variable numbers of arguments if the final macro ends in ..., for example: #define error(string, args...) fprintf(stderr, string, args); Here the first argument given becomes string and all other arguments will become args. If called as: error("%d,%s", i, string) it will give fprintf(stderr, "%d,%s", i, string); Also, if a macro with a variable number of arguments is passed no arguments for the variable argument, then commas can be optionally removed from the definition by preceding the definition with "##". For example: #define error(string, args...) fprintf(stderr, string, ##args); If this is called as: error("empty") then result will be: fprintf(stderr, "empty"); The comma immediately before ##args has been removed.

#if expr
    A conditional statement, expr will be evaluated to true (1) or false (0). If expr evaluates to true, the text between the #if and the next #else or #endif will be included. If expr evaluates to false, the text between the #if and the next #else or #endif will be ignored. expr can use all the usual cpp style comparisons (==, !=, <, >, etc.). Multiple comparisons can be combined with and (&&) and or (||). The defined keyword can also be used to check if macros are defined. For example: #if defined macro && macro == defn Note: filepp's #if does not work in exactly the same way as CPP1's #if. CPP1's #if only does numerical style comparisons. Filepp's #if statement can also compare strings and regular expressions using PERL1's full range of comaprison operations. For example, to test if two strings are exactly equal use: #if "MACRO" eq "string" To test if strings are not equal use ne instead of eq. Regular expressions can also be tested, for example to test if a macro has any whitespace in it use: #if "MACRO" =~ /\s/ To test if a macro does not have any whitespace in it =~ can be replaced with !~. Perl experts: #if works by first parsing expr for the defined keyword and checking if the macro it refers to is defined, replacing it with 1 if it is and 0 if it isn't. It then checks expr for any other macros and replaces them with their definition. Finally it passes expr through Perl's eval function, which returns true or false.

#elif expr
    #elif stands for "else if". Like #else, it goes in the middle of a #if[n][def]-#endif pair and subdivides it; it does not require a matching #endif of its own. Like #if, the #elif directive includes an expression to be tested.

#ifdef macro
    A conditional statement, if macro has been defined the text between the #ifdef and the next #else or #endif will be included. If macro has not been defined the text between the #ifdef and the next #else or #endif will be ignored.

#ifndef macro
    The reverse case of the #ifdef conditional.

#else
    The #else directive can be added to a conditional to provide alternative text to be used if the condition is false.

#endif
    Used to terminate a conditional statement. Normal processing resumes following the #endif.

#undef macro
    Undefine a previously defined macro.

#error mesg
    Causes FILEPP1 to exit with the error message mesg.

#warning mesg
    Causes FILEPP1 to issue the warning message mesg.

#comment mesg
    As FILEPP1 is supposed to be a generic file preprocessor, it cannot support any known comment styles, therefore it defines its own with this keyword. All lines starting with #comment are treated as comments and removed by FILEPP1.

#pragma filepp function arg1, arg2, ...
    The #pragma keyword immediately followed by the word filepp allows the user to execute a Perl function during parsing. The word immediately following filepp is taken as the name of the function and the remainder of the line is taken to be a comma separated list of arguments to the function. Any of the FILEPP1 internal functions (see section FILEPP MODULE API) can be called with the #pragma keyword. Warning: There are obvious security risks with allowing arbitrary functions to be run, so the -s (safe mode) command line option has been added which turns the #pragma keyword off.

PREDEFINED MACROS
FILEPP1 supports a set of predefined macros. All the predefined macros are of the form __MACRO__, where MACRO is:

FILE
    This macro expands to the name of the current input file.

LINE
    This macro expands to the current input line number.

DATE
    This macro expands to a string that describes the date on which the preprocessor is being run. The string contains eleven characters and looks like "__DATE__".

ISO_DATE
    This macro expands to a string that describes the date on which the preprocessor is being run. The string is in the format specified by ISO 8601 (YYYY-MM-DD) and looks like "__ISO_DATE__".

TIME
    This macro expands to a string that describes the time at which the preprocessor is being run. The string contains eight characters and looks like "__TIME__".

BASE_FILE
    This macro expands to the name of the main input file.

INCLUDE_LEVEL
    This macro expands to a decimal integer constant that represents the depth of nesting in include files. The value of this macro is incremented on every #include directive and decremented at every end of file.

NEWLINE
    This macro expands to a newline.

TAB This macro expands to a tab.

NULL
    This macro expands to nothing. It is useful if you want to define something to be nothing.

VERSION
    This macro expands to a string constant which describes the version number of FILEPP1. The string is a sequence of decimal numbers separated by periods and looks like "__VERSION__".

FILEPP_INPUT
    This macro expands to a string constant which says the file was generated automatically from the current BASE_FILE and looks like "__FILEPP_INPUT__".

FILEPP MODULES
The following modules are included with the main filepp distribution:

#include "modules/for/for.1.in"

#include "modules/foreach/foreach.1.in"

#include "modules/literal/literal.1.in"

#include "modules/case/case.1.in"

#include "modules/comment/comment.1.in"

#include "modules/function/function.1.in"

#include "modules/maths/maths.1.in"

#include "modules/format/format.1.in"

#include "modules/bigdef/bigdef.1.in"

#include "modules/bigfunc/bigfunc.1.in"

#include "modules/defplus/defplus.1.in"

#include "modules/regexp/regexp.1.in"

#include "modules/blc/blc.1.in"

#include "modules/cmacros/cmacros.1.in"

#include "modules/cpp/cpp.1.in"

#include "modules/grab/grab.1.in"

#define EXISTING $existing #define FILENAME $filename #define FUNCTION $function #define NUMBER $number #define POSITION $pos #define STRING $string #define TYPE $type

FILEPP MODULE API
The behaviour of FILEPP1 can be modified or extended through the use of modules. FILEPP1 modules are in fact PERL1 modules, and the rest of this section assumes the reader has a knowledge of Perl. FILEPP1 modules are PERL1 modules which extend or modify FILEPP1's behaviour by either calling or replacing FILEPP1's internal functions. FILEPP1 has the Perl package name Filepp so its internal functions can be called within modules either as Filepp::function() or just function(). Any of FILEPP1's internal functions can be called or replaced from within a FILEPP1 module, the most useful ones are:

Debug(STRING,NUMBER)
    Print STRING as debugging information if debugging is enabled. NUMBER is optional and can be used to set the debugging level at which STRING should be printed, lower numbers being higher priority. Command line option d prints all debugging info for 2 and below, option dd prints all debugging information for 3 and below and option dl prints all debugging information for 1 and below. If NUMBER is not provided, defaults to 1.

AddProcessor(FUNCTION,POSITION,TYPE)
    Allows the module to add a function named FUNCTION to FILEPP1's processing chain. The processing chain is a set of functions which are run on each line of a file as it is processed. The default functions in the processing chain are ParseKeywords which does keyword parsing and ReplaceDefines which does macro replacement. Further functions can be added to the chain, with each function taking a string (the current line) as input and returning the processed string as output. By default, or if POSITION is set to 0, the processor is added to the end of the processing chain. If POSITION is set to 1 the processor is added to the start of the processing chain. TYPE controls what the processor is run on. There are three options for this, 0 (default): the processor runs on everything passed to the processing chain; 1: the processor runs on full lines only; 2: the processor runs on part lines only (a part line is the text following a keyword such as if which needs to be parsed for macros). Both POSITION and TYPE are optional parameters.

AddProcessorAfter(FUNCTION,EXISTING,TYPE)
    Adds function FUNCTION to the processing chain directly after existing processor EXISTING. If EXISTING is not found then FUNCTION is added to the end of the processing chain. Regular expression matching is used to compare EXISTING with the names of the functions in the processing chain. TYPE is optional.

AddProcessorBefore(FUNCTION,EXISTING,TYPE)
    Adds function FUNCTION to the processing chain directly before existing processor EXISTING. If EXISTING is not found then FUNCTION is added to the start of the processing chain. Regular expression matching is used to compare EXISTING with the names of the functions in the processing chain. TYPE is optional.

RemoveProcessor(FUNCTION)
    Removes the processor function FUNCTION from the processing chain.

STRING=ReplaceDefines(STRING)
    Replaces all macros in STRING with their definitions and returns the processed string.

AddKeyword(STRING,FUNCTION)
    Add the keyword named STRING. When the keyword is found in text processing the function named FUNCTION will be run with everything following the keyword passed as a single argument.

RemoveKeyword(STRING)
    Removes the keyword named STRING.

RemoveAllKeywords()
    Removes all the keywords currently defined for FILEPP1 (used for the -k command line option).

AddIfword(STRING)
    Adds keyword named STRING to Ifword list. An Ifword takes in the string following the keyword and optionally parses it, returning a 1 if the string parses to true and 0 for false. The default Ifwords are if, ifdef and ifndef.

RemoveIfword(STRING)
    Removes keyword named STRING from Ifword list (note: this does NOT remove the keyword, use RemoveKeyword for that).

AddElseword(STRING)
    Adds keyword named STRING to Elseword list. An Elseword takes in the string following the keyword and optionally parses it, returning a 1 if the string parses to true and 0 for false. The default Elsewords are else and elif.

RemoveElseword(STRING)
    Removes keyword named STRING from Elseword list.

AddEndifword(STRING)
    Adds keyword named STRING to Endifword list. An Endifword should return a 1 to indicate successful termination of the if block. If the Endifword returns 0 the Endifword is ignored and filepp assumes the current if block carries on after the Endifword. The default Endifword is endif.

RemoveEndifword(STRING)
    Removes keyword named STRING from Endifword list.

AddIncludePath(STRING)
    Adds the include path STRING to the list of directories to search for include files (used for the -I command line option).

AddModulePath(STRING)
    Adds the path STRING to the list of directories to search for filepp modules (used for the -M command line option).

AddOpenInputFunc(FUNCTION)
    Adds a FUNCTION to a list of functions to be run each time a new base input file is opened.

AddCloseInputFunc(FUNCTION)
    Adds a FUNCTION to a list of functions to be run each time a new base input file is closed.

AddOpenOutputFunc(FUNCTION)
    Adds a FUNCTION to a list of functions to be run each time an output file is opened.

AddCloseOutputFunc(FUNCTION)
    Adds a FUNCTION to a list of functions to be run each time an output file is closed.

AddInputFile(STRING)
    Adds another input file to the list of files to be processed (used for adding input files at the command line).

ChangeOutputFile(STRING)
    Closes the current output file and attempts to open a new one named STRING.

SetKeywordchar(STRING)
    Set the initial keyword char to STRING (used for the -kc command line option).

SetContchar(STRING)
    Set the line continuation char to STRING (used for the -lc command line option).

SetContrepchar(STRING)
    Set the line continuation replacement char to STRING (used for the -lr command line option).

SetOptLineEndchar(STRING)
    Set the optional keyword line end character to STRING (used for the -lec command line option).

SetBlankSupp(1/0)
    Turns blank-line suppression on/off (1 = suppress, 0 = don't suppress). When blank-line suppression is on, blank lines in input files will not be copied to the output. Unlike the corresponding command-line option (-b), this function can also have effect in the top-level file. The setting of blank-line suppression applies to the current file being processed and all files included in the current file.

ResetBlankSupp()
    Resets blank-line suppression to the command-line specified value. This only affects the output of blank lines from the current file being processed and all files included in the current file. In the top-level file, this always turns blank-line suppression off.

SetEatTrail(STRING)
    If STRING is a macro, whenever the macro is replaced all blank space between the macro's replacement and the next character on the line will be eaten. For example, if macro foo is defined to bar and foo has been set to have it's trail eaten, the following:
     eat my foo trail is replaced with
     eat my bartrail

CheckEatTrail(STRING)
    Returns 1 if macro STRING will have it's tail eaten, 0 otherwise.

SetEnvchar(STRING)
    Set the prefix of environment variables converted to macros (envchar) to STRING (used for -ec and -ecn command line options).

DefineEnv()
    Define all environment variables as macros with prefix envchar (used for -e command line option).

SetOutput(1/0)
    Turns writing of parsed input file to output file on/off. This takes either 1 (output on) or 0 (output off) as input. When the output is turned off, the only output produced from FILEPP1 will be that generated by modules.

SetWordBoundaries(1/0)
    Turns on(1) or off(0) word boundary checking when replacing macros (used for the -w command line option).

SetCharPerlre(1/0)
    Turns on(1) or off(0) allowing of keyword prefix char and line continuation char to be Perl regular expressions (used for the -re command line option).

UndefAll()
    Undefines all currently defined macros, including predefined ones (used for the -u command line option).

UseModule(STRING)
    Loads a PERL1 module named STRING using the Perl command require (used for the -m command line option).

SetParseLineEnd(FUNCTION)
    Sets the function to determine if line continuation should be done on current line to FUNCTION.

STRING=GetNextLine()
    Returns the next line (after line continuation has been dealt with) of the input file currently being processed. Returns NULL for end of file.

Write(STRING)
    Writes STRING to the current output file.

Output(STRING)
    Conditionally writes STRING to the current output file. If output is turned on then writes STRING. Output is toggled off/on using SetOutput function.

In addition all the standard FILEPP1 keywords have equivalent functions which optionally take a single argument. The functions have the same name as the keyword, only with a capital first letter (eg: #define string calls the function Define(string)). A full description of the Parse function and all the other FILEPP1 internal functions is beyond the scope of this man page. The FILEPP1 script is well commented and hopefully readable by a Perl programmer, so use the source Luke!

BUGS
FILEPP1 has no known bugs, only "features". If you find any "features", please report them to the author.

COPYING
Copyright (C) 2000-2007 Darren Miller

FILEPP1 is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; see the file COPYING. If not, write to the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.

SEE ALSO
CPP1, PERL1

AUTHOR
Darren Miller <darren@cabaret.demon.co.uk>.