Lisp translation of C++ common expressions
==========================================

General
=======
  The tanslation of a RAC is done in three steps:

    * preprocess: it is done by g++ and it is the same as C or C++ but the
      comments needs to be kept in order to keep the guards (RAC begin/end).

    * C++ parser: parse the preprocess file and output a partially translated
      AST. Implemented under src/.

    * lisp/translate.lisp: Take the S-Statements generated by the parser and
      finises the translation.

  All those step are performed by the RAC script, which is generated from
  src/rac-skel.

C++ parser
==========

Here the global process of the parse:

* First an AST (defined in program/parser/ast) is generated from the file using
  flex and bison (implemented in program/parser).

* Then, the AST is modified (for example, expressions are typed) and semantic
  checks are performed using the vistor design pattern (all are implemented in
  program/process).

* And last, the program is display as S-expressions by using the ACL2Expr
  methods defined in the AST. This will be moved to its own pass later.

AST
---

All the nodes have a NodesID used for the visitor dispatch. When a new
AST node is added, astnodes.def and visitor.h/hxx must always be updated.

To avoid repetitive code, x-macros (see astnodes.def) are used to generate code
for the operators (operator.def define the C++ symbol and their enumeration
names), types (types.def containts the hierarchy) and ASTNodes (astnodes.def
same as types.def).

They are two types of nodes: statement and expression. The expressions are
always pure (they don't have side effects) and have an associated type. The
statements are everything else (control flow, variable and function
declaration, ...). A SimpleStatement is a statement that does not include
substatements.

Here a quick overview of the AST nodes hierarchy (Expression and Statement are
disjointed):
```
Expression                        Statement
  |                                 |
  +--Constant                       +--SimpleStatement
  |   |                             |    |
  |   +--Integer                    |    +--SymDec
  |   +--Boolean                    |    |    |
  |                                 |    |    +--EnumConstDec
  +--Parenthesis                    |    |    +--VarDec
  +--SymRef                         |    |    +--ConstDec
  +--FunCall                        |    |    +--TempParamDec
  |    |                            |    |
  |    +--TempCall                  |    +--MulVarDec
  |                                 |    +--MulConstDec
  +--Initializer                    |    +--BreakStmt
  +--ArrayRef                       |    +--ReturnStmt
  +--StructRef                      |    +--NullStmt
  +--Subrange                       |    +--Assertion
  +--PrefixExpr                     |    +--Assignment
  +--CastExpr                       |    +--MultipleAssignment
  +--BinaryExpr                     |
  +--CondExpr                       +--Block
  +--MultipleValue                  +--IfStmt
                                    +--ForStmt
                                    +--CaseSwitchStmt
                                    +--FunDef
                                         |
                                         +--Builtin
                                         +--Template
```

Parser
------
  flex/bison
  TODO C++ mode

Process
-------

Appart from basic parsing (done by bison) and some very specific work, most of
the translation and checks should be implemented in separate passes using the
vistor provided (program/process/process/visitor.h).

To create to new pass, the documentation in visitor.h, and the assertions.h and 
astdumper.h/cpp examples can be useful references. This visitor is greatly
inspired by Clang's RecursiveASTVisitor, and as such Clang's documentation
[3][4] can be useful as well. After all this reading, the next step to
understand the visitor, is to practice by writing an easy visitor printing when
visiting a node and then writing a visitor to print the AST (in dot, ascii, ...).

Instead of returning `false` when an error is detected, the error() function,
provided in returnfalse.h should be used (this is how error bypass works).

When modifying the AST, program/parser/ast/astnodes.def and the visitor
traversal MUST be updated (in visitor.hxx). Be careful to not add loops or
multiple pass on the same nodes. Also, check if the leaves are still leaves, if
not, it changes the traversal.

Currently, these pass are applied (in this order):

* TypingAction: This step is mostly for expressions. Types are checked (for
  example a `+` between an array and a int does not make sense) and the node is
  typed. All of this is done according to the typing rules found in the AC
  types reference manual (especially the section 2.3) and cppreference.

* RacConstraints and ForConstraints: Those contains the extra checks enabled by
  -pedantic. They help ensuring that the translation is correct by avoiding
  undefined behaviors (for example a variable defined but not initialized) or
  things translate.lisp/acl2 can't use or prove (like a for loop which
  termination can't be proven).

* MarkAssertions: This step is the pass where the benefit of the visitor are
  the most obvious. This step search for every assertions and set, as error
  message, the name of the parent functions. In the older version of the parser,
  there was a function (markAssertion) defined for every node. This function
  was only doing the traversal of the AST and forward the function name except
  for the Assertion node in which the name is set. This meant, adding a
  "simple" action on only two nodes (functions definition and assertion)
  implied reimplementing the full traversal everywhere. With
  RecursiveASTVisitor, it is only 30 lines of code (without the boilerplate,
  there are only 3 lines of logic).

Code generation
---------------

The parser's goal is to generate a partially translated AST usable by the Lisp
script. The parser will translate expression in Lisp but the program stays
imperative (for example, the program still contains for loops).

SExpression are generated by the ACL2Expr() functions and are implemented for
all AST nodes (see under program/parser/ast/, expression.[cpp,h],
statement.[cpp,h] and functions.[cpp,h]). Those functions are recursive: by
running ACL2Expr() on each roots, the full translation is generated (in
practice, the AST is composed of several trees, one for each functions/constant.
The root itself is not a "true" AST node. This is not elegant and one day, it
will change).

The cast translation is the hardest part of the parser. Cast are handled in two
stages: during the typing phase, every nodes is typed (types need to be
resolved before casting). Then, cast are translated in the ACL2Expr() functions
using the type information determined before. An example of cast can be found
in statement.cpp at line 376 (assignment). The actual cast logic is handled in
types.cpp.

They are separated in two cases: when it is an actual cast (for example: a
large value is assigned in a smaller storage or when a value is casted with an
explicit cast). The other, the "false" cast, are used for translating operation
(for example: when shifting an unsigned int the correct translation is
(bits a n)), where the cast logic is reused. As example of the second case, at
lines 867-882 in expressions.cpp, an untyped value (the result of the operation)
is beeing "casted" to the desired type determined by the typing rules.

Utils
-----

Errors in the parser or in a pass are reported using DiagnosticHandler, see
program/parser/utils/diagnostics.h for more details.

Also, program/parser/utils/utils.h contains some useful functions. format()
should be replace the standard's one when switching to C++20.

Tips for test and debug
--------------

To know how to run the regression test, see tests/README.md. Even if thoose
tests should covered most parts of the parse, it a good idea to test the
translations on some real life models (like the ones in examples). Every new
feature/bug discovered must be covered by a test. If a bug is discovered but no
fix are implemented, it should still be a test covering it with the "bug" flag
set in the yaml.

By default, the parser will be compile in release mode (with all optimisation
and without the debuging symbols), to  debug use 'make debug' instead.

For more in depth debuging, the -dump-ast option is really useful to understand
what's happening. By running `./bin/parse file.cpp -dump-ast > ast.dot`, the
parser prints the AST in dot [1]. Then to convert it to an image using either
by running `dot -T png ast.dot > ast.png` or one of the many online visualizers
(like [2]). For now, it only display the name of the nodes and the expression's
types. This is very useful to learn how the parser translates a program,
learning the AST "shapes" and also, since it has the type info, it helps
debugging type/cast issues. Note, the dot dump works but is still limited: some
useful features or info are probably missing. This is not a "real" feature so,
it is kept minimalistic. Feel free to add more information as needed (It is
also a super exercise to test and learn how to use the ASTRecursiveVisitor !).

The parser's -trace-parser option, can be used to enable bison debug logs. For
more details, see bison documentation. [5]

Also, the diagnostics (program/parser/utils/diagnostics.h) can be really
usefull to know which parts of the input code cause a bug.

Version
-------

The version is generated from the ACL2 version (grep from the README) and the
commit hash. If build outside of a git repository, only the ACL2 version is kept
(anyway it should be from a relase). If something fails, the header
should be still generated with an "unknown version". If the project has not
been built yet, version.h does not exist (it is normal if your editor reports
an error !).

TODOs
=====

* switch bison parser to C++ mode
* continue to refactore
* lift for restriction
* C++20 format
* generate a list of type usable in Lisp
* track location of types (deref typedef)
* display "true" C++ type
* better error message when the type is a typedef.

[1] https://graphviz.org/doc/info/lang.html
[2] https://dreampuf.github.io/GraphvizOnline
[3] https://clang.llvm.org/doxygen/RecursiveASTVisitor_8h_source.html
[4] https://clang.llvm.org/docs/RAVFrontendAction.html.
[5] https://www.gnu.org/software/bison/manual/bison.html
