# Top

* PL/0; PASCAL-S

* UX findings from PASCAL-S:
  * Have more parsing-debug tools available in sample apps
  * Write up: Sharp edge if this isn't first b/c first-found & it matches identifier
  * Write up: Better error messages when semicolons are missing
  * Write up: Root must come first

* Iterate on data languages
  * `pgpg-experiments` show parsing too heavy ... lexers maybe?
  * Try to adapt Go CSV -> DKVP

* Error out on this:
```
$ rlwrap ./apps/go/pemdas-eval -mode f2poly -mod-poly 1f -l
```

# Language ideas

## Programming languages

* PEMDAS & VIC obv
* LISP/ish
* PASCAL-S and/or PL/0
  * BNFs
  * Manual recursive descent
* Miller/millerish ...
  * Re-think `emit`: make it a function with reshaping-helper functions

## Data languages

* JSON obv
* CSV
* DKVP
  * Note: for JSON newlines are irrelevant so there is accept-and-yield logic.
  * For DKVP, newlines are record boundaries.

## Utility languages

* Config-file language for Miller -- ? Presumably there are standard config-file packages already out there.
* Something shell-like -- ?

# Perf-analyze the JSON lexer/parser

```
$ justtime jq . ~/data/big.json > /dev/null
TIME IN SECONDS 8.835 -- jq . /Users/kerl/data/big.json

$ justtime mlr -j nothing ~/data/big.json > /dev/null
TIME IN SECONDS 7.668 -- mlr -j nothing /Users/kerl/data/big.json

$ justtime mlr -j cat ~/data/big.json > /dev/null
TIME IN SECONDS 9.599 -- mlr -j cat /Users/kerl/data/big.json

$ justtime apps/go/tryparse -multi -noast g:json ~/data/big.jsonl > /dev/null
TIME IN SECONDS 10.502 -- apps/go/tryparse -multi -noast g:json /Users/kerl/data/big.jsonl

$ justtime apps/go/tryparse -multi g:json ~/data/big.jsonl > /dev/null
TIME IN SECONDS 15.796 -- apps/go/tryparse -multi g:json /Users/kerl/data/big.jsonl

...
```

# Miller to-do

* Let `-f` for `cut` et al. handle `foo.bar` etc.
* Re-think `emit`: make it a function with reshaping-helper functions
* `print` update ...
* Syntax errors / expected-one-of
* Group-by-and-apply
* CSV error/weirdness-handling: die immediately; skip and continue; accept more gracefully

# Tools to-do

* CI:
  * Copilot reviews
* UX:
  * No D on ^D at CLI
* Efficiency (memory and/or CPU):
  * Scale-test everything for perf early on -- especially channel-switching
  * DFA minimization
* Docs:
  * Doc `tryparse` with `-tokens`, `-states`, `-stack`.
  * ??  `GOPROXY=direct go mod tidy`
* Code-neatens:
  * Move types to tops of files
  * Dedupe
  * Re-organize and grok the lexer/parser tables etc.
  * Prohibit any parent/child indices the same
  * Allow separate `.bnf` files for lex and parse? In case of all parser bits identical (e.g. `PEMDAS*`).
  * AST to-string factored out of printer
    * Use for UT
  * [Maybe] Wrap lexers in LA1 and LA2 etc for lookahead level:
    * Hide direct calls to Scan``
    * `.First()`
    * `.Second()`
    * `.Advance()`
* Implmementation languages:
  * Catch more things up to Python
  * Catch more things up to JS
* Misc:
  * Hand-written recursive-ascent impls for simple grammars?

# Credits

* The Dragon book
* https://github.com/goccmack/gocc
* https://go.dev/talks/2011/lex.slide#1
* Cursor
