# arbtt – The Automatic Rule-Base Time Tracker

arbtt is a background daemon that stores which windows are open, which one has the focus and how long since your last action (and possibly more sources later), and stores this. It is also a program that will, based on expressive rules you specify, derive what you were doing, and what for.

It is comparable to the window trackers RescueTime, selfspy, TimeSnapper, and Productive Peach; but it differs from the manual timetrackers like Project Hamster which require the user to type a description of their activities.

The log file might contain very sensitive private data. Make sure you understand the consequences of a full-time logger and be careful with this data.

## Installation

### Building with `cabal-install`

arbtt comes in the form of a Cabalized

Cabal is the common software packaging for Haskell programs and libraries, see .

package, and is available from hackage. The easiest way of obtaining and installing arbtt is via `cabal-install`. If you have `cabal-install` available, just run

    $ cabal install arbtt

to download, build and install arbtt.

### Building without `cabal-install`

You can fetch the latest arbtt source tarball from hackage, at . Extract the tarball and run the following commands to build and install the arbtt binaries:

    $ runhaskell Setup.hs configure
    $ runhaskell Setup.hs build
    $ runhaskell Setup.hs install

### Setting up the capture program

To have arbtt gather useful data, you need to make sure that `arbtt-capture` is started with your X session. If you use GNOME or KDE, you can copy the file `arbtt-capture.desktop` to `~/.config/autostart/`. You might need to put the full path to `arbtt-capture` in the Exec line there, if you did not do a system wide installation.

By default, `arbtt-capture` will save one data sample per minute. If you want to change that, you can pass `--sample-rate `RATE`` to `arbtt-capture`, where `RATE` specifies the sample rate in seconds.

### Building the documentation

Obviously, you can already read the documentation. If you still want to build it yourself, enter the directory `doc/` and run `make` for the documentation in HTML and PDF format.

### Development version

If you want to try the latest unreleased state of the code, or want to contribute to arbtt, you can fetch the code with

    darcs get

## Configuring the arbtt categorizer (`arbtt-stats`)

Once `arbtt-capture` is running, it will record data without any configuration. And only to analyze the recorded data, one needs to configure the categorizer. Everytime the categorizer (`arbtt-stats`) runs, it applies categorization rules to all recorded data and tags it accordingly. Thus, if you improve your categorization rules later, they will apply also to all previous data samples!

### Configuration example

The configuration file needs to be placed in `~/.arbtt/categorize.cfg`. An example is included in the source distribution, and it is reproduced here: see catex. It should be more enlightening than a formal description.

#### `categorize.cfg`

### The semantics (informal)

A data sample consists of the time of recording, the time passed since the user’s last action, the name of the current workspace and the list of windows. For each window this information is available:

- the window title

- the program name

- whether the window was the active window

Based on this information and on the rules in `categorize.cfg`, the categorizer (`arbtt-stats`) assigns tags to each sample.

A simple rule consists of a condition followed by an arrow (`==>`) and a tag expression (`tag` keyword followed by tag name). The rule ends with a coma (`,`).

The keyword `tag`, usually preceded with a condition, assigns a tag to the sample; `tag` keyword is followed by a tag name (any sequence of alphanumeric symbols, underscores and hyphens). If tag name contains a colon (`:`), the first part of the name before the colon, is considered to be tag category.

For example, this rule

    month $date == 1 ==> tag month:January,

if it succeeds, assigns a the tag `January` in the category `month`.

If the tag has a category, it will only be assigned if no other tag of that category has been assigned. This means that for each sample and each category, there can be at most only one tag in that category. Tags can contain references to group matches in the regular expressions used in conditions (`$1`, `$2`)...). Tags can also reference some variables such as window title (`$current.title`) or program name (`$current.program`).

The variable `$idle` contains the idle time of the user, measured in seconds. Usually, it is used to assign the tag `inactive`, which is handled specially by `arbtt-stats`, as can be seen in catex.

When applying the rules, the categorizer has a notion of the window in scope, and the variables `$title`, `$program` and `$active` always refer to the window in scope. By default, there is no window is in scope. Condition should be prefixed with either `current window` or `any window`, to define scope of these variables.

The name of the current desktop (or workspace) is available as `$desktop`.

For `current window`, the currently active window is in scope. If there is no such window, the condition is false.

For `any window`, the condition is applied to each window, in turn, and if any of the windows matches, the result is true. If more than one window matches it is not defined from which match the variables `$1`... are taken from (see more about regular expressions below).

The variable `$time` refers to the time-of-day of the sample (i.e. the time since 00:00 that day, local time), while `$sampleage` refers to the time span from when the sample was recored until now, the time of evaluating the statistics. The latter variable is especially useful when passed to the `--filter` option of `arbtt-stats`. They can be compared with expressions like "hh:mm", for example

    $time >=  8:00 && $time < 12:00 ==> tag time-of-day:morning

The variable `$date` refers to the date and time of the recorded sample. It can be compared with date literals in the form YYYY-MM-DD (which stand for midnight, so

    $date ==
          2001-01-01

will not do what you want, but

    $date >= 2001-01-01 && $date <= 2001-01-02

would). All dates are evaluated in local time.

Expression `format $date` evaluates to a string with a date formatted according to ISO 8601, i.e. like "YYYY-MM-DD". The 19th of March 2010 is formatted as "2010-03-19". Formatted date can be compared to strings. Formatted dates may be useful to tag particular date ranges. But also note that this is a rather expensive operation that can slow down your data processing.

Expression `month $date` evaluates to an integer, from 1 to 12, corresponding to the month number. Expression `year $date` evaluates to an integer which is a year number. Expression `day of month $date` evaluates to an integer, from 1 to 31, corresponding to the day of month. Expression `day of week $date` evaluates to an integer, from 1 to 7, corresponding to the day of week, Monday is 1, Sunday is 7. These expressions can be compared to integers.

Expressions can be compared to literal values with `==` (equal), `/=` (not equal), `<`, `<=`, `>=`, `>` operators. String expressions (`$program`, `$title`) can be matched against regular expressions with `=~` operator. With these operatorions, the right hand side can be a comma-separated list of literals enclosed in square brackets (`[` ..., ..., `]`), which succeeds if any of them succeeds.

Regular expressions are written either between slashes (`/` regular expression `/`), or after a letter `m` followed by any symbol (`m` c regular expression c, where c is any symbol). The second appearance of that symbol ends the expression. You can find both variants in catex.

Complex conditions may be constructed from the simpler ones, using Boolean AND (`&&`), OR (`||`), and NOT (`!`) functions and parentheses.

You can define short-hand names for conditions using `condition`:

            condition arbtt = current window $title =~ m/arbtt/ in {
              $arbtt && $time < 14:00 ==> tag arbtt-morning,
              $arbtt && $time > 14:00 ==> tag arbtt-afternoon
            }

Everything that is a valid condition is assignable and you can reference bound variables in rules by prefixing them with a dollar (`$`).

### The syntax

`categorize.cfg` is a plain text file. Whitespace is insignificant and Haskell-style comments are allowed. A formal grammar is provided in grammar.

#### The formal grammar of `categorize.cfg`

Rules [ ] ( (, )* | ( ; )* ) AliasSpec aliases ( (, )* ) Alias Literal -> Literal Rule { } ==> | if then else tag Cond ( ) ! | && | || $active [ ] =~ =~ [ ] current window any window $ Literal String $title $program $desktop format " string literal " ListOfString " string literal " " string literal " , Number $idle day of week day of month month year number literal Date $date $now TimeDiff $time $sampleage ( Digit )* Digit : Digit Digit Tag [ Literal : ] Literal RegEx / Literal / | m `c` Literal `c` Where `c` can be any character. ListOfRegex " " " " , CmpOp <= | < | == | != | > | >= ConditionBinding condition Literal = in

A `String` refers to a double-quoted string of characters, while a `Literal` is not quoted. Tags may only consist of letters, dashes and underscores, or variable interpolations. A Tag maybe be optionally prepended with a category, separated by a colon. The category itself follows he same lexical rules as the tag. A variable interpolation can be one of the following:

`$1`, `$2`,...

- will be replaced by the respective group in the last successfully applied regular expression in the conditions enclosing the current rule.

`$current.title`

`$current.program`

- will be replaced by title the currently active window, resp. by the name of the currently active program. If no window happens to be active, this tag will be ignored.

A regular expression is, like in perl, either enclosed in forward slashes or, alternatively, in any character of your choice with an `m` (for match) in front. This is handy if you need to use regular expressions that match directory names. Otherwise, the syntax of the regular expressions is that of perl-compatible regular expressions.

## Effective Use of Arbtt

Now that the syntax has been described and the toolbox laid out, how do you practically go about using and configuring arbtt?

### Enabling data collection

After installing arbtt, you need to configure it to run. There are many ways you can run the `arbtt-capture` daemon. One standard way is to include the command

    arbtt-capture &

in your desktop environments startup script, e.g. `~/.xinitrc` or similar.

Another trick is add it as a `cron` job. To do so, edit your crontab file (`crontab -e`) and add a line like this:

    DISPLAY=:0
    @reboot arbtt-capture --logfile=/home/username/doc/arbtt/capture.log

At boot, `arbtt-capture` will be run in the background and will capture a snapshot of the X metadata for active windows every 60 seconds (the default). If you want more fine-grained time data at the expense of doubling storage use, you could increase the sampling rate with an option like `--sample-rate=30`. To be resilient to any errors or segfaults, you could also wrap it in an infinite loop to restart the daemon should it ever crash, with a command like

    DISPLAY=:0
    @reboot while true; do arbtt-capture --sample-rate=30; sleep 1m; done

### Checking data availability

arbtt tracks X properties like window title, class, and running program, and you write rules to classify those strings as you wish; but this assumes that the necessary data is present in those properties.

For some programs, this is the case. For example, web browsers like Firefox typically set the X title to the HTML `<title>` element of the web page in the currently-focused tab, which is enough for classification.

Some programs have title-setting available as plugins. The IRC client irssi in a GNU screen or X terminal usually sets the title to just "`irssi`", which blocks more accurate time-classification based on IRC channel (one channel may be for recreation, another for programming, and yet another for work), but can be easily configured to set the title using the extension `title.pl`.

Some programs do not set titles or class, and all arbtt sees is empty strings like `""`; or they may set the title/class to a constant like `"Liferea"`, which may be acceptable if that program is used for only one purpose, but if it is used for many purposes, then you cannot write a rule matching it without producing highly-misleading time analyses. (For example, a web browser may be used for countless purposes, ranging from work to research to music to writing to programming; but if the web browser's title/class were always just `"Web browser"`, how would you classify 5 hours spent using the web browser? If the 5 hours are classified as any or all of those purposes, then the results will be misleading garbage - you probably did not spend 5 hours just listening to music, but a mixture of those purposes, which changes from day to day.)

You should check for such problematic programs upon starting using arbtt. It would be unfortunate if you were to log for a few months, go back for a detailed report for some reason, and discover that the necessary data was never available for arbtt to log!

These programs can sometimes be customized internally, a bug report filed with the maintainers, or their titles can be externally set by `wmctrl` or `xprop`.

#### `xprop`

You can check the X properties of a running window by running the command `xprop` and clicking on the window; `xprop` will print out all the relevant X information. For example, the output for Emacs might look like this

    $ xprop | tail -5
    WM_CLASS(STRING) = "emacs", "Emacs"
    WM_ICON_NAME(STRING) = "emacs@elan"
    _NET_WM_ICON_NAME(UTF8_STRING) = "emacs@elan"
    WM_NAME(STRING) = "emacs@elan"
    _NET_WM_NAME(UTF8_STRING) = "emacs@elan"

This is not very helpful: it does not tell us the filename being edited, the mode being used, or anything. You could classify time spent in Emacs as "programming" or "writing", but this would be imperfect, especially if you do both activities regularly. However, Emacs can be customized by editing `~/.emacs`, and after some searching with queries like "setting Emacs window title", the Emacs wiki and manual advise us to put something like this Elisp in our `.emacs` file:

    (setq frame-title-format "%f")

Now the output looks different:

    $ xprop | tail -5
    WM_CLASS(STRING) = "emacs", "Emacs"
    WM_ICON_NAME(STRING) = "/home/gwern/arbtt.page"
    _NET_WM_ICON_NAME(UTF8_STRING) = "/home/gwern/arbtt.page"
    WM_NAME(STRING) = "/home/gwern/arbtt.page"
    _NET_WM_NAME(UTF8_STRING) = "/home/gwern/arbtt.page"

With this, we can usefully classify all such time samples as being “writing”:

    current window $title == "/home/gwern/arbtt.page" ==> tag Writing,

Another common gap is terminals/shells: they often do not include information in the title like the current working directory or last shell command. For example, urxvt/Bash:

    WM_COMMAND(STRING) = { "urxvt" }
    _NET_WM_ICON_NAME(UTF8_STRING) = "urxvt"
    WM_ICON_NAME(STRING) = "urxvt"
    _NET_WM_NAME(UTF8_STRING) = "urxvt"
    WM_NAME(STRING) = "urxvt"

Programmers may spend many hours in the shell doing a variety of things (like Emacs), so this is a problem. Fortunately, this is also solvable by customizing one's `.bashrc` to set the prompt to emit an escape code interpreted by the terminal (baroque, but it works). The following will include the working directory, a timestamp, and the last command:

    trap 'echo -ne "\033]2;$(pwd); $(history 1 | sed "s/^[ ]*[0-9]*[ ]*//g")\007"' DEBUG

Now the urxvt samples are useful:

    _NET_WM_NAME(UTF8_STRING) = "/home/gwern/wiki; 2014-09-03 13:39:32 arbtt-stats --help"

Some distributions (e.g. Debian) already provide the relevant configuration for this to happen. If it does not work for you, you can try to add

    . /etc/profile.d/vte.sh

to your `~/.bashrc`.

A rule could classify based on the directory you are working in, the command one ran, or both. Other shells like zsh can be fixed this way too but the exact command may differ; you will need to research and experiment.

Some programs can be tricky to set. The X image viewer feh has a `--title` option but it cannot be set in the configuration file, `.config/feh/themes`, because it needs to be specified dynamically; so you need to set up a shell alias or script to wrap the command like `feh --title "$(pwd) / %f / %n"`.

#### Raw samples

`xprop` can be tedious to use on every running window and you may forget to check seldomly used programs. A better approach is to use `arbtt-stats`’s `--dump-samples` option: this option will print out the collected data for specified time periods, allowing you to examine the X properties en masse. This option can be used with the `--exclude=` option to print the samples for samples not matched by existing rules as well, which is indispensable for improving coverage and suggesting ideas for new rules. A good way to figure out what customizations to make is to run arbtt as a daemon for a day or so, and then begin examining the raw samples for problems.

##### An initial configuration session

An example: suppose I create a simple category file named `foo` with just the line

    $idle > 30 ==> tag inactive

I can then dump all my arbtt samples for the past day with a command like this:

    arbtt-stats --categorizefile=foo --m=0 --filter='$sampleage <24:00' --dump-samples

Because there are so many open windows, this produces a large amount (26586 lines) of hard-to-read output:

    ...
    ( ) Navigator:      /r/Touhou's Favorite Arranges! Part 71: Retribution for the Eternal Night ~ Imperishable Night : touhou - Iceweasel
    ( ) Navigator:      Configuring the arbtt categorizer (arbtt-stats) - Iceweasel
    ( ) evince:         ATTACHMENT02
    ( ) evince:         2009-geisler.pdf — Heart rate variability predicts self-control in goal pursuit
    ( ) urxvt:          /home/gwern; arbtt-stats --categorizefile=foo --m=0 --filter='$sampleage <24:00' --dump-samples
    ( ) mnemosyne:      Mnemosyne
    ( ) urxvt:          /home/gwern; 2014-09-03 13:11:45 xprop
    ( ) urxvt:          /home/gwern; 2014-09-03 13:42:17 history 1 | cut --delimiter=' ' --fields=5-
    ( ) urxvt:          /home/gwern; 2014-09-03 13:12:21 git log -p .emacs
    (*) emacs:          emacs@elan
    ( ) urxvt:          /home/gwern; 2014-09-01 14:50:30 while true; do cd ~/ && getmail_fetch --ssl pop.gmail.com gwern0 'ugaozoumbhwcijxb' ./mail/; done
    ( ) urxvt:          /home/gwern/blackmarket-mirrors/silkroad2-forums; 2014-08-31 23:20:10 mv /home/gwern/cookies.txt ./; http_proxy="localhost:8118" wget...
    ( ) urxvt:          /home/gwern/blackmarket-mirrors/agora; 2014-08-31 23:15:50 mv /home/gwern/cookies.txt ./; http_proxy="localhost:8118" wget --mirror ...
    ( ) urxvt:          /home/gwern/blackmarket-mirrors/evolution-forums; 2014-08-31 23:04:10 mv ~/cookies.txt ./; http_proxy="localhost:8118" wget --mirror ...
    ( ) puddletag:      puddletag: /home/gwern/music

Active windows are denoted by an asterisk, so I can focus & simplify by adding a pipe like `| fgrep '(*)'`, producing more manageable output like

    (*) urxvt:          irssi
    (*) urxvt:          irssi
    (*) urxvt:          irssi
    (*) Navigator:      Pyramid of Technology - NextNature.net - Iceweasel
    (*) Navigator:      Search results - gwern0@gmail.com - Gmail - Iceweasel
    (*) Navigator:      [New comment] The Wrong Path - gwern0@gmail.com - Gmail - Iceweasel
    (*) Navigator:      Iceweasel
    (*) Navigator:      Litecoin Exchange Rate - $4.83 USD - litecoinexchangerate.org - Iceweasel
    (*) Navigator:      PredictionBook: LiteCoin will trade at >=10 USD per ltc in 2 years, - Iceweasel
    (*) urxvt:          irssi
    (*) Navigator:      Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...)
    (*) Navigator:      Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...)
    (*) Navigator:      Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...)
    (*) urxvt:          /home/gwern; 2014-09-02 14:25:17 man s3cmd
    (*) evince:         bayesiancausality.pdf
    (*) evince:         bayesiancausality.pdf
    (*) puddletag:      puddletag: /home/gwern/music
    (*) puddletag:      puddletag: /home/gwern/music
    (*) evince:         bayesiancausality.pdf
    (*) Navigator:      ▶ Umineko no Naku Koro ni Music Box 4 - オルガン小曲 第2億番 ハ短調 - YouTube - Iceweasel
    ...

This is better. We can see a few things: the windows all now produce enough information to be usefully classified (Gmail can be classified under email, irssi can be classified as IRC, the urxvt usage can clearly be classified as programming, the PDF being read is statistics, etc) in part because of customizations to bash/urxvt. The duplication still impedes focus, and we don't know what's most common. We can use another pipeline to sort, count duplicates, and sort by number of duplicates (`| sort | uniq --count | sort --general-numeric-sort`), yielding:

     ...
     14     (*) Navigator:      A Bluer Shade of White Chapter 4, a frozen fanfic | FanFiction - Iceweasel
     14     (*) Navigator:      Iceweasel
     15     (*) evince:         2009-geisler.pdf — Heart rate variability predicts self-control in goal pursuit
     15     (*) Navigator:      Tool use by animals - Wikipedia, the free encyclopedia - Iceweasel
     16     (*) Navigator:      Hacker News | Add Comment - Iceweasel
     17     (*) evince:         bayesiancausality.pdf
     17     (*) Navigator:      Comments - Less Wrong Discussion - Iceweasel
     17     (*) Navigator:      Keith Gessen · Why not kill them all?: In Donetsk · LRB 11 September 2014 - Iceweasel
     17     (*) Navigator:      Notes on the Celebrity Data Theft | Hacker News - Iceweasel
     18     (*) Navigator:      A Bluer Shade of White Chapter 1, a frozen fanfic | FanFiction - Iceweasel
     19     (*) gl:             mplayer2
     19     (*) Navigator:      Neural networks and deep learning - Iceweasel
     20     (*) Navigator:      Harry Potter and the Philosopher's Zombie, a harry potter fanfic | FanFiction - Iceweasel
     20     (*) Navigator:      [OBNYC] Time tracking app - gwern0@gmail.com - Gmail - Iceweasel
     25     (*) evince:         ps2007.pdf — untitled
     35     (*) emacs:          /home/gwern/arbtt.page
     43     (*) Navigator:      CCC comments on The Octopus, the Dolphin and Us: a Great Filter tale - Less Wrong - Iceweasel
     62     (*) evince:         The physics of information processing superobjects - Anders Sandberg - 1999.pdf — Brains2
     69     (*) liferea:        Liferea
     82     (*) evince:         BMS_raftery.pdf — untitled
     84     (*) emacs:          emacs@elan
     87     (*) Navigator:      overview for gwern - Iceweasel
    109     (*) puddletag:      puddletag: /home/gwern/music
    150     (*) urxvt:          irssi

Put this way, we can see what rules we should write to categorize: we could categorize the activities here into a few categories of "recreational", "statistics", "music", "email", "IRC", "research", and "writing"; and add to the `categorize.cfg` some rules like thus:

    $idle > 30 ==> tag inactive,

    current window $title =~ [/.*Hacker News.*/, /.*Less Wrong.*/, /.*overview for gwern.*/, /.*[fF]an[fF]ic.*/, /.* LRB .*/]
      || current window $program == "liferea" ==> tag Recreation,
    current window $title =~ [/.*puddletag.*/, /.*mplayer2.*/] ==> tag Music,
    current window $title =~ [/.*[bB]ayesian.*/, /.*[nN]eural [nN]etworks.*/, /.*ps2007.pdf.*/, /.*[Rr]aftery.*/] ==> tag Statistics,
    current window $title =~ [/.*Wikipedia.*/, /.*Heart rate variability.*/, /.*Anders Sandberg.*/] ==> tag Research,
    current window $title =~ [/.*Gmail.*/] ==> tag Email,
    current window $title =~ [/.*arbtt.*/] ==> tag Writing,
    current window $title == "irssi" ==> tag IRC,

If we reran the command, we'd see the same output, so we need to leverage our new rules and exclude any samples matching our current tags, so now we run a command like:

    arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples --exclude=Recreation --exclude=Music --exclude=Statistics
                 --exclude=Research --exclude=Email --exclude=Writing --exclude=IRC |
                 fgrep '(*)' | sort | uniq --count | sort --general-numeric-sort

Now the previous samples disappear, leaving us with a fresh batch of unclassified samples to work with:

      9     (*) Navigator:      New Web Order > Nik Cubrilovic - - Notes on the Celebrity Data Theft - Iceweasel
      9     ( ) urxvt:          /home/gwern; arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples | fgrep '(*)' | less
     10     (*) evince:         ATTACHMENT02
     10     (*) Navigator:      These Giant Copper Orbs Show Just How Much Metal Comes From a Mine | Design | WIRED - Iceweasel
     12     (*) evince:         [Jon_Elster]_Alchemies_of_the_Mind_Rationality_an(BookFi.org).pdf — Alchemies of the mind
     12     (*) Navigator:      Morality Quiz/Test your Morals, Values & Ethics - YourMorals.Org - Iceweasel
     33     ( ) urxvt:          /home/gwern; arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples | fgrep '(*)'...

We can add rules categorizing these as 'Recreational', 'Writing', 'Research', 'Recreational', 'Research', 'Writing', and 'Writing' respectively; and we might decide at this point that 'Writing' is starting to become overloaded, so we'll split it into two tags, 'Writing' and 'Programming'. And then after tossing another `--exclude=Programming` into our rules, we can repeat the process.

As we refine our rules, we will quickly spot instances where the title/class/program are insufficient to allow accurate classification, and we will figure out the best collection of tags for our particular purposes. A few iterations is enough for most purposes.

### Categorizing advice

When building up rules, a few rules of thumb should be kept in mind:

#### Categorize by purpose, not by program

This leads to misleading time reports. Avoid, for example, lumping all web browser time into a single category named 'Internet'; this is more misleading than helpful. Good categories describe an activity or goal, such as 'Work' or 'Recreation', not a tool, like 'Emacs' or 'Vim'.

#### When in doubt, write narrow rules and generalize later

Regexps are tricky and it can be easy to write rules far broader than one intended. The `--exclude` filters mean that one will never see samples which are matched accidentally. If one is in doubt, it can be helpful to take a specific sample one wants to match and several similar strings and look at how well one's regexp rule works in Emacs's regexp-builder or online regexp-testers like regexpal.

#### Don't try to classify everything

You will never classify 100% of samples because sometimes programs do not include useful X properties and cannot be fixed, you have samples from before you fixed them, or they are too transient (like popups and dialogues) to be worth fixing. It is not necessary to classify 100% of your time, since as long as the most common programs and, say, 80% of your time is classified, then you have most of the value. It is easy to waste more time tweaking arbtt than one gains from increased accuracy or more finely-grained tags.

#### Avoid large and microscopic tags

If a tag takes up more than a third or so of your time, it is probably too large, masks variation, and can be broken down into more meaningful tags. Conversely, a tag too narrow to show up regularly in reports (because it is below the default 1% filter) may not be helpful because it is usually tiny, and can be combined with the most similar tag to yield more compact and easily interpreted reports.

### Long-term storage

Each halving of the sampling rate doubles the number of samples taken and hence the storage requirement; sampling rates below 20s are probably wasteful. But even the default 60s can accumulate into a nontrivial amount of data over a year. A constantly-changing binary file can interact poorly with backup systems, may make arbtt analyses slower, and if one's system occasionally crashes or experiences other problems, cause some corruption of the log and be a nuisance in having to run `arbtt-recover`.

Thus it may be a good idea to archive one's `capture.log` on an annual basis. If one needs to query the historical data, the particular log file can be specified as an option like `--logfile=/home/gwern/doc/arbtt/2013-2014.log`

### External processing of arbtt statistics

arbtt supports CSV export of time by category in various levels of granularity in a 'long' format (multiple rows for each day, with n row specifying a category's value for that day). These CSV exports can be imported into statistical programs like R or Excel and manipulated as desired.

R users may prefer to have their time data in a 'wide' format (each row is 1 day, with n columns for each possible category); this can be done with the `reshape` default library. After reading in the CSV, the time-intervals can be converted to counts and the data to a wide data-frame with R code like the following:

    arbtt <- read.csv("arbtt.csv")
    interval <- function(x) { if (!is.na(x)) { if (grepl(" s",x)) as.integer(sub(" s","",x))
                                              else { y <- unlist(strsplit(x, ":"));
                                                     as.integer(y[[1]])*3600 + as.integer(y[[2]])*60 + as.integer(y[[3]]); }
                                                     }
                             else NA
                             }
    arbtt$Time <- sapply(as.character(arbtt$Time), interval)
    library(reshape)
    arbtt <- reshape(arbtt, v.names="Time", timevar="Tag", idvar="Day", direction="wide")

### Selecting the current day

This can be done easily with a bit of help from the command line:

    arbtt-stats  --filter='$date>='`date +"%Y-%m-%d"`

## Contributed tools

Due to the import and export facilities of arbtt (as explained in external-processing), tools producing and analyzing arbtt’s data can be developed independently. This section lists those that we are aware of. If you create a tool of your own, or find one somewhere, please tell us on the mailing list (see copyright).

### arbtt-graph

Jayesh Kumar Gupta created a nice d3-based visualization of your arbtt data, including daily pie charts and barcode graphs.

You can find his tool on https://github.com/rejuvyesh/arbtt-graph.

### arbtt-capture for GNOME Shell

An arbtt-capture implemented as a GNOME Shell extension. Its main value proposition is compatibility with Wayland (but it is compatible with traditional X Window System as well). You can find it on https://github.com/tmiasko/arbtt-capture.

## Program references

arbtt consists of a few command line tools, the most important one is `arbtt-stats`. This overview is followed by their manual pages.

To generate statistics about the data that `arbtt-capture` has gathered, use the program `arbtt-stats`. A detailed description of the possible options is given in arbtt-stats.

The collection of data is done by `arbtt-capture`. Usually, you only set it up once to run automatically, as described in installation, and do not have to worry about it again. Its command line reference is given in arbtt-capture.

Because the data log file is binary, a tool names `arbtt-dump` can bee used to dump the data in various textual formats. Its command line reference is given in arbtt-dump.

The textual form can be imported back using `arbtt-import`. Its command line reference is given in arbtt-import.

If `arbtt-capture` crashes it might be that the log file is not readable any more. In some cases, this can be fixed using the (relatively raw) `arbtt-recover` tool. Its command line reference is given in arbtt-recover.

arbtt-stats

1 arbtt manual

arbtt-stats

generate statistics from the arbtt data samples

            arbtt-stats
            OPTION

#### Description

`arbtt-stats` reads the samples that were recorded so far by `arbtt-capture` from the log file, filters them according to the users specifications and generates one or more reports from the data.

When run without any options, `--total-time` is assumed.

The order in which filters (`--exclude`, `--only`, `--also-inactive` and `--filter`) and reports are passed to the program is irrelevant: All filters given on the command line are active for all reports.

#### Options

`-h`

`-?`

`--help`

- shows a short summary of the available options, and exists.

`-V`

`--version`

- shows the version number, and exists.

`--logfile` `FILE`

- logfile to use instead of `~/.arbtt/capture.log`

`--categorizefile`

- categorize file to use instead of `~/.arbtt/categorize.cfg`

##### Filtering options

`-x` `TAG`

`--exclude` `TAG`

- Ignore any data samples that have been assigned this tag or category. To distinguish tags and categories, the latter have to be entered followed by a colon. Can be given more than once.

`-o` `TAG`

`--only` `TAG`

- Ignore any data samples that have not been assigned this tag. To distinguish tags and categories, the latter have to be entered followed by a colon. Can be given more than once.

`--also-inactive`

- by default, `arbtt-stats` ignores any samples which have been assigned the tag `inactive`. This flag disables this behaviour.

`-f` `CONDITION`

`--filter` `CONDITION`

- Only consider samples matching the given condition, which follows the same syntax as in `categorize.cfg` (Nonterminal in grammar the formal grammar specification found in the user guide).

##### Report options

`-m` `PERCENTAGE`

`--min-percentage` `PERCENTAGE`

- Ignore tags whose percentage is less than the value specified here. Default percentage: 1%.

`--output-exclude` `TAG`

- Skip this tag or category when printing statistics. Only affects the reports `--total-time` and `--category`. To distinguish tags and categories, the latter have to be entered followed by a colon. Can be given more than once.

`--output-only` `TAG`

- Prints statistics only for the specified tag or category. Only affects the reports `--total-time` and `--category`. To distinguish tags and categories, the latter have to be entered followed by a colon. Can be given more than once.

`--output-format` `FORMAT`

- Specify the report output format, may be one of: text, csv (comma-separated values), tsv (TAB-separated values). Default format: text.

##### Reports

`-i`

`--information`

- Various bits of information about the recorded data, such as total time recorded, number of records etc. In this report, time recorded is the sum of all samples, including inactive and those that are disabled by the filter, while time selected is the sum of the samples that are matched by the given filters.

`-t`

`--total-time`

- For all tags, print the part of the selected time with this tag applied to, both as an absolute value and a percentage of the selected time.

`-c` `CATEGORY`

`--category` `CATEGORY`

- For the given category, give the textual equivalent of a pie chart: For each possible value of the category, including one for no tag of this category present, give the absolute time and fraction. Entries which are not displayed because of the option `--min-percentage` are aggregated.

`--each-category`

- This is just a shortcut for a series of `--category` options, one for each category found in the data.

`--intervals` [`TAG`|`CATEGORY`:]

- This report lists all periods of consecutive time intervals where the given tag has been applied to, or where the given category has the same value.

  To distinguish tags and categories, the latter have to be entered followed by a colon.

  This report will give wrong results if an activity has been carried out at the end of a session and right at the beginning, as the intermediate time is thought to be part of the interval. Inactive times while `arbtt-capture` is running will separate the results as expected.

`--for-each` `PERIOD`

- This is not a report of its own, but causes the selected report to be executed for each of the given PERIOD (which can be minute, hour,day, month or year) where there exist selected samples. All the reports will then be printed one after another or, in the case of CSV output, with an additional column.

  Note that if this option is enabled, samples that are filtered out are completely ignored (to avoid empty reports for periods with only filtered samples). Therefore, the `--information` report will print the numbers for the samples selected also for the totals.

#### Examples

Some useful examples of what you can do with `arbtt-stats` are provided here:

    # Only consider the time when I was programming in Haskell
    arbtt-stats -o Editing-Haskell

    # Tell me what evolution folders I spend my time in when I actually do
    # work with e-Mail
    arbtt-stats -o Program:evolution -c Evo-Folder

    # Generate statistics about the last hour
    arbtt-stats -f '$sampleage < 1:00'

#### Files

`~/.arbtt/capture.log`

- binary file, storing the arbtt data samples

`~/.arbtt/categorize.cfg`

- specification of the arbtt categorizer syntax. A detailed description is given in configuration

#### See also

See the arbtt manual for more information and the arbtt hackage page for newer versions of arbtt.

arbtt-capture

1 arbtt manual

arbtt-capture

collect data samples for arbtt

            arbtt-capture
            OPTION

#### Description

`arbtt-capture` runs continuously and saves at the given sample rate, usually once per minute, the collected data to `~/.arbtt/capture.log`.

#### Options

`-h`

`-?`

`--help`

- shows a short summary of the available options, and exists.

`-V`

`--version`

- shows the version number, and exists.

`-r` `RATE`

`--sample-rate` `RATE`

- sets the sample rate in seconds (default: 60)

`-f` `FILE`

`--logfile` `FILE`

- logfile to use instead of `~/.arbtt/capture.log`

`-d`

`--dump`

- dump one current sample instead of modifying the logfile

#### Files

`~/.arbtt/capture.log`

- binary file, storing the arbtt data samples

#### See also

See the arbtt manual for more information and the arbtt hackage page for newer versions of arbtt.

arbtt-dump

1 arbtt manual

arbtt-dump

dumps arbtt data samples

            arbtt-dump
            OPTION

#### Description

`arbtt-dump` reads the data samples recorded by arbtt-capture and writes them so the standard output in an ascii based format.

#### Options

`-h`

`-?`

`--help`

- shows a short summary of the available options, and exists.

`-V`

`--version`

- shows the version number, and exists.

`-f` `FILE`

`--logfile` `FILE`

- logfile to use instead of `~/.arbtt/capture.log`

`-t` `FORMAT`

`--format` `FORMAT`

- dumping format to use, where `FORMAT` is one of `human` (the default), `show` or `JSON`. Case in-sensitive.

`-l` `NUMBER`

`--last` `NUMBER`

- dump only the last `NUMBER` of samples.

#### Files

`~/.arbtt/capture.log`

- binary file, storing the arbtt data samples

#### Formats

##### Human

This format is intended for human inspection, but not for further processing. Hence, it may change in new versions of arbtt without notice. Example output:

    2013-06-20 14:53:50 (48ms inactive):
        ( ) Navigator:      arbtt-dump - Iceweasel
        ( ) gnome-terminal-server: jojo@kirk:~/projekte/programming/arbtt/doc
        (*) gvim:           arbtt.xml + (~/projekte/programming/arbtt/doc) - GVIM2

The line with a star indicates the currently active window.

##### Show

This is the default serialization format of Haskell's `Show` type class, one entry per line. This can be useful if the data is to be processed by further Haskell code. Example output, with indentation added manually:

    TimeLogEntry
        { tlTime = 2013-06-20 14:53:50.957763 UTC
        , tlRate = 60000
        , tlData = CaptureData
            { cWindows =
                [ (False,"arbtt-dump - Iceweasel","Navigator")
                , (False,"jojo@kirk:~/projekte/programming/arbtt/doc","gnome-terminal-server")
                , (True,"arbtt.xml + (~/projekte/programming/arbtt/doc) - GVIM2","gvim")
                ]
            , cLastActivity = 48
            }
        }

##### JSON

For interoperability, arbtt supports dumping its data to JSON, which can easily be parsed by many different programming languages. Some level of backward-compatibility will be provided, as far as possible. Default output, again with indentation and spacing added manually:

    [ ...,
      { "windows": [
          { "program": "arbtt-dump - Iceweasel",
            "title": "Navigator",
            "active": false},
          { "program": "jojo@kirk:~/projekte/programming/arbtt/doc",
            "title":" gnome-terminal-server",
            "active": false},
          { "program": "arbtt.xml + (~/projekte/programming/arbtt/doc) - GVIM2",
            "title": "gvim",
            "active":true
          }],
        "inactive": 48,
        "date": "2013-06-20T14:53:50.957Z",
        "rate": 60000},
      ...
    ]

#### See also

See the arbtt manual for more information and the arbtt hackage page for newer versions of arbtt.

arbtt-import

imports dumped arbtt data samples

            arbtt-import
            OPTION

#### Description

`arbtt-import` reads data from standard input and adds the entries to the logfile. It supports the `Show` format as well as the `JSON` format, and can be used in a streaming fashion (i.e. it will write entries as they are read from standard input).

By default, this program completely overrides the existing file, therefore it will refuse to work if the log file already exists. If you want to overwrite a file, please delete it before running `arbtt-import`. If you want to extend an existing file, use the `--append` option.

#### Options

`-h`

`-?`

`--help`

- shows a short summary of the available options, and exists.

`-V`

`--version`

- shows the version number, and exists.

`-f` `FILE`

`--logfile` `FILE`

- logfile to use instead of `~/.arbtt/capture.log`

`-a`

`--append`

- append the log file, if it already exists

`-t` `FORMAT`

`--format` `FORMAT`

- format to use, where `FORMAT` is one of `show` (the default) or `JSON`. The JSON format can either be a list of records (as dumped by arbtt-dump), or just one record after another (useful in streaming mode).

#### Files

`~/.arbtt/capture.log`

- binary file, storing the arbtt data samples

#### See also

See the arbtt manual for more information and the arbtt hackage page for newer versions of arbtt.

arbtt-recover

1 arbtt manual

arbtt-recover

tries to recover a broken arbtt data log

            arbtt-recover
            OPTION

#### Description

`arbtt-recover` tries to read the data samples recorded by arbtt-capture, skipping over possible broken entries. A fixed log file is written to `~/.arbtt/capture.log.recovered`. If the recovery was successful, you should stop `arbtt-capture` and move the file to `~/.arbtt/capture.log`.

As a side effect, `arbtt-recover` applies the log compression method implemented in version 0.4.5 to the samples created by an earlier version. If you have a large logfile written by older versions, running `arbtt-recover` is recommended.

#### Options

`-h`

`-?`

`--help`

- shows a short summary of the available options, and exists.

`-V`

`--version`

- shows the version number, and exists.

`-i`

`--infile`

- logfile to use instead of `~/.arbtt/capture.log`

`-o`

`--outfile`

- where to save the recovered file, instead of `~/.arbtt/capture.log.recovered`

#### Files

`~/.arbtt/capture.log`

- binary file, storing the arbtt data samples

`~/.arbtt/capture.log.recovered`

- binary file, storing the fixed arbtt data samples

#### See also

See the arbtt manual for more information and the arbtt hackage page for newer versions of arbtt.

## Troubleshooting

### arbtt and xmonad

If you are using the xmonad window manager and arbtt does ont record any windows, ensure that your xmonad configuration includes the EWMH-Hints extensions in the module XMonad.Hooks.EwmhDesktops.

## Copyright and Contact

arbtt is Copyright © 2009-2010 Joachim Breitner

arbtt does not have a bug tracker yet. If you have bug reports, suggestions or questions, please send an email to the arbtt mailing list at arbtt@lists.nomeata.de, which you can subscribe at http://lists.nomeata.de/mailman/listinfo/arbtt.

### arbtt License

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

## Release Notes

The version history with changes relevant for the user is documented here.

### Version 0.10.2

- The category report now also has an entry `(total time)`, especially useful with `--for-each`.

- Support for GHC-8.0 and older has been dropped.

### Version 0.10.1

- Rodney Lorrimar contributed fixed to build against newer versions of dependencies.

- Michal J. Gajda ensured that intervals are also printed as local times.

### Version 0.10

- Markus Hauck contributed the `!=` operator, the $now variable and the `condition` construct (as well as other code cleanup).

- The development repository migrated from darcs to git.

- The command arbtt-import now supports reading JSON records, and can be used to append to an exising log file in a steaming fasion.

### Version 0.9

- The `--for-each` option of `arbtt-stats` now supports grouping results by minute or hour.

- Gwern Branwen contributed the effective-use.

### Version 0.8.1

- The syntax now allows for time differences larger than 99:99. (issue #14)

### Version 0.8

- `arbtt-dump` can now show the data in other formats as well, as suggested by Waldir Pimenta (option `--format`). This includes a human-readale output and JSON.

- New option `--last` of `arbtt-dump`.

- `arbtt-recover` can handle larger datasets with a reasonable amount of memory.

- When dumping samples with `arbtt-stats` or `arbtt-dump` (in human-readable mode), times are printed in the local time zone, as suggested by Oren Gampel.

- `arbtt-capture` now supports the option `--dump` for development and debugging purposes.

- The name of the current desktop (or workspace) is also recorded, and available as `$desktop` (issue #1).

- Two bugs in the parser were fixed (issue #4 and issue #5).

- `arbtt-stats` can print reports split by time interval, using the `--for-each` option.

- `arbtt-stats` can print the actual samples selected, with `--dump-samples`.

- Support for GHC-7.8 was added, with help from Jayesh Kumar Gupta (issue #8).

- Arbtt now has a proper testsuite integrated into Cabal, and an automated test infrastructure on Travis.

- Waldir Pimenta contributed a new web page, hosted at arbtt.nomeata.de.

### Version 0.7

- Make sure that the log file is only readable by the current user. Thanks to Joey Hess for pointing that out.

- Show a progress bar in `arbtt-stats`.

- GHC-7.6 compatibility, thanks to Isaac Dupree.

### Version 0.6.4.1

- Added missing module to the packages.

### Version 0.6.4

- Massive memory footprint reduction, due to processing the data in one run. See my blog post for technical information.

### Version 0.6.3

- Performance improvements.

- Support comparing a string to a list of strings, or matching it against a list ofregular expressions.

### Version 0.6.2

- Add a warning whtn the system locale is not supported.

- Allwo RTS options to be passed to the arbtt binaries.

- GHC 7.4 compatibility.

### Version 0.6.1

- Performance improvements.

### Version 0.6

- The command `arbtt-capture` now supports the `--logfile`.

- New report “intervals”, available using `arbtt-stats` `--intervals`.

- The paramters `--exclude` and `--include` of `arbtt-stats` can match categories as well as tags.

- Bugfix: Numbers in tag names are no longer replaced by an underscore.

- New paramters `--output-exclude` and `--output-include` of `arbtt-stats`.

### Version 0.5 (The ZuriHac-Release)

- New command `arbtt-import`, which imports the output from `arbtt-dump`.

- The command `arbtt-stats` now supports the `--logfile` and `--categorizefile` as well. (martin)

- The command `arbtt-stats` now supports the csv (comma-separated values) and tsv (TAB-separated values) report output formats in addition to text. (muharem)

- Unicode is handled correctly in regular expressions.

- Improved date-handling functions for `categorize.cfg`. (sergey)

### Version 0.4.5.1

- Bugfix: Added missing modules to the cabal file.

### Version 0.4.5

- Implement a custom compression method greatly reduce the file size of the log file. Run `arbtt-capture` to compress the previous samples as well.

### Version 0.4.4

- Bugfix: Correctly parse a tag name containing a colon when passed to `arbtt-stats` `--exclude`.

- Bugfix: Only warn once when the _NET_CLIENT_LIST X property is not set for the root window.

### Version 0.4.3

- Use fetchName from xmonad instead of xFetchName, as it works with unicode characters in window titles.

### Version 0.4.2

- Implement option `--logfile` to `arbtt-dump`.

- New command `arbtt-recover` to rescue data from a proken data log file.

- Actually include this documentation in the released tarball.

### Version 0.4.1

- Write this documentation

- Drop dependency on setlocale: Copy the SetLocale module.

- Drop dependency on tabular: Implement custom table rendering code.

- In the absence of _NET_CLIENT_LIST, look for application windows as children of the root windows. This should work for users of window managers like i3 without EWHM support.

### Version 0.4

- Implement option `--each-categories` to `arbtt-stats`

- Eliminate one possible cause for crashes of `arbtt-capture`.

### Version 0.3.0

- Switch to binary log file format, for greatly increased speed

- `arbtt-capture` will automatically detect and convert log files in the old format.

### Version 0.2.0

- Add option `--filter` to `arbtt-stats`

- Add option `--sample-rate` to `arbtt-capture`

- Introduce time-base variables `$time` and `$sampleage`

### Version 0.1.5

- Use `setlocale` to get umlauts in window titles correctly

### Version 0.1.4

- Be smarter when figuring out what window is active. Thanks to CJ van den Berg for investigating the issue.

### Version 0.1.3

- Read `_NET_CLIENT_LIST` for the list of applications, for compatibility with window managers such as metacity

### Version 0.1.2

- Off-By-Ten error in the time display

- Correctly show total number of records in `--information`

### Version 0.1.1

Rename files to allow building on MacOS.

### Version 0.1

Initial release of arbtt