Readme for analog -- other reports

[ Top | Up | Prev | Next | Map | Index ]

# Analog CE 6.0: Other reports

This section deals with the non-time reports. There are quite a lot of commands which control these reports, although we've seen some of them already.

First, these reports have `COLS` commands, just like the time reports. (See the section on Time reports for how to use these commands.) But for these reports, several additional columns are available. Here is the full list of columns for the non-time reports

`R`
  Number of requests
`r`
  Percentage of the requests
`S`
  Number of requests in the last 7 days
`s`
  Percentage of the requests in the last 7 days
`P`
  Number of page requests
`p`
  Percentage of the page requests
`Q`
  Number of page requests in the last 7 days
`q`
  Percentage of the page requests in the last 7 days
`B`
  Number of bytes transferred
`b`
  Percentage of the bytes
`C`
  Number of bytes transferred in the last 7 days
`c`
  Percentage of the bytes in the last 7 days
`d`
  Date of last access
`D`
  Date and time of last access
`e`
  Date of first access
`E`
  Date and time of first access
`N`
  The number of the item in the list

So, for example,

    REQCOLS NRSD

counts the files in the Request Report, listing the number of requests for each, the number of requests for each in the last 7 days, and the time when each was last requested. The full list of `COLS` commands for non-time reports is `HOSTCOLS`, `REDIRHOSTCOLS`, `FAILHOSTCOLS`, `ORGCOLS`, `DOMCOLS`, `REQCOLS`, `DIRCOLS`, `TYPECOLS`, `SIZECOLS`, `PROCTIMECOLS`, `REDIRCOLS`, `FAILCOLS`, `REFCOLS`, `REFSITECOLS`, `SEARCHQUERYCOLS`, `SEARCHWORDCOLS`, `INTSEARCHQUERYCOLS`, `INTSEARCHWORDCOLS`, `REDIRREFCOLS`, `FAILREFCOLS`, `BROWREPCOLS`, `BROWSUMCOLS`, `OSCOLS`, `VHOSTCOLS`, `REDIRVHOSTCOLS`, `FAILVHOSTCOLS`, `USERCOLS`, `REDIRUSERCOLS`, `FAILUSERCOLS` and `STATUSCOLS`. Not every column is allowed in every report, but if you specify an illegal one, analog will warn you about it.

Next you need to know how use a `SORTBY` command to specify how the reports should be sorted. There are ten possible ways of sorting reports:

`REQUESTS`
  total number of requests
`REQUESTS7`
  requests within the last 7 days
`PAGES`
  total requests for pages
`PAGES7`
  requests for pages within the last 7 days
`BYTES`
  total bytes transferred
`BYTES7`
  bytes transferred within the last 7 days
`FIRSTDATE`
  time of first request
`DATE`
  time of most recent request
`ALPHABETICAL`
  alphabetically
`RANDOM`
  unsorted, sometimes useful for speed in very long reports

For example, the command

    HOSTSORTBY ALPHABETICAL

will sort the Host Report alphabetically. The full list of `SORTBY` commands is `HOSTSORTBY`, `REDIRHOSTSORTBY`, `FAILHOSTSORTBY`, `ORGSORTBY`, `DOMSORTBY`, `REQSORTBY`, `DIRSORTBY`, `TYPESORTBY`, `REDIRSORTBY`, `FAILSORTBY`, `REFSORTBY`, `REFSITESORTBY`, `SEARCHQUERYSORTBY`, `SEARCHWORDSORTBY`, `INTSEARCHQUERYSORTBY`, `INTSEARCHWORDSORTBY`, `REDIRREFSORTBY`, `FAILREFSORTBY`, `BROWREPSORTBY`, `BROWSUMSORTBY`, `OSSORTBY`, `VHOSTSORTBY`, `REDIRVHOSTSORTBY`, `FAILVHOSTSORTBY`, `USERSORTBY`, `REDIRUSERSORTBY`, `FAILUSERSORTBY` and `STATUSSORTBY`. Again, not every sort method is possible in every report, but you'll be warned if you choose an illegal one.

There is one known bug concerned with `SORTBY ALPHABETICAL`. The report is sorted before any output alias is applied. This means that if an output alias has been specified for the report, then the report may appear not to be sorted correctly.

You can also specify a `FLOOR` for most reports, saying how much activity an item needs before it is listed on the report. (Other items will just be accumulated together in the "not listed" line at the bottom of the report.) There are lots of possible ways of specifying floors, which I'll list here, using the `DOMFLOOR` (Domain Report `FLOOR`) command as an example. Essentially each one consists of a number indicating the level of the floor, followed by a letter indicating the floor criterion.

    DOMFLOOR 1000r       # all domains with at least 1000 requests
    DOMFLOOR 100s        # at least 100 requests within the last 7 days
    DOMFLOOR 1000p       # at least 1000 requests for pages
    DOMFLOOR 100q        # at least 100 requests for pages within the last 7 days
    DOMFLOOR 1000000b    # at least 1,000,000 bytes transferred
    DOMFLOOR 1kb         # at least 1 kilobyte (1024 bytes)
    DOMFLOOR 10.5Mc      # at least 10.5Mb within the last 7 days
    DOMFLOOR 0.5%r       # 0.5% of the total requests in the Domain Report
                         # (ditto %s, %p etc.)
    DOMFLOOR 0.5:r       # 0.5% of the maximum number of requests for any domain
                         # (ditto :s, :p etc.)
    DOMFLOOR 970701d     # last access since 1st July 1997
    DOMFLOOR 970701e     # first access since 1st July 1997
    DOMFLOOR -00-01-00d  # last access in last month (see
                         # documentation on FROM and TO commands)
    DOMFLOOR -100r       # domains with top 100 number of requests
                         # (ditto -100s, p, q, b, c, d, or e)

The full list of `FLOOR` commands is `HOSTFLOOR`, `REDIRHOSTFLOOR`, `FAILHOSTFLOOR`, `DOMFLOOR`, `ORGFLOOR`, `REQFLOOR`, `DIRFLOOR`, `TYPEFLOOR`, `REDIRFLOOR`, `FAILFLOOR`, `REFFLOOR`, `REFSITEFLOOR`, `SEARCHQUERYFLOOR`, `SEARCHWORDFLOOR`, `INTSEARCHQUERYFLOOR`, `INTSEARCHWORDFLOOR`, `REDIRREFFLOOR`, `FAILREFFLOOR`, `BROWREPFLOOR`, `BROWSUMFLOOR`, `OSFLOOR`, `VHOSTFLOOR`, `REDIRVHOSTFLOOR`, `FAILVHOSTFLOOR`, `USERFLOOR`, `REDIRUSERFLOOR`, `FAILUSERFLOOR` and `STATUSFLOOR`. Once again, not every floor method is legal for every report, but you'll be warned if you try and choose an illegal one.

I've already told you about how to turn each report on and off from the command line using its code letter. In fact, you can specify the `SORTBY` and the `FLOOR` in the same command. Take the example of the Referrer Report. If you follow the `+f` (to turn the report on) with a letter, it represents the sort method according to the following code:

`r`
  `REQUESTS`
`s`
  `REQUESTS7`
`p`
  `PAGES`
`q`
  `PAGES7`
`b`
  `BYTES`
`c`
  `BYTES7`
`d`
  `DATE`
`e`
  `FIRSTDATE`
`a`
  `ALPHABETICAL`
`x`
  `RANDOM`

You can then, or alternatively, use one of the above `FLOOR` formats to specify the floor. If you specify a `SORTBY`, you can also leave off the last letter of the floor, and analog will guess it according to the sort method: the floor will be the same as the sort method, or by requests if the sort method is `ALPHABETICAL` or `RANDOM`. Here are four examples:

`+fp`
  means turn the Referrer Report on and sort it by page requests, but says nothing about the floor;
`+f100s`
  means list all referrers with at least 100 requests in the last 7 days, but says nothing about the sort method;
`+fb10000`
  means list all referrers with at least 10,000 bytes, sorted by bytes;
`+fa-000101d`
  means list all referrers with accesses this year, sorted alphabetically.

Each of these reports can have a pie chart drawn at the top of it. The charts can be turned on and off, or plotted by a different criterion, using the `CHART` commands. For example,

    REQCHART OFF

will stop you getting a pie chart on the Request Report, whereas

    REQCHART ON

will turn it back on. You can also use the following arguments to plot the chart by a specific variable.

`REQUESTS`
  total number of requests
`REQUESTS7`
  requests within the last 7 days
`PAGES`
  total requests for pages
`PAGES7`
  requests for pages within the last 7 days
`BYTES`
  total bytes transferred
`BYTES7`
  bytes transferred within the last 7 days

By default, the chart will be plotted by the `SORTBY` if that's one of the above six variables, or else by the `FLOOR` if that's one of the six, or failing both of those by `REQUESTS`. Usually you just want to stick to the default, and analog will warn you if you could have used the `SORTBY` or `FLOOR` and requested something different.

The full list of `CHART` commands is `HOSTCHART`, `REDIRHOSTCHART`, `FAILHOSTCHART`, `ORGCHART`, `DOMCHART`, `REQCHART`, `DIRCHART`, `TYPECHART`, `REDIRCHART`, `FAILCHART`, `REFCHART`, `REFSITECHART`, `SEARCHQUERYCHART`, `SEARCHWORDCHART`, `INTSEARCHQUERYCHART`, `INTSEARCHWORDCHART`, `REDIRREFCHART`, `FAILREFCHART`, `BROWREPCHART`, `BROWSUMCHART`, `OSCHART`, `VHOSTCHART`, `REDIRVHOSTCHART`, `FAILVHOSTCHART`, `USERCHART`, `REDIRUSERCHART`, `FAILUSERCHART`, `STATUSCHART`, `SIZECHART` and `PROCTIMECHART`. Again, not every chart method is available for every report. You can also use

    ALLCHART ON
    ALLCHART OFF

to turn them all on or off simultaneously.

The pie charts are normally written to the same directory as the `OUTFILE`. But you can specify a different location using the `CHARTDIR` and `LOCALCHARTDIR` commands. If the `OUTFILE` is standard output, you must use these commands, or you will not get any pie charts. Also, if you are writing two output files to the same directory, you must use these commands, or one set of images will overwrite the other. And you also need to use these commands if you are using the form interface, but there are additional issues in that case.

You have to use both of the commands before they have any effect. The `CHARTDIR` is the location of the pie chart directory on your server, similar to the `IMAGEDIR`; it's used for putting in the <img> tag to include the image. The `LOCALCHARTDIR` is the location on your local disk; it's where the image is written to. For example, you might have

    CHARTDIR /images/
    LOCALCHARTDIR /usr/local/apache/htdocs/images/

to put the pie charts in locations like `/usr/local/apache/htdocs/images/dom.png` and link to them like `<img src="/images/dom.png">`. On Windows, you will need to put the `LOCALCHARTDIR` in quotes, to stop the backslash at the end of the line being interpreted to mean that the command continues on the next line. For example,

    LOCALCHARTDIR "\webimages\analog\"

Actually, the `CHARTDIR` and `LOCALCHARTDIR` are just prefixes to the filename, so you can specify something like

    CHARTDIR /stats/rep1
    LOCALCHARTDIR /usr/local/htdocs/stats/rep1

to put the pie charts in locations like `/usr/local/htdocs/stats/rep1dom.png` and link to them like `<img src="/stats/rep1dom.png">`. Also the names can contain date codes in the same way as the `LOGFILE` and `OUTFILE`.

If you compiled your own version of analog, and you used your version of libgd instead of mine (see the Makefile for how to do this), then you can use the command

    JPEGCHARTS ON

to make the charts in JPEG format instead of PNG. (See the `PNGIMAGES` command for how to make the bar charts in the time reports into gif's instead of png's.)

Here are the exact rules for which wedges are plotted in the pie chart. Up to ten wedges, plus "Other", are drawn, but wedges are only drawn if they are large enough. Also, wedges are only drawn if the item is listed in the main table for the report. And the whole chart will not be plotted if it would contain only one wedge.

If you want to plot more detail on the pie charts, for example Windows 98 & Windows NT instead of just Windows on the Operating System Report chart, see the next section.

You can list the time period covered by each report. This is off by default because it uses a lot of memory to calculate it, but if different reports cover different time periods (which can happen if your log format has changed at some point), it's useful to turn it on with the command

    REPORTSPAN ON

There is also a command called `REPORTSPANTHRESHOLD` (which can be abbreviated `RSTHRESH`). This says that each report span should only be listed if it differs from the overall span of the whole report -- listed at the top of the page -- by at least this many minutes at one end. For example,

    REPORTSPANTHRESHOLD 60

will only list a report span if that report starts at least an hour after the start of the logfile, or ends at least an hour before the end of the logfile. You can set

    REPORTSPANTHRESHOLD 0

to make sure that the report span is listed on all the reports.

We've already seen some other commands affecting what was listed in the non-time reports. The output `INCLUDE` and `EXCLUDE` commands specified lines to omit from each report, and the output alias commands specified some aliasing to do on the names before they were listed. There were also `LINKINCLUDE` and `LINKEXCLUDE` commands to control what was linked to in the reports. You might want to have another look at these paragraphs.

There's one other command which affects the links in the Request Report, Redirection Report and Failure Report. The command `BASEURL` prepends an additional string to the URLs in the target of the link. For example, after the command

    BASEURL http://www.statslab.cam.ac.uk

`/~sret1/` will be linked to `http://www.statslab.cam.ac.uk/~sret1/`, not just to `/~sret1/`. This is very useful if you want to display the statistics on a different server from the server they refer to. If you want the file to be listed as `http://www.statslab.cam.ac.uk/~sret1/`, rather than just to be linked to that address, you need to use the second argument to the `LOGFILE` command instead.

In the next section, we'll look at commands for generating hierarchical reports, which are closely related to the commands in this section.

Go to the Analog CE home page.

[ Top | Up | Prev | Next | Map | Index ]