SUBJECT:     Metering Tools

PRODUCT:     ObjectStore

PLATFORM:    ALL

LANGUAGE:    C++

VERSION:     R4.0

DATE:        May 13, 1996

EXPIRATION:  May 13, 1997

KEY WORDS:   metering, client, performance

Question:  

Are there any client performance metering tools available for
ObjectStore?

Answer:

There is a document and some sample source code for a cmeter class
in a tar file on our ftp server ftp.odi.com (198.3.16.26).  When
you connect via ftp use the login "odiftp" with password "odiftp".
Please read the README file for guidelines on using this connection.

The tar file is in the directory, /support/tools/meters.tar.Z.  The
cmeter class is a wrapper for the client read counter functions and
a few UNIX timing tools.

Additionally, a document (meter.doc) describes what the various 
counters are and which ones are significant.  The meter.doc
is below for your convenience.

One last suggestion we offer a Performance Tuning class that 
covers meters in great detail with hands on exercises in tuning
applications.  For more information please send e-mail to 
ooclass@odi.com.

~~~~~~~~~~~~~~~~~~~~~~~  Meter.doc  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This file documents the meters produced by the meter_data class.  It
has not been completely verified as to its accuracy.  Caveat implementor.


Unix meters
--------------

Real time:

	Seconds as returned from gettimeofday().


Client user time:

	The total amount of time spent executing in user mode
	(getrusage: ru_utime).


Client system time:

	The total amount of time spent in the system executing on
	behalf of the process(es) (getrusage: ru_stime).


Client soft page faults:

	The number of page faults serviced without any I/O activity;
	here I/O activity is avoided by "reclaiming" a page frame from
	the list of pages awaiting reallocation (getrusage:
	ru_minflt).


Client hard page faults:

	The number of page faults serviced which required I/O activity
	(getrusage: ru_majflt).


ObjectStore Client meters
-------------------------

n_sigsegv:
	
	The number of segmentation faults taken by the process which
	have been trapped by ObjectStore.


n_page_fault:

	The number of page faults taken by the client which have been
	handled by the VMM component of the operating system.


n_fetch_page:

	This meter measures the number of pages actually fetched from
	the server by the client.


n_touch_page:

	


n_page_fault_active:

	Significant.  If this number is very high (more than 3x page
	faults) it indicates the ObjectStore page replacement
	algorithm is working too hard.  This could be caused by:

		* transactions that are too long
		* a cache that is too small
		* a cache that is too large with respect to available physical
		memory


n_snapshot:

	


n_fetch_segment:

	Significant.  This meter should be zero if
	objectstore::read_whole_segment(0) has been called (i.e.
	fetching a page-at-a-time).  Otherwise it measures the number
	of segments fetched from the server by the client.  Using
	page-at-a-time fetching can greatly improve performance if
	database segments are large.


n_return_some_pages:

	


n_return_pages:

	Significant.  The number of pages evicted to the server.  If
	this number is large it means the cache contained pages which
	had to be sent back to the server to make room for new pages.
	This could be caused by:

		* transactions which touch a lot of data
		* a small cache

	For transactions which modify a large number of pages,
	forcing page evictions from the cache can improve overall
	performance.  This is done by using a smaller cache, which
	causes the client to return modified pages to the server
	before the actual commit.  When the commit actually happens,
	fewer pages must be transmitted across the network.

	For transactions which mostly read data, performance can be
	improved by enlarging the cache.  Beware however that the
	cache competes with the process for real memory; overly
	enlarging the cache can cause thrashing due to increased
	paging.


n_evict:

	


n_rel_page_in and n_rel_page_out:

	These are incremented every time a page is relocated.  Relocation
	for the "inward" direction happens at the time a page gets
	mapped in.  Relocation for the "outward" direction happens when
	pages get sent back to the server (either at commit time or
	when pages are evicted).  They should each be less than the
	number of pages fetched (n_fetch_page) unless accessing many
	databases in the same transaction.


n_wire and n_unwire:

	Insignificant.  These meters tend to be high due to fact that
	ObjectStore fixes some database pages in memory for its own
	use.


n_grow_commseg:

	This meter counts the number of times the communications
	segment for client-to-cache-manager messages is expanded.  It
	should change rarely.

n_pages_committed:

	Significant.  This meter counts the number of dirtied pages
	written during a commit.  If this number seems excessive for a
	single transaction it could indicate:

		* poor locality of reference during the transaction
		* a transaction that is too long

	The former can be addressed by examining the data referenced in the
	transaction and grouping that data together via clustering.  The
	latter can be addressed by adding transactions.


n_top_level_commits:

	This is the number of transactions committed which were not nested
	within another transaction.


n_nested_commits:

	This is the number of transactions committed which were nested within
	another transaction.


n_top_level_aborts:

	This is the number of transactions aborted which were not nested
	within another transaction.


n_nested_aborts:
	
	This is the number of transactions aborted which were nested within
	another transaction.


n_returned_flushed:

	The number of evicted pages which were not dirty and thus did
	not need to be returned to the server.


n_returned_locked:

	This means that most of the pages being evicted are pages that
	have been referenced, but not modified, during this
	transaction.  It doesn't directly tell you how many pages he
	is modifying.  If he wants to know how many pages were
	modified, he can do

	   objectstore::read_counter("n_upgrade_lock")

	and/or

	   objectstore::read_counter("n_fault_upgrade")



n_returned_dirtied:

	The number of modified pages sent back to the server while a
	transaction was still in progress.


NOTE:   n_return_pages == (  n_returned_flushed
			   + n_returned_locked
			   + n_returned_dirtied)


n_upgrade_cache:

	


n_upgrade_lock:

	This meter counts the number of read to write upgrades for
	pages.  If it is a significant portion of the number of pages
	fetched (n_fetch_page), you can bypass the upgrade step and
	get write locks as a default by calling either of two functions:

		* objectstore::set_opt_cache_lock_mode(1) sets the
		default	globally;

		* database::set_opt_cache_lock_mode(1) - sets the mode
		for a single database.


n_real_read_page:

	


n_real_read_segment:

	


n_real_read_segment_pages:

	


n_fault_nested_ro:

	


n_fault_detect_usage:

	


n_fault_lock:

	


n_fault_upgrade:

	


n_fault_ewlw:

	


n_return_hard_pages:

	This meter measures the number of modified or snapshotted
	pages evicted from the cache.  If the number is high it
	indicates the cache is full of dirty pages.


n_return_easy_pages:

	


ObjectStore Server Meters
-------------------------

Server user time:

	The total amount of time the server spent executing in user mode.


Server system time:

	The total amount of time spent in system executing on behalf
	of the server.


Server soft page faults:

	The number of page faults serviced without any I/O activity;
	here I/O activity is avoided by "reclaiming" a page frame from
	the list pages awaiting reallocation (ru_minflt).


Server hard page faults:

	The number of page faults serviced which required I/O activity.


log_buffer_flushes:

	


log_buffer_sectors_written:

	


uncommitted_log_reads:

	


committed_log_reads:

	


material_database_reads:

	The number of pages read from the database as opposed to from
	the server log.


commits:

	This meter doesn't necessarily correspond to client activity.
	It is an approximation of the number of transactions committed
	to the server.


aborts:

	This meter doesn't necessarily correspond to client activity.
	It is an approximation of the number of transactions aborted
	while accessing data from the server.


transactions:

	This meter doesn't necessarily correspond to client activity.
	It is an approximation of the number of transactions started
	against the server.


read_only_commits:

	This meter doesn't necessarily correspond to client activity.
	It is an approximation of the number of read-only transactions
	which accessed data from the server and did not abort.


checkpoints:

	Very significant.  The number of checkpoints should be
	relatively low compared to the number of transactions which
	should be fast.  In other words, the number of checkpoints
	should be a fraction of the number of transactions; and the
	more performance critical the transactions, the smaller the
	fraction should be.  As the server log size increases, fewer
	checkpoints are required.  Checkpoints can be forced with the
	ossvrchkpt utility.


log_copies:

	


log_skipped_copies:

	


log_propagations:

	


log_skipped_propagations:

	


log_propagation_writes:

	


log_full_pool_writes:

	


checkpoint_log_fg_percentage and checkpoint_log_bg_percentage:

	These are internal server parameters which should not change.
	The default values are correct.


buffer_hits and buffer_misses:

	These meters are related to the server cache.  Unless you are
	running concurrent applicationss or are rerunning the same
	application many times, these meters can be ignored.
	Otherwise a high miss:hit ratio indicates performance could be
	improved with a larger server buffer.






FAQ_reference:  metering

