From xemacs-m  Sun Dec  8 21:39:07 1996
Received: from altair.xemacs.org (steve@xemacs.miranova.com [206.190.83.19]) by xemacs.cs.uiuc.edu (8.8.3/8.8.3) with ESMTP id VAA17324 for <xemacs-beta@xemacs.org>; Sun, 8 Dec 1996 21:39:05 -0600 (CST)
Received: (from steve@localhost)
          by altair.xemacs.org (8.8.4/8.8.4)
	  id TAA02981; Sun, 8 Dec 1996 19:48:58 -0800
Sender: steve@xemacs.org
To: xemacs-beta@xemacs.org
Subject: Benchmarking XEmacs versions (was Re: Additions to Shane Holder's bench.el)
References: <199612090006.SAA17017@xemacs.cs.uiuc.edu>
X-Url: http://www.miranova.com/%7Esteve/
Mail-Copies-To: never
X-Face: #!T9!#9s-3o8)*uHlX{Ug[xW7E7Wr!*L46-OxqMu\xz23v|R9q}lH?cRS{rCNe^'[`^sr5"
 f8*@r4ipO6Jl!:Ccq<xoV[Qz2u8<8-+Vwf2gzJ44lf_/y9OaQ`@#Q65{U4/TC)i2`~/M&QI$X>p:9I
 OSS'2{-)-4wBnVeg0S\O4Al@)uC[pD|+
X-Attribution: sb
From: Steven L Baur <steve@miranova.com>
In-Reply-To: Chuck Thompson's message of Sun, 08 Dec 1996 18:06:23 -0600
Mime-Version: 1.0 (generated by tm-edit 7.95)
Content-Type: text/plain; charset=US-ASCII
Date: 08 Dec 1996 19:48:58 -0800
Message-ID: <m2d8wkl6mt.fsf@altair.xemacs.org>
Lines: 172
X-Mailer: Red Gnus v0.72/XEmacs 20.0

>>>>> "Chuck" == Chuck Thompson <cthomp@xemacs.org> writes:
>>>>> "John" == John Turner <turner@xdiv.lanl.gov> writes:

Chuck> The beta workspaces turn the extra debug code on by default in
Chuck> configure.  If you didn't turn it off in the binaries you
Chuck> generated these numbers from then the results are extremely
Chuck> inaccurate.  If that is the case you might be in for a pleasant
Chuck> surprise.  I thought the extra debug code was creating more
Chuck> than a 10% performance degradation.

No, Chuck you are wrong, `pleasant surprise' doesn't begin to describe
the feeling.  :-)

John> Maybe --error-checking=none?  Or does --debug=no do this as well?

Yup.  Thanks John.

The results are shocking (at least for Martin, perhaps).  My comments
are attached separately.

The cast:

All versions compiled and tested on a Linux 2.0/i586/ELF system, run
against kernel version 2.0.27, libc 5.4.16, XFree86 3.2 (X11R6.1), and
compiled with gcc-2.7.2.

19.14/Net Release	compiled `-O4'
19.15-beta3/Unreleased	compiled `-O2'[1] -malign-loops=2
				-malign-functions=2 -malign-jumps=2'
20.0-beta31/Unreleased	compiled `-O4'[1] -malign-loops=2
				-malign-functions=2 -malign-jumps=2'
				without Mule support.

The numbers:

		  Test 1, bytecompilation of tm-7.95
		User	System	Total[2]
19.14		41.00	2.22	43.22
19.15-b3	41.89	1.89	43.78
20.0-b31	44.87	1.62	46.46

		  Test 2, Shane Holder's Bench Mark
                             XEmacs 19.14
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   1           86.241615999  86.241615999
bench-mark-6   1           56.106330999  56.106330999
bench-mark-1   1           7.2148149999  7.2148149999
bench-mark-2   1           6.0626679999  6.0626679999
bench-mark-4   1           0.1291999999  0.1291999999

			   XEmacs 19.15-b3
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   1           87.710748999  87.710748999
bench-mark-6   1           53.041410999  53.041410999
bench-mark-1   1           7.3050970000  7.3050970000
bench-mark-2   1           6.2974949999  6.2974949999
bench-mark-4   1           0.1557390000  0.1557390000

XEmacs 20.0-b31
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   1           86.813706999  86.813706999
bench-mark-6   1           57.934419000  57.934419000
bench-mark-1   1           7.7907109999  7.7907109999
bench-mark-2   1           5.7858679999  5.7858679999
bench-mark-4   1           0.1459620000  0.1459620000

Results sorted by benchmark.

	Bench Mark 1 [Tower of Hanoi]
19.14		7.2148149999
19.15-b3	7.3050970000
20.0-b31	7.7907109999

        Bench Mark 2 [Fontify a large file]
20.0-b31	5.7858679999
19.14		6.0626679999
19.15-b3	6.2974949999

	Bench Mark 3 [Scroll through large file]
19.14		86.241615999
20.0-b31	86.813706999
19.15-b3	87.710748999

	Bench Mark 4 [Frame creation]
19.14		0.1291999999
20.0-b31	0.1459620000
19.15-b3	0.1557390000

	Bench Mark 6 [Byte compile a large file]
19.15-b3	53.041410999
19.14		56.106330999
20.0-b31	57.934419000


	Test 3, sb-bench.elc[3]

XEmacs 19.14
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   3           57.435617999  19.145205999
bench-mark-1   3           21.305076999  7.1016923333
bench-mark-6   3           12.688550999  4.2295169999
bench-mark-2   3           5.6227170000  1.8742390000
bench-mark-4   3           0.3995079999  0.1331693333

XEmacs 19.15-b3
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   3           60.557077000  20.185692333
bench-mark-1   3           21.460382999  7.1534609999
bench-mark-6   3           12.785633999  4.2618779999
bench-mark-2   3           5.3598389999  1.7866129999
bench-mark-4   3           0.4078620000  0.1359540000

XEmacs 20.0-b31
Function Name  Call Count  Elapsed Time  Average Time
=============  ==========  ============  ============
bench-mark-3   3           56.165821999  18.721940666
bench-mark-1   3           21.045211000  7.0150703333
bench-mark-6   3           14.456910999  4.8189703333
bench-mark-2   3           5.4400430000  1.8133476666
bench-mark-4   3           0.4354110000  0.1451370000

	Bench Mark 1 [Tower of Hanoi]
20.0-b31	7.0150703333
19.14		7.1016923333
19.15-b3	7.1534609999

        Bench Mark 2 [Fontify a large file]
19.15-b3	1.7866129999
20.0-b31	1.8133476666
19.14		1.8742390000

	Bench Mark 3 [Scroll through large file]
20.0-b31	18.721940666
19.14		19.145205999
19.15-b3	20.185692333

	Bench Mark 4 [Frame creation]
19.14		0.1331693333
19.15-b3	0.1359540000
20.0-b31	0.1451370000

	Bench Mark 6 [Byte compile a large file]
19.14		4.2295169999
19.15-b3	4.2618779999
20.0-b31	4.8189703333

Notes:
[1]
It's unclear to me whether -O4 is faster than -O2, other tests didn't
show much of a difference.  The Linux kernel is compiled `-O2'.

[2]
Times in seconds from zsh 3.0.1 `time' builtin.

[3]
(load "sb-bench.elc")
(setq bench-enabled-profiling nil)
(bench 3)

[4]
Timings with elp are subject to variance due to the fact they measure
wall clock time and not system resources used.
-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be billed at $250/message.
"Bill Clinton is a bore.  He doesn't have a creative bone in his
body."  -- David Brinkley

