/* $XConsortium: README /main/4 1996/07/15 14:18:49 drk $ */
1. Performance Test Suite Organization and Procedures
This readme presents an overview of the organization of the
OSF/Motif Performance Test Suite and the procedures for building and
running the tests.
1.1 Identifying the Tests
This Performance Test Suite consists of seven directories, the name of
each directory reflecting either the region of Motif whose performance
test cases it contains, or the functionality of Motif whose performance
test cases it contains. They are:
Post - to test posting/unposting of dialog and selection box
Expose - to test complete reexpose time
Menu - to test posting/unposting of various menus
Traversal - to test traversal latency
Startup - to test application start up time
Text - to test text entry response, assigning of large amounts of text
to text widget, scrolling through large amounts of text.
These scenarios can be tested for both XmCSText and XmText widgets.
Mwm - to test enter/leave compression (FocusIn, FocusOut) in mwm
DataSpace - to test memory performance
1.3 Installing/Building the Tests
Issue the command:
make
from tests/Performance.
On the other hand, you can issue the make command from inside each of
the subdirectories to build them individually.
1.4 Running the Tests
In general the tests in each directory are run individually by hand.
But in directories where there are more than two test cases, a shell
script file called "run_all" exists, and the tests can be run in one
shot just by typing "run-all" from the command line.
1.5 A brief description of the tests under each directory
(i) Post : This contains a series of tests for posting/unposting
different dialog and selection boxes. The tests are :
postBB - To post a BulletinBoardDialog
postFSel - To post a FileSelectionDialog
postForm - To post a FormDialog
postMess - To post a MessageDialog
postSW1 - To post ScrolledWindow with no children
postSW2 - To post ScrolledWindow with a Drawing Area child
postSW3 - To post ScrolledWindow with a Drawing Area child
having an arrow button as its child
postSel - To post a SelectionDialog
postText - To post a Text widget under a BBDialog
postTextF - To post a TextField widget under a BBDialog
To run the tests, either type "run_all" from command line, or
type the name of each test separately. To exit from each of
these tests, or to run the next test in the series, select
"Close" from the window menu of the test window.
(ii) Expose : This contains the demo periodic, which displays the
motif widget list in the form of a periodic table. The purpose
of this test is to measure the complete reexpose time for
a typical motif application. After starting the test, the test
window should be covered as much as possible with other
windows on the screen, and then it should be exposed.
To run the test, type "periodic" from the command line. To
exit from the test, select "Quit" from the File menu.
(iii) Menu : This contains a series of tests for posting/unposting
different menus. The tests are :
MenuBar1 - To create and post a Menu bar. The test when run,
is self explanatory in instructions.
OptionMenu1 - To post a simple option menu with some push
buttons and toggle buttons. To post the menu, click mouse
button 2 in the push button region of the test window.
OptionMenu2 - To post a slightly complex option menu with some
push buttons and toggle buttons. To post the menu, click mouse
button 2 in the push button region of the test window.
PopupMenu1 - To post a simple popup menu with upto three
cascade levels. To post the menu, click mouse button 3 on the
"Select Me" push button. To post further cascade menus,
click mouse button 3 on the cascade button of the posted menu.
PopupMenu2 - To post a popup menu with a long list of
push buttons with no cascade menus. To post the menu, click
mouse button 3 on the "Select Me" push button.
PopupMenu3 - This is PopupMenu2 with two levels of cascade
menus added.
To run the tests, either type "run_all" from command line, or
type the name of each test separately. To exit from each of
these tests, or to run the next test in the series, select
"Close" from the window menu of the test window.
(iv) Traversal : This contains a simple test to measure the
traversal performance. To run the test, type "Traversal1"
from the command line, and after the window comes up, try
traversing through different tab groups (using <Tab> and
Shift<Tab> keys) and within the tab groups (using the
arrow keys). To exit, click on "Quit" button.
(v) Startup : This contains a test to measure the application
startup time when all the widgets/menus that are used in
the application are created at startup, against the
startup time when the application is started with the
minimum required widgets created (with all the other
widgets being created when needed inside the application).
This is in fact, a simple text editor with a File menu
and Edit menu and a scrolled text widget. By default, the
application is started with just the push buttons "File",
"Edit", "Help", and two push buttons "Create File Menu",
and "Create Edit Menu". To create the menus, click on
the appropriate create buttons. The corresponding menu
gets created, and the create push button disappears. The
scrolled text widget gets created the first time "Open"
or "New" item of the "File" menu is selected. But, from
command line, the menus/widgets to be created at startup
can be specified. To run the program with default startup,
type "editor". To get a list of command line options,
type "editor -h". To exit, select "Exit" from "File" menu.
(vi) Text : This directory contains six tests.
textedit - A simple test to measure the
performance of text input. By default, it displays a "Quit"
pushbutton widget and a text consisting of 200 lines of 100
characters each in a text widget of 24 rows and 80 columns
(these are also the maximum allowed values in the test at
this time.) with wordWrap on. These values can be controlled
through command line options. To run the program with default
values, type "text_edit". To get a list of command line
options, type "text_edit -h". To type in text, position the
text cursor by clicking inside the text widget to get a
blinking beam, and then start typing. To exit, click on
"Quit".
scrolledText - measures scrolling through a text widget
of 80 columns and 24 rows.
CSTextEnter - measures performance of entering large numbers
of text characters into a text widget. Test can be run
using an XmCSText or XmText widget with wordWrap resource
on or off. Test configuration is specified using command
line options. Test uses automation performance monitor
record time elapsed.
CSTextScroll - measures performance of a scrolled XmCSText
widget scrolling through a large volume of text. There
are a number of command line options which can be specified
for controlling the wordWrap, resizeHeight, resizeWidth
resource values as well as the text file loaded into the widget.
TextScroll - measures performance of a scrolled XmText
widget scrolling through a large volume of text. There
are a number of command line options which can be specified
for controlling the wordWrap, resizeHeight, resizeWidth
resource values as well as the text file loaded into the widget.
TextWorst - measures the performance of an XmText widget
when the widget is large enough to fill the screen.
(vii) Mwm : This contains two tests to measure the performance of
enter/leave compression in mwm. The test focus_icon, tests
this by moving through icons, while the test focus_win tests
this by moving through windows on the screen.
focus_icon - Before running this test, set up your environment
as follows :
Type "xrdb -merge ./focus_icon.Xdefaults"
Then restart mwm by selecting the "Restart" item from
the mwm menu.
Then run the test by typing "focus_icon".
By default, the test moves the pointer from (20, screen_ht -20)
to (800, screen_ht - 20) and back in steps of 95 pixels in X
direction, for 10 times. These values can be specified from
command line also. To get a list of command line options, type
"focus_icon -h".
focus_win - Before running this test, set up your environment
as follows :
Type "xrdb -merge ./focus_win.Xdefaults"
Then restart mwm by selecting the "Restart" item from
the mwm menu.
Then run the test by typing "focus_win".
By default, the test moves the pointer from
(20, (scrren_ht / scrren_width) * 20) to (scrren_width, screen_ht)
and back in steps of 100 pixels in X direction, for 10 times.
These values can be specified from command line also. To get a
list of command line options, type "focus_win -h".
To make the test result more visible, arrange the windows on
the screen, in a staggered manner (for example, position the
top corner of each window about 100 pixels below and to the
right of the top corner of the previous window), and execute
the test from the window which is nearest to the top left
corner of the screen.
After running these two tests, to return to your original
environment :
Type "xrdb $HOME/.Xdefaults"
Then restart mwm by selecting the "Restart" item from
the mwm menu.
(viii) DataSpace : This contains two tests to measure the performance
in terms of memory used by typical applications. Before
building any of the tests in this directory, make the Makefile
so as to pull in the instrumented malloc library. To do this,
issue the command:
make IMAKE_DEFINES=-DDebugLib Makefile
and then make all the tests.
TextData1 - This test creates a text widget with 80
columns, 24 rows, with vertical scrolling, with no
horizontal scrolling, and no wordwrap. The data size
will be measured against the file RowCol.c of 1.1.
The test will begin by setting text value to "". Then
the special malloc is turned on. The file is read in,
the instrumented malloc turned on, and the text value
set to the file read in. Then the text is scrolled
page by page from the keyboard to the end of the file.
Memory is measured. This is compared between 1.2
and 1.1.1. Then the text value is again set to
"" and the memory measured. This is to check for
memory leaks.
TextData2 - This test also creates a text widget with 80
columns, 24 rows, with vertical scrolling, and no
wordwrap. This test will read in files of different
sizes and measure the data space in each case. Typical
cases will be files containing 10 chars, 20 chars, 50
chars, and 100 chars per line, and each case will have
10 files with varying number of lines (1000, 2000,
..., 10000). This test can be run in different locales,
for example, the standard 8-bit 'C' locale, the 16-bit
'JP' locale, etc. The test will generate a data file
containing data which will help to draw a graph of the
ratio of data space to the number of characters in the
file versus the number of lines in the file.
MotifData - This test gives a measurement of total memory
allocated and total memory leaked in a typical motif
application, in this case, periodic table application.
To run the test, type "MotifData" from the command line. To
exit from the test, select "Quit" from the File menu.
========================================================================
Here is the "raw" video measurement data conducted on HP platform, for
performance comparison between Motif 1.1.1 and Motif 1.2S1.
General comments: All times are 1/30th second intervals (30 = 1 sec).
"x0" suffix means time to post a dialog/menu the first time, "x10"
means time to post a dialog/menu the tenth time.
Dialogs Motif 1.1.1 Motif 1.2(Snapshot)
_____________ ___________ _________
PostBBx0 23.6 23
PostBBx10 22.7 20.2
PostFSelx0 64.2 98.8
PostFSelx10 37.5 46.1
PostFormx0 23.2 22.8
PostFormx10 22.9 19.4
PostMessx0 28.6 27.2
PostMessx10 24.7 21.6
PostSelx0 48 66.2
PostSelx10 28 27.2
Menus
_____________
MenuBar1x0 80.4 132
MenuBar1x10 141.6 164.2
OptionMenu1x0 18.6 16.2
OptionMenu1x10 17.8 15.5
OptionMenu2x0 17 15.6
OptionMenu2x10 15 17.7
PopupMenu1x0 8 7.8
PopupMenu1x10 8.1 6.5
#2 10 9
#3 10 10.1
#4 9.2 8.1
PopupMenu2x0 18.2 18.2
PopupMenu2x10 18.9 18.6
PopupMenu3x0 18 20.6
PopupMenu3x10 17 17.8
#2 25.7 24.1
#3 24.4 23.2
MWM Focus
______________
Focus-Window 2.2 1
Focus-Icon 4 3.2
Periodic
______________
Expose-Periodic 885.86 849.1
Traversal Cycling
_________________
<1-2> 2.6 2.6
<2-3> 2.6 2.9
<3-4> 2.4 3.2
<4-5> 4.2 3
<5-1> 2.5 2.1
Text Performance
_________________
Key-hit-expose 4.5 dnf dnf = did not finish -
Wordwrap-expose 5.5 dnf mechanical problems here.
CTRL-f7 8.5 dnf See text measurement data below.
Home 19.2 dnf for some timed measurements.
Arrow-up 6.1 dnf
Arrow-down 16 dnf
"editor" program
_________________
Editor 112.6 108.4
a- 325.2 385.8
e- 62.3 62.7
f- 325.8 309.6
T- 150.2 185.8
________________________________________________________________________
The following is performance data to demonstrate time savings in
text scrolling in the use of the latest line table vs. Motif 1.1 computing
on the fly. It shows the performance numbers for wordwrap and non-wordwrap.
The test is basically creating a scrolled text widget in a shell. Then
it reads a file (RowColumn.c) and using XmTextSetString(), reads the contents
of the file into the text widget. Then the text widget is realized.
Using 'wc' we should note that the source file RowColumn.c is 10,849 lines
and 310,129 characters. This is an average of 28.586 characters per line.
The scrolled text widget is approximately 72 characters wide excluding the
scrollbars.
1.2 Line Table 1.1 computational scrolling
(Snapshot)
wordwrap mode:
lapsed time 0.000112 sec 25.380819 sec <- down
lapsed time 0.000116 sec 42. 59501 sec <- up
lapsed time 0.000116 sec 25.222599 sec
lapsed time 0.000116 sec 42.395001 sec
lapsed time 0.000112 sec 25.103496 sec
lapsed time 0.000116 sec 47.425299 sec
lapsed time 0.000116 sec 25.067684 sec
lapsed time 0.000120 sec 42.510134 sec
lapsed time 0.000112 sec 25.392159 sec
lapsed time 0.000116 sec 42.043402 sec
read in file 6.068295 sec 44.726430 sec
realize 55.637317 sec 44.903979 sec
additional
bytes per line 4 bytes/line 16 bytes/line 0 bytes/line
-------------------------------------------------------------------------------
non-wordwrap mode:
lapsed time 0.000116 sec 5.983795 sec <- down
lapsed time 0.000112 sec 5.301253 sec <- up
lapsed time 0.000116 sec 5.791826 sec
lapsed time 0.000112 sec 5.270487 sec
lapsed time 0.000116 sec 5.715798 sec
lapsed time 0.000112 sec 5.287976 sec
lapsed time 0.000116 sec 5.741155 sec
lapsed time 0.000112 sec 5.271324 sec
lapsed time 0.000116 sec 5.731970 sec
lapsed time 0.000116 sec 5.535520 sec
read in file 6.649300 sec 2.072336 sec
realize 0.271131 sec 0.320328 sec
additional
bytes per line 4 bytes/line 16 bytes/line 0 bytes/line