README
/* $XConsortium: README /main/4 1996/07/15 14:18:49 drk $ */
1.  Performance Test Suite Organization and Procedures

This readme presents an overview of the organization of the
OSF/Motif Performance Test Suite and the procedures for building and
running the tests.

1.1  Identifying the Tests

This Performance Test Suite consists of seven directories, the name of
each directory reflecting either the region of Motif whose performance
test cases it contains, or the functionality of Motif whose performance
test cases it contains. They are:

   Post - to test posting/unposting of dialog and selection box

   Expose - to test complete reexpose time

   Menu - to test posting/unposting of various menus

   Traversal - to test traversal latency

   Startup - to test application start up time

   Text - to test text entry response, assigning of large amounts of text
          to text widget, scrolling through large amounts of text.
          These scenarios can be tested for both XmCSText and XmText widgets.

   Mwm - to test enter/leave compression (FocusIn, FocusOut) in mwm

   DataSpace - to test memory performance

1.3  Installing/Building the Tests

Issue the command:

		make

from tests/Performance.

On the other hand, you can issue the make command from inside each of 
the subdirectories to build them individually.

1.4  Running the Tests

In general the tests in each directory are run individually by hand.
But in directories where there are more than two test cases, a shell
script file called "run_all" exists, and the tests can be run in one
shot just by typing "run-all" from the command line.

1.5 A brief description of the tests under each directory

    (i) Post : This contains a series of tests for posting/unposting
		different dialog and selection boxes. The tests are :

		postBB - To post a BulletinBoardDialog

		postFSel - To post a FileSelectionDialog

		postForm - To post a FormDialog

		postMess - To post a MessageDialog

		postSW1 - To post ScrolledWindow with no children

		postSW2 - To post ScrolledWindow with a Drawing Area child

		postSW3 - To post ScrolledWindow with a Drawing Area child
			  having an arrow button as its child

		postSel - To post a SelectionDialog

		postText - To post a Text widget under a BBDialog

		postTextF - To post a TextField widget under a BBDialog

		To run the tests, either type "run_all" from command line, or
		type the name of each test separately. To exit from each of
		these tests, or to run the next test in the series, select
		"Close" from the window menu of the test window.

   (ii) Expose : This contains the demo periodic, which displays the
		motif widget list in the form of a periodic table. The purpose
		of this test is to measure the complete reexpose time for
		a typical motif application. After starting the test, the test
		window should be covered as much as possible with other 
		windows on the screen, and then it should be exposed.

		To run the test, type "periodic" from the command line. To
		exit from the test, select "Quit" from the File menu.

  (iii) Menu : This contains a series of tests for posting/unposting
		different menus. The tests are :

		MenuBar1 - To create and post a Menu bar. The test when run,
		is self explanatory in instructions.

		OptionMenu1 - To post a simple option menu with some push
		buttons and toggle buttons. To post the menu, click mouse
		button 2 in the push button region of the test window.

		OptionMenu2 - To post a slightly complex option menu with some 
		push buttons and toggle buttons. To post the menu, click mouse
		button 2 in the push button region of the test window.

		PopupMenu1 - To post a simple popup menu with upto three
		cascade levels. To post the menu, click mouse button 3 on the
		"Select Me" push button. To post further cascade menus,
		click mouse button 3 on the cascade button of the posted menu.

		PopupMenu2 - To post a popup menu with a long list of 
		push buttons with no cascade menus. To post the menu, click
		mouse button 3 on the "Select Me" push button.

		PopupMenu3 - This is PopupMenu2 with two levels of cascade
		menus added.

		To run the tests, either type "run_all" from command line, or
		type the name of each test separately. To exit from each of
		these tests, or to run the next test in the series, select
		"Close" from the window menu of the test window.

   (iv) Traversal : This contains a simple test to measure the
		traversal performance. To run the test, type "Traversal1"
		from the command line, and after the window comes up, try
		traversing through different tab groups (using <Tab> and
		Shift<Tab> keys) and within the tab groups (using the
		arrow keys). To exit, click on "Quit" button.

	(v) Startup : This contains a test to measure the application
		startup time when all the widgets/menus that are used in
		the application are created at startup, against the 
		startup time when the application is started with the
		minimum required widgets created (with all the other 
		widgets being created when needed inside the application).
		This is in fact, a simple text editor with a File menu
		and Edit menu and a scrolled text widget. By default, the 
		application is started with just the push buttons "File",
		"Edit", "Help", and two push buttons "Create File Menu",
		and "Create Edit Menu". To create the menus, click on
		the appropriate create buttons. The corresponding menu
		gets created, and the create push button disappears. The
		scrolled text widget gets created the first time "Open"
		or "New" item of the "File" menu is selected. But, from
		command line, the menus/widgets to be created at startup
		can be specified. To run the program with default startup,
		type "editor". To get a list of command line options,
		type "editor -h". To exit, select "Exit" from "File" menu.

   (vi) Text : This directory contains six tests. 
               textedit - A simple test to measure the
		performance of text input. By default, it displays a "Quit"
		pushbutton widget and a text consisting of 200 lines of 100 
		characters each in a text widget of 24 rows and 80 columns 
		(these are also the maximum allowed values in the test at 
		this time.) with wordWrap on. These values can be controlled 
		through command line options. To run the program with default 
		values, type "text_edit". To get a list of command line 
		options, type "text_edit -h". To type in text, position the 
		text cursor by clicking inside the text widget to get a 
		blinking beam, and then start typing. To exit, click on 
		"Quit".

               scrolledText - measures scrolling through a text widget
                of 80 columns and 24 rows.

               CSTextEnter - measures performance of entering large numbers
                of text characters into a text widget.  Test can be run
                using an XmCSText or XmText widget with wordWrap resource
                on or off. Test configuration is specified using command
                line options.  Test uses automation performance monitor
                record time elapsed. 
                
               CSTextScroll - measures performance of a scrolled XmCSText
                widget scrolling through a large volume of text. There 
                are a number of command line options which can be specified
                for controlling the wordWrap, resizeHeight, resizeWidth
                resource values as well as the text file loaded into the widget.
                
               TextScroll - measures performance of a scrolled XmText
                widget scrolling through a large volume of text. There 
                are a number of command line options which can be specified
                for controlling the wordWrap, resizeHeight, resizeWidth
                resource values as well as the text file loaded into the widget.

               TextWorst - measures the performance of an XmText widget
                when the widget is large enough to fill the screen.
                

  (vii) Mwm : This contains two tests to measure the performance of
		enter/leave compression in mwm. The test focus_icon, tests
		this by moving through icons, while the test focus_win tests
		this by moving through windows on the screen.

		focus_icon - Before running this test, set up your environment
		as follows :

			Type "xrdb -merge ./focus_icon.Xdefaults"
			Then restart mwm by selecting the "Restart" item from
			the mwm menu.
			Then run the test by typing "focus_icon".

		By default, the test moves the pointer from (20, screen_ht -20)
		to (800, screen_ht - 20) and back in steps of 95 pixels in X
		direction, for 10 times. These values can be specified from
		command line also. To get a list of command line options, type
		"focus_icon -h".

		focus_win - Before running this test, set up your environment
		as follows :

			Type "xrdb -merge ./focus_win.Xdefaults"
			Then restart mwm by selecting the "Restart" item from
			the mwm menu.
			Then run the test by typing "focus_win".

		By default, the test moves the pointer from 
		(20, (scrren_ht / scrren_width) * 20) to (scrren_width, screen_ht) 
		and back in steps of 100 pixels in X direction, for 10 times. 
		These values can be specified from command line also. To get a 
		list of command line options, type "focus_win -h". 
		To make the test result more visible, arrange the windows on 
		the screen, in a staggered manner (for example, position the 
		top corner of each window about 100 pixels below and to the 
		right of the top corner of the previous window), and execute 
		the test from the window which is nearest to the top left 
		corner of the screen.


		After running these two tests, to return to your original
		environment :

			Type "xrdb $HOME/.Xdefaults"
			Then restart mwm by selecting the "Restart" item from
			the mwm menu.

 (viii) DataSpace : This contains two tests to measure the performance
		in terms of memory used by typical applications. Before
		building any of the tests in this directory, make the Makefile
		so as to pull in the instrumented malloc library. To do this,
		issue the command:

			make IMAKE_DEFINES=-DDebugLib Makefile

		and then make all the tests.

        TextData1 - This test creates a text widget with 80 
        columns, 24 rows, with vertical scrolling, with no
        horizontal scrolling, and no wordwrap. The data size 
        will be measured against the file RowCol.c of 1.1. 
        The test will begin by setting text value to "". Then 
        the special malloc is turned on. The file is read in,
        the instrumented malloc turned on, and the text value
        set to the file read in.  Then the text is scrolled 
        page by page from the keyboard to the end of the file.
        Memory is measured. This is compared between 1.2 
        and 1.1.1. Then the text value is again set to 
        "" and the memory measured. This is to check for 
        memory leaks.

        TextData2 - This test also creates a text widget with 80
        columns, 24 rows, with vertical scrolling, and no
        wordwrap. This test will read in files of different
        sizes and measure the data space in each case. Typical 
        cases will be files containing 10 chars, 20 chars, 50 
        chars, and 100 chars per line, and each case will have 
        10 files with varying number of lines (1000, 2000,
        ..., 10000). This test can be run in different locales, 
		for example, the standard 8-bit 'C' locale, the 16-bit 
		'JP' locale, etc. The test will generate a data file 
		containing data which will help to draw a graph of the 
		ratio of data space to the number of characters in the 
		file versus the number of lines in the file.

        MotifData - This test gives a measurement of total memory
		allocated and total memory leaked in a typical motif
		application, in this case, periodic table application.

		To run the test, type "MotifData" from the command line. To
		exit from the test, select "Quit" from the File menu.


========================================================================

Here is the "raw" video measurement data conducted on HP platform, for
performance comparison between Motif 1.1.1 and Motif 1.2S1.

General comments: All times are 1/30th second intervals (30 = 1 sec).
"x0" suffix means time to post a dialog/menu the first time, "x10"
means time to post a dialog/menu the tenth time.
    

Dialogs          Motif 1.1.1   Motif 1.2(Snapshot)
_____________    ___________   _________

PostBBx0             23.6         23
PostBBx10            22.7         20.2
PostFSelx0           64.2         98.8
PostFSelx10          37.5         46.1
PostFormx0           23.2         22.8
PostFormx10          22.9         19.4
PostMessx0           28.6         27.2
PostMessx10          24.7         21.6
PostSelx0            48           66.2
PostSelx10           28           27.2

Menus
_____________
MenuBar1x0           80.4         132      
MenuBar1x10          141.6        164.2    
OptionMenu1x0        18.6         16.2     
OptionMenu1x10       17.8         15.5     
OptionMenu2x0        17           15.6     
OptionMenu2x10       15           17.7     
PopupMenu1x0         8            7.8      
PopupMenu1x10        8.1          6.5      
#2                   10           9        
#3                   10           10.1     
#4                   9.2          8.1      
PopupMenu2x0         18.2         18.2     
PopupMenu2x10        18.9         18.6     
PopupMenu3x0         18           20.6     
PopupMenu3x10        17           17.8     
#2                   25.7         24.1     
#3                   24.4         23.2     

MWM Focus
______________
Focus-Window         2.2          1        
Focus-Icon           4            3.2      


Periodic
______________
Expose-Periodic      885.86       849.1


Traversal Cycling
_________________
<1-2>                2.6          2.6      
<2-3>                2.6          2.9      
<3-4>                2.4          3.2      
<4-5>                4.2          3        
<5-1>                2.5          2.1      

Text Performance
_________________
Key-hit-expose       4.5          dnf      dnf = did not finish -
Wordwrap-expose      5.5          dnf      mechanical problems here.
CTRL-f7              8.5          dnf      See text measurement data below.
Home                 19.2         dnf      for some timed measurements.
Arrow-up             6.1          dnf      
Arrow-down           16           dnf      

"editor" program
_________________
Editor               112.6        108.4    
a-                   325.2        385.8
e-                   62.3         62.7     
f-                   325.8        309.6    
T-                   150.2        185.8    


________________________________________________________________________

The following is performance data to demonstrate time savings in
text scrolling in the use of the latest line table vs. Motif 1.1 computing 
on the fly.   It shows the performance numbers for wordwrap and non-wordwrap.

The test is basically creating a scrolled text widget in a shell.  Then
it reads a file (RowColumn.c) and using XmTextSetString(), reads the contents
of the file into the text widget.  Then the text widget is realized.

Using 'wc' we should note that the source file RowColumn.c is 10,849 lines
and 310,129 characters.  This is an average of 28.586 characters per line.
The scrolled text widget is approximately 72 characters wide excluding the
scrollbars.

             1.2 Line Table  1.1 computational scrolling
                (Snapshot)
wordwrap mode:

lapsed time     0.000112 sec     25.380819 sec  <- down
lapsed time     0.000116 sec     42. 59501 sec  <- up

lapsed time     0.000116 sec     25.222599 sec
lapsed time     0.000116 sec     42.395001 sec

lapsed time     0.000112 sec     25.103496 sec
lapsed time     0.000116 sec     47.425299 sec

lapsed time     0.000116 sec     25.067684 sec
lapsed time     0.000120 sec     42.510134 sec

lapsed time     0.000112 sec     25.392159 sec
lapsed time     0.000116 sec     42.043402 sec

read in file    6.068295 sec     44.726430 sec
realize        55.637317 sec     44.903979 sec

additional
bytes per line         4 bytes/line       16 bytes/line         0 bytes/line
-------------------------------------------------------------------------------
non-wordwrap mode:

lapsed time     0.000116 sec     5.983795 sec  <- down
lapsed time     0.000112 sec     5.301253 sec  <- up

lapsed time     0.000116 sec     5.791826 sec
lapsed time     0.000112 sec     5.270487 sec

lapsed time     0.000116 sec     5.715798 sec
lapsed time     0.000112 sec     5.287976 sec

lapsed time     0.000116 sec     5.741155 sec
lapsed time     0.000112 sec     5.271324 sec

lapsed time     0.000116 sec     5.731970 sec
lapsed time     0.000116 sec     5.535520 sec

read in file    6.649300 sec     2.072336 sec
realize         0.271131 sec     0.320328 sec

additional
bytes per line         4 bytes/line       16 bytes/line         0 bytes/line