/* $XConsortium: README /main/4 1996/07/15 14:18:49 drk $ */ 1. Performance Test Suite Organization and Procedures This readme presents an overview of the organization of the OSF/Motif Performance Test Suite and the procedures for building and running the tests. 1.1 Identifying the Tests This Performance Test Suite consists of seven directories, the name of each directory reflecting either the region of Motif whose performance test cases it contains, or the functionality of Motif whose performance test cases it contains. They are: Post - to test posting/unposting of dialog and selection box Expose - to test complete reexpose time Menu - to test posting/unposting of various menus Traversal - to test traversal latency Startup - to test application start up time Text - to test text entry response, assigning of large amounts of text to text widget, scrolling through large amounts of text. These scenarios can be tested for both XmCSText and XmText widgets. Mwm - to test enter/leave compression (FocusIn, FocusOut) in mwm DataSpace - to test memory performance 1.3 Installing/Building the Tests Issue the command: make from tests/Performance. On the other hand, you can issue the make command from inside each of the subdirectories to build them individually. 1.4 Running the Tests In general the tests in each directory are run individually by hand. But in directories where there are more than two test cases, a shell script file called "run_all" exists, and the tests can be run in one shot just by typing "run-all" from the command line. 1.5 A brief description of the tests under each directory (i) Post : This contains a series of tests for posting/unposting different dialog and selection boxes. The tests are : postBB - To post a BulletinBoardDialog postFSel - To post a FileSelectionDialog postForm - To post a FormDialog postMess - To post a MessageDialog postSW1 - To post ScrolledWindow with no children postSW2 - To post ScrolledWindow with a Drawing Area child postSW3 - To post ScrolledWindow with a Drawing Area child having an arrow button as its child postSel - To post a SelectionDialog postText - To post a Text widget under a BBDialog postTextF - To post a TextField widget under a BBDialog To run the tests, either type "run_all" from command line, or type the name of each test separately. To exit from each of these tests, or to run the next test in the series, select "Close" from the window menu of the test window. (ii) Expose : This contains the demo periodic, which displays the motif widget list in the form of a periodic table. The purpose of this test is to measure the complete reexpose time for a typical motif application. After starting the test, the test window should be covered as much as possible with other windows on the screen, and then it should be exposed. To run the test, type "periodic" from the command line. To exit from the test, select "Quit" from the File menu. (iii) Menu : This contains a series of tests for posting/unposting different menus. The tests are : MenuBar1 - To create and post a Menu bar. The test when run, is self explanatory in instructions. OptionMenu1 - To post a simple option menu with some push buttons and toggle buttons. To post the menu, click mouse button 2 in the push button region of the test window. OptionMenu2 - To post a slightly complex option menu with some push buttons and toggle buttons. To post the menu, click mouse button 2 in the push button region of the test window. PopupMenu1 - To post a simple popup menu with upto three cascade levels. To post the menu, click mouse button 3 on the "Select Me" push button. To post further cascade menus, click mouse button 3 on the cascade button of the posted menu. PopupMenu2 - To post a popup menu with a long list of push buttons with no cascade menus. To post the menu, click mouse button 3 on the "Select Me" push button. PopupMenu3 - This is PopupMenu2 with two levels of cascade menus added. To run the tests, either type "run_all" from command line, or type the name of each test separately. To exit from each of these tests, or to run the next test in the series, select "Close" from the window menu of the test window. (iv) Traversal : This contains a simple test to measure the traversal performance. To run the test, type "Traversal1" from the command line, and after the window comes up, try traversing through different tab groups (using <Tab> and Shift<Tab> keys) and within the tab groups (using the arrow keys). To exit, click on "Quit" button. (v) Startup : This contains a test to measure the application startup time when all the widgets/menus that are used in the application are created at startup, against the startup time when the application is started with the minimum required widgets created (with all the other widgets being created when needed inside the application). This is in fact, a simple text editor with a File menu and Edit menu and a scrolled text widget. By default, the application is started with just the push buttons "File", "Edit", "Help", and two push buttons "Create File Menu", and "Create Edit Menu". To create the menus, click on the appropriate create buttons. The corresponding menu gets created, and the create push button disappears. The scrolled text widget gets created the first time "Open" or "New" item of the "File" menu is selected. But, from command line, the menus/widgets to be created at startup can be specified. To run the program with default startup, type "editor". To get a list of command line options, type "editor -h". To exit, select "Exit" from "File" menu. (vi) Text : This directory contains six tests. textedit - A simple test to measure the performance of text input. By default, it displays a "Quit" pushbutton widget and a text consisting of 200 lines of 100 characters each in a text widget of 24 rows and 80 columns (these are also the maximum allowed values in the test at this time.) with wordWrap on. These values can be controlled through command line options. To run the program with default values, type "text_edit". To get a list of command line options, type "text_edit -h". To type in text, position the text cursor by clicking inside the text widget to get a blinking beam, and then start typing. To exit, click on "Quit". scrolledText - measures scrolling through a text widget of 80 columns and 24 rows. CSTextEnter - measures performance of entering large numbers of text characters into a text widget. Test can be run using an XmCSText or XmText widget with wordWrap resource on or off. Test configuration is specified using command line options. Test uses automation performance monitor record time elapsed. CSTextScroll - measures performance of a scrolled XmCSText widget scrolling through a large volume of text. There are a number of command line options which can be specified for controlling the wordWrap, resizeHeight, resizeWidth resource values as well as the text file loaded into the widget. TextScroll - measures performance of a scrolled XmText widget scrolling through a large volume of text. There are a number of command line options which can be specified for controlling the wordWrap, resizeHeight, resizeWidth resource values as well as the text file loaded into the widget. TextWorst - measures the performance of an XmText widget when the widget is large enough to fill the screen. (vii) Mwm : This contains two tests to measure the performance of enter/leave compression in mwm. The test focus_icon, tests this by moving through icons, while the test focus_win tests this by moving through windows on the screen. focus_icon - Before running this test, set up your environment as follows : Type "xrdb -merge ./focus_icon.Xdefaults" Then restart mwm by selecting the "Restart" item from the mwm menu. Then run the test by typing "focus_icon". By default, the test moves the pointer from (20, screen_ht -20) to (800, screen_ht - 20) and back in steps of 95 pixels in X direction, for 10 times. These values can be specified from command line also. To get a list of command line options, type "focus_icon -h". focus_win - Before running this test, set up your environment as follows : Type "xrdb -merge ./focus_win.Xdefaults" Then restart mwm by selecting the "Restart" item from the mwm menu. Then run the test by typing "focus_win". By default, the test moves the pointer from (20, (scrren_ht / scrren_width) * 20) to (scrren_width, screen_ht) and back in steps of 100 pixels in X direction, for 10 times. These values can be specified from command line also. To get a list of command line options, type "focus_win -h". To make the test result more visible, arrange the windows on the screen, in a staggered manner (for example, position the top corner of each window about 100 pixels below and to the right of the top corner of the previous window), and execute the test from the window which is nearest to the top left corner of the screen. After running these two tests, to return to your original environment : Type "xrdb $HOME/.Xdefaults" Then restart mwm by selecting the "Restart" item from the mwm menu. (viii) DataSpace : This contains two tests to measure the performance in terms of memory used by typical applications. Before building any of the tests in this directory, make the Makefile so as to pull in the instrumented malloc library. To do this, issue the command: make IMAKE_DEFINES=-DDebugLib Makefile and then make all the tests. TextData1 - This test creates a text widget with 80 columns, 24 rows, with vertical scrolling, with no horizontal scrolling, and no wordwrap. The data size will be measured against the file RowCol.c of 1.1. The test will begin by setting text value to "". Then the special malloc is turned on. The file is read in, the instrumented malloc turned on, and the text value set to the file read in. Then the text is scrolled page by page from the keyboard to the end of the file. Memory is measured. This is compared between 1.2 and 1.1.1. Then the text value is again set to "" and the memory measured. This is to check for memory leaks. TextData2 - This test also creates a text widget with 80 columns, 24 rows, with vertical scrolling, and no wordwrap. This test will read in files of different sizes and measure the data space in each case. Typical cases will be files containing 10 chars, 20 chars, 50 chars, and 100 chars per line, and each case will have 10 files with varying number of lines (1000, 2000, ..., 10000). This test can be run in different locales, for example, the standard 8-bit 'C' locale, the 16-bit 'JP' locale, etc. The test will generate a data file containing data which will help to draw a graph of the ratio of data space to the number of characters in the file versus the number of lines in the file. MotifData - This test gives a measurement of total memory allocated and total memory leaked in a typical motif application, in this case, periodic table application. To run the test, type "MotifData" from the command line. To exit from the test, select "Quit" from the File menu. ======================================================================== Here is the "raw" video measurement data conducted on HP platform, for performance comparison between Motif 1.1.1 and Motif 1.2S1. General comments: All times are 1/30th second intervals (30 = 1 sec). "x0" suffix means time to post a dialog/menu the first time, "x10" means time to post a dialog/menu the tenth time. Dialogs Motif 1.1.1 Motif 1.2(Snapshot) _____________ ___________ _________ PostBBx0 23.6 23 PostBBx10 22.7 20.2 PostFSelx0 64.2 98.8 PostFSelx10 37.5 46.1 PostFormx0 23.2 22.8 PostFormx10 22.9 19.4 PostMessx0 28.6 27.2 PostMessx10 24.7 21.6 PostSelx0 48 66.2 PostSelx10 28 27.2 Menus _____________ MenuBar1x0 80.4 132 MenuBar1x10 141.6 164.2 OptionMenu1x0 18.6 16.2 OptionMenu1x10 17.8 15.5 OptionMenu2x0 17 15.6 OptionMenu2x10 15 17.7 PopupMenu1x0 8 7.8 PopupMenu1x10 8.1 6.5 #2 10 9 #3 10 10.1 #4 9.2 8.1 PopupMenu2x0 18.2 18.2 PopupMenu2x10 18.9 18.6 PopupMenu3x0 18 20.6 PopupMenu3x10 17 17.8 #2 25.7 24.1 #3 24.4 23.2 MWM Focus ______________ Focus-Window 2.2 1 Focus-Icon 4 3.2 Periodic ______________ Expose-Periodic 885.86 849.1 Traversal Cycling _________________ <1-2> 2.6 2.6 <2-3> 2.6 2.9 <3-4> 2.4 3.2 <4-5> 4.2 3 <5-1> 2.5 2.1 Text Performance _________________ Key-hit-expose 4.5 dnf dnf = did not finish - Wordwrap-expose 5.5 dnf mechanical problems here. CTRL-f7 8.5 dnf See text measurement data below. Home 19.2 dnf for some timed measurements. Arrow-up 6.1 dnf Arrow-down 16 dnf "editor" program _________________ Editor 112.6 108.4 a- 325.2 385.8 e- 62.3 62.7 f- 325.8 309.6 T- 150.2 185.8 ________________________________________________________________________ The following is performance data to demonstrate time savings in text scrolling in the use of the latest line table vs. Motif 1.1 computing on the fly. It shows the performance numbers for wordwrap and non-wordwrap. The test is basically creating a scrolled text widget in a shell. Then it reads a file (RowColumn.c) and using XmTextSetString(), reads the contents of the file into the text widget. Then the text widget is realized. Using 'wc' we should note that the source file RowColumn.c is 10,849 lines and 310,129 characters. This is an average of 28.586 characters per line. The scrolled text widget is approximately 72 characters wide excluding the scrollbars. 1.2 Line Table 1.1 computational scrolling (Snapshot) wordwrap mode: lapsed time 0.000112 sec 25.380819 sec <- down lapsed time 0.000116 sec 42. 59501 sec <- up lapsed time 0.000116 sec 25.222599 sec lapsed time 0.000116 sec 42.395001 sec lapsed time 0.000112 sec 25.103496 sec lapsed time 0.000116 sec 47.425299 sec lapsed time 0.000116 sec 25.067684 sec lapsed time 0.000120 sec 42.510134 sec lapsed time 0.000112 sec 25.392159 sec lapsed time 0.000116 sec 42.043402 sec read in file 6.068295 sec 44.726430 sec realize 55.637317 sec 44.903979 sec additional bytes per line 4 bytes/line 16 bytes/line 0 bytes/line ------------------------------------------------------------------------------- non-wordwrap mode: lapsed time 0.000116 sec 5.983795 sec <- down lapsed time 0.000112 sec 5.301253 sec <- up lapsed time 0.000116 sec 5.791826 sec lapsed time 0.000112 sec 5.270487 sec lapsed time 0.000116 sec 5.715798 sec lapsed time 0.000112 sec 5.287976 sec lapsed time 0.000116 sec 5.741155 sec lapsed time 0.000112 sec 5.271324 sec lapsed time 0.000116 sec 5.731970 sec lapsed time 0.000116 sec 5.535520 sec read in file 6.649300 sec 2.072336 sec realize 0.271131 sec 0.320328 sec additional bytes per line 4 bytes/line 16 bytes/line 0 bytes/line