<chapter id="gtest" status="draft">

<title>Testing</title>

<para>

There are several points of review to conduct before declaring an application accessible. Over the course of development you may want to consider automated testing techniques. <ulink url="http://ldtp.freedesktop.org/">LDTP</ulink>, for example, may complement your automated testing plan.

</para>

<para>

This section describes a number of tests you can perform manually on an application to test its accessibility. Passing all the tests does not necessarily imply that the application is fully accessible, but if the application fails any of these tests, then further work may need to be done to improve that aspect of its accessibility.

</para>

<section>

<title>Keyboard Navigation</title>

<para>

The following keyboard operations should be tested. Do not use the mouse in any part of this test.

</para>

<itemizedlist>

<listitem>

<para>

Using only keyboard commands, move the focus through all menu bars in the application.

</para>

</listitem>

<listitem>

<para>Confirm that:</para>

<itemizedlist>

<listitem>

<para>

Context sensitive menus display correctly.

</para>

</listitem>

<listitem>

<para>

Any functions listed on the toolbar can be performed using the keyboard.

</para>

</listitem>

<listitem>

<para>

You can operate every control in the client area of the application and dialog boxes.

</para>

</listitem>

<listitem>

<para>

Text and objects within the client area can be selected.

</para>

</listitem>

<listitem>

<para>

Any keyboard enhancements or shortcuts are working as designed.

</para>

</listitem>

</itemizedlist>

</listitem>

</itemizedlist>

</section>

<section>

<title>Graphical Elements</title>

<para>

Test the application using a screen reader and confirm that:

</para>

<itemizedlist>

<listitem>

<para>

Labels and text are being read correctly, including menus and toolbars.

</para>

</listitem>

<listitem>

<para>

Object information is read correctly.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Visual Focus Indicator</title>

<itemizedlist>

<listitem>

<para>

Verify that when moving among objects that the visual focus indicator is easy to identify.

</para>

</listitem>

<listitem>

<para>

Keyboard navigation through the software and menus should be clearly visible when the focus moves.

</para>

</listitem>

<listitem>

<para>

Confirm that the screen reader is tracking the visual focus indicator as you navigate using a keyboard.

</para>

</listitem>

<listitem>

<para>

Run a screen magnification program (if available) and verify that the magnifier can track the visual focus indicator as you navigate using the keyboard and mouse.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Fonts and Text</title>

<itemizedlist>

<listitem>

<para>

Change the font in the application and confirm that the settings are maintained.

</para>

</listitem>

<listitem>

<para>

Test the application by changing colors and confirm that all settings are maintained.

</para>

</listitem>

<listitem>

<para>

If magnification is available, test the font, color, and size using the magnification option.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Color and Contrast</title>

<itemizedlist>

<listitem>

<para>

Print screenshots to a black and white printer and confirm that all information is visible.

</para>

</listitem>

<listitem>

<para>

Test applications using only black and white, high-contrast settings and confirm that all information is conveyed correctly.

</para>

</listitem>

<listitem>

<para>

Test that the application provides at least three combinations of color schemes and that high-contrast schemes are available (e.g. white on black or yellow on blue).

</para>

</listitem>

<listitem>

<para>

Turn on high-contrast settings in the GNOME Control Center and confirm that the application respects these settings.

</para>

</listitem>

<listitem>

<para>

Test various themes to ensure that the software is working for all the available settings.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Audio</title>

<para>

There should be an option in the application to show audio alerts visually.

</para>

<para>

Test that the audio is working correctly by enabling sound in the GNOME Control Center and then perform the following actions:

</para>

<itemizedlist>

<listitem>

<para>

Perform an action that should generate an audio alert and confirm that the application is working as designed.

</para>

</listitem>

<listitem>

<para>

Verify that the application works correctly when increasing or decreasing the volume.

</para>

</listitem>

<listitem>

<para>

Confirm that warning messages and alerts can be heard correctly in a noisy work environment.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Animation</title>

<para>

Verify that an option is available to stop animation and that it is working as designed.

</para>

<para>

Turn the animation off. Confirm that all information is still conveyed correctly.

</para>

</section>

<section>

<title>Keyboard Focus</title>

<itemizedlist>

<listitem>

<para>

Test all messages to confirm that the user is notified before a message times out and is given the option to indicate that more time is needed.

</para>

</listitem>

<listitem>

<para>

Make sure an option has been included to adjust the response time and confirm that it is working as designed.

</para>

</listitem>

</itemizedlist>

</section>

<section>

<title>Documentation</title>

<para>

Test ASCII text documentation with a screen reader to confirm that it is clear and precise and can be read by assistive technologies.

</para>

<para>

Test HTML applications using a web browser and screen reader to confirm that the documentation is accessible to assistive technologies.

</para>

<para>

Note: There are web accessibility guidelines available at <ulink url="http://www.w3.org/TR/WAI-WEBCONTENT/">http://www.w3.org/TR/WAI-WEBCONTENT/</ulink>.

</para>

<para>

Confirm the following information is included in the documentation:

</para>

<itemizedlist>

<listitem>

<para>

State if the application does not support the standard keyboard access used by the OS.

</para>

</listitem>

<listitem>

<para>

Identify if there are unique keyboard commands.

</para>

</listitem>

<listitem>

<para>

Identify any unique accessibility features.

</para>

</listitem>

<listitem>

<para>

If an action is documented for the mouse, make sure there is an alternative for using the keyboard.

</para>

</listitem>

</itemizedlist>

</section>

<section id="gad-checklist">

<title>User Interface Checklist</title>

<para>

This section summarizes the guidelines given in <link linkend="gad-ui-guidelines">User Interface Guidelines for Supporting Accessibility</link>. You should refer to that section of the guide for more detailed information on any of the checklist items given here.

</para>

<para>

When testing an application for accessibility, you should go through each of the items in the list. Note whether the application passes or fails each test, or does not apply to that application.

</para>

<title>General Principles checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>GP</entry>

<entry>General Principles</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row>

<entry>GP.1</entry>

<entry>

Every action that alters the user's data or application's settings can be undone.

</entry>

</row>

<row>

<entry>GP.2</entry>

<entry>

All application settings can be restored to their defaults without the user having to remember what those defaults were.

</entry>

</row>

<row>

<entry>GP.3</entry>

<entry>

After installation, the application can be used without the user having to insert a disk or CD at any time.

</entry>

</row>

<row><entry>GP.4</entry>

<entry>

The most frequently used functions are found at the top level of the menu structure.

</entry>

</row>

</tgroup>

<title>Keyboard navigation checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>KN</entry>

<entry>Keyboard Navigation</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row>

<entry>KN.1</entry>

<entry>Efficient keyboard access is provided to all application features.

</entry>

</row>

<row>

<entry>KN.2</entry>

<entry>All windows have a logical keyboard navigation order.

</entry>

</row>

<row><entry>KN.3</entry>

<entry>

The correct tab order is used for controls whose enabled state is dependent on checkboxes, radio buttons or toggle buttons.

</entry>

</row>

<row><entry>KN.4</entry>

<entry>

Keyboard access to application-specific functions does not override existing system accessibility features.

</entry>

</row>

<row><entry>KN.5</entry>

<entry>

The application provides more than one method to perform keyboard tasks whenever possible.

</entry>

</row>

<row><entry>KN.6</entry>

<entry>

There are alternative key combinations wherever possible.

</entry>

</row>

<row><entry>KN.7</entry>

<entry>

There are no awkward reaches for frequently performed keyboard operations.

</entry>

</row>

<row><entry>KN.8</entry>

<entry>

The application does not use repetitive, simultaneous keypresses.

</entry>

</row>

<row><entry>KN.9</entry>

<entry>

The application provides keyboard equivalents for all mouse functions.

</entry>

</row>

<row><entry>KN.10</entry>

<entry>

Any text or object that can be selected with the mouse can also be selected with the keyboard alone.

</entry>

</row>

<row><entry>KN.11</entry>

<entry>

Any object that can be resized or moved with the mouse can also be resized or moved with the keyboard alone.

</entry>

</row>

<row><entry>KN.12</entry>

<entry>

The application does not use any general navigation functions to trigger operations.

</entry>

</row>

<row><entry>KN.13</entry>

<entry>

All keyboard-invoked menus, windows and tooltips appear near the object they relate to.

</entry>

</row>

</tgroup>

<title>Mouse Interaction checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>MI</entry>

<entry>Mouse Interaction</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>MI.1</entry>

<entry>No operations depend on input from the <mousebutton>right</mousebutton> or <mousebutton>middle</mousebutton> mouse buttons.

</entry>

</row>

<row><entry>MI.2</entry>

<entry>All mouse operations can be cancelled before they are complete.

</entry>

</row>

<row><entry>MI.3</entry>

<entry>

Visual feedback is provided throughout drag and drop operations

</entry>

</row>

<row><entry>MI.4</entry>

<entry>

The mouse pointer is never warped under application control, or its movement restricted to part of the screen by the application.

</entry>

</row>

</tgroup>

<title>Graphical Elements checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>GE</entry>

<entry>Graphical Elements</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>GE.1</entry>

<entry>

There are no hard-coded graphical attributes such as line, border or shadow thickness.

</entry>

</row>

<row><entry>GE.2</entry>

<entry>

All multi-color graphical elements can be shown in monochrome only, where possible.

</entry>

</row>

<row><entry>GE.3</entry>

<entry>

All interactive GUI elements are easily distinguishable from static GUI elements.

</entry>

</row>

<row><entry>GE.4</entry>

<entry>

An option to hide non-essential graphics is provided.

</entry>

</row>

</tgroup>

<title>Fonts and Text checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>FT</entry>

<entry>Fonts and Text</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>FT.1</entry>

<entry>No font styles or sizes are hard-coded.</entry>

</row>

<row><entry>FT.2</entry>

<entry>

An option to turn off graphical backdrops behind text is provided.

</entry>

</row>

<row><entry>FT.3</entry>

<entry>

All labels have names that make sense when taken out of context.

</entry>

</row>

<row><entry>FT.4</entry>

<entry>

No label names are used more than once in the same window.

</entry>

</row>

<row><entry>FT.5</entry>

<entry>

Label positioning is consistent throughout the application.

</entry>

</row>

<row><entry>FT.6</entry>

<entry>

All static text labels that identify other controls end in a colon (:).

</entry>

</row>

<row><entry>FT.7</entry>

<entry>

Static text labels that identify other controls immediately precede those controls in the tab order.

</entry>

</row>

<row><entry>FT.8</entry>

<entry>

An alternative to WYSIWYG is provided. For example, the ability to specify different screen and printer fonts in a text editor.

</entry>

</row>

</tgroup>

<title>Color and Contrast checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>CC</entry>

<entry>Color and Contrast</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>CC.1</entry>

<entry>

Application colors are not hard-coded, but are drawn either from the current desktop theme or an application setting.

</entry>

</row>

<row><entry>CC.2</entry>

<entry>

Color is only used as an enhancement, and not as the only means to convey information or actions.

</entry>

</row>

<row>

<entry>CC.3</entry>

<entry>

The application supports all available high- contrast themes and settings.

</entry>

</row>

<row><entry>CC.4</entry>

<entry>

The software is not dependent on any particular high-contrast themes or settings.

</entry>

</row>

</tgroup>

<title>Magnification checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>MG</entry>

<entry>Magnification</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>MG.1</entry>

<entry>

The application provides the ability to magnify the work area.

</entry>

</row>

<row><entry>MG.2</entry>

<entry>

The application provides the option to scale the work area.

</entry>

</row>

<row><entry>MG.3</entry>

<entry>

The application's functionality is not affected by changing the magnification or scale settings.

</entry>

</row>

</tgroup>

<title>Audio checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>AU</entry>

<entry>Audio</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>AU.1</entry>

<entry>

Sound is not used as the only means of conveying any items of information.

</entry>

</row>

<row><entry>AU.2</entry>

<entry>

The user can configure the frequency and volume of all sounds and warning beeps.

</entry>

</row>

</tgroup>

<title>Animation checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>AN</entry>

<entry>Animation</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>AN.1</entry>

<entry>

There are no flashing or blinking elements with a frequency greater than 2Hz or lower than 55Hz.

</entry>

</row>

<row><entry>AN.2</entry>

<entry>

Any flashing or blinking is confined to small areas of the screen.

</entry>

</row>

<row><entry>AN.3</entry>

<entry>

If animation is used, an option is available to turn it off before it is first shown.

</entry>

</row>

</tgroup>

<title>Keyboard Focus checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>KF</entry>

<entry>Keyboard Focus</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>KF.1</entry>

<entry>

When a window is opened, focus starts at the most commonly-used control.

</entry>

</row>

<row><entry>KF.2</entry>

<entry>

Current input focus position is clearly displayed at all times.

</entry>

</row>

<row><entry>KF.3</entry>

<entry>

Input focus is shown in exactly one window at all times.

</entry>

</row>

<row><entry>KF.4</entry>

<entry>

Appropriate audio or visual feedback is provided when the user attempts to navigate past either end of a group of related objects.

</entry>

</row>

<row><entry>KF.5</entry>

<entry>

The default audio or visual warning signal is played when the user presses an inappropriate key.

</entry>

</row>

<row><entry>KF.6</entry>

<entry>

There is sufficient audio information for the visual focus that the user can figure out what to do next.

</entry>

</row>

<row><entry>KF.7</entry>

<entry>

When using assistive technologies, such as a screen reader or braille device, the current program indicates the position and content of the visual focus indicator.

</entry>

</row>

</tgroup>

<title>Timing checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>TM</entry>

<entry>Timing</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>TM.1</entry>

<entry>

There are no hard-coded time-outs or time-based features in the application.

</entry>

</row>

<row><entry>TM.2</entry>

<entry>

The display or hiding of important information is not triggered solely by movement of the mouse pointer.

</entry>

</row>

</tgroup>

<title>Documentation checklist</title>

<tgroup cols="3" align="left">

<row>

<entry>DC</entry>

<entry>Documentation</entry>

<entry>Pass/Fail/NA</entry>

</row>

<row><entry>DC.1</entry>

<entry>

All documentation is in an accessible format, with textual alternate descriptions provided for all figures and diagrams.

</entry>

</row>

<row><entry>DC.2</entry>

<entry>

The documentation includes a section that covers all the application's accessibility features.

</entry>

</row>

</tgroup>

</section>

<section>

<title>GOK (GNOME Onscreen Keyboard)</title>

<note>

<para>

The information on this page is partially outdated: GNOME 3's <application><ulink url="http://wiki.gnome.org/Caribou">Caribou</ulink></application> has effectively replaced GNOME 2's <application>gok</application>.

</para>

</note>

<para>

Your application should be usable via <application>gok</application>; key input should be generated entirely by <application>gok</application>, not the keyboard. The aim here would be to work with your application and the desktop in general, ensuring any type of character input can be performed with the on-screen keyboard.

</para>

<para>

The <application>gok</application> application ships with the GNOME Desktop so should already be present. For full documentation, refer to <ulink url="http://www.gok.ca">the official gok site</ulink>.

</para>

<para>

Follow these steps to verify the correct operation of <application>gok</application> with your application:

</para>

<procedure>

<step>

<para>

Login into the GNOME desktop

</para>

</step>

<step>

<para>

Run <application>gok</application>

</para>

</step>

<step>

<para>

Start your application

</para>

</step>

<step>

<para>

Provide input to your application with a pointing device (e.g., mouse or head-tracker) and <application>gok</application>.

</para>

</step>

<step>

<para>

Work using the auto-completion and word prediction features of <application>gok</application>.

</para>

</step>

<step>

<para>

Verify that <application>gok</application> enables and disables the <guibutton>Menus</guibutton> and <guibutton>Toolbars</guibutton> buttons based on the kind of application invoked; for example, the <guibutton>Menus</guibutton> and <guibutton>Toolbars</guibutton> buttons are disabled for the 'Font properties' capplet, but the same buttons are enabled for the <application>Gedit</application> application.

</para>

</step>

<step>

<para>

Verify that the <application>gok</application> on-screen keyboard provided by the <guibutton>Compose</guibutton> button can be used to type in any text for the selected application; run <application>Gedit</application>, click on the text area, and then click on the <guibutton>Compose</guibutton> button in <application>gok</application>. Select the required keys from the on-screen keyboard. The characters should appear in the <application>Gedit</application> text area.

</para>

</step>

<step>

<para>

Verify that the <guibutton>Launcher</guibutton> button allows the user to launch any of the <application>Terminal</application>, <application>Web Browser</application> or <application>Text Editor</application> applications.

</para>

</step>

<step>

<para>

Verify that the <guibutton>Activate</guibutton> button allows the user to activate any of the currently running application windows on the user's desktop, including GNOME panels and the GNOME desktop.

</para>

</step>

<step>

<para>

Verify that the <guibutton>Menus</guibutton> button lists all the available menus in the current application. Verify that clicking on a menu button displays the sub-menu and menu items contained within the sub-menu. Finally, verify that clicking on a menu item activates the menu item. For example, click on the <application>Help Browser</application> application and click on the <guibutton>Menus</guibutton> button. The <application>GOK</application> window now displays the <guibutton>File</guibutton>, <guibutton>Go</guibutton> and <guibutton>Help</guibutton> buttons (the <application>Help Browser</application> menus). Click on the <guibutton>File</guibutton> button and it displays the <guibutton>New Window</guibutton> and <guibutton>Close Window</guibutton> buttons (menu items).

</para>

</step>

<step>

<para>

Verify that the <guibutton>Toolbars</guibutton> button lists all the available buttons in the application toolbar. For example, click on the <application>Help Browser</application> application and then click on the <guibutton>Toolbars</guibutton> button. The <application>GOK</application> window now displays the <guibutton>Back</guibutton>, <guibutton>Forward</guibutton> and <guibutton>Home</guibutton> buttons.

</para>

</step>

<step>

<para>

Verify that the <guibutton>UI Grab</guibutton> button displays all the button objects for the selected application window. For example, open the 'Font Properties' capplet and click on the <guibutton>UI Grab</guibutton> button in the <application>GOK</application> window. The <application>GOK</application> window should now display the names of the buttons in the capplet - <guibutton>Sans</guibutton>, <guibutton>Sans-serif</guibutton>, <guibutton>Close</guibutton> and <guibutton>Help</guibutton>.

</para>

</step>

</procedure>

</section>

<section>

<title>Accerciser</title>

<screenshot>

<mediaobject>

<imageobject>

<imagedata fileref="figures/at-arch.png" format="PNG"/>

</imageobject>

<textobject>

<phrase>

Accerciser and the GNOME Accessibility Architecture

</phrase>

</textobject>

</mediaobject>

</screenshot>

<para>

<application>Accerciser</application> is an interactive Python accessibility explorer for the GNOME Desktop. It uses AT-SPI to inspect and control widgets, allowing you to check if an application is providing correct     information to assistive technologies and automated test frameworks. <application>Accerciser</application> has a simple plugin framework which you can use     to create custom views of accessibility information. Full documentation can be found <ulink url="http://library.gnome.org/devel/accerciser/stable">in the Official Accerciser Manual</ulink>. For a demonstration of <application>Accerciser</application> and <application>PyATSPI</application> (Python-wrappered access and usage of AT-SPI), see <ulink url="http://live.gnome.org/Accessibility/PythonPoweredAccessibility">this article</ulink>. For an excellent walkthrough from the author, see the article titled <ulink url="http://www.linuxjournal.com/article/9991">Make Your Application Accessible with Accerciser</ulink>.

</para>

<note>

<para>

<application>Accerciser</application> has effectively replaced the older <application>at-poke</application> tool.

</para>

</note>

</section>

</chapter>