$Id: overview.txt,v 1.2 2004/07/17 00:30:49 mikpe Exp $

AN OVERVIEW OF PERFCTR

======================

The perfctr package adds support to the Linux kernel for using

the performance-monitoring counters found in many processors.

Perfctr is internally organised in three layers:

- The low-level drivers, one for each supported architecture.

  Currently there are two, one for 32 and 64-bit x86 processors,

  and one for 32-bit PowerPC processors.

  low-level-api.txt documents the model of the performance counters

  used in this package, and the internal API to the low-level drivers.

  low-level-{x86,ppc}.txt provide documentation specific for those

  architectures and their low-level drivers.

- The high-level services.

  There is currently one, a kernel extension adding support for

  virtualised per-process performance counters.

  See virtual.txt for documentation on this kernel extension.

  [There used to be a second high-level service, a simple driver

  to control and access all performance counters in all processors.

  This driver is currently removed, pending an acceptable new API.]

- The top-level, which performs initialisation and implements

  common procedures and system calls.

Rationale

---------

The perfctr package solves three problems:

- Hardware invariably restricts programming of the performance

  counter registers to kernel-level code, and sometimes also

  restricts reading the counters to kernel-level code.

  Perfctr adds APIs allowing user-space code access the counters.

  In the case of the per-process counters kernel extension,

  even non-privileged processes are allowed access.

- Hardware often limits the precision of the hardware counters,

  making them unsuitable for storing total event counts.

  The counts are instead maintained as 64-bit values in software,

  with the hardware counters used to derive increments over given

  time periods.

- In a non-modified kernel, the thread state does not include the

  performance monitoring counters, and the context switch code

  does not save and restore them. In this situation the counters

  are system-wide, making them unreliable and inaccurate when used

  for monitoring specific processes or specific segments of code.

  The per-process counters kernel extension treats the counter state as

  part of the thread state, solving the reliability and accuracy problems.

Non-goals

---------

Providing high-level interfaces that abstract and hide the

underlying hardware is a non-goal. Such abstractions can

and should be implemented in user-space, for several reasons:

- The complexity and variability of the hardware means that

  any abstraction would be inaccurate. There would be both

  loss of functionality, and presence of functionality which

  isn't supportable on any given processor. User-space tools

  and libraries can implement this, on top of the processor-

  specific interfaces provided by the kernel.

- The implementation of such an abstraction would be large

  and complex. (Consider ESCR register assignment on P4.)

  Performing complex actions in user-space simplifies the

  kernel, allowing it to concentrate on validating control

  data, managing processes, and driving the hardware.

  (C.f. the role of compilers.)

- The abstraction is purely a user-convenience thing. The

  kernel-level components have no need for it.

Common System Calls

===================

This lists those system calls that are not tied to

a specific high-level service/driver.

Querying CPU and Driver Information

-----------------------------------

int err = sys_perfctr_info(struct perfctr_info *info,

			   struct perfctr_cpu_mask *cpus,

			   struct perfctr_cpu_mask *forbidden);

This operation retrieves information from the kernel about

the processors in the system.

If non-NULL, '*info' will be updated with information about the

capabilities of the processor and the low-level driver.

If non-NULL, '*cpus' will be updated with a bitmask listing the

set of processors in the system. The size of this bitmask is not

statically known, so the protocol is:

1. User-space initialises cpus->nrwords to the number of elements

   allocated for cpus->mask[].

2. The kernel reads cpus->nrwords, and then writes the required

   number of words to cpus->nrwords.

3. If the required number of words is less than the original value

   of cpus->nrwords, then an EOVERFLOW error is signalled.

4. Otherwise, the kernel converts its internal cpumask_t value

   to the external format and writes that to cpus->mask[].

If non-NULL, '*forbidden' will be updated with a bitmask listing

the set of processors in the system on which users must not try

to use performance counters. This is currently only relevant for

hyper-threaded Pentium 4/Xeon systems. The protocol is the same

as for '*cpus'.

Notes:

- The internal representation of a cpumask_t is as an array of

  unsigned long. This representation is unsuitable for user-space,

  because it is not binary-compatible between 32 and 64-bit

  variants of a big-endian processor. The 'struct perfctr_cpu_mask'

  type uses an array of unsigned 32-bit integers.

- The protocol for retrieving a 'struct perfctr_cpu_mask' was

  designed to allow user-space to quickly determine the correct

  size of the 'mask[]' array. Other system calls use weaker protocols,

  which force user-space to guess increasingly larger values in a

  loop, until finally an acceptable value was guessed.