Blame doc/roots.rst

Packit 67cb25
.. index::
Packit 67cb25
   single: root finding
Packit 67cb25
   single: zero finding
Packit 67cb25
   single: finding roots
Packit 67cb25
   single: finding zeros
Packit 67cb25
   single: roots
Packit 67cb25
   single: solving a nonlinear equation
Packit 67cb25
   single: nonlinear equation, solutions of
Packit 67cb25
Packit 67cb25
****************************
Packit 67cb25
One Dimensional Root-Finding
Packit 67cb25
****************************
Packit 67cb25
Packit 67cb25
This chapter describes routines for finding roots of arbitrary
Packit 67cb25
one-dimensional functions.  The library provides low level components
Packit 67cb25
for a variety of iterative solvers and convergence tests.  These can be
Packit 67cb25
combined by the user to achieve the desired solution, with full access
Packit 67cb25
to the intermediate steps of the iteration.  Each class of methods uses
Packit 67cb25
the same framework, so that you can switch between solvers at runtime
Packit 67cb25
without needing to recompile your program.  Each instance of a solver
Packit 67cb25
keeps track of its own state, allowing the solvers to be used in
Packit 67cb25
multi-threaded programs.
Packit 67cb25
Packit 67cb25
The header file :file:`gsl_roots.h` contains prototypes for the root
Packit 67cb25
finding functions and related declarations.
Packit 67cb25
Packit 67cb25
.. index::
Packit 67cb25
   single: root finding, overview
Packit 67cb25
Packit 67cb25
Overview
Packit 67cb25
========
Packit 67cb25
Packit 67cb25
One-dimensional root finding algorithms can be divided into two classes,
Packit 67cb25
*root bracketing* and *root polishing*.  Algorithms which proceed
Packit 67cb25
by bracketing a root are guaranteed to converge.  Bracketing algorithms
Packit 67cb25
begin with a bounded region known to contain a root.  The size of this
Packit 67cb25
bounded region is reduced, iteratively, until it encloses the root to a
Packit 67cb25
desired tolerance.  This provides a rigorous error estimate for the
Packit 67cb25
location of the root.
Packit 67cb25
Packit 67cb25
The technique of *root polishing* attempts to improve an initial
Packit 67cb25
guess to the root.  These algorithms converge only if started "close
Packit 67cb25
enough" to a root, and sacrifice a rigorous error bound for speed.  By
Packit 67cb25
approximating the behavior of a function in the vicinity of a root they
Packit 67cb25
attempt to find a higher order improvement of an initial guess.  When the
Packit 67cb25
behavior of the function is compatible with the algorithm and a good
Packit 67cb25
initial guess is available a polishing algorithm can provide rapid
Packit 67cb25
convergence.
Packit 67cb25
Packit 67cb25
In GSL both types of algorithm are available in similar frameworks.  The
Packit 67cb25
user provides a high-level driver for the algorithms, and the library
Packit 67cb25
provides the individual functions necessary for each of the steps.
Packit 67cb25
There are three main phases of the iteration.  The steps are,
Packit 67cb25
Packit 67cb25
* initialize solver state, :data:`s`, for algorithm :data:`T`
Packit 67cb25
Packit 67cb25
* update :data:`s` using the iteration :data:`T`
Packit 67cb25
Packit 67cb25
* test :data:`s` for convergence, and repeat iteration if necessary
Packit 67cb25
Packit 67cb25
The state for bracketing solvers is held in a :type:`gsl_root_fsolver`
Packit 67cb25
struct.  The updating procedure uses only function evaluations (not
Packit 67cb25
derivatives).  The state for root polishing solvers is held in a
Packit 67cb25
:type:`gsl_root_fdfsolver` struct.  The updates require both the function
Packit 67cb25
and its derivative (hence the name :code:`fdf`) to be supplied by the
Packit 67cb25
user.
Packit 67cb25
Packit 67cb25
.. index::
Packit 67cb25
   single: root finding, caveats
Packit 67cb25
Packit 67cb25
Caveats
Packit 67cb25
=======
Packit 67cb25
Packit 67cb25
Note that root finding functions can only search for one root at a time.
Packit 67cb25
When there are several roots in the search area, the first root to be
Packit 67cb25
found will be returned; however it is difficult to predict which of the
Packit 67cb25
roots this will be. *In most cases, no error will be reported if
Packit 67cb25
you try to find a root in an area where there is more than one.*
Packit 67cb25
Packit 67cb25
Care must be taken when a function may have a multiple root (such as 
Packit 67cb25
:math:`f(x) = (x-x_0)^2` or
Packit 67cb25
:math:`f(x) = (x-x_0)^3`.
Packit 67cb25
It is not possible to use root-bracketing algorithms on
Packit 67cb25
even-multiplicity roots.  For these algorithms the initial interval must
Packit 67cb25
contain a zero-crossing, where the function is negative at one end of
Packit 67cb25
the interval and positive at the other end.  Roots with even-multiplicity
Packit 67cb25
do not cross zero, but only touch it instantaneously.  Algorithms based
Packit 67cb25
on root bracketing will still work for odd-multiplicity roots
Packit 67cb25
(e.g. cubic, quintic, ...). 
Packit 67cb25
Root polishing algorithms generally work with higher multiplicity roots,
Packit 67cb25
but at a reduced rate of convergence.  In these cases the *Steffenson
Packit 67cb25
algorithm* can be used to accelerate the convergence of multiple roots.
Packit 67cb25
Packit 67cb25
While it is not absolutely required that :math:`f` have a root within the
Packit 67cb25
search region, numerical root finding functions should not be used
Packit 67cb25
haphazardly to check for the *existence* of roots.  There are better
Packit 67cb25
ways to do this.  Because it is easy to create situations where numerical
Packit 67cb25
root finders can fail, it is a bad idea to throw a root finder at a
Packit 67cb25
function you do not know much about.  In general it is best to examine
Packit 67cb25
the function visually by plotting before searching for a root.
Packit 67cb25
Packit 67cb25
Initializing the Solver
Packit 67cb25
=======================
Packit 67cb25
Packit 67cb25
.. type:: gsl_root_fsolver
Packit 67cb25
Packit 67cb25
   This is a workspace for finding roots using methods which do not require
Packit 67cb25
   derivatives.
Packit 67cb25
Packit 67cb25
.. type:: gsl_root_fdfsolver
Packit 67cb25
Packit 67cb25
   This is a workspace for finding roots using methods which require
Packit 67cb25
   derivatives.
Packit 67cb25
Packit 67cb25
.. function:: gsl_root_fsolver * gsl_root_fsolver_alloc (const gsl_root_fsolver_type * T)
Packit 67cb25
Packit 67cb25
   This function returns a pointer to a newly allocated instance of a
Packit 67cb25
   solver of type :data:`T`.  For example, the following code creates an
Packit 67cb25
   instance of a bisection solver::
Packit 67cb25
Packit 67cb25
      const gsl_root_fsolver_type * T = gsl_root_fsolver_bisection;
Packit 67cb25
      gsl_root_fsolver * s = gsl_root_fsolver_alloc (T);
Packit 67cb25
Packit 67cb25
   If there is insufficient memory to create the solver then the function
Packit 67cb25
   returns a null pointer and the error handler is invoked with an error
Packit 67cb25
   code of :macro:`GSL_ENOMEM`.
Packit 67cb25
Packit 67cb25
.. function:: gsl_root_fdfsolver * gsl_root_fdfsolver_alloc (const gsl_root_fdfsolver_type * T)
Packit 67cb25
Packit 67cb25
   This function returns a pointer to a newly allocated instance of a
Packit 67cb25
   derivative-based solver of type :data:`T`.  For example, the following
Packit 67cb25
   code creates an instance of a Newton-Raphson solver::
Packit 67cb25
Packit 67cb25
      const gsl_root_fdfsolver_type * T = gsl_root_fdfsolver_newton;
Packit 67cb25
      gsl_root_fdfsolver * s = gsl_root_fdfsolver_alloc (T);
Packit 67cb25
Packit 67cb25
   If there is insufficient memory to create the solver then the function
Packit 67cb25
   returns a null pointer and the error handler is invoked with an error
Packit 67cb25
   code of :macro:`GSL_ENOMEM`.
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_fsolver_set (gsl_root_fsolver * s, gsl_function * f, double x_lower, double x_upper)
Packit 67cb25
Packit 67cb25
   This function initializes, or reinitializes, an existing solver :data:`s`
Packit 67cb25
   to use the function :data:`f` and the initial search interval
Packit 67cb25
   [:data:`x_lower`, :data:`x_upper`].
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_fdfsolver_set (gsl_root_fdfsolver * s, gsl_function_fdf * fdf, double root)
Packit 67cb25
Packit 67cb25
   This function initializes, or reinitializes, an existing solver :data:`s`
Packit 67cb25
   to use the function and derivative :data:`fdf` and the initial guess
Packit 67cb25
   :data:`root`.
Packit 67cb25
Packit 67cb25
.. function:: void gsl_root_fsolver_free (gsl_root_fsolver * s)
Packit 67cb25
              void gsl_root_fdfsolver_free (gsl_root_fdfsolver * s)
Packit 67cb25
Packit 67cb25
   These functions free all the memory associated with the solver :data:`s`.
Packit 67cb25
Packit 67cb25
.. function:: const char * gsl_root_fsolver_name (const gsl_root_fsolver * s)
Packit 67cb25
              const char * gsl_root_fdfsolver_name (const gsl_root_fdfsolver * s)
Packit 67cb25
Packit 67cb25
   These functions return a pointer to the name of the solver.  For example::
Packit 67cb25
Packit 67cb25
      printf ("s is a '%s' solver\n", gsl_root_fsolver_name (s));
Packit 67cb25
Packit 67cb25
   would print something like :code:`s is a 'bisection' solver`.
Packit 67cb25
Packit 67cb25
.. index::
Packit 67cb25
   single: root finding, providing a function to solve
Packit 67cb25
Packit 67cb25
.. _providing-function-to-solve:
Packit 67cb25
Packit 67cb25
Providing the function to solve
Packit 67cb25
===============================
Packit 67cb25
Packit 67cb25
You must provide a continuous function of one variable for the root
Packit 67cb25
finders to operate on, and, sometimes, its first derivative.  In order
Packit 67cb25
to allow for general parameters the functions are defined by the
Packit 67cb25
following data types:
Packit 67cb25
Packit 67cb25
.. type:: gsl_function 
Packit 67cb25
Packit 67cb25
   This data type defines a general function with parameters. 
Packit 67cb25
Packit 67cb25
   :code:`double (* function) (double x, void * params)`
Packit 67cb25
Packit 67cb25
      this function should return the value
Packit 67cb25
      :math:`f(x,params)` for argument :data:`x` and parameters :data:`params`
Packit 67cb25
Packit 67cb25
   :code:`void * params`
Packit 67cb25
Packit 67cb25
      a pointer to the parameters of the function
Packit 67cb25
Packit 67cb25
Here is an example for the general quadratic function,
Packit 67cb25
Packit 67cb25
.. math:: f(x) = a x^2 + b x + c
Packit 67cb25
Packit 67cb25
with :math:`a = 3`, :math:`b = 2`, :math:`c = 1`.  The following code
Packit 67cb25
defines a :type:`gsl_function` :code:`F` which you could pass to a root
Packit 67cb25
finder as a function pointer::
Packit 67cb25
Packit 67cb25
  struct my_f_params { double a; double b; double c; };
Packit 67cb25
Packit 67cb25
  double
Packit 67cb25
  my_f (double x, void * p)
Packit 67cb25
    {
Packit 67cb25
      struct my_f_params * params = (struct my_f_params *)p;
Packit 67cb25
      double a = (params->a);
Packit 67cb25
      double b = (params->b);
Packit 67cb25
      double c = (params->c);
Packit 67cb25
Packit 67cb25
      return  (a * x + b) * x + c;
Packit 67cb25
    }
Packit 67cb25
Packit 67cb25
  gsl_function F;
Packit 67cb25
  struct my_f_params params = { 3.0, 2.0, 1.0 };
Packit 67cb25
Packit 67cb25
  F.function = &my_f;
Packit 67cb25
  F.params = ¶m;;
Packit 67cb25
Packit 67cb25
The function :math:`f(x)` can be evaluated using the macro
Packit 67cb25
:code:`GSL_FN_EVAL(&F,x)` defined in :file:`gsl_math.h`.
Packit 67cb25
Packit 67cb25
.. type:: gsl_function_fdf
Packit 67cb25
Packit 67cb25
   This data type defines a general function with parameters and its first
Packit 67cb25
   derivative.
Packit 67cb25
Packit 67cb25
   :code:`double (* f) (double x, void * params)`
Packit 67cb25
Packit 67cb25
      this function should return the value of
Packit 67cb25
      :math:`f(x,params)` for argument :data:`x` and parameters :data:`params`
Packit 67cb25
Packit 67cb25
   :code:`double (* df) (double x, void * params)`
Packit 67cb25
Packit 67cb25
      this function should return the value of the derivative of :data:`f` with
Packit 67cb25
      respect to :data:`x`,
Packit 67cb25
      :math:`f'(x,params)`, for argument :data:`x` and parameters :data:`params`
Packit 67cb25
Packit 67cb25
   :code:`void (* fdf) (double x, void * params, double * f, double * df)`
Packit 67cb25
Packit 67cb25
      this function should set the values of the function :data:`f` to 
Packit 67cb25
      :math:`f(x,params)`
Packit 67cb25
      and its derivative :data:`df` to
Packit 67cb25
      :math:`f'(x,params)`
Packit 67cb25
      for argument :data:`x` and parameters :data:`params`.  This function
Packit 67cb25
      provides an optimization of the separate functions for :math:`f(x)` and
Packit 67cb25
      :math:`f'(x)`---it is always faster to compute the function and its
Packit 67cb25
      derivative at the same time.
Packit 67cb25
Packit 67cb25
   :code:`void * params`
Packit 67cb25
Packit 67cb25
      a pointer to the parameters of the function
Packit 67cb25
Packit 67cb25
Here is an example where 
Packit 67cb25
:math:`f(x) = \exp(2x)`::
Packit 67cb25
Packit 67cb25
  double
Packit 67cb25
  my_f (double x, void * params)
Packit 67cb25
  {
Packit 67cb25
     return exp (2 * x);
Packit 67cb25
  }
Packit 67cb25
Packit 67cb25
  double
Packit 67cb25
  my_df (double x, void * params)
Packit 67cb25
  {
Packit 67cb25
     return 2 * exp (2 * x);
Packit 67cb25
  }
Packit 67cb25
Packit 67cb25
  void
Packit 67cb25
  my_fdf (double x, void * params, 
Packit 67cb25
          double * f, double * df)
Packit 67cb25
  {
Packit 67cb25
     double t = exp (2 * x);
Packit 67cb25
Packit 67cb25
     *f = t;
Packit 67cb25
     *df = 2 * t;   /* uses existing value */
Packit 67cb25
  }
Packit 67cb25
Packit 67cb25
  gsl_function_fdf FDF;
Packit 67cb25
Packit 67cb25
  FDF.f = &my_f;
Packit 67cb25
  FDF.df = &my_df;
Packit 67cb25
  FDF.fdf = &my_fdf;
Packit 67cb25
  FDF.params = 0;
Packit 67cb25
Packit 67cb25
The function :math:`f(x)` can be evaluated using the macro
Packit 67cb25
:code:`GSL_FN_FDF_EVAL_F(&FDF,x)` and the derivative :math:`f'(x)` can
Packit 67cb25
be evaluated using the macro :code:`GSL_FN_FDF_EVAL_DF(&FDF,x)`.  Both
Packit 67cb25
the function :math:`y = f(x)` and its derivative :math:`dy = f'(x)` can
Packit 67cb25
be evaluated at the same time using the macro
Packit 67cb25
:code:`GSL_FN_FDF_EVAL_F_DF(&FDF,x,y,dy)`.  The macro stores
Packit 67cb25
:math:`f(x)` in its :data:`y` argument and :math:`f'(x)` in its :data:`dy`
Packit 67cb25
argument---both of these should be pointers to :code:`double`.
Packit 67cb25
Packit 67cb25
.. index::
Packit 67cb25
   single: root finding, search bounds
Packit 67cb25
   single: root finding, initial guess
Packit 67cb25
Packit 67cb25
Search Bounds and Guesses
Packit 67cb25
=========================
Packit 67cb25
Packit 67cb25
You provide either search bounds or an initial guess; this section
Packit 67cb25
explains how search bounds and guesses work and how function arguments
Packit 67cb25
control them.
Packit 67cb25
Packit 67cb25
A guess is simply an :math:`x` value which is iterated until it is within
Packit 67cb25
the desired precision of a root.  It takes the form of a :code:`double`.
Packit 67cb25
Packit 67cb25
Search bounds are the endpoints of an interval which is iterated until
Packit 67cb25
the length of the interval is smaller than the requested precision.  The
Packit 67cb25
interval is defined by two values, the lower limit and the upper limit.
Packit 67cb25
Whether the endpoints are intended to be included in the interval or not
Packit 67cb25
depends on the context in which the interval is used.
Packit 67cb25
Packit 67cb25
Iteration
Packit 67cb25
=========
Packit 67cb25
Packit 67cb25
The following functions drive the iteration of each algorithm.  Each
Packit 67cb25
function performs one iteration to update the state of any solver of the
Packit 67cb25
corresponding type.  The same functions work for all solvers so that
Packit 67cb25
different methods can be substituted at runtime without modifications to
Packit 67cb25
the code.
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_fsolver_iterate (gsl_root_fsolver * s)
Packit 67cb25
              int gsl_root_fdfsolver_iterate (gsl_root_fdfsolver * s)
Packit 67cb25
Packit 67cb25
   These functions perform a single iteration of the solver :data:`s`.  If the
Packit 67cb25
   iteration encounters an unexpected problem then an error code will be
Packit 67cb25
   returned,
Packit 67cb25
Packit 67cb25
   :code:`GSL_EBADFUNC`
Packit 67cb25
Packit 67cb25
      the iteration encountered a singular point where the function or its
Packit 67cb25
      derivative evaluated to :code:`Inf` or :code:`NaN`.
Packit 67cb25
Packit 67cb25
   :code:`GSL_EZERODIV`
Packit 67cb25
Packit 67cb25
      the derivative of the function vanished at the iteration point,
Packit 67cb25
      preventing the algorithm from continuing without a division by zero.
Packit 67cb25
Packit 67cb25
The solver maintains a current best estimate of the root at all
Packit 67cb25
times.  The bracketing solvers also keep track of the current best
Packit 67cb25
interval bounding the root.  This information can be accessed with the
Packit 67cb25
following auxiliary functions,
Packit 67cb25
Packit 67cb25
.. function:: double gsl_root_fsolver_root (const gsl_root_fsolver * s)
Packit 67cb25
              double gsl_root_fdfsolver_root (const gsl_root_fdfsolver * s)
Packit 67cb25
Packit 67cb25
   These functions return the current estimate of the root for the solver :data:`s`.
Packit 67cb25
Packit 67cb25
.. function:: double gsl_root_fsolver_x_lower (const gsl_root_fsolver * s)
Packit 67cb25
              double gsl_root_fsolver_x_upper (const gsl_root_fsolver * s)
Packit 67cb25
Packit 67cb25
   These functions return the current bracketing interval for the solver :data:`s`.
Packit 67cb25
Packit 67cb25
.. index::
Packit 67cb25
   single: root finding, stopping parameters
Packit 67cb25
Packit 67cb25
Search Stopping Parameters
Packit 67cb25
==========================
Packit 67cb25
Packit 67cb25
A root finding procedure should stop when one of the following conditions is
Packit 67cb25
true:
Packit 67cb25
Packit 67cb25
* A root has been found to within the user-specified precision.
Packit 67cb25
* A user-specified maximum number of iterations has been reached.
Packit 67cb25
* An error has occurred.
Packit 67cb25
Packit 67cb25
The handling of these conditions is under user control.  The functions
Packit 67cb25
below allow the user to test the precision of the current result in
Packit 67cb25
several standard ways.
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_test_interval (double x_lower, double x_upper, double epsabs, double epsrel)
Packit 67cb25
Packit 67cb25
   This function tests for the convergence of the interval [:data:`x_lower`,
Packit 67cb25
   :data:`x_upper`] with absolute error :data:`epsabs` and relative error
Packit 67cb25
   :data:`epsrel`.  The test returns :macro:`GSL_SUCCESS` if the following
Packit 67cb25
   condition is achieved,
Packit 67cb25
Packit 67cb25
   .. only:: not texinfo
Packit 67cb25
Packit 67cb25
      .. math:: |a - b| < \hbox{\it epsabs} + \hbox{\it epsrel\/}\, \min(|a|,|b|)
Packit 67cb25
Packit 67cb25
   .. only:: texinfo
Packit 67cb25
Packit 67cb25
      ::
Packit 67cb25
Packit 67cb25
         |a - b| < epsabs + epsrel min(|a|,|b|) 
Packit 67cb25
Packit 67cb25
   when the interval :math:`x = [a,b]` does not include the origin.  If the
Packit 67cb25
   interval includes the origin then :math:`\min(|a|,|b|)` is replaced by
Packit 67cb25
   zero (which is the minimum value of :math:`|x|` over the interval).  This
Packit 67cb25
   ensures that the relative error is accurately estimated for roots close
Packit 67cb25
   to the origin.
Packit 67cb25
Packit 67cb25
   This condition on the interval also implies that any estimate of the
Packit 67cb25
   root :math:`r` in the interval satisfies the same condition with respect
Packit 67cb25
   to the true root :math:`r^*`,
Packit 67cb25
Packit 67cb25
   .. only:: not texinfo
Packit 67cb25
Packit 67cb25
      .. math:: |r - r^*| < \hbox{\it epsabs} + \hbox{\it epsrel\/}\, r^*
Packit 67cb25
Packit 67cb25
   .. only:: texinfo
Packit 67cb25
Packit 67cb25
      ::
Packit 67cb25
Packit 67cb25
         |r - r^*| < epsabs + epsrel r^*
Packit 67cb25
Packit 67cb25
   assuming that the true root :math:`r^*` is contained within the interval.
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_test_delta (double x1, double x0, double epsabs, double epsrel)
Packit 67cb25
Packit 67cb25
   This function tests for the convergence of the sequence :data:`x0`,
Packit 67cb25
   :data:`x1` with absolute error :data:`epsabs` and relative error
Packit 67cb25
   :data:`epsrel`.  The test returns :macro:`GSL_SUCCESS` if the following
Packit 67cb25
   condition is achieved,
Packit 67cb25
Packit 67cb25
   .. only:: not texinfo
Packit 67cb25
Packit 67cb25
      .. math:: |x_1 - x_0| < \hbox{\it epsabs} + \hbox{\it epsrel\/}\, |x_1|
Packit 67cb25
Packit 67cb25
   .. only:: texinfo
Packit 67cb25
Packit 67cb25
      ::
Packit 67cb25
Packit 67cb25
         |x_1 - x_0| < epsabs + epsrel |x_1|
Packit 67cb25
Packit 67cb25
   and returns :macro:`GSL_CONTINUE` otherwise.
Packit 67cb25
Packit 67cb25
.. function:: int gsl_root_test_residual (double f, double epsabs)
Packit 67cb25
Packit 67cb25
   This function tests the residual value :data:`f` against the absolute
Packit 67cb25
   error bound :data:`epsabs`.  The test returns :macro:`GSL_SUCCESS` if the
Packit 67cb25
   following condition is achieved,
Packit 67cb25
Packit 67cb25
   .. only:: not texinfo
Packit 67cb25
Packit 67cb25
      .. math:: |f| < \hbox{\it epsabs}
Packit 67cb25
Packit 67cb25
   .. only:: texinfo
Packit 67cb25
Packit 67cb25
      ::
Packit 67cb25
Packit 67cb25
         |f| < epsabs
Packit 67cb25
Packit 67cb25
   and returns :macro:`GSL_CONTINUE` otherwise.  This criterion is suitable
Packit 67cb25
   for situations where the precise location of the root, :math:`x`, is
Packit 67cb25
   unimportant provided a value can be found where the residual,
Packit 67cb25
   :math:`|f(x)|`, is small enough.
Packit 67cb25
Packit 67cb25
Root Bracketing Algorithms
Packit 67cb25
==========================
Packit 67cb25
Packit 67cb25
The root bracketing algorithms described in this section require an
Packit 67cb25
initial interval which is guaranteed to contain a root---if :math:`a`
Packit 67cb25
and :math:`b` are the endpoints of the interval then :math:`f(a)` must
Packit 67cb25
differ in sign from :math:`f(b)`.  This ensures that the function crosses
Packit 67cb25
zero at least once in the interval.  If a valid initial interval is used
Packit 67cb25
then these algorithm cannot fail, provided the function is well-behaved.
Packit 67cb25
Packit 67cb25
Note that a bracketing algorithm cannot find roots of even degree, since
Packit 67cb25
these do not cross the :math:`x`-axis.
Packit 67cb25
Packit 67cb25
.. type:: gsl_root_fsolver_type
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: bisection algorithm for finding roots
Packit 67cb25
      single: root finding, bisection algorithm
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fsolver_bisection
Packit 67cb25
Packit 67cb25
      The *bisection algorithm* is the simplest method of bracketing the
Packit 67cb25
      roots of a function.   It is the slowest algorithm provided by
Packit 67cb25
      the library, with linear convergence.
Packit 67cb25
Packit 67cb25
      On each iteration, the interval is bisected and the value of the
Packit 67cb25
      function at the midpoint is calculated.  The sign of this value is used
Packit 67cb25
      to determine which half of the interval does not contain a root.  That
Packit 67cb25
      half is discarded to give a new, smaller interval containing the
Packit 67cb25
      root.  This procedure can be continued indefinitely until the interval is
Packit 67cb25
      sufficiently small.
Packit 67cb25
Packit 67cb25
      At any time the current estimate of the root is taken as the midpoint of
Packit 67cb25
      the interval.
Packit 67cb25
Packit 67cb25
      .. eps file "roots-bisection.eps"
Packit 67cb25
      .. @iftex
Packit 67cb25
      .. @sp 1
Packit 67cb25
      .. @center @image{roots-bisection,3.4in}
Packit 67cb25
Packit 67cb25
      .. @quotation
Packit 67cb25
      .. Four iterations of bisection, where :math:`a_n` is :math:`n`-th position of
Packit 67cb25
      .. the beginning of the interval and :math:`b_n` is the :math:`n`-th position
Packit 67cb25
      .. of the end.  The midpoint of each interval is also indicated.
Packit 67cb25
      .. @end quotation
Packit 67cb25
      .. @end iftex
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: false position algorithm for finding roots
Packit 67cb25
      single: root finding, false position algorithm
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fsolver_falsepos
Packit 67cb25
Packit 67cb25
      The *false position algorithm* is a method of finding roots based on
Packit 67cb25
      linear interpolation.  Its convergence is linear, but it is usually
Packit 67cb25
      faster than bisection.
Packit 67cb25
Packit 67cb25
      On each iteration a line is drawn between the endpoints :math:`(a,f(a))`
Packit 67cb25
      and :math:`(b,f(b))` and the point where this line crosses the
Packit 67cb25
      :math:`x`-axis taken as a "midpoint".  The value of the function at
Packit 67cb25
      this point is calculated and its sign is used to determine which side of
Packit 67cb25
      the interval does not contain a root.  That side is discarded to give a
Packit 67cb25
      new, smaller interval containing the root.  This procedure can be
Packit 67cb25
      continued indefinitely until the interval is sufficiently small.
Packit 67cb25
Packit 67cb25
      The best estimate of the root is taken from the linear interpolation of
Packit 67cb25
      the interval on the current iteration.
Packit 67cb25
Packit 67cb25
      .. eps file "roots-false-position.eps"
Packit 67cb25
      .. @iftex
Packit 67cb25
      .. @image{roots-false-position,4in}
Packit 67cb25
      .. @quotation
Packit 67cb25
      .. Several iterations of false position, where :math:`a_n` is :math:`n`-th
Packit 67cb25
      .. position of the beginning of the interval and :math:`b_n` is the
Packit 67cb25
      .. :math:`n`-th position of the end.
Packit 67cb25
      .. @end quotation
Packit 67cb25
      .. @end iftex
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: Brent's method for finding roots
Packit 67cb25
      single: root finding, Brent's method
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fsolver_brent
Packit 67cb25
Packit 67cb25
      The *Brent-Dekker method* (referred to here as *Brent's method*)
Packit 67cb25
      combines an interpolation strategy with the bisection algorithm.  This
Packit 67cb25
      produces a fast algorithm which is still robust.
Packit 67cb25
Packit 67cb25
      On each iteration Brent's method approximates the function using an
Packit 67cb25
      interpolating curve.  On the first iteration this is a linear
Packit 67cb25
      interpolation of the two endpoints.  For subsequent iterations the
Packit 67cb25
      algorithm uses an inverse quadratic fit to the last three points, for
Packit 67cb25
      higher accuracy.  The intercept of the interpolating curve with the
Packit 67cb25
      :math:`x`-axis is taken as a guess for the root.  If it lies within the
Packit 67cb25
      bounds of the current interval then the interpolating point is accepted,
Packit 67cb25
      and used to generate a smaller interval.  If the interpolating point is
Packit 67cb25
      not accepted then the algorithm falls back to an ordinary bisection
Packit 67cb25
      step.
Packit 67cb25
Packit 67cb25
      The best estimate of the root is taken from the most recent
Packit 67cb25
      interpolation or bisection.
Packit 67cb25
Packit 67cb25
Root Finding Algorithms using Derivatives
Packit 67cb25
=========================================
Packit 67cb25
Packit 67cb25
The root polishing algorithms described in this section require an
Packit 67cb25
initial guess for the location of the root.  There is no absolute
Packit 67cb25
guarantee of convergence---the function must be suitable for this
Packit 67cb25
technique and the initial guess must be sufficiently close to the root
Packit 67cb25
for it to work.  When these conditions are satisfied then convergence is
Packit 67cb25
quadratic.
Packit 67cb25
Packit 67cb25
These algorithms make use of both the function and its derivative. 
Packit 67cb25
Packit 67cb25
.. type:: gsl_root_fdfsolver_type
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: Newton's method for finding roots
Packit 67cb25
      single: root finding, Newton's method
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fdfsolver_newton
Packit 67cb25
Packit 67cb25
      Newton's Method is the standard root-polishing algorithm.  The algorithm
Packit 67cb25
      begins with an initial guess for the location of the root.  On each
Packit 67cb25
      iteration, a line tangent to the function :math:`f` is drawn at that
Packit 67cb25
      position.  The point where this line crosses the :math:`x`-axis becomes
Packit 67cb25
      the new guess.  The iteration is defined by the following sequence,
Packit 67cb25
Packit 67cb25
      .. only:: not texinfo
Packit 67cb25
Packit 67cb25
         .. math:: x_{i+1} = x_i - {f(x_i) \over f'(x_i)}
Packit 67cb25
Packit 67cb25
      .. only:: texinfo
Packit 67cb25
Packit 67cb25
         ::
Packit 67cb25
Packit 67cb25
            x_{i+1} = x_i - f(x_i)/f'(x_i)
Packit 67cb25
Packit 67cb25
      Newton's method converges quadratically for single roots, and linearly
Packit 67cb25
      for multiple roots.
Packit 67cb25
Packit 67cb25
      .. eps file "roots-newtons-method.eps"
Packit 67cb25
      .. @iftex
Packit 67cb25
      .. @sp 1
Packit 67cb25
      .. @center @image{roots-newtons-method,3.4in}
Packit 67cb25
Packit 67cb25
      .. @quotation
Packit 67cb25
      .. Several iterations of Newton's Method, where :math:`g_n` is the
Packit 67cb25
      .. :math:`n`-th guess.
Packit 67cb25
      .. @end quotation
Packit 67cb25
      .. @end iftex
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: secant method for finding roots
Packit 67cb25
      single: root finding, secant method
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fdfsolver_secant
Packit 67cb25
Packit 67cb25
      The *secant method* is a simplified version of Newton's method which does
Packit 67cb25
      not require the computation of the derivative on every step.
Packit 67cb25
Packit 67cb25
      On its first iteration the algorithm begins with Newton's method, using
Packit 67cb25
      the derivative to compute a first step,
Packit 67cb25
Packit 67cb25
      .. only:: not texinfo
Packit 67cb25
Packit 67cb25
         .. math:: x_1 = x_0 - {f(x_0) \over f'(x_0)}
Packit 67cb25
Packit 67cb25
      .. only:: texinfo
Packit 67cb25
Packit 67cb25
         ::
Packit 67cb25
Packit 67cb25
            x_1 = x_0 - f(x_0)/f'(x_0)
Packit 67cb25
Packit 67cb25
      Subsequent iterations avoid the evaluation of the derivative by
Packit 67cb25
      replacing it with a numerical estimate, the slope of the line through
Packit 67cb25
      the previous two points,
Packit 67cb25
Packit 67cb25
      .. only:: not texinfo
Packit 67cb25
Packit 67cb25
         .. math::
Packit 67cb25
Packit 67cb25
            x_{i+1} = x_i - {f(x_i) \over f'_{est}}
Packit 67cb25
             ~\hbox{where}~
Packit 67cb25
             f'_{est} =  {f(x_{i}) - f(x_{i-1}) \over x_i - x_{i-1}}
Packit 67cb25
Packit 67cb25
      .. only:: texinfo
Packit 67cb25
      
Packit 67cb25
         ::
Packit 67cb25
Packit 67cb25
            x_{i+1} = x_i f(x_i) / f'_{est} where
Packit 67cb25
             f'_{est} = (f(x_i) - f(x_{i-1})/(x_i - x_{i-1})
Packit 67cb25
Packit 67cb25
      When the derivative does not change significantly in the vicinity of the
Packit 67cb25
      root the secant method gives a useful saving.  Asymptotically the secant
Packit 67cb25
      method is faster than Newton's method whenever the cost of evaluating
Packit 67cb25
      the derivative is more than 0.44 times the cost of evaluating the
Packit 67cb25
      function itself.  As with all methods of computing a numerical
Packit 67cb25
      derivative the estimate can suffer from cancellation errors if the
Packit 67cb25
      separation of the points becomes too small.
Packit 67cb25
Packit 67cb25
      On single roots, the method has a convergence of order :math:`(1 + \sqrt
Packit 67cb25
      5)/2` (approximately :math:`1.62`).  It converges linearly for multiple
Packit 67cb25
      roots.  
Packit 67cb25
Packit 67cb25
      .. eps file "roots-secant-method.eps"
Packit 67cb25
      .. @iftex
Packit 67cb25
      .. @tex
Packit 67cb25
      .. \input epsf
Packit 67cb25
      .. \medskip
Packit 67cb25
      .. \centerline{\epsfxsize=5in\epsfbox{roots-secant-method.eps}}
Packit 67cb25
      .. @end tex
Packit 67cb25
      .. @quotation
Packit 67cb25
      .. Several iterations of Secant Method, where :math:`g_n` is the :math:`n`-th
Packit 67cb25
      .. guess.
Packit 67cb25
      .. @end quotation
Packit 67cb25
      .. @end iftex
Packit 67cb25
Packit 67cb25
   .. index::
Packit 67cb25
      single: Steffenson's method for finding roots
Packit 67cb25
      single: root finding, Steffenson's method
Packit 67cb25
Packit 67cb25
   .. var:: gsl_root_fdfsolver_steffenson
Packit 67cb25
Packit 67cb25
      The *Steffenson Method* [#f1]_
Packit 67cb25
      provides the fastest
Packit 67cb25
      convergence of all the routines.  It combines the basic Newton
Packit 67cb25
      algorithm with an Aitken "delta-squared" acceleration.  If the
Packit 67cb25
      Newton iterates are :math:`x_i` then the acceleration procedure
Packit 67cb25
      generates a new sequence :math:`R_i`,
Packit 67cb25
Packit 67cb25
      .. only:: not texinfo
Packit 67cb25
Packit 67cb25
         .. math:: R_i = x_i - {(x_{i+1} - x_i)^2 \over (x_{i+2} - 2 x_{i+1} + x_i)}
Packit 67cb25
Packit 67cb25
      .. only:: texinfo
Packit 67cb25
Packit 67cb25
         ::
Packit 67cb25
Packit 67cb25
            R_i = x_i - (x_{i+1} - x_i)^2 / (x_{i+2} - 2 x_{i+1} + x_{i})
Packit 67cb25
Packit 67cb25
      which converges faster than the original sequence under reasonable
Packit 67cb25
      conditions.  The new sequence requires three terms before it can produce
Packit 67cb25
      its first value so the method returns accelerated values on the second
Packit 67cb25
      and subsequent iterations.  On the first iteration it returns the
Packit 67cb25
      ordinary Newton estimate.  The Newton iterate is also returned if the
Packit 67cb25
      denominator of the acceleration term ever becomes zero.
Packit 67cb25
Packit 67cb25
      As with all acceleration procedures this method can become unstable if
Packit 67cb25
      the function is not well-behaved. 
Packit 67cb25
Packit 67cb25
Examples
Packit 67cb25
========
Packit 67cb25
Packit 67cb25
For any root finding algorithm we need to prepare the function to be
Packit 67cb25
solved.  For this example we will use the general quadratic equation
Packit 67cb25
described earlier.  We first need a header file (:file:`demo_fn.h`) to
Packit 67cb25
define the function parameters,
Packit 67cb25
Packit 67cb25
.. include:: examples/demo_fn.h
Packit 67cb25
   :code:
Packit 67cb25
Packit 67cb25
We place the function definitions in a separate file (:file:`demo_fn.c`),
Packit 67cb25
Packit 67cb25
.. include:: examples/demo_fn.c
Packit 67cb25
   :code:
Packit 67cb25
Packit 67cb25
The first program uses the function solver :data:`gsl_root_fsolver_brent`
Packit 67cb25
for Brent's method and the general quadratic defined above to solve the
Packit 67cb25
following equation,
Packit 67cb25
Packit 67cb25
.. math:: x^2 - 5 = 0
Packit 67cb25
Packit 67cb25
with solution :math:`x = \sqrt 5 = 2.236068...`
Packit 67cb25
Packit 67cb25
.. include:: examples/roots.c
Packit 67cb25
   :code:
Packit 67cb25
Packit 67cb25
Here are the results of the iterations::
Packit 67cb25
Packit 67cb25
  $ ./a.out 
Packit 67cb25
  using brent method
Packit 67cb25
   iter [    lower,     upper]      root        err  err(est)
Packit 67cb25
      1 [1.0000000, 5.0000000] 1.0000000 -1.2360680 4.0000000
Packit 67cb25
      2 [1.0000000, 3.0000000] 3.0000000 +0.7639320 2.0000000
Packit 67cb25
      3 [2.0000000, 3.0000000] 2.0000000 -0.2360680 1.0000000
Packit 67cb25
      4 [2.2000000, 3.0000000] 2.2000000 -0.0360680 0.8000000
Packit 67cb25
      5 [2.2000000, 2.2366300] 2.2366300 +0.0005621 0.0366300
Packit 67cb25
  Converged:                            
Packit 67cb25
      6 [2.2360634, 2.2366300] 2.2360634 -0.0000046 0.0005666
Packit 67cb25
Packit 67cb25
If the program is modified to use the bisection solver instead of
Packit 67cb25
Brent's method, by changing :data:`gsl_root_fsolver_brent` to
Packit 67cb25
:data:`gsl_root_fsolver_bisection` the slower convergence of the
Packit 67cb25
Bisection method can be observed::
Packit 67cb25
Packit 67cb25
  $ ./a.out 
Packit 67cb25
  using bisection method
Packit 67cb25
   iter [    lower,     upper]      root        err  err(est)
Packit 67cb25
      1 [0.0000000, 2.5000000] 1.2500000 -0.9860680 2.5000000
Packit 67cb25
      2 [1.2500000, 2.5000000] 1.8750000 -0.3610680 1.2500000
Packit 67cb25
      3 [1.8750000, 2.5000000] 2.1875000 -0.0485680 0.6250000
Packit 67cb25
      4 [2.1875000, 2.5000000] 2.3437500 +0.1076820 0.3125000
Packit 67cb25
      5 [2.1875000, 2.3437500] 2.2656250 +0.0295570 0.1562500
Packit 67cb25
      6 [2.1875000, 2.2656250] 2.2265625 -0.0095055 0.0781250
Packit 67cb25
      7 [2.2265625, 2.2656250] 2.2460938 +0.0100258 0.0390625
Packit 67cb25
      8 [2.2265625, 2.2460938] 2.2363281 +0.0002601 0.0195312
Packit 67cb25
      9 [2.2265625, 2.2363281] 2.2314453 -0.0046227 0.0097656
Packit 67cb25
     10 [2.2314453, 2.2363281] 2.2338867 -0.0021813 0.0048828
Packit 67cb25
     11 [2.2338867, 2.2363281] 2.2351074 -0.0009606 0.0024414
Packit 67cb25
  Converged:                            
Packit 67cb25
     12 [2.2351074, 2.2363281] 2.2357178 -0.0003502 0.0012207
Packit 67cb25
Packit 67cb25
The next program solves the same function using a derivative solver
Packit 67cb25
instead.
Packit 67cb25
Packit 67cb25
.. include:: examples/rootnewt.c
Packit 67cb25
   :code:
Packit 67cb25
Packit 67cb25
Here are the results for Newton's method::
Packit 67cb25
Packit 67cb25
  $ ./a.out 
Packit 67cb25
  using newton method
Packit 67cb25
  iter        root        err   err(est)
Packit 67cb25
      1  3.0000000 +0.7639320 -2.0000000
Packit 67cb25
      2  2.3333333 +0.0972654 -0.6666667
Packit 67cb25
      3  2.2380952 +0.0020273 -0.0952381
Packit 67cb25
  Converged:      
Packit 67cb25
      4  2.2360689 +0.0000009 -0.0020263
Packit 67cb25
Packit 67cb25
Note that the error can be estimated more accurately by taking the
Packit 67cb25
difference between the current iterate and next iterate rather than the
Packit 67cb25
previous iterate.  The other derivative solvers can be investigated by
Packit 67cb25
changing :data:`gsl_root_fdfsolver_newton` to
Packit 67cb25
:data:`gsl_root_fdfsolver_secant` or
Packit 67cb25
:data:`gsl_root_fdfsolver_steffenson`.
Packit 67cb25
Packit 67cb25
References and Further Reading
Packit 67cb25
==============================
Packit 67cb25
Packit 67cb25
For information on the Brent-Dekker algorithm see the following two
Packit 67cb25
papers,
Packit 67cb25
Packit 67cb25
* R. P. Brent, "An algorithm with guaranteed convergence for finding a
Packit 67cb25
  zero of a function", *Computer Journal*, 14 (1971) 422--425
Packit 67cb25
Packit 67cb25
* J. C. P. Bus and T. J. Dekker, "Two Efficient Algorithms with Guaranteed
Packit 67cb25
  Convergence for Finding a Zero of a Function", *ACM Transactions of
Packit 67cb25
  Mathematical Software*, Vol.: 1 No.: 4 (1975) 330--345
Packit 67cb25
Packit 67cb25
.. rubric:: Footnotes
Packit 67cb25
Packit 67cb25
.. [#f1] J.F. Steffensen (1873--1961). The spelling used in the name of the
Packit 67cb25
         function is slightly incorrect, but has been preserved to avoid incompatibility.