.. default-domain:: cpp

.. cpp:namespace:: ceres

.. _`chapter-nnls_modeling`:

=================================

Modeling Non-linear Least Squares

=================================

Introduction

============

Ceres solver consists of two distinct parts. A modeling API which

provides a rich set of tools to construct an optimization problem one

term at a time and a solver API that controls the minimization

algorithm. This chapter is devoted to the task of modeling

optimization problems using Ceres. :ref:`chapter-nnls_solving` discusses

the various ways in which an optimization problem can be solved using

Ceres.

Ceres solves robustified bounds constrained non-linear least squares

problems of the form:

.. math:: :label: ceresproblem

   \min_{\mathbf{x}} &\quad \frac{1}{2}\sum_{i}

   \rho_i\left(\left\|f_i\left(x_{i_1},

   ... ,x_{i_k}\right)\right\|^2\right)  \\

   \text{s.t.} &\quad l_j \le x_j \le u_j

In Ceres parlance, the expression

:math:`\rho_i\left(\left\|f_i\left(x_{i_1},...,x_{i_k}\right)\right\|^2\right)`

is known as a **residual block**, where :math:`f_i(\cdot)` is a

:class:`CostFunction` that depends on the **parameter blocks**

:math:`\left\{x_{i_1},... , x_{i_k}\right\}`.

In most optimization problems small groups of scalars occur

together. For example the three components of a translation vector and

the four components of the quaternion that define the pose of a

camera. We refer to such a group of scalars as a **parameter block**. Of

course a parameter block can be just a single scalar too.

:math:`\rho_i` is a :class:`LossFunction`. A :class:`LossFunction` is

a scalar valued function that is used to reduce the influence of

outliers on the solution of non-linear least squares problems.

:math:`l_j` and :math:`u_j` are lower and upper bounds on the

parameter block :math:`x_j`.

As a special case, when :math:`\rho_i(x) = x`, i.e., the identity

function, and :math:`l_j = -\infty` and :math:`u_j = \infty` we get

the more familiar unconstrained `non-linear least squares problem

<http://en.wikipedia.org/wiki/Non-linear_least_squares>`_.

.. math:: :label: ceresproblemunconstrained

   \frac{1}{2}\sum_{i} \left\|f_i\left(x_{i_1}, ... ,x_{i_k}\right)\right\|^2.

:class:`CostFunction`

=====================

For each term in the objective function, a :class:`CostFunction` is

responsible for computing a vector of residuals and if asked a vector

of Jacobian matrices, i.e., given :math:`\left[x_{i_1}, ... ,

x_{i_k}\right]`, compute the vector

:math:`f_i\left(x_{i_1},...,x_{i_k}\right)` and the matrices

 .. math:: J_{ij} = \frac{\partial}{\partial

           x_{i_j}}f_i\left(x_{i_1},...,x_{i_k}\right),\quad \forall j

           \in \{1, \ldots, k\}

.. class:: CostFunction

   .. code-block:: c++

    class CostFunction {

     public:

      virtual bool Evaluate(double const* const* parameters,

                            double* residuals,

                            double** jacobians) = 0;

      const vector<int32>& parameter_block_sizes();

      int num_residuals() const;

     protected:

      vector<int32>* mutable_parameter_block_sizes();

      void set_num_residuals(int num_residuals);

    };

The signature of the :class:`CostFunction` (number and sizes of input

parameter blocks and number of outputs) is stored in

:member:`CostFunction::parameter_block_sizes_` and

:member:`CostFunction::num_residuals_` respectively. User code

inheriting from this class is expected to set these two members with

the corresponding accessors. This information will be verified by the

:class:`Problem` when added with :func:`Problem::AddResidualBlock`.

.. function:: bool CostFunction::Evaluate(double const* const* parameters, double* residuals, double** jacobians)

   Compute the residual vector and the Jacobian matrices.

   ``parameters`` is an array of pointers to arrays containing the

   various parameter blocks. ``parameters`` has the same number of

   elements as :member:`CostFunction::parameter_block_sizes_` and the

   parameter blocks are in the same order as

   :member:`CostFunction::parameter_block_sizes_`.

   ``residuals`` is an array of size ``num_residuals_``.

   ``jacobians`` is an array of size

   :member:`CostFunction::parameter_block_sizes_` containing pointers

   to storage for Jacobian matrices corresponding to each parameter

   block. The Jacobian matrices are in the same order as

   :member:`CostFunction::parameter_block_sizes_`. ``jacobians[i]`` is

   an array that contains :member:`CostFunction::num_residuals_` x

   :member:`CostFunction::parameter_block_sizes_` ``[i]``

   elements. Each Jacobian matrix is stored in row-major order, i.e.,

   ``jacobians[i][r * parameter_block_size_[i] + c]`` =

   :math:`\frac{\partial residual[r]}{\partial parameters[i][c]}`

   If ``jacobians`` is ``NULL``, then no derivatives are returned;

   this is the case when computing cost only. If ``jacobians[i]`` is

   ``NULL``, then the Jacobian matrix corresponding to the

   :math:`i^{\textrm{th}}` parameter block must not be returned, this

   is the case when a parameter block is marked constant.

   **NOTE** The return value indicates whether the computation of the

   residuals and/or jacobians was successful or not.

   This can be used to communicate numerical failures in Jacobian

   computations for instance.

:class:`SizedCostFunction`

==========================

.. class:: SizedCostFunction

   If the size of the parameter blocks and the size of the residual

   vector is known at compile time (this is the common case),

   :class:`SizeCostFunction` can be used where these values can be

   specified as template parameters and the user only needs to

   implement :func:`CostFunction::Evaluate`.

   .. code-block:: c++

    template

             int N0 = 0, int N1 = 0, int N2 = 0, int N3 = 0, int N4 = 0,

             int N5 = 0, int N6 = 0, int N7 = 0, int N8 = 0, int N9 = 0>

    class SizedCostFunction : public CostFunction {

     public:

      virtual bool Evaluate(double const* const* parameters,

                            double* residuals,

                            double** jacobians) const = 0;

    };

:class:`AutoDiffCostFunction`

=============================

.. class:: AutoDiffCostFunction

   Defining a :class:`CostFunction` or a :class:`SizedCostFunction`

   can be a tedious and error prone especially when computing

   derivatives.  To this end Ceres provides `automatic differentiation

   <http://en.wikipedia.org/wiki/Automatic_differentiation>`_.

   .. code-block:: c++

     template 

            int kNumResiduals,  // Number of residuals, or ceres::DYNAMIC.

            int N0,       // Number of parameters in block 0.

            int N1 = 0,   // Number of parameters in block 1.

            int N2 = 0,   // Number of parameters in block 2.

            int N3 = 0,   // Number of parameters in block 3.

            int N4 = 0,   // Number of parameters in block 4.

            int N5 = 0,   // Number of parameters in block 5.

            int N6 = 0,   // Number of parameters in block 6.

            int N7 = 0,   // Number of parameters in block 7.

            int N8 = 0,   // Number of parameters in block 8.

            int N9 = 0>   // Number of parameters in block 9.

     class AutoDiffCostFunction : public

     SizedCostFunction<kNumResiduals, N0, N1, N2, N3, N4, N5, N6, N7, N8, N9> {

      public:

       explicit AutoDiffCostFunction(CostFunctor* functor);

       // Ignore the template parameter kNumResiduals and use

       // num_residuals instead.

       AutoDiffCostFunction(CostFunctor* functor, int num_residuals);

     };

   To get an auto differentiated cost function, you must define a

   class with a templated ``operator()`` (a functor) that computes the

   cost function in terms of the template parameter ``T``. The

   autodiff framework substitutes appropriate ``Jet`` objects for

   ``T`` in order to compute the derivative when necessary, but this

   is hidden, and you should write the function as if ``T`` were a

   scalar type (e.g. a double-precision floating point number).

   The function must write the computed value in the last argument

   (the only non-``const`` one) and return true to indicate success.

   For example, consider a scalar error :math:`e = k - x^\top y`,

   where both :math:`x` and :math:`y` are two-dimensional vector

   parameters and :math:`k` is a constant. The form of this error,

   which is the difference between a constant and an expression, is a

   common pattern in least squares problems. For example, the value

   :math:`x^\top y` might be the model expectation for a series of

   measurements, where there is an instance of the cost function for

   each measurement :math:`k`.

   The actual cost added to the total problem is :math:`e^2`, or

   :math:`(k - x^\top y)^2`; however, the squaring is implicitly done

   by the optimization framework.

   To write an auto-differentiable cost function for the above model,

   first define the object

   .. code-block:: c++

    class MyScalarCostFunctor {

      MyScalarCostFunctor(double k): k_(k) {}

      template <typename T>

      bool operator()(const T* const x , const T* const y, T* e) const {

        e[0] = k_ - x[0] * y[0] - x[1] * y[1];

        return true;

      }

     private:

      double k_;

    };

   Note that in the declaration of ``operator()`` the input parameters

   ``x`` and ``y`` come first, and are passed as const pointers to arrays

   of ``T``. If there were three input parameters, then the third input

   parameter would come after ``y``. The output is always the last

   parameter, and is also a pointer to an array. In the example above,

   ``e`` is a scalar, so only ``e[0]`` is set.

   Then given this class definition, the auto differentiated cost

   function for it can be constructed as follows.

   .. code-block:: c++

    CostFunction* cost_function

        = new AutoDiffCostFunction<MyScalarCostFunctor, 1, 2, 2>(

            new MyScalarCostFunctor(1.0));              ^  ^  ^

                                                        |  |  |

                            Dimension of residual ------+  |  |

                            Dimension of x ----------------+  |

                            Dimension of y -------------------+

   In this example, there is usually an instance for each measurement

   of ``k``.

   In the instantiation above, the template parameters following

   ``MyScalarCostFunction``, ``<1, 2, 2>`` describe the functor as

   computing a 1-dimensional output from two arguments, both

   2-dimensional.

   :class:`AutoDiffCostFunction` also supports cost functions with a

   runtime-determined number of residuals. For example:

   .. code-block:: c++

     CostFunction* cost_function

         = new AutoDiffCostFunction<MyScalarCostFunctor, DYNAMIC, 2, 2>(

             new CostFunctorWithDynamicNumResiduals(1.0),   ^     ^  ^

             runtime_number_of_residuals); <----+           |     |  |

                                                |           |     |  |

                                                |           |     |  |

               Actual number of residuals ------+           |     |  |

               Indicate dynamic number of residuals --------+     |  |

               Dimension of x ------------------------------------+  |

               Dimension of y ---------------------------------------+

   The framework can currently accommodate cost functions of up to 10

   independent variables, and there is no limit on the dimensionality

   of each of them.

   **WARNING 1** A common beginner's error when first using

   :class:`AutoDiffCostFunction` is to get the sizing wrong. In particular,

   there is a tendency to set the template parameters to (dimension of

   residual, number of parameters) instead of passing a dimension

   parameter for *every parameter block*. In the example above, that

   would be ``<MyScalarCostFunction, 1, 2>``, which is missing the 2

   as the last template argument.

:class:`DynamicAutoDiffCostFunction`

====================================

.. class:: DynamicAutoDiffCostFunction

   :class:`AutoDiffCostFunction` requires that the number of parameter

   blocks and their sizes be known at compile time. It also has an

   upper limit of 10 parameter blocks. In a number of applications,

   this is not enough e.g., Bezier curve fitting, Neural Network

   training etc.

     .. code-block:: c++

      template <typename CostFunctor, int Stride = 4>

      class DynamicAutoDiffCostFunction : public CostFunction {

      };

   In such cases :class:`DynamicAutoDiffCostFunction` can be

   used. Like :class:`AutoDiffCostFunction` the user must define a

   templated functor, but the signature of the functor differs

   slightly. The expected interface for the cost functors is:

     .. code-block:: c++

       struct MyCostFunctor {

         template<typename T>

         bool operator()(T const* const* parameters, T* residuals) const {

         }

       }

   Since the sizing of the parameters is done at runtime, you must

   also specify the sizes after creating the dynamic autodiff cost

   function. For example:

     .. code-block:: c++

       DynamicAutoDiffCostFunction<MyCostFunctor, 4>* cost_function =

         new DynamicAutoDiffCostFunction<MyCostFunctor, 4>(

           new MyCostFunctor());

       cost_function->AddParameterBlock(5);

       cost_function->AddParameterBlock(10);

       cost_function->SetNumResiduals(21);

   Under the hood, the implementation evaluates the cost function

   multiple times, computing a small set of the derivatives (four by

   default, controlled by the ``Stride`` template parameter) with each

   pass. There is a performance tradeoff with the size of the passes;

   Smaller sizes are more cache efficient but result in larger number

   of passes, and larger stride lengths can destroy cache-locality

   while reducing the number of passes over the cost function. The

   optimal value depends on the number and sizes of the various

   parameter blocks.

   As a rule of thumb, try using :class:`AutoDiffCostFunction` before

   you use :class:`DynamicAutoDiffCostFunction`.

:class:`NumericDiffCostFunction`

================================

.. class:: NumericDiffCostFunction

  In some cases, its not possible to define a templated cost functor,

  for example when the evaluation of the residual involves a call to a

  library function that you do not have control over.  In such a

  situation, `numerical differentiation

  <http://en.wikipedia.org/wiki/Numerical_differentiation>`_ can be

  used.

  .. NOTE ::

    TODO(sameeragarwal): Add documentation for the constructor and for

    NumericDiffOptions. Update DynamicNumericDiffOptions in a similar

    manner.

  .. code-block:: c++

      template 

                NumericDiffMethodType method = CENTRAL,

                int kNumResiduals,  // Number of residuals, or ceres::DYNAMIC.

                int N0,       // Number of parameters in block 0.

                int N1 = 0,   // Number of parameters in block 1.

                int N2 = 0,   // Number of parameters in block 2.

                int N3 = 0,   // Number of parameters in block 3.

                int N4 = 0,   // Number of parameters in block 4.

                int N5 = 0,   // Number of parameters in block 5.

                int N6 = 0,   // Number of parameters in block 6.

                int N7 = 0,   // Number of parameters in block 7.

                int N8 = 0,   // Number of parameters in block 8.

                int N9 = 0>   // Number of parameters in block 9.

      class NumericDiffCostFunction : public

      SizedCostFunction<kNumResiduals, N0, N1, N2, N3, N4, N5, N6, N7, N8, N9> {

      };

  To get a numerically differentiated :class:`CostFunction`, you must

  define a class with a ``operator()`` (a functor) that computes the

  residuals. The functor must write the computed value in the last

  argument (the only non-``const`` one) and return ``true`` to

  indicate success.  Please see :class:`CostFunction` for details on

  how the return value may be used to impose simple constraints on the

  parameter block. e.g., an object of the form

  .. code-block:: c++

     struct ScalarFunctor {

      public:

       bool operator()(const double* const x1,

                       const double* const x2,

                       double* residuals) const;

     }

  For example, consider a scalar error :math:`e = k - x'y`, where both

  :math:`x` and :math:`y` are two-dimensional column vector

  parameters, the prime sign indicates transposition, and :math:`k` is

  a constant. The form of this error, which is the difference between

  a constant and an expression, is a common pattern in least squares

  problems. For example, the value :math:`x'y` might be the model

  expectation for a series of measurements, where there is an instance

  of the cost function for each measurement :math:`k`.

  To write an numerically-differentiable class:`CostFunction` for the

  above model, first define the object

  .. code-block::  c++

     class MyScalarCostFunctor {

       MyScalarCostFunctor(double k): k_(k) {}

       bool operator()(const double* const x,

                       const double* const y,

                       double* residuals) const {

         residuals[0] = k_ - x[0] * y[0] + x[1] * y[1];

         return true;

       }

      private:

       double k_;

     };

  Note that in the declaration of ``operator()`` the input parameters

  ``x`` and ``y`` come first, and are passed as const pointers to

  arrays of ``double`` s. If there were three input parameters, then

  the third input parameter would come after ``y``. The output is

  always the last parameter, and is also a pointer to an array. In the

  example above, the residual is a scalar, so only ``residuals[0]`` is

  set.

  Then given this class definition, the numerically differentiated

  :class:`CostFunction` with central differences used for computing

  the derivative can be constructed as follows.

  .. code-block:: c++

    CostFunction* cost_function

        = new NumericDiffCostFunction<MyScalarCostFunctor, CENTRAL, 1, 2, 2>(

            new MyScalarCostFunctor(1.0));                    ^     ^  ^  ^

                                                              |     |  |  |

                                  Finite Differencing Scheme -+     |  |  |

                                  Dimension of residual ------------+  |  |

                                  Dimension of x ----------------------+  |

                                  Dimension of y -------------------------+

  In this example, there is usually an instance for each measurement

  of `k`.

  In the instantiation above, the template parameters following

  ``MyScalarCostFunctor``, ``1, 2, 2``, describe the functor as

  computing a 1-dimensional output from two arguments, both

  2-dimensional.

  NumericDiffCostFunction also supports cost functions with a

  runtime-determined number of residuals. For example:

   .. code-block:: c++

     CostFunction* cost_function

         = new NumericDiffCostFunction<MyScalarCostFunctor, CENTRAL, DYNAMIC, 2, 2>(

             new CostFunctorWithDynamicNumResiduals(1.0),               ^     ^  ^

             TAKE_OWNERSHIP,                                            |     |  |

             runtime_number_of_residuals); <----+                       |     |  |

                                                |                       |     |  |

                                                |                       |     |  |

               Actual number of residuals ------+                       |     |  |

               Indicate dynamic number of residuals --------------------+     |  |

               Dimension of x ------------------------------------------------+  |

               Dimension of y ---------------------------------------------------+

  The framework can currently accommodate cost functions of up to 10

  independent variables, and there is no limit on the dimensionality

  of each of them.

  There are three available numeric differentiation schemes in ceres-solver:

  The ``FORWARD`` difference method, which approximates :math:`f'(x)`

  by computing :math:`\frac{f(x+h)-f(x)}{h}`, computes the cost

  function one additional time at :math:`x+h`. It is the fastest but

  least accurate method.

  The ``CENTRAL`` difference method is more accurate at the cost of

  twice as many function evaluations than forward difference,

  estimating :math:`f'(x)` by computing

  :math:`\frac{f(x+h)-f(x-h)}{2h}`.

  The ``RIDDERS`` difference method[Ridders]_ is an adaptive scheme

  that estimates derivatives by performing multiple central

  differences at varying scales. Specifically, the algorithm starts at

  a certain :math:`h` and as the derivative is estimated, this step

  size decreases.  To conserve function evaluations and estimate the

  derivative error, the method performs Richardson extrapolations

  between the tested step sizes.  The algorithm exhibits considerably

  higher accuracy, but does so by additional evaluations of the cost

  function.

  Consider using ``CENTRAL`` differences to begin with. Based on the

  results, either try forward difference to improve performance or

  Ridders' method to improve accuracy.

  **WARNING** A common beginner's error when first using

  :class:`NumericDiffCostFunction` is to get the sizing wrong. In

  particular, there is a tendency to set the template parameters to

  (dimension of residual, number of parameters) instead of passing a

  dimension parameter for *every parameter*. In the example above,

  that would be ``<MyScalarCostFunctor, 1, 2>``, which is missing the

  last ``2`` argument. Please be careful when setting the size

  parameters.

Numeric Differentiation & LocalParameterization

-----------------------------------------------

   If your cost function depends on a parameter block that must lie on

   a manifold and the functor cannot be evaluated for values of that

   parameter block not on the manifold then you may have problems

   numerically differentiating such functors.

   This is because numeric differentiation in Ceres is performed by

   perturbing the individual coordinates of the parameter blocks that

   a cost functor depends on. In doing so, we assume that the

   parameter blocks live in an Euclidean space and ignore the

   structure of manifold that they live As a result some of the

   perturbations may not lie on the manifold corresponding to the

   parameter block.

   For example consider a four dimensional parameter block that is

   interpreted as a unit Quaternion. Perturbing the coordinates of

   this parameter block will violate the unit norm property of the

   parameter block.

   Fixing this problem requires that :class:`NumericDiffCostFunction`

   be aware of the :class:`LocalParameterization` associated with each

   parameter block and only generate perturbations in the local

   tangent space of each parameter block.

   For now this is not considered to be a serious enough problem to

   warrant changing the :class:`NumericDiffCostFunction` API. Further,

   in most cases it is relatively straightforward to project a point

   off the manifold back onto the manifold before using it in the

   functor. For example in case of the Quaternion, normalizing the

   4-vector before using it does the trick.

   **Alternate Interface**

   For a variety of reasons, including compatibility with legacy code,

   :class:`NumericDiffCostFunction` can also take

   :class:`CostFunction` objects as input. The following describes

   how.

   To get a numerically differentiated cost function, define a

   subclass of :class:`CostFunction` such that the

   :func:`CostFunction::Evaluate` function ignores the ``jacobians``

   parameter. The numeric differentiation wrapper will fill in the

   jacobian parameter if necessary by repeatedly calling the

   :func:`CostFunction::Evaluate` with small changes to the

   appropriate parameters, and computing the slope. For performance,

   the numeric differentiation wrapper class is templated on the

   concrete cost function, even though it could be implemented only in

   terms of the :class:`CostFunction` interface.

   The numerically differentiated version of a cost function for a

   cost function can be constructed as follows:

   .. code-block:: c++

     CostFunction* cost_function

         = new NumericDiffCostFunction<MyCostFunction, CENTRAL, 1, 4, 8>(

             new MyCostFunction(...), TAKE_OWNERSHIP);

   where ``MyCostFunction`` has 1 residual and 2 parameter blocks with

   sizes 4 and 8 respectively. Look at the tests for a more detailed

   example.

:class:`DynamicNumericDiffCostFunction`

=======================================

.. class:: DynamicNumericDiffCostFunction

   Like :class:`AutoDiffCostFunction` :class:`NumericDiffCostFunction`

   requires that the number of parameter blocks and their sizes be

   known at compile time. It also has an upper limit of 10 parameter

   blocks. In a number of applications, this is not enough.

     .. code-block:: c++

      template <typename CostFunctor, NumericDiffMethodType method = CENTRAL>

      class DynamicNumericDiffCostFunction : public CostFunction {

      };

   In such cases when numeric differentiation is desired,

   :class:`DynamicNumericDiffCostFunction` can be used.

   Like :class:`NumericDiffCostFunction` the user must define a

   functor, but the signature of the functor differs slightly. The

   expected interface for the cost functors is:

     .. code-block:: c++

       struct MyCostFunctor {

         bool operator()(double const* const* parameters, double* residuals) const {

         }

       }

   Since the sizing of the parameters is done at runtime, you must

   also specify the sizes after creating the dynamic numeric diff cost

   function. For example:

     .. code-block:: c++

       DynamicNumericDiffCostFunction<MyCostFunctor>* cost_function =

         new DynamicNumericDiffCostFunction<MyCostFunctor>(new MyCostFunctor);

       cost_function->AddParameterBlock(5);

       cost_function->AddParameterBlock(10);

       cost_function->SetNumResiduals(21);

   As a rule of thumb, try using :class:`NumericDiffCostFunction` before

   you use :class:`DynamicNumericDiffCostFunction`.

   **WARNING** The same caution about mixing local parameterizations

   with numeric differentiation applies as is the case with

   :class:`NumericDiffCostFunction`.

:class:`CostFunctionToFunctor`

==============================

.. class:: CostFunctionToFunctor

   :class:`CostFunctionToFunctor` is an adapter class that allows

   users to use :class:`CostFunction` objects in templated functors

   which are to be used for automatic differentiation. This allows

   the user to seamlessly mix analytic, numeric and automatic

   differentiation.

   For example, let us assume that

   .. code-block:: c++

     class IntrinsicProjection : public SizedCostFunction<2, 5, 3> {

       public:

         IntrinsicProjection(const double* observation);

         virtual bool Evaluate(double const* const* parameters,

                               double* residuals,

                               double** jacobians) const;

     };

   is a :class:`CostFunction` that implements the projection of a

   point in its local coordinate system onto its image plane and

   subtracts it from the observed point projection. It can compute its

   residual and either via analytic or numerical differentiation can

   compute its jacobians.

   Now we would like to compose the action of this

   :class:`CostFunction` with the action of camera extrinsics, i.e.,

   rotation and translation. Say we have a templated function

   .. code-block:: c++

      template<typename T>

      void RotateAndTranslatePoint(const T* rotation,

                                   const T* translation,

                                   const T* point,

                                   T* result);

   Then we can now do the following,

   .. code-block:: c++

    struct CameraProjection {

      CameraProjection(double* observation)

      : intrinsic_projection_(new IntrinsicProjection(observation)) {

      }

      template <typename T>

      bool operator()(const T* rotation,

                      const T* translation,

                      const T* intrinsics,

                      const T* point,

                      T* residual) const {

        T transformed_point[3];

        RotateAndTranslatePoint(rotation, translation, point, transformed_point);

        // Note that we call intrinsic_projection_, just like it was

        // any other templated functor.

        return intrinsic_projection_(intrinsics, transformed_point, residual);

      }

     private:

      CostFunctionToFunctor<2,5,3> intrinsic_projection_;

    };

   Note that :class:`CostFunctionToFunctor` takes ownership of the

   :class:`CostFunction` that was passed in to the constructor.

   In the above example, we assumed that ``IntrinsicProjection`` is a

   ``CostFunction`` capable of evaluating its value and its

   derivatives. Suppose, if that were not the case and

   ``IntrinsicProjection`` was defined as follows:

   .. code-block:: c++

    struct IntrinsicProjection

      IntrinsicProjection(const double* observation) {

        observation_[0] = observation[0];

        observation_[1] = observation[1];

      }

      bool operator()(const double* calibration,

                      const double* point,

                      double* residuals) {

        double projection[2];

        ThirdPartyProjectionFunction(calibration, point, projection);

        residuals[0] = observation_[0] - projection[0];

        residuals[1] = observation_[1] - projection[1];

        return true;

      }

     double observation_[2];

    };

  Here ``ThirdPartyProjectionFunction`` is some third party library

  function that we have no control over. So this function can compute

  its value and we would like to use numeric differentiation to

  compute its derivatives. In this case we can use a combination of

  ``NumericDiffCostFunction`` and ``CostFunctionToFunctor`` to get the

  job done.

  .. code-block:: c++

   struct CameraProjection {

     CameraProjection(double* observation)

       intrinsic_projection_(

         new NumericDiffCostFunction<IntrinsicProjection, CENTRAL, 2, 5, 3>(

           new IntrinsicProjection(observation)) {

     }

     template <typename T>

     bool operator()(const T* rotation,

                     const T* translation,

                     const T* intrinsics,

                     const T* point,

                     T* residuals) const {

       T transformed_point[3];

       RotateAndTranslatePoint(rotation, translation, point, transformed_point);

       return intrinsic_projection_(intrinsics, transformed_point, residual);

     }

    private:

     CostFunctionToFunctor<2,5,3> intrinsic_projection_;

   };

:class:`DynamicCostFunctionToFunctor`

=====================================

.. class:: DynamicCostFunctionToFunctor

   :class:`DynamicCostFunctionToFunctor` provides the same functionality as

   :class:`CostFunctionToFunctor` for cases where the number and size of the

   parameter vectors and residuals are not known at compile-time. The API

   provided by :class:`DynamicCostFunctionToFunctor` matches what would be

   expected by :class:`DynamicAutoDiffCostFunction`, i.e. it provides a

   templated functor of this form:

   .. code-block:: c++

    template<typename T>

    bool operator()(T const* const* parameters, T* residuals) const;

   Similar to the example given for :class:`CostFunctionToFunctor`, let us

   assume that

   .. code-block:: c++

     class IntrinsicProjection : public CostFunction {

       public:

         IntrinsicProjection(const double* observation);

         virtual bool Evaluate(double const* const* parameters,

                               double* residuals,

                               double** jacobians) const;

     };

   is a :class:`CostFunction` that projects a point in its local coordinate

   system onto its image plane and subtracts it from the observed point

   projection.

   Using this :class:`CostFunction` in a templated functor would then look like

   this:

   .. code-block:: c++

    struct CameraProjection {

      CameraProjection(double* observation)

          : intrinsic_projection_(new IntrinsicProjection(observation)) {

      }

      template <typename T>

      bool operator()(T const* const* parameters,

                      T* residual) const {

        const T* rotation = parameters[0];

        const T* translation = parameters[1];

        const T* intrinsics = parameters[2];

        const T* point = parameters[3];

        T transformed_point[3];

        RotateAndTranslatePoint(rotation, translation, point, transformed_point);

        const T* projection_parameters[2];

        projection_parameters[0] = intrinsics;

        projection_parameters[1] = transformed_point;

        return intrinsic_projection_(projection_parameters, residual);

      }

     private:

      DynamicCostFunctionToFunctor intrinsic_projection_;

    };

   Like :class:`CostFunctionToFunctor`, :class:`DynamicCostFunctionToFunctor`

   takes ownership of the :class:`CostFunction` that was passed in to the

   constructor.

:class:`ConditionedCostFunction`

================================

.. class:: ConditionedCostFunction

   This class allows you to apply different conditioning to the residual

   values of a wrapped cost function. An example where this is useful is

   where you have an existing cost function that produces N values, but you

   want the total cost to be something other than just the sum of these

   squared values - maybe you want to apply a different scaling to some

   values, to change their contribution to the cost.

   Usage:

   .. code-block:: c++

       //  my_cost_function produces N residuals

       CostFunction* my_cost_function = ...

       CHECK_EQ(N, my_cost_function->num_residuals());

       vector<CostFunction*> conditioners;

       //  Make N 1x1 cost functions (1 parameter, 1 residual)

       CostFunction* f_1 = ...

       conditioners.push_back(f_1);

       CostFunction* f_N = ...

       conditioners.push_back(f_N);

       ConditionedCostFunction* ccf =

         new ConditionedCostFunction(my_cost_function, conditioners);

   Now ``ccf`` 's ``residual[i]`` (i=0..N-1) will be passed though the

   :math:`i^{\text{th}}` conditioner.

   .. code-block:: c++

      ccf_residual[i] = f_i(my_cost_function_residual[i])

   and the Jacobian will be affected appropriately.

:class:`GradientChecker`

================================

.. class:: GradientChecker

    This class compares the Jacobians returned by a cost function against

    derivatives estimated using finite differencing. It is meant as a tool for

    unit testing, giving you more fine-grained control than the check_gradients

    option in the solver options.