|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<html class="no-js" lang="en" >
|
|
Packit |
ea1746 |
<head>
|
|
Packit |
ea1746 |
<meta charset="utf-8">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<title>On Derivatives — Ceres Solver</title>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<link rel="top" title="Ceres Solver" href="index.html"/>
|
|
Packit |
ea1746 |
<link rel="next" title="Spivak Notation" href="spivak_notation.html"/>
|
|
Packit |
ea1746 |
<link rel="prev" title="General Unconstrained Minimization" href="gradient_tutorial.html"/>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<script src="_static/js/modernizr.min.js"></script>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
</head>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<body class="wy-body-for-nav" role="document">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Ceres Solver
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
1.13
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
|
|
Packit |
ea1746 |
<input type="text" name="q" placeholder="Search docs" />
|
|
Packit |
ea1746 |
<input type="hidden" name="check_keywords" value="yes" />
|
|
Packit |
ea1746 |
<input type="hidden" name="area" value="default" />
|
|
Packit |
ea1746 |
</form>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Why?
|
|
Packit |
ea1746 |
Installation
|
|
Packit |
ea1746 |
Tutorial
|
|
Packit |
ea1746 |
On Derivatives
|
|
Packit |
ea1746 |
Spivak Notation
|
|
Packit |
ea1746 |
Analytic Derivatives
|
|
Packit |
ea1746 |
Numeric derivatives
|
|
Packit |
ea1746 |
Automatic Derivatives
|
|
Packit |
ea1746 |
Interfacing with Automatic Differentiation
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Modeling Non-linear Least Squares
|
|
Packit |
ea1746 |
Solving Non-linear Least Squares
|
|
Packit |
ea1746 |
Covariance Estimation
|
|
Packit |
ea1746 |
General Unconstrained Minimization
|
|
Packit |
ea1746 |
FAQS, Tips & Tricks
|
|
Packit |
ea1746 |
Users
|
|
Packit |
ea1746 |
Contributing
|
|
Packit |
ea1746 |
Version History
|
|
Packit |
ea1746 |
Bibliography
|
|
Packit |
ea1746 |
License
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
</nav>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Ceres Solver
|
|
Packit |
ea1746 |
</nav>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Docs »
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
On Derivatives
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
On DerivativesΒΆ
|
|
Packit |
ea1746 |
Ceres Solver, like all gradient based optimization algorithms, depends
|
|
Packit |
ea1746 |
on being able to evaluate the objective function and its derivatives
|
|
Packit |
ea1746 |
at arbitrary points in its domain. Indeed, defining the objective
|
|
Packit |
ea1746 |
function and its Jacobian is
|
|
Packit |
ea1746 |
the principal task that the user is required to perform when solving
|
|
Packit |
ea1746 |
an optimization problem using Ceres Solver. The correct and efficient
|
|
Packit |
ea1746 |
computation of the Jacobian is the key to good performance.
|
|
Packit |
ea1746 |
Ceres Solver offers considerable flexibility in how the user can
|
|
Packit |
ea1746 |
provide derivatives to the solver. She can use:
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Analytic Derivatives: The user figures out the
|
|
Packit |
ea1746 |
derivatives herself, by hand or using a tool like Maple or Mathematica, and implements them in a
|
|
Packit |
ea1746 |
CostFunction.
|
|
Packit |
ea1746 |
Numeric derivatives: Ceres numerically computes
|
|
Packit |
ea1746 |
the derivative using finite differences.
|
|
Packit |
ea1746 |
Automatic Derivatives: Ceres automatically computes
|
|
Packit |
ea1746 |
the analytic derivative using C++ templates and operator
|
|
Packit |
ea1746 |
overloading.
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Which of these three approaches (alone or in combination) should be
|
|
Packit |
ea1746 |
used depends on the situation and the tradeoffs the user is willing to
|
|
Packit |
ea1746 |
make. Unfortunately, numerical optimization textbooks rarely discuss
|
|
Packit |
ea1746 |
these issues in detail and the user is left to her own devices.
|
|
Packit |
ea1746 |
The aim of this article is to fill this gap and describe each of these
|
|
Packit |
ea1746 |
three approaches in the context of Ceres Solver with sufficient detail
|
|
Packit |
ea1746 |
that the user can make an informed choice.
|
|
Packit |
ea1746 |
For the impatient amongst you, here is some high level advice:
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Use Automatic Derivatives.
|
|
Packit |
ea1746 |
In some cases it maybe worth using
|
|
Packit |
ea1746 |
Analytic Derivatives.
|
|
Packit |
ea1746 |
Avoid Numeric derivatives. Use it as a measure of
|
|
Packit |
ea1746 |
last resort, mostly to interface with external libraries.
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
For the rest, read on.
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Spivak Notation
|
|
Packit |
ea1746 |
Analytic Derivatives
|
|
Packit |
ea1746 |
Numeric derivatives
|
|
Packit |
ea1746 |
Automatic Derivatives
|
|
Packit |
ea1746 |
Interfacing with Automatic Differentiation
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<footer>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Next
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
Previous
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
© Copyright 2016 Google Inc.
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
</footer>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
</section>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<script type="text/javascript">
|
|
Packit |
ea1746 |
var DOCUMENTATION_OPTIONS = {
|
|
Packit |
ea1746 |
URL_ROOT:'./',
|
|
Packit |
ea1746 |
VERSION:'1.13.0',
|
|
Packit |
ea1746 |
COLLAPSE_INDEX:false,
|
|
Packit |
ea1746 |
FILE_SUFFIX:'.html',
|
|
Packit |
ea1746 |
HAS_SOURCE: true
|
|
Packit |
ea1746 |
};
|
|
Packit |
ea1746 |
</script>
|
|
Packit |
ea1746 |
<script type="text/javascript" src="_static/jquery.js"></script>
|
|
Packit |
ea1746 |
<script type="text/javascript" src="_static/underscore.js"></script>
|
|
Packit |
ea1746 |
<script type="text/javascript" src="_static/doctools.js"></script>
|
|
Packit |
ea1746 |
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML">
|
|
Packit |
ea1746 |
MathJax.Hub.Config({
|
|
Packit |
ea1746 |
"HTML-CSS": {
|
|
Packit |
ea1746 |
availableFonts: ["TeX"]
|
|
Packit |
ea1746 |
}
|
|
Packit |
ea1746 |
});
|
|
Packit |
ea1746 |
</script>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<script type="text/javascript" src="_static/js/theme.js"></script>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<script type="text/javascript">
|
|
Packit |
ea1746 |
jQuery(function () {
|
|
Packit |
ea1746 |
SphinxRtdTheme.StickyNav.enable();
|
|
Packit |
ea1746 |
});
|
|
Packit |
ea1746 |
</script>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
<script>
|
|
Packit |
ea1746 |
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
|
Packit |
ea1746 |
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
|
Packit |
ea1746 |
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
|
Packit |
ea1746 |
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
|
|
Packit |
ea1746 |
ga('create', 'UA-49769510-1', 'ceres-solver.org');
|
|
Packit |
ea1746 |
ga('send', 'pageview');
|
|
Packit |
ea1746 |
</script>
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
|
|
Packit |
ea1746 |
</body>
|
|
Packit |
ea1746 |
</html>
|