This class implements a gradient descent optimizer with adaptive gain. More...

#include <itkAdaptiveStochasticGradientDescentOptimizer.h>

Inheritance diagram for itk::AdaptiveStochasticGradientDescentOptimizer:

Collaboration diagram for itk::AdaptiveStochasticGradientDescentOptimizer:

Public Types
typedef SmartPointer< const Self >	ConstPointer
typedef Superclass::CostFunctionType	CostFunctionType
typedef Superclass::DerivativeType	DerivativeType
typedef Superclass::MeasureType	MeasureType
typedef Superclass::ParametersType	ParametersType
typedef SmartPointer< Self >	Pointer
typedef Superclass::ScaledCostFunctionPointer	ScaledCostFunctionPointer
typedef Superclass::ScaledCostFunctionType	ScaledCostFunctionType
typedef Superclass::ScalesType	ScalesType
typedef AdaptiveStochasticGradientDescentOptimizer	Self
typedef Superclass::StopConditionType	StopConditionType
typedef StandardGradientDescentOptimizer	Superclass
Public Member Functions
virtual const char *	GetClassName () const
virtual double	GetSigmoidMax () const
virtual double	GetSigmoidMin () const
virtual double	GetSigmoidScale () const
virtual bool	GetUseAdaptiveStepSizes () const
virtual void	SetSigmoidMax (double _arg)
virtual void	SetSigmoidMin (double _arg)
virtual void	SetSigmoidScale (double _arg)
virtual void	SetUseAdaptiveStepSizes (bool _arg)
Static Public Member Functions
static Pointer	New ()
Protected Member Functions
	AdaptiveStochasticGradientDescentOptimizer ()
virtual void	UpdateCurrentTime (void)
virtual	~AdaptiveStochasticGradientDescentOptimizer ()
Protected Attributes
DerivativeType	m_PreviousGradient
Private Member Functions
	AdaptiveStochasticGradientDescentOptimizer (const Self &)
void	operator= (const Self &)
Private Attributes
double	m_SigmoidMax
double	m_SigmoidMin
double	m_SigmoidScale
bool	m_UseAdaptiveStepSizes

Detailed Description

This class implements a gradient descent optimizer with adaptive gain.

If $C(x)$ is a costfunction that has to be minimised, the following iterative algorithm is used to find the optimal parameters $x$ :

$x(k+1) = x(k) - a(t_k) dC/dx$

The gain $a(t_k)$ at each iteration $k$ is defined by:

$a(t_k) = a / (A + t_k + 1)^alpha$

.

And the time $t_k$ is updated according to:

$t_{k+1} = [ t_k + sigmoid( -g_k^T g_{k-1} ) ]^+$

where $g_k$ equals $dC/dx$ at iteration $k$ . For $t_0$ the InitialTime is used, which is defined in the the superclass (StandardGradientDescentOptimizer). Whereas in the superclass this parameter is superfluous, in this class it makes sense.

This method is described in the following reference:

P. Cruz, Almost sure convergence and asymptotical normality of a generalization of Kesten's stochastic approximation algorithm for multidimensional case. Technical Report, 2005. http://hdl.handle.net/2052/74

It is very suitable to be used in combination with a stochastic estimate of the gradient $dC/dx$ . For example, in image registration problems it is often advantageous to compute the metric derivative ( $dC/dx$ ) on a new set of randomly selected image samples in each iteration. You may set the parameter NewSamplesEveryIteration to "true" to achieve this effect. For more information on this strategy, you may have a look at:

See also:: AdaptiveStochasticGradientDescent, StandardGradientDescentOptimizer

Definition at line 63 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

Member Typedef Documentation

typedef SmartPointer<const Self> itk::AdaptiveStochasticGradientDescentOptimizer::ConstPointer

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 73 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::CostFunctionType itk::AdaptiveStochasticGradientDescentOptimizer::CostFunctionType

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 86 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::DerivativeType itk::AdaptiveStochasticGradientDescentOptimizer::DerivativeType

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 85 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::MeasureType itk::AdaptiveStochasticGradientDescentOptimizer::MeasureType

Typedefs inherited from the superclass.

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 80 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::ParametersType itk::AdaptiveStochasticGradientDescentOptimizer::ParametersType

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 84 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef SmartPointer<Self> itk::AdaptiveStochasticGradientDescentOptimizer::Pointer

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 72 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::ScaledCostFunctionPointer itk::AdaptiveStochasticGradientDescentOptimizer::ScaledCostFunctionPointer

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 89 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::ScaledCostFunctionType itk::AdaptiveStochasticGradientDescentOptimizer::ScaledCostFunctionType

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 88 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::ScalesType itk::AdaptiveStochasticGradientDescentOptimizer::ScalesType

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 87 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef AdaptiveStochasticGradientDescentOptimizer itk::AdaptiveStochasticGradientDescentOptimizer::Self

Standard ITK.

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 69 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef Superclass::StopConditionType itk::AdaptiveStochasticGradientDescentOptimizer::StopConditionType

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

Definition at line 90 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

typedef StandardGradientDescentOptimizer itk::AdaptiveStochasticGradientDescentOptimizer::Superclass

Reimplemented from itk::StandardGradientDescentOptimizer.

Definition at line 70 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

Constructor & Destructor Documentation

itk::AdaptiveStochasticGradientDescentOptimizer::AdaptiveStochasticGradientDescentOptimizer ( ) [protected]

virtual itk::AdaptiveStochasticGradientDescentOptimizer::~AdaptiveStochasticGradientDescentOptimizer ( ) [inline, protected, virtual]

Definition at line 114 of file itkAdaptiveStochasticGradientDescentOptimizer.h.

itk::AdaptiveStochasticGradientDescentOptimizer::AdaptiveStochasticGradientDescentOptimizer ( const Self & ) [private]

Member Function Documentation

virtual const char* itk::AdaptiveStochasticGradientDescentOptimizer::GetClassName ( ) const [virtual]

Run-time type information (and related methods).

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

virtual double itk::AdaptiveStochasticGradientDescentOptimizer::GetSigmoidMax ( ) const [virtual]

virtual double itk::AdaptiveStochasticGradientDescentOptimizer::GetSigmoidMin ( ) const [virtual]

virtual double itk::AdaptiveStochasticGradientDescentOptimizer::GetSigmoidScale ( ) const [virtual]

virtual bool itk::AdaptiveStochasticGradientDescentOptimizer::GetUseAdaptiveStepSizes ( ) const [virtual]

static Pointer itk::AdaptiveStochasticGradientDescentOptimizer::New ( ) [static]

Method for creation through the object factory.

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

void itk::AdaptiveStochasticGradientDescentOptimizer::operator= ( const Self & ) [private]

Reimplemented from itk::StandardGradientDescentOptimizer.

Reimplemented in elastix::AdaptiveStochasticGradientDescent< TElastix >.

virtual void itk::AdaptiveStochasticGradientDescentOptimizer::SetSigmoidMax ( double _arg ) [virtual]

Set/Get the maximum of the sigmoid. Should be >0. Default: 1.0

virtual void itk::AdaptiveStochasticGradientDescentOptimizer::SetSigmoidMin ( double _arg ) [virtual]

Set/Get the maximum of the sigmoid. Should be <0. Default: -0.8

virtual void itk::AdaptiveStochasticGradientDescentOptimizer::SetSigmoidScale ( double _arg ) [virtual]

Set/Get the scaling of the sigmoid width. Large values cause a more wide sigmoid. Default: 1e-8. Should be >0.

virtual void itk::AdaptiveStochasticGradientDescentOptimizer::SetUseAdaptiveStepSizes ( bool _arg ) [virtual]

Set/Get whether the adaptive step size mechanism is desired. Default: true

virtual void itk::AdaptiveStochasticGradientDescentOptimizer::UpdateCurrentTime ( void ) [protected, virtual]

Function to update the current time If UseAdaptiveStepSizes is false this function just increments the CurrentTime by $E_0 = (sigmoid_{max} + sigmoid_{min})/2$ . Else, the CurrentTime is updated according to:
time = max[ 0, time + sigmoid( -gradient*previousgradient) ]
In that case, also the m_PreviousGradient is updated.