The ZOOM Minimization PackageThe ZOOM Minimization Package

David SachsMark Fischler

Fermi National Accelerator Laboratory

Minimization PackageMinimization Package Object-oriented C++ library for minimization

– Minuit capabilities, and more

Suitable for use– In stand-alone programs– As part of Root applications– In other frameworks

The OO design allows the Minimizer to ‘play well with others’

– As central part of a fitter; provided as examples will be: an unbinned maximum likelihood fitter a binned chi-squared fitter,

Benefits of OO DesignBenefits of OO Design User convenience

– Domain concepts reflected more directly in code– Natural to encapsulate function to be minimized– Avoidance of global variables allows

multiple simultaneous minimizations use in multi-threaded applications

Extensibility– Separation of concepts allows for independent

replacement Benefits from any revision of code:

– Find features that “should have” been there– Sometimes you uncover flaws… yes, even in Minuit

Brief Survey of C++ MinimizersBrief Survey of C++ Minimizers Other C++ packages are available, but do not use

modern C++– and so lack the OO advantages

Other Projects include– The MINUIT Project [SEAL/MathLib]

Hand-translation of Minuit (James and Winkler)

– ROOT minimization/fitting Hand-smoothed automated translation of Minuit

An Example of UseAn Example of Use

double f(const std::vector<double>& x) { return x[0]*x[0]+ //… some function of

// x[0] thru x[4]}int main() { using namespace Minimization; const int Ndimensions = 5; Problem m(UserFunction(Ndimensions, f)); m.minimize(); cout << m.currentPoint() << endl;}

Directions of ExtensibilityDirections of Extensibility class Terminator

– When do we stop improving the minimum?

class Domain– How do we restrict the allowed values of

the parameters?

class Algorithm– How do we refine existing minimization methods?– How do we add new minimization techniques?

TerminatorsTerminators A terminator is an object that can be asked

whether or not minimization is complete– it must be an instance of a class derived from class

Terminator

Several standard terminators are provided– These can be combined by ORs and ANDs

Terminators typically make use of the state of the minimization

Users can easily create their own terminator classes

An Example An Example TerminatorTerminator

class NCalls : public Terminator { int n;public: explicit NCalls(int limit):n(limit) {} ...

TerminationType finished (const ProblemState& p) { return p.functionCallCount >= n ? TTStop : TTContinue; }};

DomainsDomains A domain is an object that maps internal

coordinates to external coordinates and vice versa– It must be an instance of a class derived from Domain

Custom domain classes must provide– Mapping from external M-space to “interior”

unrestricted N-space, and vice versa– Gradient calculation

One domain is provided: RectilinearDomain– independent [-, ], [a,b], [-, b], [a, ] intervals– maps for [a,b] intervals match those in Minuit

Some Possible Custom DomainsSome Possible Custom Domains Orthogonal domain, but with sigmoid mappings

instead of arcsine mappings– Superior for some cases, worse for others

Multiple probability space– All parameters must be non-negative– Parameters must sum to 1

Interior (or surface) of N-Sphere– Lots of possible mappings– But watch out if minimum lies very near a mapping

singularity

AlgorithmsAlgorithms An algorithm is an object that encapsulates a minimization

technique– It must be an instance of a class derived from Algorithm

Custom algorithm classes can be written by experts. Two algorithms are provided:

– Migrad– Simplex

Other algorithms will be provided:– Seek and others completing Minuit collection

Further algorithm implementation possibilities: – Fumili– biConjGradStab – LEAMAX

Adding a New AlgorithmAdding a New Algorithm We might wish to add a Simplex-like algorithm

but with: – improved estimation of the distance to the minimum

(edm)– strategies to avoid false convergence

The mathematics of finding the best strategy is still hard

OO design makes adding the algorithm straightforward, once the proper strategy is discovered

The Simplex AlgorithmThe Simplex Algorithm The algorithm:

– N+1 points in N-space form a simplex– Repeatedly reflect the “worst” point about the centroid

of the remaining points Sometimes shrink if that improves function Sometimes expand if reflected point is best point

– When estimated distance to minimum is small, stop

Problem: The edm estimate formed by simplex “is largely fantasy.”

Discovery of a Flaw in MinuitDiscovery of a Flaw in Minuit Simplex can converge to a non-minimum

– Even on well-behaved (quadratic) functions!– When starting a large distance from the true minimum

How does this happen?– Simplex becomes thin, aligned with an equipotential – There is a good direction to expand off that line but…– simplex shrinks before it finds that direction and…– triggers convergence criterion based on the false edm

If the edm were big at that point, we would at least know something was wrong– The fix is still not trivial but knowing a meaningful edm is a

start

Apparent ConvergenceApparent Convergence

MINUIT

Minimization

The simplex algorithm is stopped when it indicates it is near the function minimum.

Convergence FailureConvergence Failure

MINUIT’s simplex indicates convergence while still far from it.The error is fixed in Minimization

MINUIT

Minimization

ConclusionConclusion Minimization package available from

– http://cepa.fnal.gov/aps/minimization.shtml

Standard build mechanism (configure/make) Modern, Object Oriented C++

– Suitable for some multi-threaded operation– Designed to be extensible to new

Termination criteria Domains (parameter restrictions) Algorithms

Corrects a flaw in Minuit– In the Simplex algorithm

The ZOOM Minimization PackageThe ZOOM Minimization Package

David SachsMark Fischler

Fermi National Accelerator Laboratory