armadillo: template metaprogramming for linear...

1

IntroductionTemplate Metaprogramming

TMP in ArmadilloConclusion

Armadillo: Template Metaprogramming forLinear Algebra

Leo Antunes and Markus Kruber

Seminar on Automation, Compilers, and Code-GenerationHPAC

June 2014

Leo Antunes and Markus Kruber Armadillo

http://hpac.rwth-aachen.de

2

What is Armadillo?Code example

What is Armadillo?

. C++ Library for Linear Algebra

. Aims for ease of use

. Still wants performance

. Abstract interface to BLAS and LAPACK

. Pure template library (only headers)

2

What is Armadillo?

2

What is Armadillo?

2

What is Armadillo?

3

Code example

Calculating A−1BvvT :

mat A =

4

C++ TemplatesOverviewExamples

Refresher: C++ Templates

. Allow type-agnostic development (e.g.: class Array)”generics” in Java, C#; ”parametric polymorphism” in Haskell, Scala

. Realized at compile-time → code generation

. Generate different class for each type used in code(thus: ”template”)

. Therefore: no dynamic typing

. Allows for further optimizations by compiler

4

5

C++ Template example

Type-dependent:

class Adder{

int base;

public:

Adder(int x):base(x){}

int doit(int y){

return base+y;

}

};

Type-agnostic:

template

class Adder{

TYPE base;

public:

Adder(TYPE x):base(x){}

TYPE doit(TYPE y){

return base+y;

}

};

6

Template Metaprogramming

. Use C++ templates as a language in itself

. Type specification as pattern matching → Branching(independently of C++’s control structures)

. Template used inside template → Recursion

. Branching and Recursion → Turing complete

. Allows offloading of computation to compile-time

6

7

Factorial: normal C++

int factorial(int n) {

if(n < 1){

return 1;

}

else {

return n*factorial(n-1);

}

8

Factorial: Template metaprogramming

template

struct factorial {

const static int result =

n * factorial :: result;

};

template

struct factorial {

const static int result = 1 ;

};

. Types used for pattern matching.

. Structs used as functions.

. Most ”specific” template chosen first. (cf. Haskell)

9

OverviewExamples

TMP in Armadillo

. Classes for common LA objects: matrix, vector, etc

. Literal expression in code converted to template tree, e.g.:A*inv(B) → Glue

. Infers information on arguments via pattern matching

. User can provide information about objects.e.g.: diagmat(A) or trimatu(A)

. Actual calculation delayed until runtime request for results

9

OverviewExamples

TMP in Armadillo

9

OverviewExamples

TMP in Armadillo

9

OverviewExamples

TMP in Armadillo

9

OverviewExamples

TMP in Armadillo

10

OverviewExamples

Simple example: inv(inv(A))

. Rationale: should not compute anything; just return A

. Inner call to inv() returns a Op object.

. Outter call matches the definition of inv() withOp argument. Returns A.

. Advantages:

. Obvious: no actual inversion.

. Not obvious: multiple-dispatch without the runtime overhead.

. Allows compiler to get rid of unneeded function calls.

10

OverviewExamples

Simple example: inv(inv(A))

. Rationale: should not compute anything; just return A

. Inner call to inv() returns a Op object.

. Outter call matches the definition of inv() withOp argument. Returns A.

. Advantages:

. Obvious: no actual inversion.

. Not obvious: multiple-dispatch without the runtime overhead.

. Allows compiler to get rid of unneeded function calls.

11

OverviewExamples

Matrix Chain Multiplication

. Matrix multiplication is associative.(ABC )D = A(BCD) = . . .

. Order affects the number of operations.

. A ∈ R60×5, B ∈ R5×30, C ∈ R30×10

. (AB)C = (60 · 5 · 30) + (60 · 30 · 10) = 27.000

. A(BC ) = (5 · 30 · 10) + (60 · 5 · 10) = 4.500

. E.g. Matlab: naive left to right ordering

. Complexity: O(n3) with dynamic programmingSmarter approach: O(n · log(n))

11

OverviewExamples

. A ∈ R60×5, B ∈ R5×30, C ∈ R30×10

. (AB)C = (60 · 5 · 30) + (60 · 30 · 10) = 27.000

. A(BC ) = (5 · 30 · 10) + (60 · 5 · 10) = 4.500

11

OverviewExamples

. A ∈ R60×5, B ∈ R5×30, C ∈ R30×10

. (AB)C = (60 · 5 · 30) + (60 · 30 · 10) = 27.000

. A(BC ) = (5 · 30 · 10) + (60 · 5 · 10) = 4.500

12

OverviewExamples

Complex example: A*B*C*D

. Instantiated as:Glue

. Determines ”depth” of expression: N = 3(amount of operators with same precedence)

. Pattern matches specialized function for N = 3 which:

. Decide (ABC )D vs. A(BCD)

. Heuristic: cost(ABC ) := columns(A) · rows(C )

. Recursively calls reordering function with N = 2

. Thus: linear (but sub-optimal) reordering of operations

. Remember: all this at compile-time!

12

OverviewExamples

12

OverviewExamples

13

OverviewExamples

Complex example: A*B*C*D (cont.)

. ((AB)C )D 3

⇒ O(n3)

. (A(BC ))D 3

⇒ O(n3)

. A((BC )D) 3

⇒ O(n3)

. A(B(CD)) 3

⇒ O(n3)

. (AB)(CD) 7

⇒ O(n2)

n Armadillo All

3 2 24 4 55 8 146 16 427 32 132

n

A

1

n

B

n1

C

n

D

13

OverviewExamples

. ((AB)C )D 3

⇒ O(n3)

. (A(BC ))D 3

⇒ O(n3)

. A((BC )D) 3

⇒ O(n3)

. A(B(CD)) 3

⇒ O(n3)

. (AB)(CD) 7

⇒ O(n2)

n Armadillo All

3 2 24 4 55 8 146 16 427 32 132

n

A

1

n

B

n1

C

n

D

13

OverviewExamples

. ((AB)C )D 3 ⇒ O(n3)

. (A(BC ))D 3 ⇒ O(n3)

. A((BC )D) 3 ⇒ O(n3)

. A(B(CD)) 3 ⇒ O(n3)

. (AB)(CD) 7 ⇒ O(n2)

n Armadillo All

3 2 24 4 55 8 146 16 427 32 132

n

A

1

n

B

n1

C

n

D

13

OverviewExamples

. ((AB)C )D 3 ⇒ O(n3)

. (A(BC ))D 3 ⇒ O(n3)

. A((BC )D) 3 ⇒ O(n3)

. A(B(CD)) 3 ⇒ O(n3)

. (AB)(CD) 7 ⇒ O(n2)

n Armadillo All

3 2 24 4 55 8 146 16 427 32 132

n

A

1

n

B

n1

C

n

D

14

OverviewExamples

Other improvements

. A = A1 + A2 + . . . + AnNo temporary matrix besides output

. y = A−1x → y = solve(A, x)Faster and more accurate

. inv(diag(A))Elementwise inverse

. as scalar(r ∗ X ∗ q) with r ∈ R1×n,X ∈ Rn×n, q ∈ Rn×1Result of computation is 1× 1 matrix

. Lots more!

14

OverviewExamples

Other improvements

. Lots more!

14

OverviewExamples

Other improvements

. Lots more!

14

OverviewExamples

Other improvements

. Lots more!

14

OverviewExamples

Other improvements

. Lots more!

15

Conclusion

3 Easy to use for non-LA-experts

3 Some optimization comes built-in

7 Cannot optimize all cases

7 Debugging for more complex code can be a problem (C++)

. Our recommendation: Use it! Don’t try to understand it.

15

Conclusion

15

Conclusion

16

Thanks! Questions?

Sources:

i Conrad Sanderson, Armadillo: An Open Source C++ Linear AlgebraLibrary for Fast Prototyping and Computationally Intensive Experiments,Technical Report, NICTA, 2010.

i Armadillo source code.i C++ Programming, chapter Template Meta-Programming, Wikibooksi Matrix chain multiplication, Wikipedia

http://arma.sourceforge.net/armadillo_nicta_2010.pdfhttp://arma.sourceforge.net/armadillo_nicta_2010.pdfhttp://arma.sourceforge.net/armadillo_nicta_2010.pdfhttp://sourceforge.net/projects/arma/files/https://en.wikibooks.org/wiki/C++_Programming/Templates/Template_Meta-Programminghttps://en.wikipedia.org/wiki/Matrix_chain_multiplication

IntroductionWhat is Armadillo?Code example

Template MetaprogrammingC++ TemplatesOverviewExamples

TMP in ArmadilloOverviewExamples

Conclusion

armadillo: template metaprogramming for linear...

Documents