Brief SciPy Introduction Jacob B. SchroderUniversity of New Mexico

1

Introduction to SciPy—Numerical Python• What is SciPy? From www.scipy.org,

"SciPy is a collection of mathematical algorithms and convenience functions built on the Numpy extension of Python. It adds significant power to the interactive Python session by providing the user with high-level commands and classes for manipulating and visualizing data. With SciPy an interactive Python session becomes a data-processing and system-prototyping environment rivaling systems such as MATLAB, IDL, Octave, R-Lab, and SciLab."

• Additionally, SciPy users can take advantage of open source packages providing functionality beyond what many commercial packages provide.

Example: PyAMGhttps://github.com/pyamg/pyamg

2

Introduction to SciPy: Numerical Python• SciPy depends on NumPy

• If you know how to use SciPy, then you'll know how to use NumPy• We will also use matplotlib for plotting (as seen in the last lab)

• SciPy's organization: We will primarily use the packagesscipy.linalg Dense linear algebrascipy.sparse Sparse linear algebraà What is dense vs. sparse linear algebra, anyway?

• But, SciPy has many packages for just about any kind of scientific computing

3

Introduction to SciPy: Numerical Python

Warning: This tutorial is for interactive programming with iPython, notJupyter notebooks.

I recommend first using the interactive iPython prompt before attempting notebooks.

You'll need to first install python, ipython, scipy, numpy, and matplotlib

Then run from your command prompt,$ ipython>>> import scipy

4

Introduction to SciPy: Numerical Python• If you ever get lost, or forget

something, remember:All SciPy functions are heavily documented.

• From inside of iPython, you can get documentation with à

5

Introduction to SciPy: Numerical PythonKey Differences with Matlab:

1. Default data type is "array"

>>> from scipy import *>>> a = rand(2,2)>>> aarray([[...>>> a.shape(2,2)

2. Note, the array object is not a matrix object !!!This carries out element-wise multiplication, not matrix-matrix multiplication>>> a*aTo do matrix-matrix, or matrix-vector multiplication do>>> b = rand(2,1) >>> dot(a,b)array([[...>> dot(a,a)array([[[... 6

Introduction to SciPy: Numerical PythonKey Differences with Matlab:

1. Do not use the "matrix" object to do linear algebra. For instance, this will do a matrix-matrix multiply.

>>> mat(a)matrix([[... >>> mat(a)*mat(a)

Things can get very confusing, very fast. For instance, most routines will take a matrix and return an array

>>> from scipy.linalg import inv>>> inv(mat(a))array([[..

2. It is safest to always use dot(A,B)1. This will always do mat-mat multiply

7

Introduction to SciPy: Numerical PythonKey Differences with Matlab:

1. You have to pay close attention to the shapes of arrays

>>> a.shape(2,2)>>> b.shape(2,1)

>>> dot(a, b)array([[...>>> c = dot(a,b)>>> c.shape 2x2 * 2x1 = 2x1 (2,1)

>>> dot(b,a)Error

>>> dot(b.T, a) 1x2 * 2x2 = 1x2(1,2)

>>> dot(b, b.T) 2x1 * 1x2 = 2x2 Outer product>>> dot(b.T, b) 1x2 * 2x1 = 1x1 Inner product

Note: this result is a 1x1 array, not a scalar8

Introduction to SciPy: Numerical Python1. There are 1D and 2D arrays.

>>> b.shape This is a 2D array object that represents(2,1) a column vector>>> b.T.shape This is a 2D array object that represents a (1,2) a row vector>>> c = rand(2,) This is a 1D array object that represents>>> c.shape a vector (neither column or row)(2,)>>> dot(a, c).shape The result here inherits the 1D shape(2,)>>> dot(b,c)Error>>> dot(c,b) The 1D array "c" is acting like a row vector,array([...]) by allowing you to do an inner product. Note

the result here is a 1D array or size (1,)>>> dot(a,c) Both of these operations work, but now notice that>>> dot(c,a) c is acting like a row and a column vector.

Confusing!

2. It's safest to stick to 2D array objects for column vectors and row vectors. There's less room for confusion9

Introduction to SciPy: Numerical PythonKey Differences with Matlab:

1. Note, indexing starts at 0, and slicing x[0:K] grabs entries from 0 to (K-1), i.e., K total entries

>>> f = arange(10)>>> farray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> f[0:5] This grabs the first 5 entriesarray([0, 1, 2, 3, 4])

>>> f[5] Notice that entry at index 5 is not 5 returned by f[0:5] !!!

>>> f[0] Indexing starts at 0 !!! 0

10

Introduction to SciPy: Numerical PythonKey SciPy Tools1. You can easily reshape your data

>>> a = arange(9)>>> aarray([0, 1, 2, 3, 4, 5, 6, 7, 8])

>>> b = a.reshape(3,3) 1D array to 2D 3x3 array>>> barray([[0, 1, 2],

[3, 4, 5],[6, 7, 8]])

>>> b = a.reshape(9,1) 1D array to 2D column vector>>> barray([[0],

[1],[2],..[8]]) 11

Introduction to SciPy: Numerical PythonKey SciPy Tools1. You can write files that define useful functions for you2. And then import these new functions into your main Python script3. Create a file simple_function.py with the contents

from scipy import *

def simple_func1(x,y):''' #TheseThis defines a simple function #linesInput: x, y scalars #mustOutput: x*y #all''' #be indentedreturn x*y #once

4. Then, from inside of ipython type>>> from simple_function import simple_func1>>> simple_func1(2,3)6

12

Introduction to SciPy: Numerical PythonKey SciPy Tools1. Modify your simple_func1(x,y) so that you use some basic control structures. 2. Try this function bodies (observe how you use exponents)

def simple_func1(x,y):if(x < y):

return x**3elif (x == y):

return y**3 * x**3else:

return y**33. Or, you can try something with a loop

def simple_func1(x,y):for i in range(x): # x must be an integer for this to work

y += i + yreturn y

13

Introduction to SciPy: Numerical PythonKey SciPy Tools1. Try something with a while loop, for example

def simple_func1(x,y):while( y < x):

y += y*yreturn y

2. The basic comparison operators all work,x < y less thany > x greater thanx == y equality testx >= y greater than or equalx <= y less than or equal

3. And, you can combine these operators

if ( (x < y) and (z > y) ):Do something

4. Experiment with if-statements and comparisons in simple_func1(.)1. Maybe add a parameter z

14

Introduction to SciPy: Numerical PythonKey SciPy Tools1. SciPy has all the built in functions that you would expect

>>> pi3.141592653589793>>> sin(pi)something times 10-16

>>> cos(pi)-1.0>>> sqrt(4)2.0

>>> 4**(1.0/2.0)2.0>>> log10(100)2.0

>>> log(e)1.0>>> e2.718281828459045 15

Introduction to SciPy: Numerical PythonKey SciPy Tools

1. Python is a "dynamically typed" language, meaning the type of a variable is determined at run-time.

>>> type(pi)float

>>> a = zeros((1,), dtype=int)>>> a[0] = pi>>> aarray([3])

2. You've implicitly type-cast pi to an integer when assigning to a[0] !!That is, you've lost all of the decimal places of pi !!1. Repeat the above experiment, but let

a = zeros((1,), dtype=float)

3. You can even go so far as to specify int32, int64, float32, float64, but we won't explore that in this class.

16

Introduction to SciPy: Numerical PythonKey SciPy Tools1. Accessing array elements is fairly straight-forward

>>> a = arange(4).reshape(2,2)>>> aarray([[0, 1],

[2, 3]])>>> a[0,1] = 100>>> aarray([[0, 100],

[2, 3]])

2. Most operations are applied point-wise to arrays, for example

>>> b = rand(2,2)>>> sqrt(a) Take square root of each element of a

>>> a**3 Take the cube of each element of a

>>> a*b Element-wise multiplication of a and bThis is not a matrix-matrix multiply !!

17

Introduction to SciPy: Numerical PythonIn-place memory operations versus over-writing1. In-place memory operations: note how this function writes the existing memory of v

>>> def set_vec_inplace(v, x): v[:] = x>>> v = zeros((3,))>>> v

array([0., 0., 0.])>>> set_vec_inplace(v, 1.1) Note no return value!>>> v

array([1.1, 1.1, 1.1])

2. Overwritting memory: note how this function overwrites v, and then returns the new v>>> def set_vec(v, x): v = x*ones(v.shape); return v>>> v = zeros((3,))>>> v

array([0., 0., 0.])>>> set_vec(v, 1.1) Note the return value!

array([1.1, 1.1, 1.1])>>> v

array([0., 0., 0.])18

Introduction to SciPy: Numerical Python1. In general, SciPy does not let you dynamically increase the size of arrays.

That is, once you set an array to be (m,n), e.g.,

>>> m = 2; n = 2>>> a = zeros((m,n))

You cannot index in at locations outside of the mxn dimensions, e.g., these commands all fail

>>> a[m, 0]>>> a[m+1, 0]>>> a[0,n]>>> a[0,n+1]

2. But this fact forces you to pre-allocate all of your arrays, which is great for efficiency! 3. Pre-allocation just means that you set the array's size only once, at the start

19

Introduction to SciPy: Numerical PythonThis lab's homework (or if you finish early!) is to work through the linear algebra tutorialhttps://docs.scipy.org/doc/scipy-1.0.0/reference/tutorial/linalg.html

Only work through these sections,• Basic routines

• Finding Inverse• Solving linear system• Finding Determinant• Computing norms• Solving linear least-squares problems and pseudo-inverses• [Skip Generalized inverse ]

• Decompositions• Eigenvalues and eigenvectors• Singular value decomposition• LU decomposition• Cholesky decomposition• [Skip QR decomposition ]• [Skip Schur decomposition ]• [Skip Interpolative Decomposition]

• [Skip Entire Matrix Function Section ]

20

21