Installing on Windows

On Windows, we currently have .zip files that should be unzipped into the top of your Python distribution; there is no "Setup" to run. If you wish to build from source on Windows, the Unix procedure described below can be used, running python in a command-line tool.

In general, there may not be a prebuilt version of a particular kind available in every minor release. If you need a prebuilt version, choose the most recent version available.

Installing on Unix

The source distribution should be uncompressed and unpacked using the the tar program:

csh> tar xfz Numeric-n.m.tar.gz

Follow the instructions in the top-level directory for compilation and installation. Note that there are options you must consider before beginning. Installation is usually as simple as:

python setup_all.py install

However, please (please!) see the README itself for the latest details.

The Numeric Discussion List

You can subscribe to a discussion list about Numeric python using the project page at SourceForge. The list is a good place to ask questions and get help. Send mail to numpy-discussion@lists.sourceforge.net.

Bugs and Patches

Bug tracking and patch-management facilities is provided on the SourceForge project page.

CVS Repository

You can get the latest and greatest (albeit less tested and trustworthy) version of Numeric directly from our CVS repository.

FTP Site

The FTP Site contains this documentation in several formats, plus maybe some other goodies we have lying around.

Pyfort

One tool for connecting Fortran to Numeric and Python is Pyfort, sourceforge.net/projects/pyfortran.

Win32

>>> import NumTut

Traceback (innermost last):

[...]

ConfigurationError: view needs Tkinter on Win32, and either threads or the IDLE editor"

or:

ConfigurationError: view needs either threads or the IDLE editor to be enabled.

On Win32 (Windows 95, 98, NT), the Tk toolkit is needed to view the images. Additionally, either the Python interpreter needs to be compiled with thread support (which is true in the standard win32 distribution) or you need to call the NumTut program from the IDLE interactive development environment.

If you do not wish to modify your Python installation to match these requirements, you can simply ignore the references to the demonstrations which use the view() command later in this document. Using NumPy does not require image display tools, they just make some array operations easier to understand.

Unix

On Unix machines, NumTut will work best with a Python interpreter with Tk support (not true in the default configuration), with the Tkinter GUI framework available and optionally with the tkImaging add-on (part of the Python Imaging Library). If this is not the case, it will try to use an external viewer which is able to read PPM files. The default viewer is 'xv', a common image viewer available from ftp://ftp.cis.upenn.edu/pub/xv. If xv is not installed, you will get an error message similar to:

>>> import NumTut

Traceback (innermost last):

[...]

ConfigurationError: PPM image viewer 'xv' not found

You can configure NumTut to use a different image viewer, by typing e.g.:

>>> import NumTut

>>> NumTut.view.PPMVIEWER = 'ppmviewer'

>>> from NumTut import *

>>> view(greece)

If you do not have a PPM image viewer, you can simply ignore the references to the demonstrations which use the view() command later in this document. Using NumPy does not require image display tools, they just make some array operations easier to understand.

array() and typecodes

There are many ways to create arrays. The most basic one is the use of the array() function:

>>> a = array([1.2, 3.5, -1])

to make sure this worked, do:

>>> print a

[ 1.2 3.5 -1. ]

The array(numbers, typecode=None, savespace=0) function takes three arguments - the first one is the values, which have to be in a Python sequence object (such as a list or a tuple). The optional second argument is the typecode of the elements. If it is omitted, as in the example above, Python tries to find the one type which can represent all the elements. The third is discussed in Saving space.

Since the elements we gave our example were two floats and one integer, it chose `float' as the type of the resulting array. If one specifies the typecode, one can specify unequivocally the type of the elements - this is especially useful when, for example, one wants to make sure that an array contains floats even though in some cases all of its elements are integers:

>>> x,y,z = 1,2,3

>>> a = array([x,y,z]) # integers are enough for 1, 2 and 3

>>> print a

[1 2 3]

>>> a = array([x,y,z], Float) # not the default type

>>> print a

[ 1. 2. 3.]

Multidimensional Arrays

The following example shows one way of creating multidimensional arrays:

>>> ma = array([[1,2,3],[4,5,6]])

>>> print ma

[[1 2 3]

[4 5 6]]

The first argument to array() in the code above is a single list containing two lists, each containing three elements. If we wanted floats instead, we could specify, as discussed in the previous section, the optional typecode we wished:

>>> ma_floats = array([[1,2,3],[4,5,6]], Float)

>>> print ma_floats

[[ 1. 2. 3.]

[ 4. 5. 6.]]

This array allows us to introduce the notion of `shape'. The shape of an array is the set of numbers which define its dimensions. The shape of the array ma defined above is 2 by 3. More precisely, all arrays have a shape attribute which is a tuple of integers. So, in this case:

>>> print ma.shape

(2, 3)

Using the earlier definitions, this is a shape of rank 2, where the first axis has length 2, and the seond axis has length 3. The rank of an array A is always equal to len(A.shape) .

Note that shape is an attribute of array objects. It is the first of several which we will see throughout this tutorial. If you're not used to object-oriented programming, you can think of attributes as "features" or "qualities" of individual arrays. The relation between an array and its shape is similar to the relation between a person and their hair color. In Python, it's called an object/attribute relation.

What if one wants to change the dimensions of an array? For now, let us consider changing the shape of an array without making it "grow." Say, for example, we want to make the 2x3 array defined above ( ma ) an array of rank 1:

>>> flattened_ma = reshape(ma, (6,))

>>> print flattened_ma

[1 2 3 4 5 6]

One can change the shape of arrays to any shape as long as the product of all the lengths of all the axes is kept constant (in other words, as long as the number of elements in the array doesn't change):

>>> a = array([1,2,3,4,5,6,7,8])

[1 2 3 4 5 6 7 8]

>>> print a

>>> b = reshape(a, (2,4)) # 2*4 == 8

[[1 2 3 4]

[5 6 7 8]]

>>> print b

>>> c = reshape(b, (4,2) # 4*2 == 8

>>> print c

[[1 2]

[3 4]

[5 6]

[7 8]]

Notice that we used a new function, reshape() . It, like array() , is a function defined in the Numeric module. It expects an array as its first argument, and a shape as its second argument. The shape has to be a sequence of integers (a list or a tuple). Keep in mind that a tuple with a single element needs a comma at the end; the right shape tuple for a rank-1 array with 5 elements is (5,) , not (5) .

One nice feature of shape tuples is that one entry in the shape tuple is allowed to be -1 . The -1 will be automatically replaced by whatever number is needed to build a shape which does not change the size of the array. Thus:

>>> a = reshape(array(range(25)), (5,-1))

>>> print a, a.shape

[[ 0 1 2 3 4]

[ 5 6 7 8 9]

[10 11 12 13 14]

[15 16 17 18 19]

[20 21 22 23 24]] (5, 5)

The shape of an array is a modifiable attribute of the array. You can therefore change the shape of an array simply by assigning a new shape to it:

>>> a = array([1,2,3,4,5,6,7,8,9,10])

>>> a.shape

(10,)

>>> a.shape = (2,5)

>>> print a

[[ 1 2 3 4 5]

[ 6 7 8 9 10]]

>>> a.shape = (10,1) # second axis has length 1

>>> print a

[[ 1]

[ 2]

[ 3]

[ 4]

[ 5]

[ 6]

[ 7]

[ 8]

[ 9]

[10]]

>>> a.shape = (5,-1) # note the -1 trick described above

>>> print a

[[ 1 2]

[ 3 4]

[ 5 6]

[ 7 8]

[ 9 10]]

As in the rest of Python, violating rules (such as the one about which shapes are allowed) results in exceptions:

>>> a.shape = (6,-1)

Traceback (innermost last):

File "<stdin>", line 1, in ?

ValueError: total size of new array must be unchanged

zeros() and ones()

Often, one needs to manipulate arrays filled with numbers which aren't available beforehand. The Numeric module provides a few functions which create arrays from scratch:

zeros() and ones() simply create arrays of a given shape filled with zeros and ones respectively:

>>> z = zeros((3,3))

>>> print z

[[0 0 0]

[0 0 0]

[0 0 0]]

>>> o = ones([2,3])

>>> print o

[[1 1 1]

[1 1 1]]

Note that the first argument is a shape - it needs to be a list or a tuple of integers. Also note that the default type for the returned arrays is Int , which you can feel free to override using something like:

>>> o = ones((2,3), Float)

>>> print o

[[ 1. 1. 1.]

[ 1. 1. 1.]]

arrayrange()

The arrayrange() function is similar to the range() function in Python, except that it returns an array as opposed to a list.

>>> r = arrayrange(10)

>>> print r

[0 1 2 3 4 5 6 7 8 9]

Combining the arrayrange() with the reshape() function, we can get:

>>> big = reshape(arrayrange(100),(10,10))

>>> print big
[[ 0 1 2 3 4 5 6 7 8 9]

[10 11 12 13 14 15 16 17 18 19]

[20 21 22 23 24 25 26 27 28 29]

[30 31 32 33 34 35 36 37 38 39]

[40 41 42 43 44 45 46 47 48 49]

[50 51 52 53 54 55 56 57 58 59]

[60 61 62 63 64 65 66 67 68 69]

[70 71 72 73 74 75 76 77 78 79]

[80 81 82 83 84 85 86 87 88 89]

[90 91 92 93 94 95 96 97 98 99]]

>>> view(reshape(arrayrange(10000),(100,100)))

# array of increasing lightness from top down (slowly) and from left to

# right (faster) is shown

arange() is a shorthand for arrayrange() .

One can set the start, stop and step arguments, which allows for more varied ranges:

>>> print arrayrange(10,-10,-2)

[10 8 6 4 2 0 -2 -4 -6 -8]

An important feature of arrayrange is that it can be used with non-integer starting points and strides:

>>> print arrayrange(5.0)

[ 0. 1. 2. 3. 4.]

>>> print arrayrange(0, 1, .2)

[ 0. 0.2 0.4 0.6 0.8]

If you want to create an array with just one value, repeated over and over, you can use the * operator applied to lists

>>> a = array([[3]*5]*5)

>>> print a

[[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]]

but that is relatively slow, since the duplication is done on Python lists. A quicker way would be to start with 0's and add 3:

>>> a = zeros([5,5]) + 3

>>> print a

[[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]

[3 3 3 3 3]]

The optional typecode argument can force the typecode of the resulting array, which is otherwise the "highest" of the starting and stopping arguments. The starting argument defaults to 0 if not specified. Note that if a typecode is specified which is "lower" than that which arrayrange would normally use, the array is the result of a precision-losing cast (a round-down, as that used in the astype method for arrays.)

Creating an array from a function: fromfunction()

Finally, one may want to create an array with contents which are the result of a function evaluation. This is done using the fromfunction() function, which takes two arguments, a shape and a callable object (usually a function). For example:

>>> def dist(x,y):

... return (x-5)**2+(y-5)**2 # distance from point (5,5) squared

...

>>> m = fromfunction(dist, (10,10))

>>> print m

[[50 41 34 29 26 25 26 29 34 41]

[41 32 25 20 17 16 17 20 25 32]

[34 25 18 13 10 9 10 13 18 25]

[29 20 13 8 5 4 5 8 13 20]

[26 17 10 5 2 1 2 5 10 17]

[25 16 9 4 1 0 1 4 9 16]

[26 17 10 5 2 1 2 5 10 17]

[29 20 13 8 5 4 5 8 13 20]

[34 25 18 13 10 9 10 13 18 25]

[41 32 25 20 17 16 17 20 25 32]]

>>> view(fromfunction(dist, (100,100))

# shows image which is dark in topleft corner, and lighter away from it.

>>> m = fromfunction(lambda i,j,k: 100*(i+1)+10*(j+1)+(k+1), (4,2,3))

>>> print m

[[[111 112 113]

[121 122 123]]

[[211 212 213]

[221 222 223]]

[[311 312 313]

[321 322 323]]

[[411 412 413]

[421 422 423]]]

By examining the above examples, one can see that fromfunction() creates an array of the shape specified by its second argument, and with the contents corresponding to the value of the function argument (the first argument) evaluated at the indices of the array. Thus the value of m[3,4] in the first example above is the value of dist when x=3 and y=4 . Similarly for the lambda function in the second example, but with a rank-3 array.

The implementation of fromfunction consists of:

def fromfunction(function, dimensions):

return apply(function, tuple(indices(dimensions)))

which means that the function function is called for each element in the sequence indices(dimensions). As described in the definition of indices, this consists of arrays of indices which will be of rank one less than that specified by dimensions. This means that the function argument must accept the same number of arguments as there are dimensions in dimensions, and that each argument will be an array of the same shape as that specified by dimensions. Furthermore, the array which is passed as the first argument corresponds to the indices of each element in the resulting array along the first axis, that which is passed as the second argument corresponds to the indices of each element in the resulting array along the second axis, etc. A consequence of this is that the function which is used with fromfunction will work as expected only if it performs a separable computation on its arguments, and expects its arguments to be indices along each axis. Thus, no logical operation on the arguments can be performed, or any non-shape preserving operation. The first example below satisfies these requirements, hence works (the x and y arrays both get 10x10 arrays as input corresponding to the values of the indices along the two dimensions), while the second array attemps to do a comparison test on an array of indices, which fails.

>>> def buggy(test):

... if test > 4: return 1

... else: return 0

...

>>> print fromfunction(buggy, (10,))

Traceback (innermost last):

File "<stdin>", line 1, in ?

File "C:\PYTHON\LIB\Numeric.py", line 157, in fromfunction

return apply(function, tuple(indices(dimensions)))

File "<stdin>", line 2, in buggy

TypeError: Comparison of multiarray objects is not implemented.

identity()

The simplest array constructor is the identity(n) function, which takes a single integer argument and returns a square identity array of that size of integers:

>>> print identity(5)

[[1 0 0 0 0]

[0 1 0 0 0]

[0 0 1 0 0]

[0 0 0 1 0]

[0 0 0 0 1]]

>>> view(identity(100))

# shows black square with a single white diagonal

Automatic Coercions and Binary Operations

The rules followed by NumPy when performing binary operations on arrays mirror those used by Python in general. Operations between numeric and non-numeric types are not allowed (e.g. an array of characters can't be added to an array of numbers), and operations between mixed number types (e.g. floats and integers, floats and omplex numbers, or in the case of NumPy, operations between any two arrays with different numeric typecodes) first perform a coercion of the 'smaller' numeric type to the type of the `larger' numeric type. Finally, when scalars and arrays are operated on together, the scalar is converted to a rank-0 array first. Thus, adding a "small" integer to a "large" floating point array is equivalent to first casting the integer "up" to the typecode of the array:

>>> arange(0, 1.0, .1) + 12

array([ 12. , 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 12.8, 12.9])

The automatic coercions are described in Figure 1. Avoiding upcasting is discussed in Saving space.

Deliberate up-casting: The asarray function

One more array constructor is the asarray() function. It is used if you want to have an array of a specific typecode and you don't know what typecode array you have (for example, in a generic function which can operate on all kinds of arrays, but needs them to be converted to complex arrays). If the array it gets as an argument is of the right typecode, it will get sent back unchanged. If the array is not of the right typecode, each element of the new array will be the result of the coercion to the new type of the old elements. asarray() will refuse to operate if there might be loss of information -- in other words, asarray() only casts 'up'.

asarray is also used when you have a function that operates on arrays, but you want to allow people to call it with an arbitrary python sequence object. This gives your function a behavior similar to that of most of the builtin functions that operate on arrays.

The typecode value table

The typecodes identifiers ( Float0 , etc.) have as values single-character strings. The mapping between typecode and character strings is machine dependent. An example of the correspondences between typecode characters and the typecode identifiers for 32-bit architectures are shown in Table 3-X.

Consequences of silent upcasting

When dealing with very large arrays of floats and if precision is not important (or arrays of small integers), then it may be worthwhile to cast the arrays to "small" typecodes, such as Int8 , Int16 or Float32 . As the standard Python integers and floats correspond to the typecodes Int32 and Float64 , using them in apparently "innocent" ways will result in up-casting, which may null the benefit of the use of small typecode arrays. For example:

>>> mylargearray.typecode()

'f' # a.k.a. Float32 on a Pentium

>>> mylargearray.itemsize()

4

>>> mylargearray = mylargearray + 1 # 1 is an Int64 on a Pentium

>>> mylargearray.typecode() # see Fig. 1 for explanation.

'd'

>>> mylargearray.itemsize()

8

Note that the sizes returned by the itemsize() method are expressed in bytes.

Saving space

Numeric arrays can be created using an optional, keyworded argument to the constructor, savespace. If savespace is set to 1, Numeric will attempt to avoid the silent upcasting behavior. The status of an array can be queried with the spacesaver() method. If x.spacesaver() is true, x has its space-saving flag set. The flag can be set with the savespace method: x.savespace(1) to set it, x.savespace(0) to clear it.

Deliberate casts (potentially down): the astype method

You may also force NumPy to cast any number array to another number array. For example, to take an array of any numeric type (IntX or FloatX or ComplexX or UnsignedInt8) and convert it to a 64-bit float, one can do:

>>> floatarray = otherarray.astype(Float64)

The typecode can be any of the number typecodes, "larger" or "smaller". If it is larger, this is a cast-up, as if asarray() had been used. If it is smaller, the standard casting rules of the underlying language (C) are used, which means that truncation or loss of precision can occur:

>>> print x

[ 0. 0.4 0.8 1.2 1.6]

>>> x.astype(Int)

array([0, 0, 0, 1, 1])

If the typecode used with astype() is the original array's typecode, then a copy of the original array is returned.

Operating on Arrays