Python Numpy

Python Numpy

Previous - Python Packages

You are aware that the Python list is pretty powerful: A list can hold any type and can hold different types at the same time. You can also change, add and remove elements.

This is wonderful, but one feature is missing.

When analyzing data, you'll often want to carry out operations over entire collections of values, and you want to do this fast. With lists, this is a problem.

Let's consider we have two lists

If you now want to calculate the Body Mass Index for each family member, you'd hope that this call can work, making the calculations element-wise.
Unfortunately, Python throws an error, because it has no idea how to do calculations with lists.

You could solve this by going through each list element one after the other, and calculating the BMI for each person separately, but this is terribly inefficient and tiresome to write.

A way more elegant solution is to use NumPy, or Numeric Python. It's a Python package that, among others, provides an alternative to the regular python list: the Numpy array.

The Numpy array is pretty similar to a regular Python list, but has one additional feature: you can perform calculations over all entire arrays.

It's really easy, and super-fast as well.

Let's start with _creating_ a numpy array. You do this with Numpy's `array()` function: the input is a regular Python list. I'm using `array()` twice here, to create Numpy versions of the `height` and `weight` lists.

Let's try to calculate everybody's BMI with a single call again:

First, we tried to do calculations with regular lists, like this, but this gave us an error, because Python doesn't now how to do calculations with lists like we want them to.

Next, these regular lists were converted to Numpy arrays. The same operations now work without any problem: Numpy knows how to work with arrays as if they are single values

First of all, Numpy can do all of this so easily because it assumes that your Numpy array can only contain values of a single type. It's either an array of floats, either an array of booleans, and so on.

If you do try to create an array with different types, like this for example, the resulting Numpy array will contain a single type, string in this case. The boolean and the float were both converted to strings.

Second, you should know that a Numpy array is simply a new kind of Python type, like the float, string and list types from before
Take this Python list and this numpy array, for example:

If you do `python_list + python_list`, the list elements are pasted together, generating a list with 6 elements.

 If you do this with the numpy arrays, on the other hand, Python will do an element-wise sum of the array:

Specifically for Numpy, there's also another way to do list subsetting: using an array of booleans. Say you want to get all BMI values in the bmi arrays that are over 23. A first step is using the greater than sign, like this:

The result is a Numpy array containing booleans: True if the corresponding bmi is above 23, False if it's below. 

Next, you can use this boolean array inside square brackets to do subsetting

• ·         The Numpy Package provides the array, a data type that can be used to do element-wise calculations.
• ·         Because Numpy arrays can only hold element of a single type, calculations on Numpy arrays can be carried out way faster than regular Python lists.

Previous - Python Packages

Python Packages

Python Packages

Previous - Python Methods | Next - Python Numpy

However, adding all functions and methods that have been written up to now to the same Python distribution would be a mess. There would be tons and tons of code in there that you'll never use. Also, maintaining all of this code would be a real pain.

This is where packages come into play. You can think of package as a directory of Python scripts.
Each such script is a so-called module. These modules specify functions, methods and new Python types aimed at solving particular problems. There are thousands of Python packages available from the internet. Among them are packages for data science:

• ·         there's numpy to efficiently work with arrays,
• ·         matplotlib for data visualization,
• ·         scikit-learn for machine learning, and many others.

What are Packages

If you want to install packages on your own system, you'll want to use pip, a package maintenance system for Python.

If you go to this URL, you can download the file `get-pip.py`. Next, you go to the terminal, and execute `python3 get-pip.py`. Now you can use pip to actually install a Python package of your choosing. 

Suppose we want to install the numpy package, you type `pip3 install numpy`. You have to use the commands python3 and pip3 here to tell our system that we're working with Python version 3.

Now that the package is installed, you can actually start using it in one of your Python scripts. Before you can do this, you should import the package, or a specific module of the package. You can do this with the `import` statement.

To import the entire numpy package, you can do import numpy, like this. A commonly used function in Numpy is `array()`. It takes a list as input. Simply calling the array function like this will generate an error.

To refer to the array function from the numpy package, you'll need this:

Now, instead of numpy.array(), you'll have to use np.array() to use Numpy's array function:

There are cases in which you only need one specific function of a package. Python allows you to make this explicit in your code.

Suppose that we only want to use the array() function from the Numpy package. Instead of doing import numpy, you can instead do from numpy import array, like this:

Previous - Python Methods| Next - Python Numpy

Python Methods

Python Methods

Previous - Python Functions | Next - Python Packages

Among other Python types, we've created strings, floats and lists, like the ones you see here. Each one of these values or data structures is so-called Python object. This string is an object, this float is an object, but this list is also an object.

These objects have a specific type that you already know: string, float, and list, 

But next to that, Python objects also come with a bunch of so-called "methods". You can think of methods as _functions_ that "belong to" Python objects. 

A Python object of type string has methods, such as capitalize and replace, but also objects of type float and list have specific methods depending on the type.

Suppose you want to get the index of the string "mom" in the `fam` list. `fam` is an Python object with the type `list`, and has a method named `index()`. To call the method, you use the dot notation, like this. The only input is the string "mom", the element you want to get the index for. Python returns 4

You can call the method `capitalize()` on `sister`, without any inputs. It returns a string where the first letter is capitalized now:

In Python, everything is an object, and each object has specific methods associated. Depending on the type of the object, list, string, float, whatever, the available methods are different.

A string object like `sister` has a replace method, but a list like `fam` doesn't have this, as you can see from this error.

Some methods can change the objects they are called on. Let's retake the `fam` list, and call the `append()` method on it. As the input, we pass a string we want to add to the list:

Python doesn't generate an output, but if we check the `fam` list again, we see that it has been extended with the string `"me"`:

You have Python functions, like type(), max() and round(), that you can call like this:

There are also methods, which are functions that are specific to Python objects. Depending on the type of the Python object you're dealing with, you'll be able to use different methods and they behave differently.

You can call methods on the objects with the dot notation, like this for example.

Previous - Python Functions | Next - Python Packages

Python Functions

Python Functions

Previous - Manipulating Python Lists | Next - Python Methods

type(), for example, is a function that returns the type of a value.

Suppose you have the list containing only the heights of your family, fam: Say that you want to get the maximum value in this list.

Use Python's `max()` function. This is one of Python's built-in functions,

Another one of these built-in functions is `round()`. It takes two inputs: first, a number you want to round, and second, the precision with which to round, so how many digits behind the decimal point you want to keep.

But there's more. It's perfectly possible to call the `round()` function with only one input, like this. This time, Python figured out that you didn't specify the second input, and automatically chooses to round the number to the closest integer.

Previous - Manipulating Python Lists | Next - Python Methods

Manipulating Python Lists

Manipulating Python Lists

Previous - Python Lists| Next - Python Functions

Changing list elements is pretty straightforward. You use the same square brackets that we've used to subset lists, and then assign new elements to it using the equals sign.

To change this list element, which is at index 7, you can use this line:

To change the elements "liz" and 1.73, you access the first two elements with zero colon 2, and then assign a new list to it.

If you use the plus sign with two lists, Python simply pastes together their contents in a single list. 

Of course, you can also store this new list in a variable, `fam_ext` for example.

Finally, deleting elements from a list is also pretty straightforward; you'll have to use `del` here. Take this line, for example, that deletes the element with index 2.

Understanding how Python lists actually work behind the scenes becomes pretty important

What actually happens when you create a new list, `x`, like this?

Well, in a simplified sense, you're storing a list in your computer memory, and store the 'address' of that list, so where the list is in your computer memory, in `x`. This means that `x` does not actually contain all the list elements, it rather contains a reference to the list.

Let's store the list `x` as a new variable `y`, by simply using the equals sign:

Let's now change the element with index one in the list `y`, as follows:

The funky thing is that if you now check out `x` again, also here the second element was changed

That's because when you copied x to y with the equals sign, you copied the reference to the list, not the actual values themselves. 

When you're updating an element the list, though, it's one and the same list in the computer memory you are changing.

Both `x` and `y` point to this list, so the update is visible from both.

If you want to create a list `y` that points to a new list in the memory with the same variables, you'll need to use something else than the equals sign. You can use the `list()` function, like this, or use slicing to select all list elements explicitly.

If you now make a change to the list `y` points to, `x` is not affected:

Previous - Python Lists| Next - Python Functions

Python Lists

Python Lists

Previous- Python Variables and Types | Next - Manipulating Python Lists

As a data scientist, you'll often want to work with many data points. 

If you for example want to measure the height of everybody in your family, and store this information in python, it would be inconvenient to create a new python variable for each data point you collected right?

What you can do instead, is store all this information in a python _list_. 

You can build such a list with square brackets. Suppose you asked your two sisters and parents for their height, in meters. You can build the list as follows:

Of course, also this data structure can be referenced to with a variable. Simply put the variable name and the equals sign in front, like here:

A list is a way to give a single name to a collection of values. These values, or elements, can have any type; they can be floats, integer, booleans, strings, but also more advanced Python types, even lists.

Lists can also contain lists themselves. Instead of putting the strings in between the numbers, you can create little sublists for each member of the family. One for liz, one for emma and so on.

 Now, you can tell Python that these sublists are the elements of another list, that I named fam2: the little lists are wrapped in square brackets and separated with commas. 

We're dealing with a new Python type here. These calls show that both `fam` and `fam2` are lists. 

Subsetting Lists

You can also count backwards, using negative indexes. This is useful if you want to get some elements at the end of your list. To get your dad's height, for example, you'll need the index -1. These are the negative indexes for all list elements.

Apart from indexing, there's also something called slicing, which allows you to select multiple elements from a list, thus creating a new list. You can do this by specifying a range, using a colon.

Can you guess what it'll return?

Apparently, only the elements with index 3 and 4, get returned. The element with index 5 is not included. In general, this is the syntax: the index you specify before the colon, so where the slice starts, is included, while the index you specify after the colon, where the slice ends, is not.

You can also choose to just leave out the index before or after the colon. If you leave out the index where the slice should begin, you're telling Python to start the slice from index 0, like this example.

If you leave out the index where the slice should end, you include all elements up to and including the last element in the list, like here:

Previous- Python Variables and Types | Next - Manipulating Python Lists