Introduction to Python for sciences 2020

Acquire strong basis in Python to use it efficiently

Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Eric Maldonado (Irstea), Franck Thollard (ISTerre), Christophe Picard (LJK), Loïc Huder (ISTerre)

Standard types and basic statements

Function calls

There are built-in functions and the developers can of course define other functions. To call a function:

print("hello")

hello

Some functions return a result.

round(1.2)

1

It's common to store the result in a variable:

my_var = round(1.2)

which can then be used:

print(my_var)

1

Few standard types

• Simple types (int, float, bool, complex)
• Standard type str
• Standard type list
• Standard type tuple

int (integers)

a = 4
c = -10

# binary notation (base 2)
b = 0b010

# octal notation (base 8)
o = 0o011

# hexadecimal (base 16)
h = 0x1cf0

a = int('1')       # base 10
a = int('111', 2)  # base 2
a = int('70', 8)   # base 8
a = int('16', 16)  # base 16

Remark: int in Python 3 are impressive! No limit! See https://docs.python.org/3.1/whatsnew/3.0.html#integers

Arithmetic operations

print(10 + 3)
print(10 - 3)
print(10 * 3)
print(10 / 3)  # float division
print(10 // 3)  # integer division
print(10 % 3)

13
7
30
3.3333333333333335
3
1

bool (booleans)

b = bool('1')
b = False
b = True

Comparison operations (bool)

• == equal
• != différent
• < inferior
• <= inferior or equal
• > superior
• >= superior or equal

Keyword is: check identity

a = None
print(a is None)
print(a is not None)

True
False

Keywords and and or

are_we_12_in_the_room() and is_there_an_open_window()
are_we_12_in_the_room() or is_there_an_open_window()

True and True

True

True and False

False

False and False

False

True or True

True

True or False

True

False or False

False

float (real, double precision) and complex

# float
a = float('1')
a = 1.234
a = 1e2
a = -1e-2
a = .2

# complex (2 floats)
c = complex('1')
c = 1 + 2j
print(c, c.real, c.imag)

(1+2j) 1.0 2.0

Remark: notation var_name.attr_name to access to an attribute of an object.

Warning about floating-point arithmetic and numerical errors!

b = 1e16
c = 1.2 + b
d = c - b
print(d)

2.0

Very general issue (not Python):

see https://en.wikipedia.org/wiki/Floating-point_arithmetic

Standard type str

s = 'hello'
s = "hello"

s = ('How is it possible to write a very very '
     'very long string with lines limited to 79 characters?')

s = """Strings on 
more thand 
one line.
"""
print(s)

Strings on 
more thand 
one line.

Warning: big difference between Python 2 and Python 3. In Python 3, str are unicode and there is another type bytes.

Methods of the type str

Objects of built-in types have methods associated with their type (object oriented programming). The built-in function dir returns a list of name of the attributes. For a string, these attributes are python system attributes (with double-underscores) and several public methods:

s = 'abcdef'
print(dir(s))

['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

To access an attribute of an object (here, the method str.startswith), we use the dot:

s.startswith('a')

True

standard type str

function str.format

Docstring:
S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs.
The substitutions are identified by braces ('{' and '}').

a = 1.23456789
'a = {}'.format(a)

'a = 1.23456789'

'a = {:.4f}'.format(a)

'a = 1.2346'

'a = {:8.4f}'.format(a)

'a =   1.2346'

'a = {:.4e} (scientific notation)'.format(a)

'a = 1.2346e+00 (scientific notation)'

print('{}\t{}\t{}'.format(1, 2, 3))

1	2	3

standard type str

New in Python 3.6: format strings

a = 1.23456789
f'a = {a}'

'a = 1.23456789'

f'a = {a:.4f}'

'a = 1.2346'

f'a = {a:8.4f}'

'a =   1.2346'

f'a = {a:.4e} (scientific notation)'

'a = 1.2346e+00 (scientific notation)'

print(f'{1}\t{1+1}\t{2+1}')

1	2	3

standard type str

Strings are immutable "sequences".

• lookup

s = 'abcdef'
print('a' in s)
print('hello' not in s)

True
True

• We can get an element of a string (index starts from 0):

print(s[0])

a

• since strings are immutable, they can not be modified inplace. If we try, we get an error:

s[0] = 'b'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-55620f378bce> in <module>()
----> 1 s[0] = 'b'

TypeError: 'str' object does not support item assignment

• since strings are sequences, they can be "sliced" (we will soon study in details this powerful notation):

s[1:3]

'bc'

• it is very simple to manipulate strings in many ways:

print((s.capitalize() + ' ' + s.upper() + '\n') * 4 ) 

Abcdef ABCDEF
Abcdef ABCDEF
Abcdef ABCDEF
Abcdef ABCDEF

Slicing

Very general, can be used on all sequences as str, list, etc... Not simple for beginners but very powerfull (see here and here).

Python indexes and slices for a six-element str. Indexes enumerate the elements, slices enumerate the spaces between the elements.

Index from rear:    -6  -5  -4  -3  -2  -1
Index from front:    0   1   2   3   4   5      
                   +---+---+---+---+---+---+
                   | a | b | c | d | e | f |    
                   +---+---+---+---+---+---+    
Slice from front:  0   1   2   3   4   5   6    
Slice from rear:  -6  -5  -4  -3  -2  -1   None

s = 'abcdef'
# s[start:stop:step]
s[2:6:2]

'ce'

# s[start:stop]
s[2:6]

'cdef'

# s[start:]
s[1:]

'bcdef'

# s[:stop]
s[:2]

'ab'

# step = -1: goes through the string in reverse order
s[::-1]

'fedcba'

Do it yourself

Suppose we have a string representing a header line of the form:

s = ' wind;temperature;;pressure '

• Remove leading and ending blanks (see str.replace and str.strip)
• Extract the first field (e.g. using find method and slicing)
• Extract the last field (e.g. using rfind method and slicing)
• Check for empty field (e.g. find ";;" pattern)
• Remove empty field (e.g. using replace method)
• BONUS : Extract the second field (tip :find can take an argument that tells where to start the search)

A possible solution:

s = ' wind;temperature;;pressure '
# remove leading blanks
s = s.strip()
print("--{}--".format(s))
# extract the first field
idx = s.find(";")
s0 = s[0:idx]
print(s0)
# extract the second field
idx1 = s.find(";", idx+1) # start the search after the first ";"
s1 = s[idx+1:idx1]
print(s1)
# extract the last field
idx2 = s.rfind(";")
s2 = s[idx2+1:]
print(s2)
idx_first_empty_field = s.find(";;")
print(idx_first_empty_field)
# remove empty field
s_no_empty = s.replace(";;", ";")
print(s_no_empty)

--wind;temperature;;pressure--
wind
temperature
pressure
16
wind;temperature;pressure

standard type list

A list is a mutable sequence of (possibly inhomogeneous) elements.

type([0, 'a'])

list

# create an empty list
l = []
# fill the list (with the function append)
l.append('2')
# fill the list (with the function extend)
l.extend([6, 3.])
print(l)

['2', 6, 3.0]

# concatenate lists with the operator +
print(l + ['hello', 3])

['2', 6, 3.0, 'hello', 3]

# get values
print(l[0], l[2], l[-2])
# slicing
print(l[0:2])

2 3.0 6
['2', 6]

standard type tuple

A tuple is a immutable sequence of (possibly inhomogeneous) elements.

Remark: when you need a sequence that won't be modified, tuple is usually more efficient than list.

t = 0, 'a', 1.2
t1 = (5, 'hello')
t2 = tuple([1.1, 2])
type(t)

tuple

t[1]  # indexing

'a'

t[1:]  # slicing

('a', 1.2)

a, b = t1  # tuple assigment
print(b)

hello

Mutable and immutable objects

The objects of type str, int, float, bool are immutable. They can not be modified. Of course, a name that points towards an integer can point towards a different integer.

i = 1
i = i + 2  # (or i += 2)
print(i)
i = 10
print(i)

3
10

Here, the objects 1 and 3 have not been modified.

Mutable and immutable objects

An object of type list is mutable:

l = [0, 5]
print(l)
l.append('hello')
print(l)

[0, 5]
[0, 5, 'hello']

Here, the object list tagged by the name l has been modified inplace.

References and del keyword

del removes a reference. If an object in not binded to any names, Python can delete it from its internal memory.

l = ['a', 'b']
del l[1]
print(l)

['a']

More on slicing

Very general, can be used on all sequences as str, list, etc... Not simple for beginners but very powerfull (see here and here).

Python indexes and slices for a six-element str. Indexes enumerate the elements, slices enumerate the spaces between the elements.

Index from rear:    -6  -5  -4  -3  -2  -1
Index from front:    0   1   2   3   4   5      
                   +---+---+---+---+---+---+
                   | a | b | c | d | e | f |    
                   +---+---+---+---+---+---+    
Slice from front:  0   1   2   3   4   5   6    
Slice from rear:  -6  -5  -4  -3  -2  -1   0

s = 'abcdef'
# s[start:stop:step]
s[2:6:2]

'ce'

More on slicing

Assigment to mutable object

l = [0, 1, 2, 3, 4, 5]
l1 = l  # assigment to a new name l1 (no copy of the object).
# the names l and l1 points towards the same object.
l1.append('a')
print(l1)
print(l)

[0, 1, 2, 3, 4, 5, 'a']
[0, 1, 2, 3, 4, 5, 'a']

Shallow copy

l = [0, 1, 2, 3, 4, 5]
l1 = l[:] # shallow copy of l
l1.append('a')
print(l1)
print(l)

[0, 1, 2, 3, 4, 5, 'a']
[0, 1, 2, 3, 4, 5]

More on slicing

Other examples of slices for a six-element list. Indexes enumerate the elements, slices enumerate the spaces between the elements.

a = [0, 1, 2, 3, 4, 5]  
all([
    len(a) == 6,
    a[1:] == [1, 2, 3, 4, 5],
    a[:5] == [0, 1, 2, 3, 4],
    a[0] == 0,
    a[:-2] == [0, 1, 2, 3],
    a[5] == 5,
    a[1:2] == [1],
    a[-1] == 5,
    a[1:-1] == [1, 2, 3, 4],
    a[-2] == 4,
])

True

Do it yourself

Suppose we have the string containing header line.

s = 'wind;temperature;pressure'

• Extract the list of items (i.e. "wind", "temperature", "pressure"; see str.split).
• Add "Snow level" to the list of items (e.g. using append)
• Build a new header such that items are capitalized (e.g. using the methods str.join and str.capitalize and iterating on the list)

A possible solution:

s = 'wind;temperature;pressure'
list_items = s.split(";")
print(list_items)
list_items.append("snow level".capitalize())
list_items[0] = list_items[0].capitalize()
list_items[1] = list_items[1].capitalize()
list_items[2] = list_items[2].capitalize()

print(list_items)
";".join(list_items)

['wind', 'temperature', 'pressure']
['Wind', 'Temperature', 'Pressure', 'Snow level']

'Wind;Temperature;Pressure;Snow level'

The function range

The function returns a range object:

# start, stop, step
range(1, 8, 2)

range(1, 8, 2)

We can make a list with the range object:

# start, stop, step
list(range(1, 8, 2))

[1, 3, 5, 7]

# start, stop (step=1)
list(range(2, 8))

[2, 3, 4, 5, 6, 7]

# stop argument (start=0, step=1)
list(range(8))

[0, 1, 2, 3, 4, 5, 6, 7]

Do it yourself

Build a list of odd numbers in decreasing order.

Conditions: if, elif, else

if expression:
   statement(s)
else:
   statement(s)

The statement contains the block of code that executes if the conditional expression in the if statement resolves to 1 or a TRUE value.

a = 0
if a == 0:
    print('a is equal to 0.')

a is equal to 0.

a = 1
if a < 0:
    print('a is negative.')
elif a == 0:
    print('a is equal to 0.')
elif a > 0:
    print('a is positive.')
else:
    print("I don't know.")

a is positive.

Loops

Loops with the keyword while

i = 0
while i < 4:
    i += 1
print('i =', i)

i = 4

i = 0
while i < 4:
    i += 1
    print('i =', i)
    

i = 1
i = 2
i = 3
i = 4

Do it yourself

• Edit a script with the spyder editor that calculates the average of a set of numbers. For example numbers = [67, 12, 2, 9, 23, 5]

  • using the functions sum and len
  • manually, using the keyword while

• Run the script

  • in spyder,
  • in a ipython session opened from another terminal,
  • with the command python.

A possible solution:

numbers = [67, 12, 2, 9, 23, 5]
local_sum = 0
i = 0
while i < len(numbers):
    local_sum = local_sum + numbers[i]
    i = i+1
avg = local_sum / len(numbers)
print(avg)

19.666666666666668

Loops with the keyword for

values = range(5)
for value in values:
    print(value, end=', ')

0, 1, 2, 3, 4, 

# the built-in function enumerate is very useful
for index, value in enumerate(['a', 'b', 'c']): 
    print('({}, {})'.format(index, value))

(0, a)
(1, b)
(2, c)

Loops: keywords continue and break

• continue: passes the block in the loop and continues the loop.

for x in range(1, 8):
    if x == 5:
        continue
    print(x, end=', ')

1, 2, 3, 4, 6, 7, 

• break: stop the loop.

for x in range(1, 11):
    if x == 5:
        break
    print(x, end=', ')

1, 2, 3, 4, 

Do it yourself

• Simplify your script by using a for loops.

• In ipython, try to understand how the function enumerate works. Use it in your script.

A possible solution:

l = [67, 12, 2, 9, 23, 5]
local_sum = 0
for e in l:
    local_sum += e
avg = local_sum / len(l)
print(avg)

19.666666666666668

Do it yourself

We build a list:

from random import shuffle, randint

n = 20
i = randint(0, n-1)
print('integer remove from the list:', i)
l = list(range(n))
l.remove(i)
shuffle(l)
print('shuffled list: ', l)

integer remove from the list: 3
shuffled list:  [13, 16, 6, 10, 17, 19, 4, 2, 5, 9, 0, 7, 11, 12, 14, 8, 1, 15, 18]

One element has been removed:

• Find this element (given that you can change the ordering of l).
• Find this element (given that you cannot change the ordering of l).

A possible solution:

# we can change ordering, let's sort
print(l)
l_sorted = sorted(l)
print(l_sorted)
found = None
for idx, elem in enumerate(l_sorted):
    if elem != idx:
        found = idx
        break
if found is None:
    found = len(l)
print("missing ", idx)
    

[13, 16, 6, 10, 17, 19, 4, 2, 5, 9, 0, 7, 11, 12, 14, 8, 1, 15, 18]
[0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
missing  3

# we cannot sort -> higher complexity
for elem in range(len(l)+1):
    if elem not in l:
        break
print("missing ", elem)

missing  3

# another solution
actual_sum = sum(l)
len_l = len(l)
original_sum = (len_l + 1) * (len_l) // 2
print("missing ", original_sum - actual_sum)

missing  3

Exceptions and try, except syntax

Exceptions and errors are common in all codes. There is a good system to handle them in Python. Let's first see what gives such buggy code

l = ['a']
i = 1
print(l[i])

When these lines are executed, Python stops its execution and print a traceback:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-30-8df9cec1a0ec> in <module>()
      1 l = ['a']
      2 i = 1
----> 3 print(l[i])

IndexError: list index out of range

Exceptions and try, except syntax

Handling exception:

l = ['a']
i = 1
try:
    print(l[i])
except IndexError as e:
    print(e)

list index out of range

Remark: never use

except:

It means "except all errors and exceptions". A user Control-C is an exception (KeyboardInterrupt) so it would be caught and have no effect.

Full syntax

try:
    ...
except <exception1> as e1:
    ...
except <exception2> as e2:
    ...
else:
    ...
finally:
    ...

Non exhaustive error list:

• ArithmeticError
• ZeroDivisionError
• IndexError
• KeyError
• AttributeError
• IOError
• ImportError
• NameError
• SyntaxError
• TypeError

Do it yourself:

For each line of this string, append a variable of correct type in a list (i.e. "hello" should stay hello, 2 should become an int and 1.5 a float). Do it by catching errors.

Hints:

• int("a") and float("a") raise ValueError
• the above str_variable can be split using "\n"

str_variables = """hello
1.5
2"""

the_list_you_should_get = ["hello", 1.5, 2]

A possible solution:

split_list = []
for value in str_variables.split("\n"):
    try:
        value = int(value)
    except ValueError: 
        print(value, "is not a int")
        try:
            value = float(value)
        except ValueError:
            print(value, 'is not a float')
    split_list.append(value)

            
print(split_list)

hello is not a int
hello is not a float
1.5 is not a int
['hello', 1.5, 2]