---
title: Linux & scripting for data wrangling
author: Jimmy Christensen
theme: Linux, Bash, Python
date: October 31, 2022
duration: 120
---

# Overview

- Linux
- Bash
- Python

Slides available at https://dusted.dk/pages/scripting/slides.html

# Linux ?
Linux is an operating system, like Windows or OSX, it allows you to run programs.

# Bash ?
Popular "command line" for Linux & OSX

- Launch programs
- Interact with files and folders
- Composition via pipes
- Scripting language
- Easy things, very easy
- Medium things, hard
- Hard things, very hard

# Python ?
Popular scripting language

- Easy to learn
- General purpose
- Easy things are easy enough
- Medium things are not so bad
- Hard things are less hard

# All of them ?

Bash, grep, awk, sed, tr, cut:

- Very fast
- Easy things are easy
- Everything else is really hard
- Use for rough extraction

Bash scripts using above programs to automate extraction.

Python scripts to remodel and consolidate extracted data, calculate and prepare results.


# Overview: Linux
- Editing with nano
- Files / Directories
- Pipes & redirects


# Editing with nano
Nano is a (bit too) simple text editor

- nano myfile.sh 
- ctrl+o  to op... save!
- ctrl+w  to wri.. search!
- ctrl+k  to k.... cut!
- ctrl+u  to yes.. insert! ;-)
- ctrl+x  to eXit

# Files / Directories

/home/chrisji/theWork/important.txt

- pwd
- cd
- ls -lah
- hd, head, tail, more, du, wc

Peek at unknown file: hd strangeFile | head

Check size: ls -lh strangeFile 

Only beginning: head strangeFile

Page through it: more strangeFile

# Pipes & redirects

- Programs will read data from standard input (stdin)
- Programs will write data to standard output (stdout)
- Pipe directs stdout into the next programs stdin
- programA | programB | programC
- \> and >> directs stdout to a file

# Pipes & redirects
Composition, also useful in bash scripting
```bash
# A pipe directs data from stdout of one program to stdin of the next
ls | wc -l 

# A redirect directs data from stdout of a program to a file.
ls > myFiles.txt

# An appending redirect will not truncate the file first.
echo "Line 1" > file.txt
echo "Line 2" >> file.txt
``` 

# Script vs program
Same difference, almost

- Scripts typically easier to write for the human
- Scripts typically slower to execute for the computer
- Scripts are "clear text"


# Overview: Bash

- Making & running a bash script
- Variables & parameters
- Conditionals & loops
- Reading data
- Functions
- Debugging
- Tips

# Two quick notes

```bash
# This line is a comment, because it starts with the hash mark
# comments are to help the humans (us)

both bash and python # ignores a hash mark and any following text
but only for rest of that line
```
- Scripts are executed by the computer one line at a time, not "all at once".

# Making & running a bash script

Creating the script with the nano editor
```bash
nano shee.sh
```
![](./bash1.png)

- ctrl +o to save the file
- ctrl +x to exit nano

# Making & running a bash script


```bash
# Look at our new script file, it is not marked as being "executable"
ls -l shee.sh

# We make it executable!
chmod +x shee.sh

# Check that it has the x flag (and pretty green color)
ls -l shee.sh

# Now we can run it
./shee.sh
Hello World!
```
![](./bash2.png)

# Bash: Variables

![](var.png)

Holds something, has a name, like box with a label on it

```bash
#!/bin/bash

placeToGreet="World"

echo "Hello $placeToGreet!"
```

```
Hello World!
```

# Bash: Variables

![](var.png)

They can be re-assigned!

```bash
#!/bin/bash

placeToGreet="World"

placeToGreet="Earth"

echo "Hello $placeToGreet!"
```
What's the result ? (remember, line-by-line)

# Bash: Variables

A bash variable can store about 2 megabytes of data, python about 64 gigabytes

```bash
#!/bin/bash

# Create some variables and assign them values
timesToCall=3
animal="dog"
name="viggo"

# Output to the screen
echo "Calling for $name the $animal $timesToCall times!"
```
```
Calling for viggo the dog 3 times!
``` 

# Bash: Parameters
Variables assigned from command-line

```bash
# The shee.sh script is executed with four parameters:
# First parameter: 3
# Second parameter: dog
# Third parameter: viggo
# Fourth parameter: heere doggie doggie! 
./shee.sh 3 dog viggo "heere doggie doggie!"
Calling for viggo the dog 3 times!
```

# Bash: Parameters
```bash
#!/bin/bash

# $1 = first parameter
# $2 = second parameter

timesToCall=$1
animal="$2"
name="$3"

echo "Calling for $name the $animal $timesToCall times!"
```

# Variables: There's more

Bash has strong string-manipulation capabilities, quickly degenerates into alphabet soup

Don't go there, use python

For the curious: https://tldp.org/LDP/abs/html/string-manipulation.html

It also has arrays and sets, bash syntax makes it painful

# Conditionals

Conditionals let's us decide if we want to do something

```bash
if CONDITION
then
    # do
    # some
    # things
fi
```

# Conditionals

The "else" word lets us do something if the condition is not met

```bash
if CONDITION
then
    # do
    # some
    # things
else
    # do some
    # other
    # things
fi
```

# Conditionals

The elif is optional, allows chains

```bash
if CONDITION_A
then
    # something to do if A..
elif CONDITION_B
then
    # This must not happen if A, but if B.
elif CONDITION_C
then
    # If neither A nor B, but C, then do this
else
    # If not A, B nor C, then do this..
fi
```
else is also optional
```bash
if CONDITION_A
then
    # Do this if A
elif CONDITION_B
then
    # Do this if B
fi

```

# Conditionals

In bash, a condition is met if the [ ] builtin or a program exit with the "success" status code.

```bash
true # A program that returns success
false # A program that returns.. not success
grep # Will return success if it finds at least one match
[ "$name" == "viggo" ] # Will success if the variable contains the text viggo
[ $a -gt $b ] # success if a is Greather Than b
[ $a -lt $b ] # success if a is Less Then b
[ $a -eq $b ] # success if a is EQual to b
[ -z ${a+x} ] # success if a is empty (weird syntax, I know)
[ "$name" == "viggo" ] && [ "$animal" == "dog" ] # success only when name is viggo AND animal is dog (both must be true)
[ "$name" == "viggo" ] || [ "$animal" == "dog" ] # success if name is viggo or animal is dog (one or both must be true)
```

# Conditionals

```bash
if echo "secret string" | grep "secret" &> /dev/null
then
    echo "The secret string was found!"
fi
```

# Conditionals

```bash
if [ "$name" == "viggo" ] && [ "$animal" == "dog" ]
then
    echo "Viggo is a very special dog!"
elif [ "$animal" == "dog" ]
then
    echo "Dogs are just great, even if they're $name instead of viggo!"
else
    echo "I'm sure $name is a great $animal!"
fi
```

# Loops

- for loop
- while loop 
- do while loop (rarely used)

# Loop: for

Repeats for each item in sequence

```bash 
for VARIABLE in SEQUENCE
do
    # things to do
done
```

# Loop: for

Example

```bash
for number in 1 2 3 4
do
    echo "Number is $number"
done
```

# Loop: for

```bash
animals="dog fish cat bird"

for animal in $animals
do
    echo "$animal is an animal"
done
```
```
dog is an animal
fish is an animal
cat is an animal
bird is an animal
```

# Loop: for

```bash
animals="dog fish cat bird"

for animal in "$animals"
do
    echo "$animal is an animal"
done
```
```
dog fish cat bird is an animal
``` 

# Loop: for

Use expansion for files, this produces a sequence of .txt files in current directory

```bash
for file in *.txt
do
    echo "Text file: $file"
done
```

# Loop: while

Repeats as long as CONDITION is met (potentially forever)

```bash
while CONDITION
do
    # thing to do
done
```

# Interjection

This is a good time to remind you that pressing ctrl+c (eventually) terminates a running script.

# Loop: while

When you need to repeat something until "things are done"

```bash
while CONDITION
do
    # Thing to do until CONDITION is no longer met
done
```

# Loop: while

```bash
while read line
do
    echo "This just in: $line"
done
```

# Reading data

- From human
- From file
- From program
- From stream

# Reading data: The read builtin

```bash
read VARIABLE # Creates a variable and reads data from standard input into it
```

# Read data from human

Useful for interactive scripts

```bash
echo "Welcome to super cool script v 24!"

# The -p parameter is a prompt to show to the human
read -p "File to corrupt: " fileName

echo "Don't worry, I was just kidding, nothing happened to $fileName"
```
```
Welcome to super cool script v 24!
File to corrupt: homework.md
Don't worry, I was just kidding, nothing happened to homework.md
```

# Read data from human (in a loop)

```bash

answer=""
while [ "$answer" != "y" ] && [ "$answer" != "n" ]
do
    read -p "Continue? [y/n] ?" answer
done

if [ "$answer" == "n" ]
then
    echo "Stopping here."
    exit
fi

echo "Continuing"
```
```
Continue? [y/n] ?yes
Continue? [y/n] ?...
Continue? [y/n] ?yeees
Continue? [y/n] ?noooo ?
Continue? [y/n] ?n
Stopping here.
```
# Read data from file

For small files, we can read the entire file into a variable by running the cat program and capturing its output in a variable

```bash
someFileContents=$(cat secrets.txt)

echo "File content: $someFileContents"
```

# Read data from program

For small outputs, we can capture the entire standard (non-error) output into a variable

```bash
gloriousResult=$(grep 'someRegex' secrets.txt)
```

# Read data from stream

Read exits with success unless standard input is closed

```bash
grep 'someRegex' secrets.txt | while read line
do
    echo "A line of the result: $line"
done
```

# Read data from stream

We can compose complex pipelines now

```bash
grep 'someRegex' secrets.txt | while read line
do
    # Extract some field from the line
    importantThing=$(echo $line | cut -f 2 -d ',' | awk magic things)
    # Look for that field in some other file, put the result in a variable
    if resultFromOtherFile=$(grep $importantThing otherSecrets.txt)
    then
        # Eureka! some correlation is interesting when the field from subset of secrets.txt is in otherSecrets.txt!
        # Maybe combine on one line and print it out?
        echo "$line,$resultFromOtherFile"
    fi
done > results.txt
```

# Read data from stream

We can also write a script that takes a stream from a pipe, for use with other scripts

ourCoolScript.sh

```bash
#!/bin/bash

while read ourLine
do
    # Do intersting things with ourLine
done
```

We'd use it like this
```bash
grep 'someRegex' someFile.txt | ./ourCoolScript.sh > results.txt
```

# Write data

We just use the > and >> after any command, or after the script itself.

```bash
> # Will create an empty file (or truncate an existing one)
>> # Will create an empty file (or append to an existing one)
```
# Functions

- A small "script inside your script"
- Can take parameters
- Useful for repetitive tasks (don't repeat yourself)

# Functions

```bash
function removeRepeatSpaces {
    tr -s " "
}

echo "Badly formatted    data right  here." | removeRepeatSpaces
```

```bash
function largestOne {
    if [ $1 -gt $2 ]
    then
        echo $1
    elif $1 -lt $2 ]
    then
        echo $2
    fi
}

largest=$(largestOne $A $B)
```

# Debugging
See _EVERYTHING_ that happens during script execution by adding set -x

```bash
#!/bin/bash
set -x
echo "My script"
read -p "What is your name ?" name

if [ "$name" == "viggo" ]
then
    echo "Good dog!"
fi
```
![](./bash3.png) 

# Tips

- Comment
- Indent
- Use good variable names
- Filter before iterating
- Do as much filtering as is convenient, but not more
- Google

# Tips

- If script takes parameters, have it write how to use it
```bash
#!/bin/bash

# This script needs 3 paramters, name, animal and number of legs

name=$1
animal=$2
nlegs=$3

if [ -z ${nlegs+x} ]
then
    echo "Usage $0 NAME ANIMAL NUM_LEGS"
    echo "    NAME - The name of the pet (example: viggo)"
    echo "    ANIMAL - What kind of animal is it? (example: dog)"
    echo "    NUM_LEGS - How many legs does it have? (example: 4)"
    exit 1
fi
```
Even if you're the only user, you might forgot how to use it later

$0 is special, it is always the name of the script in question.

# Tips

Suggestion for treating multiple files while keeping track of the source of each result

Let there be binary files data1.bin data2.bin ... dataN.bin that can be translated
by a "translatebinarytotext" program and filtered by some regular expression to find
relevant lines:

```bash
#!/bin/bash

for fileName in data*.bin
do
    translatebinarytotext "$fileName" \
        | grep 'someRegex' \
        | ./ourAwesomeScript.sh "$fileName" \
        >> result.txt
done
```
```bash
# Alternative if the files are unorderly
for fileName in data1.bin data_2.bin thirdPartOfData.bin
```
The polished version of above alternative could be used like a real program.

```bash
./processThePeskyFiles.sh data1.bin data_2.bin thirdPartOfData.bin > result.txt
```
Then the script would look like:
```bash
#!/bin/bash

for fileName in $@
do
    translatebinarytotext "$fileName" \
        | grep 'someRegex' \
        | ./ourAwesomeScript.sh "$fileName"
done
```

# Tips

It can be useful to write out such "debug" information to the standard error channel, this way it goes "around" pipes and are shown on screen.

```bash

#!/bin/bash

# This function writes a message to stderr, so our script output can be piped while we still see information/errors.
function msg {
    echo $@ >&2
}

msg "Script for doing cool stuff running in $(pwd)"
echo "Important data output"
```
```
bash coolStuffDoer.sh > test.txt
Script for doing cool stuff running in /home/chrisji/theWork/
cat test.txt 
Important data output
```

# Tips

Script and data does not need to be the same place, you can write the script as if it is next to the files.

```text
.
├── data
│   └── seta
│       └── wool.txt
└── scripts
    └── sheep.sh

```

scripts/sheep.sh:
```bash
#!/bin/bash

sheep=$(grep sheep wool.txt | wc -l)
echo "There are $sheep sheep."
```
```bash
cd data/seta
../../scripts/sheep.sh wool.txt
```

# Tips

Alternative, write script that uses parameters:

```text
.
├── data
│   └── seta
│       └── wool.txt
└── scripts
    └── sheep.sh

```

scripts/sheep.sh:
```bash
#!/bin/bash

file="$1"
sheep=$(grep sheep "$file" | wc -l)
echo "There are $sheep sheep in $file"
```
```bash
./scripts/sheep.sh data/seta/wool.txt
```

# Tips

Alternative, write script that uses pipes:

```text
.
├── data
│   └── seta
│       └── wool.txt
└── scripts
    └── sheep.sh

```

scripts/sheep.sh:
```bash
#!/bin/bash

sheep=$(grep sheep | wc -l)
echo "There are $sheep sheep."
```
```bash
cat data/seta/wool.txt | ./scripts/sheep.sh
```

# Tips
A script that reads which files to work on from a file

filesToWorkOn.txt
```
file1.txt
file2.txt
file3.txt
```

process.sh
```bash
#!/bin/bash

listFileName="$1"

filesToProcess=$(cat "$listFileName")

for fileToProcess in $filesToProcess
do
    echo "Processing file: $fileToProcess"
done
```
```
./process.sh filesToWorkOn.txt 
Processing file: file1.txt
Processing file: file2.txt
Processing file: file3.txt
```


# Overview: Python

- Make a script & run it
- Variables & parameters
- Conditionals & loops
- Functions
- Reading & writing, files & pipes

# Why?!

- Easy and powerful math
- Easy and powerful text string manipulation
- Easier syntax
- Easier to work with sets and lists
- Nice modules for science stuff

# Make a script & run it

```bash
nano hello.py
```

```python
#!/usr/bin/python3
print("Hello World!")
```

- ctrl + o 
- ctrl + x

```bash
./hello.py
Hello World!
```


# Variables

```python
#!/usr/bin/python3

placeToGreet = "Earth"

# Apologies, the f is not a typo, it tells python we want variables substituted in the text string
print(f"Hello World {placeToGreet}!")
```

# Variable

Types & objects

# Variables

- Text strings
- Numbers
- Lists
- Sets
- Dictionaries
- Powerful functions to easily work with them
- Convenient regex and sub-strings


# Variables 

```python
# integer
length=23

# floating point
pi=3.1415
tau=pi*2

# set
instances={ pi, tau, tau, 'dog', 'viggo' }
# How many tau is in the set? 1. It's a set.
# What order are the items in? None, sets are unordered.

# Convenient for acumulating unique occurences of stuff
instances.add( 42 )

if tau in instances:
    print("Tau is in the set!")

# list
aList = [ 5,3,12,5 ]
# Lists are ordered, duplicates are allowed. (5 occurs twice, at the start and end)
# In python and most other languages, the first element of a list is 0
print(f"The second element in the list is {aList[1]})
```

# Variables

Dictionaries makes it easy to structure data

```python
dogs = dict(
    viggo = dict(
        age = 4,
        legs= 4,
        tail=true
    ),
    henry = dict(
        age = 9,
        legs= 3.9,
        tail=true
    )
)

dogs.update(
    bob = dict(
        age = 2,
        legs= 4,
        tail=false # poor guy!
    )
)

print( f"Viggo is {dogs['viggo']['age']} years old." )

```
# Variables

```python

# Sets can accumulate unique things, dicts can be count them
numAnimals = dict()

# -- snip --

if 'tau' not in numAnimals:
    numAnimals['tau'] = 0

numAnimals['tau'] += 1
```

# Variables

More than everything you need to know about Python variables at 

[https://docs.python.org/3/library/stdtypes.html](https://docs.python.org/3/library/stdtypes.html)


# Conditionals

Conditionals easier to read in python, generally we don't run external programs, so the boolean result of an expression is all we worry about.

A condition is met when the result is True or any non-zero value.

A condition is not met when the result is False or 0, or empty (such as the empty string "" or list [])

# Conditionals

```python
a > b # Is True if a is greater than b
a < b # Is True if a is less than b
a == b # Is True if a is equal to b
a != b # Is true if a is not euqal t b
a < b and b > c # IS true if a is less than b AND b is greater than c
a > b or b < c # Is true if a is greater than b or if b is less than c
a in c # Is true if a is an element in c
a not in c # Is true if a is not an element in c
```

# Conditionals

```python
if CONDITION:
    # Do something that should be done
    # if CONDITION is met
```
```python
if CONDITION
    # Do something
else
    # Do something else
``` 

# Attention!
Python is indentation sensitive, that is, the number of "tabs" indicate to which code-block a line belong.

```python

if False:
    print("This is never shown")
    print("And this is also never shown")
print("This is shown")
``` 

# Conditionals

```python
name = 'viggo'

if name == 'viggo':
    print(f"The dogs name is Viggo!")
```
# Conditionals

```python
name = 'Viggo'

# 'Viggo' is not the same as 'viggo', but python strings provide a method for lowercasing for comparisons
if name.casefold() == 'viggo':
    print(f"The dogs name is Viggo!")
```

# Loops

```python
# For is great for iteration over "iterables"
for VARIABLE in ITERABLE:
    # do thing for each item
```

```python
for number in range( 10, 20 ):
    print(f"Number: {number}")
```
```
Number: 10
Number: 11
Number: 12
Number: 13
Number: 14
Number: 15
Number: 16
Number: 17
Number: 18
Number: 19
```
More about the CSV module at [https://docs.python.org/3/library/csv.html](https://docs.python.org/3/library/csv.html)

# Loops


```python
#!/usr/bin/python3

import sys
import csv

# This csv file used ; instead of ,
reader = csv.reader(sys.stdin, delimiter=';')

rowNumber=0

for row in reader:
    name=row[0]
    age=int(row[1]) # Convert from text to number 
    print(f"Row {rowNumber}, name: {name} age: {age}")
    if age > 5:
        print(f"{name} is an old doggie!")
    rowNumber += 1
```
```bash
cat csvfi | ./hello.py 
Row 1, name: viggo age: 6
viggo is an old doggie!
Row 2, name: henry age: 4
```

# Loops

Much more about loops here: [https://wiki.python.org/moin/ForLoop](https://wiki.python.org/moin/ForLoop)

# Reading from a file

With an open file, readline will read a single line from the file, useful for large files

```python
#!/usr/bin/python3

fileName="poe.txt"

with open(fileName) as textFile:
    while True:
        line = textFile.readline()
        if not line: # If the line was empty, file end reached
            break;   # so break the lop
        line = line.strip();
        print(f"{line}")
```
```
Once upon a midnight dreary,
while I pondered, weak and weary,
Over many a quaint and curious
volume of forgotten lore—
```

# Reading from a file

With an open file, readlines will read from the file into a list

```python
#!/usr/bin/python3

lineNumber=0
fileName="poe.txt"

with open(fileName) as textFile:
        lines = textFile.readlines()
        for line in lines:
            lineNumber += 1
            line = line.strip(); # remove newline from the line
            print(f"{fileName} {lineNumber}: {line}")
```

```
poe.txt 1: Once upon a midnight dreary,
poe.txt 2: while I pondered, weak and weary,
poe.txt 3: Over many a quaint and curious
poe.txt 4: volume of forgotten lore—
```


# Writing files

reverser.py
```python
#!/usr/bin/python3

inFileName="poe.txt"
outFileName="eop.txt"

with open(inFileName) as inFile:
    with open(outFileName, 'w') as outFile:
        while True:
            line = inFile.readline()
            if not line: # If the line was empty, file end reached
                break;   # so break the lop
            line = line.strip();
            line = line[::-1] # Reverse the string
            line = line + "\n"
            outFile.write(line)
```

```bash
./reverser.py
cat eop.txt 
,yraerd thgindim a nopu ecnO
,yraew dna kaew ,derednop I elihw
suoiruc dna tniauq a ynam revO
```

# Writing CSV files

```python
#!/usr/bin/python3
import csv

ourResults = []                                                                 
ourResults.append( ['Johnny', 'Cat', 4 ] )                                      
ourResults.append( ['Viggo', 'Dog', 5 ] )                                       
ourResults.append( ['Mat', 'Dog', 7 ] )                                         
ourResults.append( ['Markus', 'Cat', 7 ] )                                      

with open("out.csv", "w") as csvOutFile:
    writer = csv.writer(csvOutFile, dialect='excel')
    writer.writerow(['Name', 'Species', 'Age'])
    for row in ourResults:
        writer.writerow(row)
``` 
```bash
cat out.csv 
Name,Species,Age
Johnny,Cat,4
Viggo,Dog,5
Mat,Dog,7
Markus,Cat,7
```

# Writing CSV files

```python
#!/usr/bin/python3
import csv

csvHeader = [ 'Name', 'Species', 'Age' ]

ourResults = []
ourResults.append( { 'Name': 'Johnny', 'Species': 'Cat', 'Age': 4 } )
ourResults.append( { 'Name': 'Viggo',  'Species': 'Dog', 'Age': 5 } )
ourResults.append( { 'Name': 'Mat',    'Species': 'Dog', 'Age': 7 } )
ourResults.append( { 'Name': 'Markus', 'Species': 'Cat', 'Age': 7 } )


with open("out.csv", "w") as csvOutFile:
    writer = csv.DictWriter(csvOutFile, fieldnames=csvHeader, dialect='excel')
    writer.writeheader()
    for row in ourResults:
        writer.writerow(row)

```
# String search

```python
string.find(KEYWORD)

#rfind is the same, but goes right to left

# They return -1 if KEYWORD not found
```

```python
needle = "is"
haystack = "This is a test"

position = haystack.find(needle)  # 2
position = haystack.rfind(needle) # 5
```

# String slicing

```python
string[ BEGIN : STEP : END ]
``` 

```python
# Slicing (extract substring)

myString = "abcdefg"

subString = myString[0:3:1] # First 3 characters (character 0,1,2)
subString = myString[3:6:1] # From third character to sixth (def)
substring = myString[-2::]  # Last 2 characters
subString = myString[::-1]  # Step through string in reverse   
```

# String splitting
```python
string.split( DELIMITER )
string.split( DELIMITER, NUMBER_OF_SPLITS)

# rsplit is the same, but goes right to left
```
```python
myString = "a,b,c,d,e,f,g"

subString = myString.split(',') # ['a', 'b', 'c', 'd', 'e', 'f', 'g']
subString = myString.split(',', 1) # ['a', 'b,c,d,e,f,g']
subString = myString.rsplit(',', 1) # ['a,b,c,d,e,f', 'g']
```

# Regular expressions

```python
import re # regex module

text = "He was carefully disguised but captured quickly by police."

subString = re.split(r"but", text) # ['He was carefully disguised ', ' captured quickly by police.']

subString = re.findall(r"\w+ly\b", text) # ['carefully', 'quickly']
```

More about regex at [https://docs.python.org/3/library/re.html](https://docs.python.org/3/library/re.html)

# Searching in list

```python
#!/usr/bin/python3

listOfAnimals = [ 
        { "Name": "Viggo", "Species": "Dog", "Age": 5 },
        { "Name": "Mat", "Species": "Dog", "Age": 6 },
        { "Name": "Oliver", "Species": "Cat", "Age": 5 },
        { "Name": "Luggie", "Species": "Dog", "Age": 5 }
]

fiveYearOldDogs = []

for animal in listOfAnimals:
    if animal["Species"] == "Dog" and animal["Age"] == 5:
        fiveYearOldDogs.append(animal["Name"])

print(fiveYearOldDogs)
```
```
['Viggo', 'Luggie']
```

# Functions

```python
def FUN_NAME( ARGUMENTS ):
    BODY
```

# Functions

```python
#!/usr/bin/python3

# repeater will return a textToRepeat repeated timesToRepeat times.
# If timesToRepeat is not provided, a default value of 5 is used
def repeater( textToRepeat, timesToRepeat=5 ):
    return textToRepeat * timesToRepeat

repeatedString = repeater("test", 3)

print(repeatedString)

```
```
testtesttest

```

# Functions

```python
#!/usr/bin/python3

listOfAnimals = [ 
        { "Name": "Viggo", "Species": "Dog", "Age": 5 },
        { "Name": "Mat", "Species": "Dog", "Age": 6 },
        { "Name": "Oliver", "Species": "Cat", "Age": 5 },
        { "Name": "Luggie", "Species": "Dog", "Age": 5 }
]

def isLuggie( animal ):
    return animal["Species"] == "Dog" and animal["Age"] == 5 and animal["Name"] == "Luggie"

num = 0

for animal in listOfAnimals:
    num += 1
    if isLuggie(animal):
        print(f"Found {animal['Name']} he is number {num} in the list")

```
```
Found Luggie he is number 4 in the list
```

# Sorting lists

```python
list.sort()
list.sort( reverse=True )
list.sort( key=FUNCTION )
```

```python
#!/usr/bin/python3

myList = [ 2, 6, 1 ,4 ]
myList.sort() # Sort will sort the list "in place"
print(myList)

myList = [ 
    { 'Age': 5, 'Name': 'Viggo' },
    { 'Age' : 2, 'Name': 'Mat' }
]

def getAge(animal):
    return animal['Age']

myList.sort( key=getAge)

print(myList)
```
```
[1, 2, 4, 6]
[{'Age': 2, 'Name': 'Mat'}, {'Age': 5, 'Name': 'Viggo'}]
```

# The end

![](./end.png)