Python Starter – Exercises

Yoan Mollard

List of mini-projects

1. The basics in JupyterLab: practice lists and dictionaries
2. Hanged man: practice Python scripting, in PyCharm
3. Money transfer simulator: develop, test and distribute a complete package
4. Address book: manipulate nested data collections, and argparse, re, json
5. Choose a mini-project: (ascending difficulty)
   5A. Guess my number: practice basic Python syntax
   5B. Estimate π: practice more complex computations, and random
   5C. Create a micro web app: practice flask
   5D. Communicate with a REST API: practice requests and multiprocessing
   5E. Webscraping: practice beautifulsoup4
   5F. Plot ping durations: practice subprocess and matplotlib

Exercise 1. Practice lists and dictionaries

1. Open a System terminal in PyCharm.
   Type activate if you see no (venv) prefix.

2. Install Jupyter Lab with pip install jupyterlab in the virtual environment

3. Launch Jupyter Lab by typing jupyter lab in the same terminal

4. Right click these notebooks and Save As in your PyCharm project's folder:

1. Types.ipynb – 2. Lists.ipynb – 3. Dicts.ipynb – 4. Dataset.ipynb

Make sure that Windows does not add extension .txt on its own

5. Go back to JupyterLab, open and follow the downloaded notebook.

Tip: Shift + Enter runs a cell. Ctrl + M and then B inserts a new cell Below

Enable zoom in Pycharm

From now, we will only use Pycharm.

Enable zooming with
Ctrl+wheel in File > Settings:

Mini-project 2: Hanged man

You probably know the hanged man game:

1. The player is shown a secret word in which letters are hidden by underscores
2. Each turn, the player proposes a letter to unveil
3. If the chosen letter is part of the word, all their occurences are revealed

The goal is to reveal the secret word in less turns that there are letters in it.

___IC__S_I_U_IO___LL_M___
   ________
   |       |
  \o/      |
   |       |
  / \      |
_______________________

1. Write and test a function input_letter() that asks the user to type a letter and returns it. This functions retries in case the user types anything that is not valid (a number, punctuation, several letters, ...).

The function input("Prompt:") returns a string read from the console

2. Write a function unveil(letter, original_word, hidden_word) that browses all characters of a hidden word and reveals the requested letter at the right position if it is in the original word.

The str type is immutable

hidden word can be either fully hidden by underscores, or only partially hidden

3. Give 4 examples representing the 4 possible cases for inputs and outputs of the unveil(...) function. Use assert to make sure they pass all four.

4. Define and initialize the following variables to coherent initial values:

• words: a list of possible words to be guessed
• secret_word: a secret word randomly picked among the previous list (use for instance random.choice)
• displayed_word: the partially hidden word, i.e. the word of same length as the secret word in which every letter is replaced by an underscore
• remaining_attempts: the number of remaining attempts. For simplicity, initialize it to the number of letters in secret_word. This counter must be decremented every turn.

5. Add a game loop that:

• Displays the partially hidden word and the number of remaining attempts
• Prompts the player to enter a valid letter with input_letter()
• Replaces matches of this letter from secret_word in displayed_word, if any
• Checks the game state: exit the program with an appropriate message if the player wins or looses

You game must now be playable!

6. Optional question: Remember high scores in a JSON file:

Use the built-in json module to dump the 3 best scores into a .json file. Load that file at each startup to show high scores.

Mini-project 3. Money transfer simulator

In this exercise we are going to create a simplified Information System that is able to handle and simulate bank transactions.

In our scenario there are 4 actors: a bank (HSBC), a supermarket (Walmart), and 2 individuals Alice and Bob.

Each actor has his/her own bank account.

Part 1: The basic scenario

• 1.1. Create a class BankAccount that owns 2 attributes:
  • owner (of type str): the owner's name
  • balance (of type int): the balance (do not take care of decimals)
  • the class constructor takes in parameter, in this order, owner and initial_balance

With your class it must be possible to execute the following scenario (that has no effect so far, but it must not raise any error):

bank = BankAccount("HSBC", 10000)
walmart = BankAccount("Walmart", 5000)
alice = BankAccount("Alice Worz", 500)
bob = BankAccount("Bob Müller", 100)

• 1.2. Implement a print() method in class BankAccount that displays the name of the owner and current balance. Iterate on all accounts to print them.

• 1.3. Implement these methods :

  • _credit(value) that credits the current account with the value passed in parameter. We will explain the goal of the initial underscore later.
  • transfer_to(recipient, value) that transfers the value passed in parameter to the recipient passed in parameter

• 1.4. Run the following scenario and check that end balances are right:

  • 1.4.1. Alice buys $100 of goods at Walmart
  • 1.4.2. Bob buys $100 of goods at Walmart
  • 1.4.3. Alice makes a donation of $100 to Bob
  • 1.4.4. Bob buys $200 at Walmart

Part 2: The blocked account

Bob is currently overdrawn. To prevent this, its adviser converts his account into a blocked account: any purchase would be refused if Bob had not enough money.

• 2.1. Create the new InsufficientBalance exception type inheriting from ValueError. No code is needed into that new class: use pass to skip code.

• 2.2. Implement a class BlockedBankAccount so that:

  • the BlockedBankAccount inherits from BankAccount. Make sure you do not forget to call parent methods with super() if necessary
  • the transfer_to methods overrides the parent method, with the only difference that it raises InsufficientBalance if the balance is not sufficiently provided to execute the transfer

• 2.3. Replace Bob's account by a blocked account and check that the previous scenario actually raises an exception

• 2.4. Protect the portion of code that looks coherent with try..except in order to catch the exception without interrupting the script

• 2.5. Explain the concept of protected method and the role of the underscore in front of the method name ; and why it is preferable that _credit is protected

Part 3: The account with agios

In real life another kind of account exists: the account whose balance can actually be negative, but it that case the owner must pay agios to his(her) bank.

The proposed rule here is that, when an account is negative after an outgoing money transfer, each day will cost $1 to the owner until the next money credit.

To do so, we need to introduce transaction dates in our simulation.

3.1. Implement a class AgiosBankAccount so that:

• the AgiosBankAccount inherits from BankAccount. Make sure you do not forget to call parent method with the super() keyword if necessary
• the constructor of this account takes in parameter the account of the bank so that agios can be credited on their account.

3.2. Implements the transfer_to method overrides the parent method:

• it takes the transaction_date in parameter, of type datetime
  (also change the parent class and propagate the date paramter to the base classes and the other child class when necessary)
• it records the time from which the balance becomes negative. You need an additional attribute for this.

3.3. Implement the _credit method that overrides the method from the parent class, with the only difference that it computes the agios to be payed to the bank and transfer the money to the bank. Round agios to integer values.

3.4. Check your implementation with the previous scenario: After Bob has a negative balance, Alice makes him a transfer 5 days later: make sure that $5 of agios are payed by Bob to his bank.

Part 4: The account package

We have just coded a very simple tool simulating transactions between bank accounts in Object-Oriented Programming.

In order to use it with a lot of other scenarii and actors, we are going to structure our code within a Python package.

We will organise our accounts with the following terminology:

• bank-internal accounts do not create agios and are not blocked, there are BankAccount and only banks can own such account
• bank-external accounts are for individuals or companies, they can be either blocked or agios accounts.

We would like to be able to import the classes from than manner:

from account.external.agios import AgiosBankAccount
from account.external.blocked import BlockedBankAccount, InsufficientBalance
from account.internal import BankAccount

• 4.1. Re-organize your code in order to create this hierarchy of empty .py files first as on the figure.
  Create an empty script scenario1.pyfor the scenario.

• 4.2. Move the class declaration of AgiosBankAccount in agios.py

• 4.3. Move the class declarations of BlockedBankAccount and InsufficientBalance in blocked.py

• 4.4. Move the class declaration of BankAccount in internal.py

• 4.5. Move the scenario (i.e. the successive instanciation of all accounts of companies and individuals) in scenario1.py

• 4.6. Check each module and add missing relative import statements
  Relative imports start with . or ..

• 4.7. Check each module and add missing absolute import statements such as datetime

Import statements in the scenario must not be relative because scenario1.py will be located outside package account.

• 4.8. Add empty __init__.py files to all directories of the package.

• 4.9. Execute the scenario and check that it leads to the same result as before this refactoring

Part 5: Test your package with pytest

• 5.1. Install pytest with pip
• 5.2. Create independant test files tests/<module>.py for each module of your package
• 5.3. Add an entry in sys.path pointing to the parent folder of your package so that pytest is able to locate and import your account package (*)
• 5.4. With the documentation of pytest, implement unit tests for your classes and run the tests with pytest

*This workaround is not ideal since this path is different on each system, and the situation will be fixed once the package will be made installable in Part 6.

Part 6: Automate package building and testing with tox (Optional)

6.1. Make your package installable

Refer to the doc about package creation

Create a metadata file pyproject.toml and update its metadata (package name, author, license, description...)

Delete the sys.path workaround in test files since the package is now installable

6.2. Install, configure and run tox

Refer to the tox basic example. Create a basic tox.ini so that your package is built and tested against Python 3.10 and 3.9.

Install and run tox in your project. Make sure all tests pass in both environments.

Re-organise your project structure as proposed in the figure. In Pycharm File > Settings > Project > Project Structure, identify src as a source folder so that the linter can identify your source files.

Part 7: Distribute your package on TestPyPi (Optional)

• 7.1. Refer to the doc about package creation to create a minimal pyproject.toml
• 7.2. Name your package accounts-<MYNAME> and substitute your name
• 7.3. Install build, wheel and twine
• 7.4. Refer to the doc to build sdist and bdist_wheel distributions
• 7.5. Upload both distributions to TestPyPI using login __token__. For the password, ask for the token or create your own TestPyPI account and new token.
• 7.6. Make sure you can install your package from the TestPyPI index via pip:
  pip install accounts-MYNAME --index-url https://test.pypi.org/simple/

Mini-project 4. Address book

We are going to write a Python script to handle e-mail addresses.

This contact manager will be named contacts.py. Your address book must accept these arguments:

1. a command: add search del to add, search or delete a contact
2. a --book=<name> option: with this option the user can select which of the address book (s)he wants to target: only pro and perso books.

Here are, for instance, a few commands that your contact manager must accept:

./contacts.py add Maria --email maria@muller.me --book=pro
# Add or update a contact for name "Maria" in the professional book

./contacts.py search Maria
# Search all occurences of "Maria" in all books

./contacts.py del Maria --verbose
# Delete all occurences from all books matching exactly the name "Maria"

Part 1: Arguments and base commands

1. Declare an example dictionary storing fake e-mails associated to names in 2 pro and perso books. This is to explicit the structure of your dataset.

2. Hard-code a few contacts in your script matching the representation that you chose and implement the function add(contacts, book, name, email) to add an element

3. Thanks to the help of the argparse tutorial, declare an argument parser accepting a positional arguement command that might be add, del or search and for which the add command will call with a hardcoded name, a book and an address

4. If you run Unix, add the hashbang #!/usr/bin/env python that will tell the shell what is the interpreter to use for this script, and make it executable with chmod +x

5. In the terminal, call your script with option -h that is not implemented ; but that exists automatically as soon as we declare a parser

6. Call now your tool with a command: ./contacts.py add and check that it actually added the hardcoded contact to the hardcoded book in memory (with a print)

7. Add positional arguments name and email so that name and mail are not longer hardcoded but can be passed in arguments and test

8. Problem : by doing this, we made name and email compulsory what ever the command is. However the search command does not require these arguments. There are several ways to fix this issue, the main one is the sub-parser. However in this exercise we propose an alternative solution:

9. Transform email in optional argument (i.e.--email) and change add() in order to raise an exception if this argument is not provided

10. Add the optional argument --book (and its shortname -b) that accepts only these 2 book names: pro and perso, the latter being the default.

11. Update add() so that it uses the --book parameter

12. Implement search(..) that searches in all books and returns all found e-mail addresses, or None

13. Implement delete(..) so that it deletes all occurences in all books and returns the number of deletions.

Part 2: Regular expressions

1. Thanks to the regex cheat sheet, write a regex that matches e-mail addresses
2. In the Python console, use the re module to match your regex in a few examples
3. Observe the value and the type returned by the matching function and use it in add(..) so that it refuses adresses that do not validate the regex
4. Test your address book by adding valid and invalid addresses

Remark: There is a validate_email package that is way more efficient in the e-mail validation process, but the goal here is to train with regexes

Part 3: Recording books on disk

1. Declare an example of a JSON data structure that will hold all your books data
2. Implement a reading function read(contacts) and a writing function write(contacts), call them at the beginning and end of your script
3. Think to the special case where the file does not exist and must be created. You can use Path.exists() from the pathlib (read the doc)

Documentation that you will need to read before:

• The argparse tutorial : https://docs.python.org/3/howto/argparse.html
• Regex cheat sheet: https://www.debuggex.com/cheatsheet/regex/python
• Regex re : https://docs.python.org/3/library/re.html?highlight=re#module-re
• The pathlib: https://docs.python.org/3/library/pathlib.html

Mini-project 5A. Guess my number

Game rules: the computer randomly picks a number between 1 and 100000000 without revealing it to the player.

The player guesses a number and gives it to the computer that replies if the actual number is lesser or higher than the guess, and so on until the player guesses the right number.

Once your game is playable, you may add the following features:

• Catch exceptions: how does your code behave when the user enters text instead of an int?
• Record the gaming time in a variable and display it in the terminal
• Record and display the 3 best scores when the game starts

Mini-project 5B. Estimate π

Compute a maximum number of decimals of π has become a challenge to benchmark CPUs. In particular, the Monte-Carlo method estimates the value of pi this way:

We estimate the area of 1/4 of a circle by picking at least 1 billion random points between 0 and 1.
We count all those which are inside the circle of radius = 1 (in green). Their sum is an estimate of the green 1/4 of circle.
Then, thanks to the well known formula area = π x r², we deduce an estimate value of π!

Proceed this way:

1. Generate n = 1000000 float abscissas (x-axis) as well as n float ordinates (y-axis) in range [0; 1[
2. Group these floats by two in order to get a list of couples: [(x, y), (x, y), (x, y), …]
3. Count how many M(x,y) points comply the following equation x²+y² < 1 (call this number m)
4. The m/n is an estimate of the area of the 1/4 of circle of centre 0, 0 with radius 1. Multiply this ratio per 4 to get the area of the full circle A, and since A = n.r² this result is also the estimate of n
5. With pyplot.scatter(x, y, color=’green’), draw in green the points that are interior to the circle, and in red all others.

Use pyplot.axis(‘equal’) to get orthogonal axes

Mini-project 5C. Create a micro web app

• flask is a Web micro-server ideal for basic websites.

• petname generates funny animal names eg. crazy-rabbit, rugged-salmon

We will combine both to generate a different name at each load of a webpage:

Read the flask quick start and create a basic website serving a single HTML file containing a different animal name every time!

Proceed as follow:

• Create a HTML template with a basic CSS style and a single centred <div> containing a variable
• Create a Flask app that serves this template in endpoint / after you filled in the <div> with a new animal name
• Start the Flask development server according to the tutorial
• Open URL http://localhost:5000/ to test!

If template file is not found, place it into a templates dir alongside the script

Mini-project 5D. Communicate with a REST API

https://jsonplaceholder.typicode.com/ exposes a REST API to list/publish posts on a fake blog.

Each blog post has a title + a description + optional photo album.

Part I: Get 10 existing blog posts

Install requests and emit a GET request to get document /posts to server https://jsonplaceholder.typicode.com

Retrieve all posts and print the title of the first 10 posts.

Part II: Publish a new blog post

2.1. On the same endoint as before, emit a POST request with a payload to publish a new blog post.

You need to send a JSON payload that includes:

• title: Python training
• body: I am training to the requests module with Python
• userId: A user ID for the author (eg., user id 1)

2.2. Check in the response that the publish was successful

This API is public on the Internet, so for safety purposes, even successful requests do not actually modify the database.

Part III: Delete your blog post

3.1. With a GET, identify which blog ID has title optio dolor molestias sit.

3.2. Send a DELETE request to remove the post with the found id. Check the server response.

Part IV: 4 nested GET requests to download all photos

Get the list of albums for the user with the username 'Delphine' and then download all photos for the first album in the list:

• Get the list of users and search for the one named Delphine
• Get the list of albums for the user 'Delphine'
• Get the list of all photo URLs of all Delphine's album

Part V: Get all comments in a CSV

5.1. Get all posts with /posts
5.2. For each found post ID, get its comments with /posts/{id}/comments
5.3. Use a DictWriter to write them into comments.csv
5.4. Measure the overall execution time

Part VI: Get all comments in a CSV via multiprocessing

Start from the code of the previous part.

6.1. Add a multiprocessing Manager, a Pool of 4 workers, and a multiprocessing list to be shared into workers
6.2. Divide the list of posts into 4 chunks (the worker posts)
6.3. Isolate the job into get(worker_posts: list, comments: mp.list)
6.4. Starmap worker_posts, comments to get()
6.5. Measure the overall execution time

Mini-project 5E. Webscraping

https://books.toscrape.com/ is a public book library meant to be scraped.

Part I: Compute the mean book price of the homepage

1.1. Get the homepage: install requests and emit a GET request to document / to the https://books.toscrape.com web server.

1.2. Extract data: install beautifulsoup4 to deconstruct the HTML tree and extract all prices of the homepage

1.3. Compute the mean book price of the homepage

Part II: Download all books in CSV + thumbnails

Now focus ONLY on the Nonfiction category.

2.1. In a second script, accumulate all books names and prices from this category into a Python variable.

You have too loop over all pages as long as it contains a next button.

2.2. Modify your script so that it outputs:

• a CSV file of all found names + prices for this category (it must have 110 lines + 1 line for headers)
• all 110 images of all books into a thumbnails directory of the user homedir

Mini-project 5F. Plot ping durations

ping is a network tool sending ICMP requests to hosts. An ICMP request is sent every second and displays its round-trip duration in milliseconds.

The project aims at developing a Python tool that plots durations of ping requests.

1. Use argparse to accept arguments host (string) and iterations (int)
2. Use subprocess to call ping on the host passed in argument
   On Windows, pass iterations to -n
   On Linux, pass iterations to -c.
3. Capture the stdout stream of the subprocess into Python variables
4. Exclude header/footer lines and error lines based on their format (split strings on delimiters)
5. Convert each ping duration in millisec into a float and save them in a list
6. Terminate the ping subprocess after the specified number of iterations
7. Use matplotlib to plot all ping durations, a dashed mean plot (style="--"), and the standard deviation (fill_between) computed with numpy.

Windows console commands are CP1252 or CP437-encoded. UTF-8 for Linux.