Python Basics

Other available Jupyter kernels can be found here.

# https://ipython.readthedocs.io/en/9.2.0/install/index.html
# to make cell magic functions work
pip3 install ipython --break-system-packages
python3 -m pip install ipykernel --break-system-packages
python3 -m ipykernel install

The local kernel pulls modules from the Python virtual environment, while ipykernel uses the global Python environment. Note: Only ipykernel supports handy magic functions like %pip!

a = 1
b = 2
print(a+b)

3

Dictionary

The “Dictionary Builder” Trick: Turn a list of items into keyed dictionaries on the fly:

users = ["sam", "alex", "jordan"]
data = {u: {"active": True} for u in users}
print(data)

{'sam': {'active': True}, 'alex': {'active': True}, 'jordan': {'active': True}}

Generators and Iterators

def fibonacci_sequence(limit):
  a, b = 0, 1
  while a < limit:
    yield a
    a, b = b, a + b

for num in fibonacci_sequence(100):
  print(num)

0

1

1

2

3

5

8

13

21

34

55

89

List Comprehensions

# https://medium.com/@abdur.rahman12/11-python-myths-that-are-wasting-your-time-f52aaea0c146
# Terrible
result = [x*y for x in range(5) for y in range(5) if (x+y)%2==0 and x!=y]
print(result)

# Cleaner
result = []
for x in range(5):
    for y in range(5):
        if (x+y) % 2 == 0 and x != y:
            result.append(x*y)
print(result)

[0, 0, 3, 0, 8, 3, 0, 8]

[0, 0, 3, 0, 8, 3, 0, 8]

Type Hints

They massively help with:

• Auto-completion
• Linting
• Team communication
• Catching silly bugs early (like accidentally passing a dict to a function expecting a str — ahem, been there)

def greet(name: str) -> str:
  return f"Hello, {name}"
print(greet("World"))
print(greet(123))

Hello, World

Hello, 123

from typing import List, Union

def calculate_sum(numbers: List[Union[int, float]]) -> float:
  """Calculates the sum of a list of numbers."""
  total = 0.0
  for num in numbers:
    total += num
  return total

ages: List[int] = [20, 25, 30]
average_age: int = calculate_sum(ages) / len(ages)
print(f"Average age: {average_age}")
print(f"Total sum of ages: {calculate_sum([18, 22, 27, 35])}")

Average age: 25.0

Total sum of ages: 102.0

Decorators

Further reading: Notebook Decorators

def log_function_call(func):
  def wrapper(*args, **kwargs):
    print(f"Calling {func.__name__} with {args} and {kwargs}")
    return func(*args, **kwargs)
  return wrapper

@log_function_call
def add(a, b):
  return a + b

add(5, 3)

Calling add with (5, 3) and {}

8

Image

Further reading about image manipulation in Python can be found in the Python Notebook Image.

Context Manager

from contextlib import contextmanager

@contextmanager
def open_file(file, mode):
  f = open(file, mode)
  try:
    print(f"File {file} has been opened in {mode} mode.")
    yield f
    print(f"File {file} operations completed.")
  finally:
    f.close()
    print(f"File {file} has been closed.")

with open_file('/tmp/example.txt', 'w') as f:
  f.write('Hello, World!')

File /tmp/example.txt has been opened in w mode.

File /tmp/example.txt operations completed.

File /tmp/example.txt has been closed.

Coroutines and Asyncio

import asyncio

async def fetch_data():
  print("Fetching data...")
  await asyncio.sleep(2)
  print("Data fetched")

print("Starting...")
asyncio.run(fetch_data())
print("Finished.")

Starting...

Fetching data...

Data fetched

Finished.

Multi-threading and Multi-processing

# This script demonstrates the use of threading in Python.
import threading
import time

def print_numbers(name, delay):
  for i in range(5):
    time.sleep(delay)
    print(f"Thread {name}: {i}")

# It should be run as a main program.
# Otherwise, it will not execute the threading part.
if __name__ == "__main__":
  print("Main program: Starting threads")

  thread1 = threading.Thread(target=print_numbers, args=("One", 0.5))
  thread2 = threading.Thread(target=print_numbers, args=("Two", 0.7))

  thread1.start()
  thread2.start()

  thread1.join() # Wait for thread1 to complete
  thread2.join() # Wait for thread2 to complete

  print("Main program: All threads finished")
else:
  print("This script is intended to be run as a main program.")

This script is intended to be run as a main program.

Memory Management and Garbage Collection

import gc

def create_cycle():
  a = {}
  b = {"ref": a}
  a["ref"] = b

create_cycle()
print(gc.collect())  # Force garbage collection

2179

Using INI file with configparser

Further examples are in another notebook

import configparser

sample_config = """
[mypy]
warn_return_any = True
warn_unused_configs = True

# per-module options

[mypy-mycode.foo.*]
disallow_untyped_defs = True
"""

config = configparser.ConfigParser(allow_no_value=True)
config.read_string(sample_config)
print(config.sections())
print(config['mypy'])
print(config['mypy']['warn_return_any'])

['mypy', 'mypy-mycode.foo.*']

<Section: mypy>

True

Function things

Declare keyword-only parameters using a single *.

def func(a, b, *, c, d):
  print(f"{a=}, {b=}, {c=}, {d=}")
try:
  func(1, 2, 3, 4)
except Exception as e:
  print(e)
  print(f"`c` and `d` can only take in keyword arguments, and hence the error.")

print(f'\nto call this function, we must pass `c` and `d` as keyword arguments.')
func(1, 2, c=3, d=4)

func() takes 2 positional arguments but 4 were given

`c` and `d` can only take in keyword arguments, and hence the error.

to call this function, we must pass `c` and `d` as keyword arguments.

a=1, b=2, c=3, d=4

Declare positional-only parameters using /.

def func(a, b, /, c, d):
  print(f"{a=}, {b=}, {c=}, {d=}")
try:
  func(a=1, b=2, c=3, d=4)
except Exception as e:
  print(e)
  print(f"`a` and `b` can only take in positional arguments, and hence the error.")

print(f'\nto call this function, we must pass `a` and `b` as positional arguments.')
func(1, 2, c=3, d=4)

func() got some positional-only arguments passed as keyword arguments: 'a, b'

`a` and `b` can only take in positional arguments, and hence the error.

to call this function, we must pass `a` and `b` as positional arguments.

a=1, b=2, c=3, d=4

We can combine both / and *, but / must come before *:

def func(a, b, /, c, d, *, e, f):
  # ...

• a and b can only take in positional arguments
• c and d can take in both positional and keyword arguments
• e and f can only take in keyword arguments

Default arguments.

def greet(name, greeting="Hello"):
  print(f"{greeting}, {name}!")
greet("Tom")
greet("Tom", "Hi")
greet(name="Alice", greeting="Hi")

Hello, Tom!

Hi, Tom!

Hi, Alice!

*args & **kwargs in Python are used to pass a variable number of arguments to a function.

• *args allows to pass a variable number of non-keyword arguments to a function. It collects extra positional arguments into a tuple.
• **kwargs allows to pass a variable number of keyword arguments to a function. It collects extra keyword arguments into a dictionary.

def func(*args):
  print(args)
func(1, 2, 3)

def func2(**kwargs):
  print(kwargs)
func2(a=1, b=2, c=3)

(1, 2, 3)

{'a': 1, 'b': 2, 'c': 3}

TIPS

Use traceback to debug.

import traceback
try:
  1 / 0
except Exception:
  error_info = traceback.format_exc()
  print("An error occurred:\n", error_info.split('\n')[-2:-1][0])
  

An error occurred:

 ZeroDivisionError: division by zero

Leverage __missing__ in custom dictionaries to handle missing keys.

class AutoDict(dict):
  def __missing__(self, key):
    self[key] = []
    return self[key]
d = AutoDict()
d["python"].append("rocks")
print(d)  # {'python': ['rocks']}

{'python': ['rocks']}

Dynamically create functions with type().

# def template_func(x): return x
def make_func(name, multiplier):
  return type(name, (), {
    "__call__": lambda self, x: x * multiplier
  })()
double = make_func("Doubler", 2)
print(double(10)) # 20

20