Lecture Notes 5: Generator, Decorator, and HOFs in Python
Generator
Using generator, we can make a function behave like an iterator in Python.
def firstn(n):
num = 0
while num < n:
yield num
num += 1
first_n = firstn(10)
Above code will return a generator stored in first_n variable. This can be accessed one by one by using next
or for loop
or can all be executed using list
or another function like sum
.
This simplify the lazy evaluation in python function.
Generator can also be expressed in one-line statement using ()
. For example:
squares_generator = (x * x for x in range(5))
for square in squares_generator:
print(square)
Using generator we can create data processing pipeline, for instance:
def get_numbers(limit):
for i in range(limit):
yield i
def square_numbers(numbers):
for num in numbers:
yield num * num
def filter_even(numbers):
for num in numbers:
if num % 2 == 0:
yield num
pipeline = filter_even(square_numbers(get_numbers(10)))
for result in pipeline:
print(result) # Output: 0, 4, 16, 36, 64
We can use the generator in such conditions:
- When processing the data that is too large, infinite, or needs to be processed lazily. This fit naturally with functional pipelines and stream processing, and avoid unnecessary memory use.
- When we want to separate computation logic from consumption logic
- When we want to implement coroutines or cooperative multitasking
- When we want to keep code stateful, but without mutable variables
Decorator
A decorator is a function that takes another function as input, adds extra behavior, and returns a new function. It’s a way to “wrap” the existing code without modifying the original function’s code. It is like a higher-order function.
Why we use decorator?
- we use it for seperating the concern of functionality (for example like logging, timing, security, caching, etc) from the main logic of the code.
- to reuse behavior without copy-pasting.
- to make code cleaner and more readable for developer
Without a decorator:
def shout(func):
def wrapper(*args, **kwargs):
print("Before the function runs")
func(*args, **kwargs)
print("After the function runs")
return wrapper
def greet():
print("Hello!")
# Manually decorating
decorated_greet = shout(greet)
decorated_greet()
With decorator:
@shout
def greet():
print("Hello!")
greet()
Practical use for getting the running time:
import time
def timer(func):
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f"{func.__name__} took {end - start:.4f}s")
return result
return wrapper
@timer
def slow_function():
time.sleep(1)
slow_function()
Decorator with arguments
def abc(a):
def real_decorator(func): # the actual decorator
def wrapper(*args, **kwargs):
print(f"Decorator arg a={a}")
return func(*args, **kwargs)
return wrapper
return real_decorator # return the actual decorator
@abc(a=123)
def greet(name):
print(f"Hello {name} ...!")
greet("Amanda")
Django Framework example in using decorator:
from django.contrib.auth.decorators import login_required
from django.http import HttpResponse
@login_required
def dashboard(request):
return HttpResponse("Welcome to your dashboard!")
from django.contrib.auth.decorators import permission_required
@permission_required('blog.add_post', raise_exception=True)
def create_post(request):
return HttpResponse("You can create a post!")
from django.views.decorators.cache import cache_page
@cache_page(60) # cache for 60 seconds
def homepage(request):
return HttpResponse("Heavy homepage content")
Map
numbers = [1, 2, 3, 4]
result = map(lambda x: x * 2, numbers)
print(list(result))
Filter
numbers = [1, 2, 3, 4, 5, 6]
result = list(filter(lambda x: x % 2 == 0, numbers))
print(result)
Reduce
from functools import reduce
numbers = [1, 2, 3, 4]
result = reduce(lambda x, y: x + y, numbers)
print(result)
Example of combined solution
from functools import reduce
numbers = [1, 2, 3, 4, 5, 6]
squares = map(lambda x: x * x, numbers)
even_squares = filter(lambda x: x % 2 == 0, squares)
result = reduce(lambda x, y: x + y, even_squares)
print(result) # 56 (4 + 16 + 36)
Exercise
Let say we have this data after querying to a non-sql database receiving the json format and converted into map in python:
orders = [
{"customer": "Alice", "items": [
{"name": "Laptop", "price": 1200, "category": "electronics"},
{"name": "Book", "price": 30, "category": "stationery"}
]},
{"customer": "Bob", "items": [
{"name": "Phone", "price": 800, "category": "electronics"},
{"name": "Pen", "price": 5, "category": "stationery"}
]},
{"customer": "Charlie", "items": [
{"name": "Headphones", "price": 150, "category": "electronics"}
]}
]
Please keep only electronics with price > 500, apply 10% discount, and compute the total revenue…!