Python Advanced Guide
Master professional Python: async, concurrency, memory, performance, packaging, security, and production patterns used in real-world systems.
Start LearningAdvanced OOP
What you'll learn
- Master inheritance chains
- Use abstract base classes
- Compose objects over inheritance
Advanced OOP goes beyond basic classes. Abstract Base Classes (ABCs) force subclasses to implement methods, composition builds behaviour by combining objects, and diamond inheritance teaches you why super() follows the MRO.
from abc import ABC, abstractmethod
class Shape(ABC):
@abstractmethod
def area(self):
pass
class Circle(Shape):
def __init__(self, radius):
self.radius = radius
def area(self):
return 3.14159 * self.radius ** 2
print(Circle(5).area())
Try it yourself
Create an abstract Animal class with a speak() method and a Dog subclass.
from abc import ABC, abstractmethod
class Animal(ABC):
@abstractmethod
def speak(self):
pass
class Dog(Animal):
def speak(self):
return 'Woof!'
print(Dog().speak())Descriptors & Properties Deep Dive
What you'll learn
- Understand the descriptor protocol
- Build reusable descriptors
- Combine with properties
Descriptors are the machinery behind properties, methods, and classmethod. A descriptor implements __get__, __set__, or __delete__ to control attribute access. Building one shows you how validation, caching, and type checking work under the hood.
class PositiveNumber:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, obj, objtype=None):
return obj.__dict__.get(self.name, 0)
def __set__(self, obj, value):
if value < 0:
raise ValueError("Must be positive")
obj.__dict__[self.name] = value
class Order:
total = PositiveNumber()
o = Order()
o.total = 100
print(o.total)
Try it yourself
Create a ValidatedString descriptor that rejects empty strings.
class ValidatedString:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, obj, objtype=None):
return obj.__dict__.get(self.name, '')
def __set__(self, obj, value):
if not value.strip():
raise ValueError('Empty not allowed')
obj.__dict__[self.name] = value
class User:
name = ValidatedString()
u = User()
u.name = 'Ali'
print(u.name)Decorators Deep Dive
What you'll learn
- Write decorators with arguments
- Stack multiple decorators
- Use class-based decorators
Advanced decorators take arguments, stack in order, and can be implemented as classes with __call__. Understanding decorator order matters — decorators apply bottom-up, so the one closest to the function runs first.
def repeat(times):
def decorator(func):
def wrapper(*args, **kwargs):
for _ in range(times):
result = func(*args, **kwargs)
return result
return wrapper
return decorator
@repeat(3)
def greet(name):
print(f"Hi {name}")
greet("Sufyan")
Try it yourself
Write a log_calls decorator that prints the function name on each call.
def log_calls(func):
def wrapper(*args, **kwargs):
print(f'Calling {func.__name__}')
return func(*args, **kwargs)
return wrapper
@log_calls
def add(a, b):
return a + b
print(add(2, 3))Generators Deep Dive
What you'll learn
- Build generator pipelines
- Use generator.send()
- Handle exceptions with throw() and close()
Advanced generators communicate bidirectionally with send(), terminate with close(), and raise exceptions with throw(). Generator pipelines chain multiple generators lazily for efficient data processing.
def running_average():
total = 0
count = 0
average = None
while True:
value = yield average
total += value
count += 1
average = total / count
avg = running_average()
next(avg)
print(avg.send(10))
print(avg.send(20))
print(avg.send(30))
Try it yourself
Write a generator that yields squares of numbers sent to it.
def square_stream():
while True:
value = yield
yield value ** 2
s = square_stream()
next(s)
print(s.send(4))Async Python Basics
What you'll learn
- Understand async and await
- Create coroutines
- Run with asyncio.run()
Async Python runs tasks cooperatively without threads. async defines a coroutine, await pauses until another coroutine completes, and asyncio.run() starts the event loop. It's ideal for I/O-bound work like APIs and file reads.
import asyncio
async def fetch(name, delay):
await asyncio.sleep(delay)
print(f"Done: {name}")
return name
async def main():
results = await asyncio.gather(
fetch("A", 2),
fetch("B", 1),
fetch("C", 3),
)
print(results)
asyncio.run(main())
Try it yourself
Write two coroutines that sleep and print, then run them with asyncio.run().
import asyncio
async def one():
await asyncio.sleep(1)
print('one')
async def two():
await asyncio.sleep(0.5)
print('two')
async def main():
await asyncio.gather(one(), two())
asyncio.run(main())asyncio Tasks & Timeouts
What you'll learn
- Create tasks with create_task
- Apply timeouts with wait_for
- Await first with wait
Tasks schedule coroutines on the event loop. asyncio.wait_for() enforces timeouts, and asyncio.wait() handles multiple futures with control over return conditions.
import asyncio
async def slow_task():
await asyncio.sleep(5)
return "done"
async def main():
try:
result = await asyncio.wait_for(slow_task(), timeout=1)
print(result)
except asyncio.TimeoutError:
print("Too slow!")
asyncio.run(main())
Try it yourself
Create a task with create_task and await it.
import asyncio
async def work():
await asyncio.sleep(0.1)
return 'done'
async def main():
task = asyncio.create_task(work())
print(await task)
asyncio.run(main())Threads
What you'll learn
- Run threads with threading
- Communicate safely with locks
- Understand the GIL
Threads run code in parallel within one process. Use threading.Thread to start work and Lock to prevent races on shared data. Python's GIL means CPU-bound threads don't fully parallelise, but I/O-bound ones do.
import threading
import time
def worker(name, delay):
time.sleep(delay)
print(f"{name} finished")
threads = [threading.Thread(target=worker, args=(f"T{i}", i)) for i in range(1, 4)]
for t in threads:
t.start()
for t in threads:
t.join()
print("All done")
Try it yourself
Start two threads that print their names with a small sleep.
import threading, time
def show(name):
time.sleep(0.1)
print(name)
threads = [threading.Thread(target=show, args=('A',)), threading.Thread(target=show, args=('B',))]
for t in threads: t.start()
for t in threads: t.join()Multiprocessing
What you'll learn
- Use Process for CPU-bound work
- Share data with Queue
- Understand spawn vs fork
Multiprocessing bypasses the GIL by running separate Python processes. Use Process for CPU-heavy tasks and Queue for safe inter-process communication. Each process gets its own memory.
import multiprocessing
def square(n):
return n * n
with multiprocessing.Pool(4) as pool:
results = pool.map(square, [1, 2, 3, 4, 5])
print(results)
Try it yourself
Use multiprocessing.Pool to double a list of numbers.
import multiprocessing
def double(n):
return n * 2
with multiprocessing.Pool(2) as pool:
print(pool.map(double, [1, 2, 3]))Concurrency Patterns
What you'll learn
- Choose threads vs processes vs async
- Use concurrent.futures
- Build a clean concurrency pattern
Different concurrency tools fit different jobs: async for many I/O tasks, threads for blocking I/O, processes for CPU work. concurrent.futures gives a unified interface for thread and process pools.
from concurrent.futures import ThreadPoolExecutor
def double(n):
return n * 2
with ThreadPoolExecutor(max_workers=4) as executor:
results = list(executor.map(double, [1, 2, 3, 4]))
print(results)
Try it yourself
Use ThreadPoolExecutor to run a function three times concurrently.
from concurrent.futures import ThreadPoolExecutor
def say(x):
return f'hi {x}'
with ThreadPoolExecutor(max_workers=3) as ex:
futures = [ex.submit(say, i) for i in range(3)]
print([f.result() for f in futures])Performance Optimization
What you'll learn
- Measure before optimizing
- Use built-ins for speed
- Apply caching strategically
Performance work starts with measurement. Built-in functions (sum, map, list comprehensions) are C-implemented and fast, and @lru_cache removes repeated computation. Optimize the actual bottleneck, not guesses.
import functools, time
@functools.lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
start = time.perf_counter()
print(fib(35))
print(f"Time: {time.perf_counter() - start:.4f}s")
Try it yourself
Time how long it takes to sum a list of a million numbers.
import time start = time.perf_counter() total = sum(range(1_000_000)) print(total, time.perf_counter() - start)
Memory Management
What you'll learn
- Understand reference counting
- Use __slots__ to save memory
- Work with garbage collection
Python frees objects automatically using reference counting and a cyclic garbage collector. __slots__ reduces per-instance memory by preventing attribute dict creation. tracemalloc profiles memory allocations.
import tracemalloc
tracemalloc.start()
data = [n ** 2 for n in range(100_000)]
current, peak = tracemalloc.get_traced_memory()
print(f"Current: {current / 1024:.0f} KB")
print(f"Peak: {peak / 1024:.0f} KB")
Try it yourself
Add __slots__ to a class to reduce its memory usage.
class Person:
__slots__ = ('name', 'age')
def __init__(self, name, age):
self.name = name
self.age = age
p = Person('Ali', 30)
print(p.name)Profiling
What you'll learn
- Profile with cProfile
- Find hot spots
- Read profiler output
Profiling shows exactly where time goes. cProfile records function calls and cumulative time, helping you find the real bottleneck instead of guessing. Optimise the function that dominates the output.
import cProfile
def slow():
total = 0
for i in range(1_000_000):
total += i
return total
cProfile.run('slow()')
Try it yourself
Use cProfile.run() to profile a simple loop function.
import cProfile
def work():
total = 0
for i in range(10000):
total += i
return total
cProfile.run('work()')Advanced Type Hinting
What you'll learn
- Use typing for clarity
- Write generic types
- Use Protocol and TypeAlias
Type hints document contracts and enable static checkers like mypy. typing provides Optional, Union, Callable, and generics. Protocol defines structural typing — anything with the right methods fits.
from typing import Protocol
class Named(Protocol):
name: str
def greet(item: Named) -> str:
return f"Hello, {item.name}"
class User:
def __init__(self, name):
self.name = name
print(greet(User("Sufyan")))
Try it yourself
Add a type hint to a function that takes int and returns str.
def describe(n: int) -> str:
return f'Number: {n}'
print(describe(5))Packaging Python Projects
What you'll learn
- Understand project layout
- Write pyproject.toml
- Structure a publishable package
A proper package has a clear structure: a project folder, a package directory, pyproject.toml, and a README. Modern Python uses pyproject.toml for build metadata instead of setup.py.
# pyproject.toml (example)
# [build-system]
# requires = ["setuptools>=68"]
# build-backend = "setuptools.build_meta"
#
# [project]
# name = "mypackage"
# version = "0.1.0"
# description = "A sample package"
print("Project structure:")
print("mypackage/")
print(" mypackage/")
print(" __init__.py")
print(" core.py")
print(" pyproject.toml")
print(" README.md")
Try it yourself
Write a minimal pyproject.toml with name and version.
print('[project]')
print('name = "demo"')
print('version = "0.1.0"')Building & Publishing Packages
What you'll learn
- Build wheels and sdists
- Publish to PyPI with twine
- Version packages correctly
Publishing makes your package installable by anyone. python -m build creates distributions (wheel and sdist), and twine uploads them to PyPI. Semantic versioning keeps releases predictable.
# Build commands (run in terminal):
# python -m build
# python -m twine upload dist/*
import sys
print("Release checklist:")
print("1. Update version in pyproject.toml")
print("2. Build: python -m build")
print("3. Upload: python -m twine upload dist/*")
print("4. Verify: pip install yourpkg")
Try it yourself
Print the semantic versioning order for 1.0.0, 2.0.0, and 1.1.0.
versions = ['1.0.0', '2.0.0', '1.1.0'] print(sorted(versions))
Environment & Configuration
What you'll learn
- Load environment variables
- Use pydantic-settings
- Separate config from code
Configuration should live outside code — in environment variables or .env files. pydantic-settings validates config at startup so the app fails fast with a clear error if something is missing.
import os
os.environ["DATABASE_URL"] = "postgres://localhost/app"
database_url = os.environ.get("DATABASE_URL", "sqlite:///default.db")
print(database_url)
secret = os.environ.get("SECRET_KEY")
print("Secret loaded:", bool(secret))
Try it yourself
Read a HOME environment variable and print it.
import os
print(os.environ.get('HOME', 'not set'))Security Basics
What you'll learn
- Sanitize user input
- Use secrets for tokens
- Avoid SQL injection
Security is a mindset: never trust user input, never build SQL with string concatenation, and use the secrets module for tokens and passwords. Parameterized queries stop injection attacks.
import secrets
def make_token():
return secrets.token_urlsafe(16)
print(make_token())
print(make_token())
Try it yourself
Generate a 32-byte secure token with secrets.token_hex.
import secrets print(secrets.token_hex(32))
Architecture & Clean Code
What you'll learn
- Apply SOLID principles
- Separate concerns
- Keep functions small
Clean code is readable, testable, and easy to change. Single Responsibility says one function does one thing, dependency injection makes code testable, and descriptive names remove the need for comments.
from dataclasses import dataclass
@dataclass
class EmailSender:
smtp_host: str
def send(self, to, subject, body):
return f"Sent to {to}: {subject}"
class Notifier:
def __init__(self, sender):
self.sender = sender
def notify(self, user, message):
return self.sender.send(user, "Update", message)
n = Notifier(EmailSender("smtp.example.com"))
print(n.notify("[email protected]", "Welcome!"))
Try it yourself
Write a class that takes a logger object in __init__ and uses it in a method.
class Service:
def __init__(self, logger):
self.logger = logger
def run(self):
self.logger.log('running')
class Logger:
def log(self, msg):
print(msg)
Service(Logger()).run()Production Python
What you'll learn
- Configure logging properly
- Handle errors gracefully
- Plan observability
Production code needs proper logging, graceful error handling, and observability. Configure the logging module with levels and formats, catch errors at boundaries, and always provide useful context.
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s",
)
logging.info("Service started")
logging.warning("High memory usage")
try:
result = 10 / 0
except ZeroDivisionError:
logging.error("Division by zero attempted")
Try it yourself
Configure logging and log an info message.
import logging
logging.basicConfig(level=logging.INFO)
logging.info('Hello production')Capstone: Production API
What you'll learn
- Build a production-style app
- Combine async, classes, and config
- Structure for maintainability
This capstone combines everything: classes for services, async for I/O, environment config, and clean error handling. The goal is a small but production-quality API-like service.
import os
import logging
from dataclasses import dataclass
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
@dataclass
class Config:
max_results: int = 10
class FakeDB:
def query(self, limit):
return [f"row-{i}" for i in range(limit)]
class Service:
def __init__(self, db, config):
self.db = db
self.config = config
def search(self):
logging.info("Running search")
try:
return self.db.query(self.config.max_results)
except Exception as exc:
logging.error(f"Search failed: {exc}")
return []
service = Service(FakeDB(), Config(max_results=5))
print(service.search())
Try it yourself
Add a retry option to Config and use it in the Service.
from dataclasses import dataclass
@dataclass
class Config:
max_results: int = 10
retries: int = 3
print(Config())Advanced Projects
Professional-grade projects with real inputs, concurrency, and clean architecture. Click each project to open its full guide.
About this project
Fetch multiple URLs concurrently using asyncio. This is the real-world pattern used by web scrapers and API clients — dozens of requests at once without waiting for each one.
What you'll practice
Step-by-step: How it works
- Define a list of URLs to fetch.
- Write an async
fetch()function that waits on network I/O. - Use
asyncio.gather()to run all fetches at the same time. - Measure total time to prove concurrency beats sequential requests.
- Catch per-request errors so one failure doesn't stop the rest.
import asyncio
import time
async def fetch(url, delay):
# Simulate network I/O with an async sleep
await asyncio.sleep(delay)
return f"{url} -> done"
async def main():
urls = [("site-a.com", 2), ("site-b.com", 1), ("site-c.com", 3)]
start = time.perf_counter()
results = await asyncio.gather(
*(fetch(url, delay) for url, delay in urls)
)
elapsed = time.perf_counter() - start
for result in results:
print(result)
print(f"All fetched in {elapsed:.2f}s (not 6s!)")
asyncio.run(main())
About this project
Build a secure CLI password manager with user input, hashing, and JSON storage. It demonstrates real security: secrets module, hashlib, and file persistence — all from a terminal.
What you'll practice
Step-by-step: How it works
- User enters a username and password via
input(). - A random salt is generated with
secrets.token_hex(8). - The password is hashed with
hashlib.sha256(salt + password). - The hash + salt + username are saved to a JSON file.
- The example verifies a login by re-hashing and comparing.
import hashlib
import secrets
import json
from pathlib import Path
def hash_password(password, salt):
return hashlib.sha256((salt + password).encode()).hexdigest()
def register(username, password):
salt = secrets.token_hex(8)
digest = hash_password(password, salt)
record = {"username": username, "salt": salt, "digest": digest}
Path("vault.json").write_text(json.dumps(record))
print(f"User {username} registered.")
def login(username, password):
record = json.loads(Path("vault.json").read_text())
if record["username"] != username:
print("User not found.")
return
digest = hash_password(password, record["salt"])
if digest == record["digest"]:
print("Login successful!")
else:
print("Wrong password.")
# Simulate user input
name = input("Username: ")
secret = input("Password: ")
register(name, secret)
login(name, secret)
About this project
Analyze a log file to count errors, warnings, and info messages using regex and collections. This is the same pattern monitoring tools use in production.
What you'll practice
Step-by-step: How it works
- Read a sample log file line by line.
- Use
re.search()to extract the log level from each line. - Count occurrences with
collections.Counter. - Print a clean summary of ERROR, WARNING, and INFO counts.
- Extend it to filter errors by timestamp or keyword.
import re
from collections import Counter
LOG_SAMPLE = """
2026-08-24 10:00:00 INFO Server started
2026-08-24 10:01:00 ERROR Disk full
2026-08-24 10:02:00 WARNING High memory
2026-08-24 10:03:00 INFO User logged in
2026-08-24 10:04:00 ERROR Failed to connect
2026-08-24 10:05:00 INFO Request served
"""
pattern = r"(INFO|WARNING|ERROR)"
levels = re.findall(pattern, LOG_SAMPLE)
summary = Counter(levels)
for level, count in summary.items():
print(f"{level}: {count}")
print(f"Total log lines: {len(summary.values())}")
You've completed all 20 lessons. Ready to continue?
Solidify your skills with practice exercises, coding challenges, and debugging drills.