Concurrency and Parallelism in Python

1. Threads vs. Processes in Python

Concurrency in Python can be achieved using threads or processes, each having unique use cases and benefits:

Threads

Threads allow multiple tasks to run concurrently within the same process, sharing the same memory space. Python’s threading module is commonly used for tasks like I/O operations or tasks that require minimal CPU.

import threading

def print_numbers():
    for i in range(5):
        print(i)

# Creating a thread
thread = threading.Thread(target=print_numbers)
thread.start()
thread.join()

Processes

Processes are independent units with their own memory space, allowing true parallelism. Python’s multiprocessing module is useful for CPU-intensive tasks since each process can run on a different core.

from multiprocessing import Process

def print_numbers():
    for i in range(5):
        print(i)

# Creating a process
process = Process(target=print_numbers)
process.start()
process.join()

Key Difference: Threads share memory space and have faster context-switching, while processes have separate memory spaces and allow true parallelism on multi-core processors. The Global Interpreter Lock (GIL) in Python affects threading, making processes more suitable for CPU-bound tasks.

2. Asynchronous Programming with asyncio

Asynchronous programming allows code to run concurrently without blocking, making it ideal for I/O-bound tasks like network requests and file operations.

Introducing asyncio

The asyncio library provides the tools needed for writing asynchronous code in Python. It enables coroutines to pause and resume, allowing other tasks to run during waiting periods.

Creating an Asynchronous Function

To define an asynchronous function, use the async keyword. Use await for non-blocking calls within these functions:

import asyncio

async def say_hello():
    print("Hello")
    await asyncio.sleep(1)
    print("World")

# Running the asynchronous function
asyncio.run(say_hello())

This code runs say_hello, pausing for 1 second between "Hello" and "World" without blocking the main thread.

3. Running Multiple Tasks with asyncio

With asyncio, you can run multiple asynchronous tasks concurrently. Here’s an example that demonstrates parallel task execution:

import asyncio

async def task1():
    print("Task 1 started")
    await asyncio.sleep(2)
    print("Task 1 finished")

async def task2():
    print("Task 2 started")
    await asyncio.sleep(1)
    print("Task 2 finished")

async def main():
    await asyncio.gather(task1(), task2())

asyncio.run(main())

Here, task1 and task2 run in parallel, with task2 finishing first due to a shorter delay.

4. Summary

Python offers multiple ways to achieve concurrency, each suited for different scenarios:

• Threading: Use for I/O-bound tasks with minimal CPU use.
• Multiprocessing: Use for CPU-intensive tasks requiring parallel execution on multiple cores.
• Asyncio: Use for non-blocking I/O operations where asynchronous code execution is beneficial.

Understanding these options allows you to choose the best concurrency model for your specific tasks, optimizing performance and responsiveness.