0x02. Python - Async Comprehension

Python - Back-end - Asynchronous Comprehensions - Asynchronous Generators- type-annotate generators - asyncio.gather

Intro.

Python 3.6 added the ability to create Asynchronous Comprehensions and Asynchronous Generators.

Asynchronous Comprehensions in Python is a feature that allows you to use the await keyword inside list, set, dictionary, and generator comprehensions. This feature was introduced in Python 3.6 as part of PEP 530.

Asynchronous comprehensions are a way to work with asynchronous sequences efficiently. They allow you to perform asynchronous operations on each item of a sequence and collect the results in a new sequence.

Asynchronous Iterable: is An object that you can loop over using async for.

Here's a simple example:

In this example, fetch_data(url) is an asynchronous function that simulates fetching data from a URL. The async comprehension [await fetch_data(url) for url in urls] fetches data from each URL in the urls list concurrently. The await keyword is used to pause the execution of the async comprehension until fetch_data(url) completes for each URL. The results are collected in the results list.

This feature is particularly useful when you need to perform asynchronous operations on a collection of items, such as making network requests or reading from files.

---

aiter()

The aiter() function in Python is used to create an asynchronous iterator from an asynchronous iterable object.

Usage: You typically use aiter() in conjunction with anext() which fetches the next item from the asynchronous iterator. Together, they allow you to manually control the iteration process over asynchronous iterables. However, in most cases, an async for loop is preferred for its simplicity ^.

Example: Here's an example of using aiter()

AsyncIterable is an asynchronous iterable that yields items with a delay. aiter() is used to get an asynchronous iterator from it. The async for loop then consumes the items from this iterator.

---

Asynchronous Generators

Background: Before Python 3.6, you couldn't mix yield and await in the same function. PEP 525 removed this limitation, allowing the creation of asynchronous generator functions.

Use Cases: They are useful in scenarios where data is produced asynchronously and consumed asynchronously. For example, reading from a stream of data coming over a network or a large file being read in chunks where I/O operations are involved.

the ticker function is an asynchronous generator. It yields numbers from 0 to to with a delay of delay seconds between each yield. This generator is then consumed using an async for loop.

Asynchronous Comprehensions

In this example, aiter is an asynchronous generator. The comprehension [i async for i in aiter() if i % 2 == 0] asynchronously iterates over aiter, selecting only the even numbers.

---

How to type-annotate generators ?

Type-annotating generators in Python involves specifying the types of values that the generator yields, the type of value it receives (if any), and the type of value it returns upon completion. This is done using the Generator type from the typing module in Python.

---

Notes for practice :

1. # Generate a random integer within a range

   print(random.randint(1, 10))

   Coroutine in Python : Coroutines are basically functions whose execution can be paused/suspended at a particular point, and then we can resume the execution from that same point later whenever we want.¹

To run the coroutine, you need to use asyncio.run() if this is your main coroutine # or await it inside another coroutine.asyncio.run(wait_and_print(3, "This message appears after 3 seconds"))

---

asyncio.gather:

The asyncio.gather function is allowing for the concurrent execution of awaitable objects such as coroutines, Tasks, Futures, or a combination of these.

It is a high-level API that simplifies running and managing multiple asynchronous tasks concurrently and collecting their results once they are completed.

The results of the awaitable objects are returned as a list, in the order they were passed to asyncio.gather.

When run, this script will output ['result1', 'result2', 'result3'] after approximately 3 seconds, demonstrating that the tasks have been executed concurrently rather than sequentially.

It's important to note that asyncio.gather should be used when you want to run multiple tasks concurrently and when you need to collect their results. For situations where you don't need to collect results or when you have different handling strategies for task completion, other constructs such as asyncio.wait, asyncio.as_completed, or asyncio.TaskGroup (introduced in Python 3.11) might be more appropriate ^{1 3 5}

Note

In an I/O-bound task or task that has some sort of delay (like a sleep statement), running them concurrently should not significantly increase the total runtime compared to running a single instance. The total time is roughly equal to the time taken by the longest task.

---

Resources

https://pythonpl.com/python-aiter-function/

random.uniform to generate random floats

1. Coroutines in Python