7 Replies - 1000 Views - Last Post: 25 June 2020 - 11:15 AM Rate Topic: -----

#1 ZOMBIE!!!   User is offline

  • D.I.C Head

Reputation: 28
  • View blog
  • Posts: 216
  • Joined: 28-October 09

Asyncio : Program not running in parallel

Posted 19 June 2020 - 03:21 PM

Hi everybody,

So I've been checking out tutorial after tutorial for the asyncio library and I just can't get it to run as intended. I'm somewhat familiar with multi-threading in C but Python's version has been difficult for me to wrap my head around


The Goal :


I'm creating a program to send a lot of custom emails. It was in working order but quite slow as waiting for a request to finish to start another is quite time consuming.

The (hopefully) Solution:

Use asyncio to run email request in parallel so that I can blast through the list asap.

The Constraint:

Amazon's SES is currently restricting me to 14 emails/sec (if this rate is exceeded, the entire program fails). To adjust for this I've tried implementing this library: ratelimiter, not sure if it's working currently since (spoiler) my program won't execute in parallel. This is the less important issue to solve, I can always worry about this later.

The Problem:

Like I said, the supposed asynchronous calls are being run sequentially, meaning the original problem of being too slow is not being fixed


The Code:

Here is the code that seems relevant to the issue (as always I will happily provide other snippets if you need more context). I've gone through a few different iterations/attempts (awaiting single function calls instead of using gather, etc.), but seeing as this one I essentially inserted my code into an example I found, I figured it was most likely to work, and therefore most suitable to share.

    #this code is in the "main" function 
    loop = asyncio.get_event_loop()
    loop.run_until_complete(send_coroutine(addresses,email,text_version))
    loop.close()

#related to the RateLimiting attempt, not super important for the main question
async def limited(until):
    print('rate limit exceeded, chilling out')


async def send_coroutine(addresses, email,text_version):
    tasks = []
    for user in addresses:
        email_data={
            'recipient' : user,
            'subject' : 'icitizen - Your Weekly Update',
            'html' : email,
            'text' : text_version
        }
        tasks.append(asyncio.ensure_future(co_send(email_data)))

    await asyncio.gather(*tasks)
# I wanted to avoid making send_email a coroutine as it is used
# elsewhere in the codebase, so I thought just wrapping it in 
# an async def would work...I could be wrong?
@RateLimiter(max_calls=14, period=1, callback=limited)
async def co_send(email_data):
    send_email(email_data)




As always, I'll happily further explain anything,
and as always,
Thanks in Advanced :smile2:

This post has been edited by ZOMBIE!!!: 19 June 2020 - 03:22 PM


Is This A Good Question/Topic? 0
  • +

Replies To: Asyncio : Program not running in parallel

#2 ZOMBIE!!!   User is offline

  • D.I.C Head

Reputation: 28
  • View blog
  • Posts: 216
  • Joined: 28-October 09

Re: Asyncio : Program not running in parallel

Posted 21 June 2020 - 02:26 PM

So I believe I finally got it working after abandoning asyncio and using the native Python threading library (shoutout to DK3250 for his great tutorial here: Parallel Processing)

Not sure why I didn't just go with the native library to begin with and I'm still a bit frustrated that I couldn't get asyncio to work but hey whatever.


for the curious, here is what the code looks like now:

    rate_limiter = RateLimiter(max_calls=send_limit, period=1, callback=limited)
    threads = []
    for user in addresses:
        with rate_limiter:
            email_data={
                'recipient' : user,
                'subject' : 'icitizen - Your Weekly Update',
                'html' : email,
                'text' : text_version
            }
            new_thread = Thread(target=send_email, args=(email_data,))
            new_thread.start()
            threads.append(new_thread)

    for thread in threads:
        thread.join()

################################################
########## Helper functions ####################
################################################
def limited(until):
    print('rate limit exceeded, chilling out')



This post has been edited by ZOMBIE!!!: 21 June 2020 - 02:33 PM

Was This Post Helpful? 0
  • +
  • -

#3 xclite   User is offline

  • I wrote you an code
  • member icon


Reputation: 1474
  • View blog
  • Posts: 4,326
  • Joined: 12-May 09

Re: Asyncio : Program not running in parallel

Posted 21 June 2020 - 04:30 PM

Well, you basically ran into the "concurrency is not parallelism" thing. Async doesn't mean that you're going to do multiple things simultaneously. It means that if you're waiting on some IO, you can yield control so that another piece of work can make progress while you wait.
Was This Post Helpful? 2
  • +
  • -

#4 ZOMBIE!!!   User is offline

  • D.I.C Head

Reputation: 28
  • View blog
  • Posts: 216
  • Joined: 28-October 09

Re: Asyncio : Program not running in parallel

Posted 22 June 2020 - 10:52 AM

View Postxclite, on 21 June 2020 - 05:30 PM, said:

Well, you basically ran into the "concurrency is not parallelism" thing. Async doesn't mean that you're going to do multiple things simultaneously. It means that if you're waiting on some IO, you can yield control so that another piece of work can make progress while you wait.

That was the idea, perhaps I implemented the logic behind it incorrectly, or I'm getting my terms mixed up. The send_email function was the source of the IO, it waits on a response from the mail server. So the hope was that I could call it many times without waiting for a response for every call. Thanks for the reply, sorry if my original explanation wasn't thorough enough.

This post has been edited by ZOMBIE!!!: 22 June 2020 - 11:09 AM

Was This Post Helpful? 0
  • +
  • -

#5 xclite   User is offline

  • I wrote you an code
  • member icon


Reputation: 1474
  • View blog
  • Posts: 4,326
  • Joined: 12-May 09

Re: Asyncio : Program not running in parallel

Posted 22 June 2020 - 12:44 PM

Well and I guess I would ask - does your async function ever hit a situation where it yields control? In other words, if it's always making progress and it never yields... then nothing will ever get a chance to run. Calling something async doesn't necessarily make it so, and depends on whether the code that actually sends the email knows how to behave in an asynchronous fashion.
Was This Post Helpful? 2
  • +
  • -

#6 ZOMBIE!!!   User is offline

  • D.I.C Head

Reputation: 28
  • View blog
  • Posts: 216
  • Joined: 28-October 09

Re: Asyncio : Program not running in parallel

Posted 23 June 2020 - 10:58 PM

View Postxclite, on 22 June 2020 - 01:44 PM, said:

Well and I guess I would ask - does your async function ever hit a situation where it yields control? In other words, if it's always making progress and it never yields... then nothing will ever get a chance to run.



Yeah that makes total sense. I think I just misunderstood the how gather worked. I tried a lot of different things but at the point when I posted this I essentially copied an example from a youtube tutorial I was watching and replaced their 'slow function' with my 'slow function' and hoped it would work. I think I remember seeing (and possibly testing) something like inserting:
await asyncio.sleep(.01)


within the function as a means of handing over control to the event loop.
Is that the sort of thing you are referring to?

View Postxclite, on 22 June 2020 - 01:44 PM, said:

depends on whether the code that actually sends the email knows how to behave in an asynchronous fashion.

Not really sure I follow you there. Essentially all the code in the email function does is establish a connection to Amazon's Simple Email Service and blast out a given email to a given address. Amazon's send_email function waits on a response so that function is somewhat time consuming. It's behavior would be independent of whether it is called in a standard or asynchronous fashion would it not? At least it works that way with my threading example.

This post has been edited by ZOMBIE!!!: 23 June 2020 - 10:59 PM

Was This Post Helpful? 0
  • +
  • -

#7 xclite   User is offline

  • I wrote you an code
  • member icon


Reputation: 1474
  • View blog
  • Posts: 4,326
  • Joined: 12-May 09

Re: Asyncio : Program not running in parallel

Posted 24 June 2020 - 05:26 AM

View PostZOMBIE!!!, on 24 June 2020 - 01:58 AM, said:

View Postxclite, on 22 June 2020 - 01:44 PM, said:

depends on whether the code that actually sends the email knows how to behave in an asynchronous fashion.

Not really sure I follow you there. Essentially all the code in the email function does is establish a connection to Amazon's Simple Email Service and blast out a given email to a given address. Amazon's send_email function waits on a response so that function is somewhat time consuming. It's behavior would be independent of whether it is called in a standard or asynchronous fashion would it not? At least it works that way with my threading example.


This is actually the crux! Ignoring some tricks that various runtimes do, YES, send_email acts the same whether it is called in a standard way, or an asynchronous way. "the same" here means that it blocks (I'm assuming). That function must ALSO be written in an asynchronous fashion - it has to, inside its inner workings, know when it's waiting on IO and call the equivalent of "yield" to let another async function take over for a while. If it doesn't, then calling it async does nothing to actually make it async. A naive algorithm that they'd have to do is something like:

while bytesLeft:
  bufferFull = attemptPushBytes
  if bufferFull:
    yield



They probably do:
while bytesLeft:
  pushBytes




where pushBytes would block if the buffer were full.

This is actually a huge problem in Java-land. There is decent support for asynchronous-style programming, but most clients and drivers end up blocking somewhere (particularly database drivers), so you can't write a fully async program. You can cheat by doing what you did - hand off work to a dedicated thread pool.
Was This Post Helpful? 1
  • +
  • -

#8 ZOMBIE!!!   User is offline

  • D.I.C Head

Reputation: 28
  • View blog
  • Posts: 216
  • Joined: 28-October 09

Re: Asyncio : Program not running in parallel

Posted 25 June 2020 - 11:15 AM

That makes a lot of sense, thank you for the explanation!
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1