Asynchronous Servers & Django 3.1: An Explainer
Intuitive understanding of asynchronous servers and APIs
I love talking about asynchronicity quite a lot. In one of my previous issues, I highlighted the importance of learning async programming to scale up your applications.
The concept of asynchronicity, where things are executed flawlessly, is quite fascinating to me. We can be more efficient and get more things done this way. Async is a way of life!
Coming back to technology.
Have you heard of the C10k problem? It’s a pretty interesting one.
In the early 2000s, when RAMs were really expensive, there was this theory that servers can only serve upto 10,000 simultaneous requests. This is not 10k requests per second, mind that. But 10,000 requests processed simultaneously.
There are multiple factors to it.
One was that for every request, a thread(a very small process) is allocated and each of them consumes 1MB memory. Given the size of RAMs those days and considering the overhead of context switching, 10k was the magic number which came up.
Ryan Dahl, the game changer 🔥
Enter 2009, Ryan Dahl, an amazing amazing programmer, asked a very serious question which changed our ways of running web servers.
He asked “What are the threads doing while they are serving database queries and making external API calls?”. The answer is “they are sitting idle”. He argued that this idle time can be utilised to serve other requests.
Wow! Amazing!
He then went on to create NodeJS, which is a single threaded server, following which other frameworks also started shifting to single threaded architectures.
What is Asynchronous execution in a server?
The concept of asynchronous programming is general to programming but here we will talk only about servers and APIs.
The core idea behind asynchronous programming is that a single thread is responsible for executing multiple requests or functions.
This is counter intuitive. How can one single thread be more efficient than multiple threads?
For this, let’s blame our poor understanding of how a request works.
When a user makes a request, the underlying API code doesn’t take a lot of time to execute assuming we are using standard libraries which are written in C.
The program communicates with the database and asks for a query. Now, it’s the database which does the rest of the processing i.e. there’s a separate process or thread created at the DB’s end which queries the tables and returns the result.
So while I say that the thread executing the code on the server is idle, this is what I mean!
Okay, so what’s async?
When the thread on our server is waiting for the DB process to return the result, it can do other things like serve other requests! (Basically get the most bang for your buck from those AWS guys 😎)
Why are threads not the best option here? As mentioned in the section above, there are limits to the number of threads you can spin up because of their memory consumption. If you only have one thread executing all requests then that’s pretty cool, right!
Where async fails?
Let’s take another scenario. You are exposing an API, which serves a deep learning model. Your benchmarking shows that it takes more than a minute to complete the request because it’s a hell lot of calculation that your CPU has to do to achieve the result.
Can async help here? No!
Async only helps in situations where CPU consumption is minimal and where your API is waiting on external calls like DB queries and third party API requests.
In the above scenario, threads or multiprocessing will be a better alternative.
How async works in Django?
Async is only possible if your framework as well as your HTTP server program (like gunicorn, uvicorn) should implement this feature explicitly.
Let’s take a case where you are serving a Django application on a Gunicorn server.
In Gunicorn, the process which handles your requests is termed as a worker. Gunicorn has multiple worker types which you can choose from. The choice depends on how you want to serve your requests. It can be a threaded implementation. Or it can be asynchronous implementation. Gunicorn along with Uvicorn gives you async support. So that’s on the HTTP server end!
Now comes the framework. Earlier if you had Django, every request is executed using a thread. But with the release of Django 3.1, you can change this behavior.
Django 3.1 introduced async views. Now, if you want, you can write your views such that your service can take advantage of the asynchronicity mentioned above.
You wouldn’t really notice any difference between sync and async in your application until you reach a certain scale. Once you cross a certain number of simultaneous users, your application starts to slow down and requests will start taking more time. That’s when async’s advantages start to kick in.
If you are building for scale, adding async is gonna impact your application on a whole another level. Also async allows you to use slow streaming, long-polling, and other response types.
To learn more and get code samples, you can visit Django’s official documentation .
Read recommendations
Pysa: An open source static analysis tool to detect and prevent security issues in Python code
How Khan Academy Successfully Handled 2.5x Traffic in a Week
References:
c10k problem: https://en.wikipedia.org/wiki/C10k_problem
Django async views, explanation with code:
If you liked my newsletter, please share it with your friends! It would mean the world to me
Connect with me!
You can drop me a “Hi” on Twitter ! Let’s talk tech, my DMs are open for you 🤓
If you need help with your service architecture, you can also email me 💌 : dennysam14@gmail.com
Kudos to your amazing inputs always! Keep Shining and spreading your invaluable insights!! :))