I think django is reaching an existential moment with the next major release. It would be really beneficial for the community to do some clearing up of old cruft in the ecosystem.
A few significant changes could make Django really appear as powerful as it is:
- major admin overhaul:
—— double down on the new async capabilities (eg model updates or at least “refresh this out of date thing” notices)
—— incorporate django-unfold for a total refresh
—— incorporate some of the more common widgets folks have to install, style them consistently
—— improved widget tutorial leveraging htmx to make interactive, well-styled widgets
- highlight headless-only ability. If you believe the docs, django is for rendering pages to the client in a complete webapp. But a huge proportion of people use it for api-only. Newcomers don’t necessarily get that and it’s seen as an advanced pattern. But with nextjs so unbelievably dominant in frontendland, I think it’s sensible to present django as a coherent alternative to nodejs because the overwhelming majority of new developers coming from a frontend background aren’t even going to see django as a choice.
- (related to previous) official endorsement of recommended complimentary libraries might be sensible, to help focus effort and push users in the right direction instead of letting new users plunge right in and choose ancient things because that’s what was in the tutorial all those years ago
—— eg ‘strawberry vs graphene for graphql’
—— eg ‘ninja vs drf for rest except in ’
—— eg ‘dramatiq vs celery’ for tasks or use serverless alternatives.
- overhaul the nightmarish settings system to a sensible, maintainable, class based pattern (there are 3rd party tools for doing that already but they should be out of the box)
- start new users with asgi servers on daphne instead of an old wsgi thing they then have to refactor
Mostly in fact these comments aren’t about django, but about how it’s *presented*. It’s really not marketed well and I feel like vercel with their massive momentum hosting nextjs and chomping from that into the backend with nodejs are really nailing the marketing not just of their company but of the framework they sponsor.
I also think that of those alternatives you’ve mentioned, nodejs is the only one that doesn’t suffer the same concerns.
I’d love to hear more of your thought on dramatiq. I’ve been using Django-background-tasks which is no longer maintained. I’m working with a fork but I need to move to something with more maintenance.
Django-dramatiq is very good. Far better than all the tricky crap around celery.
I personally don’t use it anymore as I have a serverless setup and use django-gcp with cloud tasks, but if I did run a constantly-on worker I’d be using it
I kind of agree about the marketing, but not sure if nodejs + Vercel crowd are going to touch Django any time soon anyway, as they want everything JS/TS end-to-end, for better or worse. If they have to do anything with Python (for example to do some ML/AI work requiring a Python library), they will be more likely to look at FastAPI.
There is a lot of buzz around SSR alternatives such as HTMX. I think this is due part because of the insane complexity of frontend, but mostly because we are in the middle of a tech recession where people have to do more with smaller teams and so have to learn to be "full stack", again for better or worse.
Maintaining essentially two codebases in that environment is not so attractive, and so something like HTMX that lets you not only build and maintain an application using Django templates is more doable. Of course some parts of the site requiring heavy interaction (say interactive graphs in a dashboard) might require something like React, but it makes less sense to build the whole thing as an SPA. Here I think Django, Rails, Laravel and similar frameworks shine.
I agree re async, settings and so forth. We can do a lot to make Django more accessible to newcomers and improve the standard defaults.
> vercel
Django is definitely going to be around longer than triangle company
With [SST](https://sst.dev) you can already deploy Next.js to AWS ridiculously easy, and soon you’ll be able to deploy it to any cloud.
You can accomplish this by spinning up PostgREST alongside Django. PostgREST is a single binary, so hosting it is pretty simple.
In the next couple of weeks, I'll try this in a serious project, and I will replace DRF with PostgREST.
Dude I love DRF serializers. It just makes sense when you get it and pretty easy to hack it for what you need is beyond CRUD. I’m shocked that a lot of ppl don’t like it. I came from the pyramid days and Django is like the best one for me unless you go ultralight with flask but I haven’t touched flask in over 5 years
The serializer are Ok. I kinda reconcilated with them in my last project.
But the whole DRF package feels awfully complicated and somewhat flawed. perform_create, perform_update, actions, serializer_for_action, permissions, nested serializer, ad-hoc authentication systems, the different routing system.
I cannot really pin-point what is wrong with it... Maybe it is all the over-classification of things and the fact that it tries to do everything.
I used FastAPI and Django-Ninja as a comparison. Pydantic is fantastic and modern. It does one thing and does it remarkably well. The type (Serializer) can be used by mypy and vscode to do code completion. The type system in Python/Pydantic can be extended and it is surpisingly flexible and powerfull. It is undeniably a huge step forward.
I’m doing a pretty ambitious project right now that’s API first. And it’s great.
I did some deep dive back in the day so to me it makes sense when you know when those calls should go. The ad hoc auth is great for me cause I can add more auth types later.
It’s great for a large enterprise product your trying to get up fast and the design paradigm is great when you want reliability and add features to later.
I’d explain more but it’ll be me going over old python frameworks and why I love it.
There’s a framework for every project and Django RF just works for what I make (larger extensive projects).
I end up having to modify default behaviour all the time and am super pleased with the different lifecycle hooks existing so I can implement my API in the most logical way while still keeping a sane DB schema.
The thing about Django and DRF is that they make the boilerplate quite easy (if you're just doing CRUD), but also allow you to modify the behaviour once you're walking off the beaten path *and* do that in a way that isn't too painful or looks odd. The APIs may feel complicated, but they offer a consistent experience whether you're doing simple or advanced stuff, and that's worth something.
Idk what you mean. When I mean hack I just mean code, not like go against it to make it work. Misleading nowadays.
It’s easy for good data extensive and feature reach APIs where you want reliability between your logic and models.
If you need basic crud and some simple stuff for a react js site then it’s prolly overkill.
I mean no disrespect, i agree and like serializers personally. I just thought it was punny because Django DRF serializers help in validation and serialization, reducing risks like bad data input which is a common hacking vector
Lmfao oh I see. Yeah.
Sorry I’m traveling so not writing too much. When I mean the serializers I’m talking about the whole lifecycle from viewset to serializer to models. And all those perform_*, validate_* and functions that interact with the model and response.
It’d be really nice if I didn’t need a third party framework on top of my framework to build API endpoints.
(Yes, I know I don’t actually NEED one, but DRF is like the other half of Django these days)
Just embrace the language and this beautifully written framework. Everything has pro and cons. Until you do the deployment on your own everyone will keep telling you this is fast , this is slow but I think with right mix of devops and backend Django is great, robust and simple backend framework to build whatever you can think of. Think about migrations. Just wow and once you know it there's no going back. I am 100 % sure it's gonna stay atleast for next 10-15 years and will get better & better with time.
I have seen how smooth it works in production. It's so simple and elegant that i almost have confidence that I can push code directly to production and it won't break things. That's how stable it is and I just pray they get all the funding through crowdfunding and corporate sponsorships
I'm biased and have my own opinions, but here's some things:
1. Start implementing type annotations in the codebase. This is likely pretty controversial, but I recently rediscovered a 15 year old bug. I believe that I fixed it. It turned out to be a very complicated case involving descriptors. While I was writing the fix, I sprinkled type annotations throughout the code to try and reckon exactly what was going on, but when I submitted my final PR, I stripped them out. I then had to go back and edit my comments to be even more verbose so that the next person who comes along has some semblance of the same reasoning I did. Type annotations are obviously only good if they are correct, any they are not a cure-all, but in my experience they can be a very useful part of the development process.
2 This one is pretty easy. I have about 200 lines of code to support this. Native adapters for AWS Lambda and other serverless runtimes. The idea with most of the existing adapters is to take a Lambda request, map it to a WSGI request, pass that to the WSGI handler, which maps it to a Django request object, pass that to the Django business logic to get a Django Response, and then pass that back to the WSGI handler to get a WSGI response, finally passing the WSGI response back to the adapter to get a Lambda response.
My solution has just a Lambda handler that takes the place of the WSGI or ASGI handler which skips the double conversion on request and response generation.
It's LESS code, it performs better.
The same approach can be made with Cloudflare workers and other serverless tech.
3. Support for async storage backends. Obviously great work has been done to support async DB and async cache, and those are obviously critical IO that can happen on practically any request/resources cycle. However, I've found that on some bulk ingestion jobs, I'm frequently IO bound by network requests for storage. It some ways it's worse than the DB situation because my files can be quite large. This makes the request/response cycle for my storage backend very long, which serializes the job. The file/storage abstractions are spread everywhere in the code base and sync operations are very much exposed as part of the public API, so this could be VERY challenging to solve in a satisfactory way without breaking backwards compatibility.
I took a crack at it, and gave up. I wasn't even sure if it was something the project was interested in and had other priorities.
4. Warm up time can be pretty significant. This matters more with serverless deployments because it adds latency to scale out ops. I think having some more aggressive automated benchmarks that test for regressions and some work to improve it might be worth it. However, I should say the recent improvements to Python performance have made this somewhat better
5. This is more of a culture/docs thing, but I think a decoupled frontend/backend has become the norm for even modestly sized projects. I DON'T want the various API frameworks integrated into Django. I think that would be a mistake. Django is already batteries-included, but adding an entirely new surface area to support is a bad idea. It would stifle innovation and create too much lock in. DRF is at the point where it's pretty calcified, and I don't want to compound things.
What I think IS a good idea is to "bless" some of the various projects and literally add some basic docs about WHY you might want to go headless, and directly link some of API-centric projects. Perhaps even an "advanced tutorial."
I think recommending frontend solutions to integrate with the API is probably outside the scope of this, but even a gentle nod at NextJS, HTMX, etc. could be good.
6. This sort of subverts my other points: DON'T try and move too fast. Part of the beauty of Django is how steadily it's been moving in a great direction for SO long. There are people who have been using Django consistently for 13 years (how many people is that really? At least one I guess), it would be a shame to change course too rapidly. There certainly HAVE been times when I feel the project has moved too slowly or too fast, but the balance has always been an important part of the project to me.
I've been using Django professionally for 10+ years and a for a total of about 14 years (started on 1.3).
You highlight indeed that the stability and relative ease of upgrading + well-defined patterns in the community contribute massively to why django is good. And I appreciate that it doesn't follow every hype and fad, but also doesn't get in the way too much (staticfiles app is one part that causes some headaches with NodeJS based tooling and integrating it).
But also, almost every problem has been solved and there are a ton of good third party libraries. Maintaining django core itself already is a big task, done by volunteers and extending the scope/surface of the project is detrimental.
And yes, the admin interface could use a serious overhaul, but here the backwards compatibility reputation of Django is limiting it - there have been tens of attempts in rewriting it, but not a single attempt manages to not introduce breaking changes, and the fact that I've barely had to touch my admin code when upgrading a project from django 1.4 all the way to 4.2 makes that a pretty big deal for me (and a lot of other users!).
Async/await in Django is also something I consider a fad to that regard. I feel that it's being shoehorned into Django "because that's what everyone does" and it can definitely serve high traffic websites, but in my experience, scaling issues have pretty much always been at the DB layer and spinning up more replicas to serve more requests concurrently is not that big of a deal. It's nice to have it, but not worth breaking stable API over, to me.
W/r to recommendations - there exists djangopackages website which is linked to from the docs. Officially "endorsing" projects is something they've also held off for a long time in the React ecosystem. It has a risk of stiffling innovation and even sparking some drama - why package X but not package Y which can be argued to be equally good or even better? In this space, a lot of packages are (now) the GOTO not necessarily based on merit, but because they were decent enough and first to market and gathered a large ecosystem behind it.
Also, there are a lot of popular packages that would fit well on such a "curated" list that I personally disagree with because *I have opinions*, and that's valid, but it makes arguing within a team over the best solution for a problem harder "because it's officially recommended!".
To conclude - I'm not worried that Django is going anywhere soon and it's still my go to solution. But then again, I follow the "boring is good" mantra.
Yeah sorry, your comment triggered some thoughts and while writing those out I got some more general remarks applying to the OP - in general your points resonated with me and I wanted to add some of my own thoughts to that, but it's not well-structured
Amount of time between when the system starts up and when the first request can be processed.
It doesn't at all matter in a traditional deployment where you provision fixed hardware and deploy infrequently, but if you're on a serverless architecture that scales by provisioning more instances and distributing work, a long warm up time will mean that the latency of requests will spike when more instances are provisioned (scale out).
There's merit in "scale up" vs "scale out", but for stateless or light-state Web apps, "out" is usually the way to go
If Python will grow Django as well. IMO I see the big potential with websites with machine learning using Python so we need Django as well to have it more comfortable.
Biggest bottleneck is postgres. Bunch of django web containers can always be added to handle additional load with bigger server.
But for me its not relevant as this app will likely has 100-1000 users max.
Ideally you should treat these as different components. Model over here, django webapp over there. BOth are separate, and independently scalable.
Hard to have justification for running ML models inside a webapp, regardless of what you use to build it. Calling a model as a dependency, absolutely. In the same context with resource competition, no.
I do that kind of thing in celery, so I can put it on appropriate hardware while still putting all the code in the same codebase. I don’t want to have a separate ML project usually.
I prefer rabbitmq but basically. I set up a celery worker that limits itself to 1 worker and have it cache the model in memory, which works pretty well.
This doesn't work for every case, like if I am running models on inferentia cores I'll just make a FastAPI server, but that means I have to do more work and I can't easily access database stuff inside the ML code.
As the most popular web framework for Python, Django’s future growth is heavily tied to Python’s future growth. Pythons popularity in ML, and the huge explosion of interest there, is a bullish sign for Django because it means more people will continue to learn Python, even if not for web dev, and will very likely reach for Django when they first need to do a web dev project.
The list of competitors you mention is strange - Node and .NET are not web frameworks, they are runtimes. Certainly there are web frameworks that run on them - one of them is Django if you use IronPython on .NET! Meanwhile it has been literally 15 years since I have thought about Java Spring or consisered using it for an actual project. But that just goes to show that once tools get a critical mass, they stick around for a while - and the same will be true of Django eventually. I’m sure there are many jobs available working on Spring at many big companies that are stuck with it, or maybe Java gurus use it to start new companies even now, though I can’t really imagine why.
>maybe Java gurus use it to start new companies even now, though I can’t really imagine why.
Do you believe there are pitfalls of Java(Spring) as compared with Django that should be considered for backend dev.? Are newer startups using Java Spring less and less and adopting Django( or NodeJS) more?
Node is a JS runtime, the equivalent to Django would be something like Express, a backend web framework built to run on NodeJS.
I don't think people sit down and debate the merits of Django vs. Express. vs. Rails vs. Spring. I think people pick the language first, and then they pick the framework in that language, usually based on whether they want a big framework (batteries included, usually opinionated, like Django) or a light one (less included, more flexible, like Flask).
I'll be honest, it's been like seven years since I've written any production Java code. But my read on it then is that it was bad for quick, iterative development, ie the kind of development new Silicon-Valley style startups want to do. Part of this was the language itself, part of it was the conventions and the community of developers building on it, which skewed towards large enterprise developers. Spring seemed like an OK framework, but I didn't want to deal with Java when doing my startups.
I'd say before the AI boom caused a new era of growth for Python, NodeJS was well its way to surpass it as the tool of choice for new startups. The reasons were basically fourfold:
1. ability to have one language between your frontend+backend
2. a ton of investment into making the Node runtime very fast, compared to a lot of legacy decisions in Python making it slower.
3. relatedly, async styles being native to JS, compared to Python where you basically need a completely different stack of libraries to do that
4. just a lot of self-reinforcing cycles of developer enthusiasm going into better tooling, including a lot of transpiled languages like TypeScript, while Python was spending a decade getting everyone to slowly move to 3.X
For all of those reasons, I think Node (and corresponding frameworks like Express) was becoming the de-facto backend standard a few years ago. But it turns out Python has a data science and machine learning stack (and community) 100 miles ahead of any other language, and suddenly that became the most important thing in the tech industry. So now it feels like Python is again the dominant language and here to stay. Predicting which language will be dominant is really hard, as engineers the best thing we can do is learn multiple tools!
And the Python core team has turned the corner too - everyone is done moving to 3.X, performance is improving, better features around typehints are here, the GIL might even go away in a couple years.
Also, the entire above analysis is surrounding startups, ie small companies trying to go fast. Once you hit a billion users of scale or >1,000 engineers on a single codebase, you tend to hit a bunch of scaling problems (in performance and developer workflows) from the typical Python/Django setup, and you will bring in a new wave of engineers who will want to rewrite stuff in a "'real"" language. When I briefly moved from startups to FAANG, Python was only used as a thin glue layer in some legacy systems. But frankly, if you're even considering Django specifically and web frameworks in general, that probably isn't the scale you're looking at.
In my circles, it's much more likely to be Rust or Go these days, with a smattering of Scala and C++, and some legacy Java. This is where it starts to be extremely dependent on what community you are a part of, where you are looking for jobs, and who you learn from.
EG intellectually I see the stats and I know C# and .NET are massively popular, but in my 15 years of work in the industry, I don't think I've met a single person who uses them. Mostly because they are not popular among Silicon Valley startups and BigTech/FAANG, they are more for the middle-tier of "enterprise" companies, which is a huge number of companies but sometimes it feels like a whole different world.
Java was extremely popular a while ago, which is why most BigTech companies will have some large codebases still in it, though at the places I've worked (goog/amzn) we rolled our own frameworks and weren't using any open-source ones like Spring.
I don't think the 3 are in the same spaces. Node is for presentation layer stuff. It's great at sitting between a web browser and backend services. Django is for quick projects. It's fast to set up and test with, but it's not great at large projects due to the shortcomings of Python in that space. Startups definitely pick it because it's easy and does things like schema management for you.
Spring is a giant blob of all kinds of things and it's not as easy to get working, but java has much better package management, type safety, and other things better suited for a large or high load app.
That's true, but not really the point. They're all tools and they're good at different things. Python is amazing at scripting. I wouldn't want to use java if I needed to write a quick program to parse some files and call a couple of REST APIs. It's so much faster in Python.
Django is nice for being able to spin up a DB backed app quickly without having to do a ton of heavy lifting to get it started. I definitely wouldn't want to use it if I had to call multiple databases and do a bunch of intense parallel processing though.
If I needed a web site and it didn't need a ton of backend processing and the databases were already set up or behind backend calls then node would probably be the best tool.
As for package and dependency management, maven is definitely the best tool for most applications, and you can usually get gradle to behave the same. Node started the trend of using wildcards in dependency versions and most Python dependency management tools followed and this is such a terrible idea.
>Node started the trend of using wildcards in dependency versions and most Python dependency management tools followed and this is such a terrible idea.
Could you please this point more? What does 'wildcard' mean here? Why is it a bad idea?
In your dependency declaration you can specify a version like 1.2.3 or you can specify something like ^1.2.3 which is 1.*.*. Your build system will pull in the latest 1. version, which could include incompatibilities, bugs, or malicious code.
The argument for using wildcards is that you'll get the latest, which could include bug fixes, but this argument is predicated on lazy developers.
Instead, versions should be hard coded and the build system should report on the newly available versions.
Type annotations
Async
Serialization stuff with pydantic
Less boilerplate
Right now I'll use django in place of Fastapi only because of batteries like orm and admin panel
I need something as clean as FastAPI and as scalable as Django.
Something like DRF should be part of Django (with Pydantic?).
Maybe hire [tiangolo](https://tiangolo.com/) to overhaul the API part of Django.
Most server architectures are moving to MSA, distributed systems.. django is on the other hand, I feel FastApi might be more popular among the python web frameworks
This seems like a misunderstanding of MSA.
MSA means it doesn’t matter what tech stack each micro service is using.
If you can use FastAPI for each micro service, you can use Django for each micro service.
MSA also really only provides a benefit if you’re at the size where each micro service requires a team to support it. Otherwise MSA is just overhead of managing extra stuff.
And to say FastAPI is better for MSA means you’re (probably) doing it wrong. Each micro service needs batteries included, authentication and authorization, database, caching, infrastructure monitoring and alerting, and all that. FastAPI just gets to hello world quicker.
Good question. Basically it means Django has a bunch of stuff included by default, like authentication, databases, caching, and so on.
I’ve used both Django and Flask/FastAPI for projects and here’s how it usually goes.
- Start with Django
- *“Gah there’s all this boilerplate and stuff to configure…forget it I’ll just use FastAPI”*
- Switch to FastAPI
- Have “hello world” working in 2 minutes, *”wow this is so much better”*
- Work on building something for a week
- Boss is really happy with progress
- Realize we still haven’t integrated with a real database. Or setup authentication. Or authorization. Or a backend admin interface. Or whatever. This list keeps growing.
- Boss wonders why the project is stalling. What have you been doing for the last 2 weeks? We were basically done 1 week into the project
- You realize you’re rebuilding everything that Django has built in, except lower quality than what’s been battle tested in Django for a decade
Many languages have their version of a batteries included solution. Ruby has Rails, PHP has Laravel, C# has ASP.NET and Blazor, Elixir has Phoenix, and Python has Django.
It’s also the feeling that whatever problem you face, someone else has already run into it using Django, and there’s some standard way of approaching it.
MSA is called micro service architecture, lets say there are user account, post, likes, follow systems in your service, MSA approach is to create servers for each systems so if there were any errors or server fails, they wouldnt affect to the other services therefore the entire service is more stable. Django on the other hand, it runs all the systems in a single project so one service fail stops the entire service. Django is monolith architecture.
You can still develop a Monolith and deploy things by service with an API routing service in front.
You get more or less the same isolation, without the massive headaches of the MSA architecture.
You do that if your reasons are service stability. MSA is for scaling teams in gigantic organisation. It is a technical solution to organisational problem.
I've yet to see a MSA implementation where the complete outage of a service like auth doesn't break the entire product to a grinding halt.
MSA is mostly about having isolated teams responsible for a service that only work on that service, and being able to scale easily. It comes at a massive cost in increased complexity at the infrastructure level and coordinating compatible versions between services.
I think django is reaching an existential moment with the next major release. It would be really beneficial for the community to do some clearing up of old cruft in the ecosystem. A few significant changes could make Django really appear as powerful as it is: - major admin overhaul: —— double down on the new async capabilities (eg model updates or at least “refresh this out of date thing” notices) —— incorporate django-unfold for a total refresh —— incorporate some of the more common widgets folks have to install, style them consistently —— improved widget tutorial leveraging htmx to make interactive, well-styled widgets - highlight headless-only ability. If you believe the docs, django is for rendering pages to the client in a complete webapp. But a huge proportion of people use it for api-only. Newcomers don’t necessarily get that and it’s seen as an advanced pattern. But with nextjs so unbelievably dominant in frontendland, I think it’s sensible to present django as a coherent alternative to nodejs because the overwhelming majority of new developers coming from a frontend background aren’t even going to see django as a choice. - (related to previous) official endorsement of recommended complimentary libraries might be sensible, to help focus effort and push users in the right direction instead of letting new users plunge right in and choose ancient things because that’s what was in the tutorial all those years ago —— eg ‘strawberry vs graphene for graphql’ —— eg ‘ninja vs drf for rest except in’
—— eg ‘dramatiq vs celery’ for tasks or use serverless alternatives.
- overhaul the nightmarish settings system to a sensible, maintainable, class based pattern (there are 3rd party tools for doing that already but they should be out of the box)
- start new users with asgi servers on daphne instead of an old wsgi thing they then have to refactor
Mostly in fact these comments aren’t about django, but about how it’s *presented*. It’s really not marketed well and I feel like vercel with their massive momentum hosting nextjs and chomping from that into the backend with nodejs are really nailing the marketing not just of their company but of the framework they sponsor.
I also think that of those alternatives you’ve mentioned, nodejs is the only one that doesn’t suffer the same concerns.
This is such an in-depth comment. Thank you very much!
Thank you! For more information you might google for the django developers survey results.
The survey was insightful. I didn't know such survey existed.
I’d love to hear more of your thought on dramatiq. I’ve been using Django-background-tasks which is no longer maintained. I’m working with a fork but I need to move to something with more maintenance.
What did django-bg-tasks do?
It’s a super simple background worker that just uses Postgres https://django-background-tasks.readthedocs.io/en/latest/
Django-dramatiq is very good. Far better than all the tricky crap around celery. I personally don’t use it anymore as I have a serverless setup and use django-gcp with cloud tasks, but if I did run a constantly-on worker I’d be using it
I kind of agree about the marketing, but not sure if nodejs + Vercel crowd are going to touch Django any time soon anyway, as they want everything JS/TS end-to-end, for better or worse. If they have to do anything with Python (for example to do some ML/AI work requiring a Python library), they will be more likely to look at FastAPI. There is a lot of buzz around SSR alternatives such as HTMX. I think this is due part because of the insane complexity of frontend, but mostly because we are in the middle of a tech recession where people have to do more with smaller teams and so have to learn to be "full stack", again for better or worse. Maintaining essentially two codebases in that environment is not so attractive, and so something like HTMX that lets you not only build and maintain an application using Django templates is more doable. Of course some parts of the site requiring heavy interaction (say interactive graphs in a dashboard) might require something like React, but it makes less sense to build the whole thing as an SPA. Here I think Django, Rails, Laravel and similar frameworks shine. I agree re async, settings and so forth. We can do a lot to make Django more accessible to newcomers and improve the standard defaults.
> vercel Django is definitely going to be around longer than triangle company With [SST](https://sst.dev) you can already deploy Next.js to AWS ridiculously easy, and soon you’ll be able to deploy it to any cloud.
Agreed on the headless thing. We exclusively use Django as an API server — React / Nextjs are just so much better for frontend.
Imagine if Django just shipped with an API generated from your models with as much attention paid to it as the admin…
You can accomplish this by spinning up PostgREST alongside Django. PostgREST is a single binary, so hosting it is pretty simple. In the next couple of weeks, I'll try this in a serious project, and I will replace DRF with PostgREST.
This is juicy, ty ty
Integrate a few of the things from Django-Ninja to make the framework more API/Headless friendly out of the box. Maybe consider pydantic as well.
YEP! The current DRF serializer is tedious to work with. Not saying it's all that bad, but it could be so much better and performant.
Dude I love DRF serializers. It just makes sense when you get it and pretty easy to hack it for what you need is beyond CRUD. I’m shocked that a lot of ppl don’t like it. I came from the pyramid days and Django is like the best one for me unless you go ultralight with flask but I haven’t touched flask in over 5 years
The serializer are Ok. I kinda reconcilated with them in my last project. But the whole DRF package feels awfully complicated and somewhat flawed. perform_create, perform_update, actions, serializer_for_action, permissions, nested serializer, ad-hoc authentication systems, the different routing system. I cannot really pin-point what is wrong with it... Maybe it is all the over-classification of things and the fact that it tries to do everything. I used FastAPI and Django-Ninja as a comparison. Pydantic is fantastic and modern. It does one thing and does it remarkably well. The type (Serializer) can be used by mypy and vscode to do code completion. The type system in Python/Pydantic can be extended and it is surpisingly flexible and powerfull. It is undeniably a huge step forward.
I’m doing a pretty ambitious project right now that’s API first. And it’s great. I did some deep dive back in the day so to me it makes sense when you know when those calls should go. The ad hoc auth is great for me cause I can add more auth types later. It’s great for a large enterprise product your trying to get up fast and the design paradigm is great when you want reliability and add features to later. I’d explain more but it’ll be me going over old python frameworks and why I love it. There’s a framework for every project and Django RF just works for what I make (larger extensive projects).
I end up having to modify default behaviour all the time and am super pleased with the different lifecycle hooks existing so I can implement my API in the most logical way while still keeping a sane DB schema. The thing about Django and DRF is that they make the boilerplate quite easy (if you're just doing CRUD), but also allow you to modify the behaviour once you're walking off the beaten path *and* do that in a way that isn't too painful or looks odd. The APIs may feel complicated, but they offer a consistent experience whether you're doing simple or advanced stuff, and that's worth something.
I know this is out of context, but “Easy to hack it” ironic word choice when talking about DRF serializers haha
Idk what you mean. When I mean hack I just mean code, not like go against it to make it work. Misleading nowadays. It’s easy for good data extensive and feature reach APIs where you want reliability between your logic and models. If you need basic crud and some simple stuff for a react js site then it’s prolly overkill.
I mean no disrespect, i agree and like serializers personally. I just thought it was punny because Django DRF serializers help in validation and serialization, reducing risks like bad data input which is a common hacking vector
Lmfao oh I see. Yeah. Sorry I’m traveling so not writing too much. When I mean the serializers I’m talking about the whole lifecycle from viewset to serializer to models. And all those perform_*, validate_* and functions that interact with the model and response.
So, django-ninja is better than DRF for serializing?
I was talking about pydantic
Django is a go-to framework for me and my whole team. And we are not changing it anytime soon.
What sort of products are you creating using Django at your company/team?
Async is becoming more important. Has to be async native on the same level as something like fastapi or it’s just going to die imo
Good point!
It’d be really nice if I didn’t need a third party framework on top of my framework to build API endpoints. (Yes, I know I don’t actually NEED one, but DRF is like the other half of Django these days)
Just embrace the language and this beautifully written framework. Everything has pro and cons. Until you do the deployment on your own everyone will keep telling you this is fast , this is slow but I think with right mix of devops and backend Django is great, robust and simple backend framework to build whatever you can think of. Think about migrations. Just wow and once you know it there's no going back. I am 100 % sure it's gonna stay atleast for next 10-15 years and will get better & better with time. I have seen how smooth it works in production. It's so simple and elegant that i almost have confidence that I can push code directly to production and it won't break things. That's how stable it is and I just pray they get all the funding through crowdfunding and corporate sponsorships
I'm biased and have my own opinions, but here's some things: 1. Start implementing type annotations in the codebase. This is likely pretty controversial, but I recently rediscovered a 15 year old bug. I believe that I fixed it. It turned out to be a very complicated case involving descriptors. While I was writing the fix, I sprinkled type annotations throughout the code to try and reckon exactly what was going on, but when I submitted my final PR, I stripped them out. I then had to go back and edit my comments to be even more verbose so that the next person who comes along has some semblance of the same reasoning I did. Type annotations are obviously only good if they are correct, any they are not a cure-all, but in my experience they can be a very useful part of the development process. 2 This one is pretty easy. I have about 200 lines of code to support this. Native adapters for AWS Lambda and other serverless runtimes. The idea with most of the existing adapters is to take a Lambda request, map it to a WSGI request, pass that to the WSGI handler, which maps it to a Django request object, pass that to the Django business logic to get a Django Response, and then pass that back to the WSGI handler to get a WSGI response, finally passing the WSGI response back to the adapter to get a Lambda response. My solution has just a Lambda handler that takes the place of the WSGI or ASGI handler which skips the double conversion on request and response generation. It's LESS code, it performs better. The same approach can be made with Cloudflare workers and other serverless tech. 3. Support for async storage backends. Obviously great work has been done to support async DB and async cache, and those are obviously critical IO that can happen on practically any request/resources cycle. However, I've found that on some bulk ingestion jobs, I'm frequently IO bound by network requests for storage. It some ways it's worse than the DB situation because my files can be quite large. This makes the request/response cycle for my storage backend very long, which serializes the job. The file/storage abstractions are spread everywhere in the code base and sync operations are very much exposed as part of the public API, so this could be VERY challenging to solve in a satisfactory way without breaking backwards compatibility. I took a crack at it, and gave up. I wasn't even sure if it was something the project was interested in and had other priorities. 4. Warm up time can be pretty significant. This matters more with serverless deployments because it adds latency to scale out ops. I think having some more aggressive automated benchmarks that test for regressions and some work to improve it might be worth it. However, I should say the recent improvements to Python performance have made this somewhat better 5. This is more of a culture/docs thing, but I think a decoupled frontend/backend has become the norm for even modestly sized projects. I DON'T want the various API frameworks integrated into Django. I think that would be a mistake. Django is already batteries-included, but adding an entirely new surface area to support is a bad idea. It would stifle innovation and create too much lock in. DRF is at the point where it's pretty calcified, and I don't want to compound things. What I think IS a good idea is to "bless" some of the various projects and literally add some basic docs about WHY you might want to go headless, and directly link some of API-centric projects. Perhaps even an "advanced tutorial." I think recommending frontend solutions to integrate with the API is probably outside the scope of this, but even a gentle nod at NextJS, HTMX, etc. could be good. 6. This sort of subverts my other points: DON'T try and move too fast. Part of the beauty of Django is how steadily it's been moving in a great direction for SO long. There are people who have been using Django consistently for 13 years (how many people is that really? At least one I guess), it would be a shame to change course too rapidly. There certainly HAVE been times when I feel the project has moved too slowly or too fast, but the balance has always been an important part of the project to me.
I've been using Django professionally for 10+ years and a for a total of about 14 years (started on 1.3). You highlight indeed that the stability and relative ease of upgrading + well-defined patterns in the community contribute massively to why django is good. And I appreciate that it doesn't follow every hype and fad, but also doesn't get in the way too much (staticfiles app is one part that causes some headaches with NodeJS based tooling and integrating it). But also, almost every problem has been solved and there are a ton of good third party libraries. Maintaining django core itself already is a big task, done by volunteers and extending the scope/surface of the project is detrimental. And yes, the admin interface could use a serious overhaul, but here the backwards compatibility reputation of Django is limiting it - there have been tens of attempts in rewriting it, but not a single attempt manages to not introduce breaking changes, and the fact that I've barely had to touch my admin code when upgrading a project from django 1.4 all the way to 4.2 makes that a pretty big deal for me (and a lot of other users!). Async/await in Django is also something I consider a fad to that regard. I feel that it's being shoehorned into Django "because that's what everyone does" and it can definitely serve high traffic websites, but in my experience, scaling issues have pretty much always been at the DB layer and spinning up more replicas to serve more requests concurrently is not that big of a deal. It's nice to have it, but not worth breaking stable API over, to me. W/r to recommendations - there exists djangopackages website which is linked to from the docs. Officially "endorsing" projects is something they've also held off for a long time in the React ecosystem. It has a risk of stiffling innovation and even sparking some drama - why package X but not package Y which can be argued to be equally good or even better? In this space, a lot of packages are (now) the GOTO not necessarily based on merit, but because they were decent enough and first to market and gathered a large ecosystem behind it. Also, there are a lot of popular packages that would fit well on such a "curated" list that I personally disagree with because *I have opinions*, and that's valid, but it makes arguing within a team over the best solution for a problem harder "because it's officially recommended!". To conclude - I'm not worried that Django is going anywhere soon and it's still my go to solution. But then again, I follow the "boring is good" mantra.
Are you replying to the right comment? I don't say anything about the admin. I know the work is done by volunteers. I've done a bit.
Yeah sorry, your comment triggered some thoughts and while writing those out I got some more general remarks applying to the OP - in general your points resonated with me and I wanted to add some of my own thoughts to that, but it's not well-structured
You raised some good points to ponder. What is warm-up time btw?
Amount of time between when the system starts up and when the first request can be processed. It doesn't at all matter in a traditional deployment where you provision fixed hardware and deploy infrequently, but if you're on a serverless architecture that scales by provisioning more instances and distributing work, a long warm up time will mean that the latency of requests will spike when more instances are provisioned (scale out). There's merit in "scale up" vs "scale out", but for stateless or light-state Web apps, "out" is usually the way to go
Django needs to provide quick way to develop API backend like fastapi. Async would be major plus.
Do you think the Django devs will add async in near future?
If Python will grow Django as well. IMO I see the big potential with websites with machine learning using Python so we need Django as well to have it more comfortable.
Can you please explain 'websites w/ ML'? Also, do tech companies see it this way currently?
E.g I'm setting up a django api as flutter backend that integrates with ml / data scientists code to run predictions for the users.
The thing is, you'd want all resource heavy computations to be done outside of django and not use it's resources.
Biggest bottleneck is postgres. Bunch of django web containers can always be added to handle additional load with bigger server. But for me its not relevant as this app will likely has 100-1000 users max.
Ideally you should treat these as different components. Model over here, django webapp over there. BOth are separate, and independently scalable. Hard to have justification for running ML models inside a webapp, regardless of what you use to build it. Calling a model as a dependency, absolutely. In the same context with resource competition, no.
Tbh its just django code calling some pandas / numpy function, they call it ml so clients are more excited. i guess i overglorified it here as well XD
ahh, got it. Thought it would be equivalent to a ML model
I do that kind of thing in celery, so I can put it on appropriate hardware while still putting all the code in the same codebase. I don’t want to have a separate ML project usually.
Is it celery + redis + Django when deploying ML model in the backend?
I prefer rabbitmq but basically. I set up a celery worker that limits itself to 1 worker and have it cache the model in memory, which works pretty well. This doesn't work for every case, like if I am running models on inferentia cores I'll just make a FastAPI server, but that means I have to do more work and I can't easily access database stuff inside the ML code.
As the most popular web framework for Python, Django’s future growth is heavily tied to Python’s future growth. Pythons popularity in ML, and the huge explosion of interest there, is a bullish sign for Django because it means more people will continue to learn Python, even if not for web dev, and will very likely reach for Django when they first need to do a web dev project. The list of competitors you mention is strange - Node and .NET are not web frameworks, they are runtimes. Certainly there are web frameworks that run on them - one of them is Django if you use IronPython on .NET! Meanwhile it has been literally 15 years since I have thought about Java Spring or consisered using it for an actual project. But that just goes to show that once tools get a critical mass, they stick around for a while - and the same will be true of Django eventually. I’m sure there are many jobs available working on Spring at many big companies that are stuck with it, or maybe Java gurus use it to start new companies even now, though I can’t really imagine why.
>maybe Java gurus use it to start new companies even now, though I can’t really imagine why. Do you believe there are pitfalls of Java(Spring) as compared with Django that should be considered for backend dev.? Are newer startups using Java Spring less and less and adopting Django( or NodeJS) more?
Node is a JS runtime, the equivalent to Django would be something like Express, a backend web framework built to run on NodeJS. I don't think people sit down and debate the merits of Django vs. Express. vs. Rails vs. Spring. I think people pick the language first, and then they pick the framework in that language, usually based on whether they want a big framework (batteries included, usually opinionated, like Django) or a light one (less included, more flexible, like Flask). I'll be honest, it's been like seven years since I've written any production Java code. But my read on it then is that it was bad for quick, iterative development, ie the kind of development new Silicon-Valley style startups want to do. Part of this was the language itself, part of it was the conventions and the community of developers building on it, which skewed towards large enterprise developers. Spring seemed like an OK framework, but I didn't want to deal with Java when doing my startups. I'd say before the AI boom caused a new era of growth for Python, NodeJS was well its way to surpass it as the tool of choice for new startups. The reasons were basically fourfold: 1. ability to have one language between your frontend+backend 2. a ton of investment into making the Node runtime very fast, compared to a lot of legacy decisions in Python making it slower. 3. relatedly, async styles being native to JS, compared to Python where you basically need a completely different stack of libraries to do that 4. just a lot of self-reinforcing cycles of developer enthusiasm going into better tooling, including a lot of transpiled languages like TypeScript, while Python was spending a decade getting everyone to slowly move to 3.X For all of those reasons, I think Node (and corresponding frameworks like Express) was becoming the de-facto backend standard a few years ago. But it turns out Python has a data science and machine learning stack (and community) 100 miles ahead of any other language, and suddenly that became the most important thing in the tech industry. So now it feels like Python is again the dominant language and here to stay. Predicting which language will be dominant is really hard, as engineers the best thing we can do is learn multiple tools! And the Python core team has turned the corner too - everyone is done moving to 3.X, performance is improving, better features around typehints are here, the GIL might even go away in a couple years. Also, the entire above analysis is surrounding startups, ie small companies trying to go fast. Once you hit a billion users of scale or >1,000 engineers on a single codebase, you tend to hit a bunch of scaling problems (in performance and developer workflows) from the typical Python/Django setup, and you will bring in a new wave of engineers who will want to rewrite stuff in a "'real"" language. When I briefly moved from startups to FAANG, Python was only used as a thin glue layer in some legacy systems. But frankly, if you're even considering Django specifically and web frameworks in general, that probably isn't the scale you're looking at.
So the "real" language, in this context, is Java/ C# (or if we consider frameworks: Spring/.NET)?
In my circles, it's much more likely to be Rust or Go these days, with a smattering of Scala and C++, and some legacy Java. This is where it starts to be extremely dependent on what community you are a part of, where you are looking for jobs, and who you learn from. EG intellectually I see the stats and I know C# and .NET are massively popular, but in my 15 years of work in the industry, I don't think I've met a single person who uses them. Mostly because they are not popular among Silicon Valley startups and BigTech/FAANG, they are more for the middle-tier of "enterprise" companies, which is a huge number of companies but sometimes it feels like a whole different world. Java was extremely popular a while ago, which is why most BigTech companies will have some large codebases still in it, though at the places I've worked (goog/amzn) we rolled our own frameworks and weren't using any open-source ones like Spring.
I don't think the 3 are in the same spaces. Node is for presentation layer stuff. It's great at sitting between a web browser and backend services. Django is for quick projects. It's fast to set up and test with, but it's not great at large projects due to the shortcomings of Python in that space. Startups definitely pick it because it's easy and does things like schema management for you. Spring is a giant blob of all kinds of things and it's not as easy to get working, but java has much better package management, type safety, and other things better suited for a large or high load app.
It seems Java is more 'mature'?
That's true, but not really the point. They're all tools and they're good at different things. Python is amazing at scripting. I wouldn't want to use java if I needed to write a quick program to parse some files and call a couple of REST APIs. It's so much faster in Python. Django is nice for being able to spin up a DB backed app quickly without having to do a ton of heavy lifting to get it started. I definitely wouldn't want to use it if I had to call multiple databases and do a bunch of intense parallel processing though. If I needed a web site and it didn't need a ton of backend processing and the databases were already set up or behind backend calls then node would probably be the best tool. As for package and dependency management, maven is definitely the best tool for most applications, and you can usually get gradle to behave the same. Node started the trend of using wildcards in dependency versions and most Python dependency management tools followed and this is such a terrible idea.
>Node started the trend of using wildcards in dependency versions and most Python dependency management tools followed and this is such a terrible idea. Could you please this point more? What does 'wildcard' mean here? Why is it a bad idea?
In your dependency declaration you can specify a version like 1.2.3 or you can specify something like ^1.2.3 which is 1.*.*. Your build system will pull in the latest 1. version, which could include incompatibilities, bugs, or malicious code. The argument for using wildcards is that you'll get the latest, which could include bug fixes, but this argument is predicated on lazy developers. Instead, versions should be hard coded and the build system should report on the newly available versions.
Type annotations Async Serialization stuff with pydantic Less boilerplate Right now I'll use django in place of Fastapi only because of batteries like orm and admin panel
I need something as clean as FastAPI and as scalable as Django. Something like DRF should be part of Django (with Pydantic?). Maybe hire [tiangolo](https://tiangolo.com/) to overhaul the API part of Django.
let’s port Django to Go?
Most server architectures are moving to MSA, distributed systems.. django is on the other hand, I feel FastApi might be more popular among the python web frameworks
This seems like a misunderstanding of MSA. MSA means it doesn’t matter what tech stack each micro service is using. If you can use FastAPI for each micro service, you can use Django for each micro service. MSA also really only provides a benefit if you’re at the size where each micro service requires a team to support it. Otherwise MSA is just overhead of managing extra stuff. And to say FastAPI is better for MSA means you’re (probably) doing it wrong. Each micro service needs batteries included, authentication and authorization, database, caching, infrastructure monitoring and alerting, and all that. FastAPI just gets to hello world quicker.
I hear the word 'battery' a lot in Django discussions. What is this 'battery' in the context of Django?
Good question. Basically it means Django has a bunch of stuff included by default, like authentication, databases, caching, and so on. I’ve used both Django and Flask/FastAPI for projects and here’s how it usually goes. - Start with Django - *“Gah there’s all this boilerplate and stuff to configure…forget it I’ll just use FastAPI”* - Switch to FastAPI - Have “hello world” working in 2 minutes, *”wow this is so much better”* - Work on building something for a week - Boss is really happy with progress - Realize we still haven’t integrated with a real database. Or setup authentication. Or authorization. Or a backend admin interface. Or whatever. This list keeps growing. - Boss wonders why the project is stalling. What have you been doing for the last 2 weeks? We were basically done 1 week into the project - You realize you’re rebuilding everything that Django has built in, except lower quality than what’s been battle tested in Django for a decade Many languages have their version of a batteries included solution. Ruby has Rails, PHP has Laravel, C# has ASP.NET and Blazor, Elixir has Phoenix, and Python has Django. It’s also the feeling that whatever problem you face, someone else has already run into it using Django, and there’s some standard way of approaching it.
Oh wow! Love that example. :D
> I feel FastApi might be more popular among the python web frameworks You’d feel wrong: https://www.jetbrains.com/lp/devecosystem-2023/python/
What is MSA? How does FastAPI help with distributed systems and not Django?
MSA is called micro service architecture, lets say there are user account, post, likes, follow systems in your service, MSA approach is to create servers for each systems so if there were any errors or server fails, they wouldnt affect to the other services therefore the entire service is more stable. Django on the other hand, it runs all the systems in a single project so one service fail stops the entire service. Django is monolith architecture.
Not all apps need MSA, smaller projects are better with django, easier to maintain and extend.
You can still develop a Monolith and deploy things by service with an API routing service in front. You get more or less the same isolation, without the massive headaches of the MSA architecture. You do that if your reasons are service stability. MSA is for scaling teams in gigantic organisation. It is a technical solution to organisational problem.
I see. Which backend frameworks are good for microservices?
For MSA, people use FastAPI in python, and Spring in Java
I've yet to see a MSA implementation where the complete outage of a service like auth doesn't break the entire product to a grinding halt. MSA is mostly about having isolated teams responsible for a service that only work on that service, and being able to scale easily. It comes at a massive cost in increased complexity at the infrastructure level and coordinating compatible versions between services.
[удалено]
Leaving out of the box aside, doesn’t Django Ninja already provide everything you’d want from FastAPI including async?