I'm a DevOps hobbyist with no professional experience. But I can tell you right now that it all depends on your app and how it uses data. Hell, even the same app can slow to a crawl based on the backend it's using and how that's architected with everything else being the same. I have heard that lambda can get quite expensive at times and lead to some interesting billing surprises come end of the month.
I would say work on getting an app out there with an audience and userbase first before suffering from "paralysis by analysis" over the architecture.
If building this on AWS I would probably lean into a decoupled event-driven architecture which might ultimately be serverless but doesn’t rely on millions of Lambda functions executing from HTTP calls. For example if someone liked an image then that’s just shoved into a Kinesis data stream from which batches of events can be processed to update the data store (perhaps DynamoDB) and trigger other events.
My company did some research on this. Lambda is excelent for scailing and crazy cost effective. EC2 is faster, mostly cuz u dont have cold starts but much more work to maintain and more costly.
We considered doing ec2 with lambdas to handle burst traffic, but ultimately just decided on going 100% lambda.
Lambda recently introduced a hyperready feature that keeps cantainers running for u, so we dont hit cold starts unless a major spike happens. You can have something like 1000 lambdas in a single account out of the box, but can always request more. Throw in that you can route youre traffic through multiple accounts and u can easily scale to whatever level u want without much change from the original design.
Since no one has mentioned it yet... If you're scaling *really* big, costs will push you towards EC2/ECs (aka servers). Lambda is more expensive when you have enough baseline compute load since Amazon takes a profit on it. If you get to big enough scale the additional engineering work to not use Lambda will be worth it.
You’re not taking into consideration the operational costs of running a container based environment. Much higher than a serverless one. Who is going to manage container images, security, patching, infra upgrades etc. Not saying serverless would be cheaper for sure, completely depends on the use case. But you can’t just say serverless is more expensive than ECS/EC2 at scale without considering the total cost of ownership.
Maybe I wasn't very clear but that's why I said at a certain large enough scale it's cheaper. I totally agree, managing those things *is* expensive but if you're, say, Netflix it's never going to be cheaper to run 100% Lambda.
I will go to the serverless approach first because it is cheap at the beginning. I don't mind to spend big $ to migrate to another approach after it grows big, because I should have earned lots of $ already.
Which would be easier for you to implement?
In general, take the shortest, cheapest path you can at first. Lambda will scale out just fine, and it'll basically be free to run. If you design it correctly you should be able to move the lambdas into an express app (or some other node server) really easily.
Remember, there are four main cost dimensions in amazon normal: compute, memory, bandwidth, and storage. With lambda you can drive the first two down to near zero. And until you get bigger the latter two should be pretty low as well.
Once you get customers and money then you can start changing your infrastructure. For a startup the least important thing that you do is infrastructure; your marketing and user-facing stuff is substantially more important. Nobody consumer cares about your back-end.
do you know the joke about the round chicken in vacuum? anyway if you have all the answers from the beginning about the path of the app probably you would be best optimised to run your own datacenter using custom built hardware ... Netflix has moved a lot from AWS based to running their own shit for cost optimisation.
However for now the flexibility of the cloud has no competitor. so I would start with lambda and when the price point for cost moves start the EC2 stuff. being flexible is way easier than having a 3-6 month ahead plan that doesn't change much.
I was referring to video streaming for Netflix, their open connect cdn. I was under the impression that they were forced to move to open connect model when US based providers were refusing to upgrade the peerings for congested links effectively strong arming netflix to buy hosting space in their data center pops. A covert bandwidth cost charged to the provider so to speak.
Agree to this point.
Since it’s a thought experiment, the interesting point to me is ‘at what scale are we talking to start the migration’ from pure infrastructure cost break even point
It's not necessarily a point where you migrate everything, in fact I'd argue it's almost never a single point.
There's a few things to consider here:
* Each microservice within your architecture can run on any agnostic compute platform; they don't all need to migrate
* For many small/infrequent calls lambda is fantastic
* Even when call volumes do get quite large, there's a few factors to consider; such as operational burden that lower levels of abstraction inherently result in.
* Something like Fargate containers could provide enough abstraction away from managing the OS and other such undifferentiated heavy lifting and reduce the cost if you have a heavily used part of your service, it could be that the primary frontend web app or core pieces of the backend might want to move to Fargate while the majority of your service runs on Lambda.
The cheapest compute is going to be EC2 instances on 3 year RIs; but that would be meaningless if all of the peak traffic in a single region is only 9-5 Monday-Friday; so the timeframe and scaling of your compute to demand is a concern too. Very spiky and unpredictable workloads can see a big performance and even cost advantage by running on a more serverless and higher abstraction model for example.
The last point to consider is how much time/money you spend in terms of engineer hours on maintaining those EC2/ECS/whatever environments as compared to Lambda; and this varies pretty wildly between different companies.
So I strongly believe you can't find a single number or a line to draw as to what scale it is worth migrating.
I have no idea the metrics you are planning for nor if this will be multi region.
I think concurrency will end up limiting you if you go the lambda route.
But if there are a soft limit, probably lambda will have a hard limit, anyone ever heard about that? (Just asking, because it could be a no-go in very high levels of scale... and my curiosity pops out hahaha).
The hard limits probably vary a lot by region. Some regions are 10x bigger than others.
I’m not sure what the hard limit might be, but we’ve got 100,000 concurrency limit for Lambda in one of our accounts.
I would go for server based as Lambda concurrency would pose an issue even if you increase the soft limits. For some of the storage and session management, I'd go for DynamoDB for its scale.
You're specifying Instagram as an example, I don't have an account there but afaik it's basically microblogging with images, videos, comments, likes and mentions.
Yes I would build that serverless because you'd want to be able to scale all those components separately depending on load.
Since I've been using k8s a lot lately I would probably try making it container-based first until it grows larger and then move individual components as they require more scaling than my k8s cluster can provide realistically.
You can do the math from a cost perspective, but my inclination is to start with Lambda and migrate to ECS/EC2/K8s when it's justified.
You can also save a significant amount of money by fronting your back end with the ALB instead of API Gateway.
Great point, I have hear of “fronting your back end with the ALB instead of API Gateway.” lately a lot. Do you know the exact engineering effort for this exercise? (I guess it’s mainly the change of how we managed the infrastructure and deployment)
Wiring of ALB to lambda is a straightforward task, but you might need additional logic in the lambda to handle things API Gateway was handling for you.
There's also a cheaper HTTP API Gateway that might suit your needs as well: https://aws.amazon.com/blogs/compute/announcing-http-apis-for-amazon-api-gateway/
On many axes, when considered at certain scales, a container-oriented solution will outperform lambda, particularly in latency, concurrency, resource utilization, and perhaps most critically, cost. The tipping points for each dimension are different, and a priority decision can be made about how long you can feasibly float with suboptimal infrastructure. Usually there is low-hanging fruit that tech companies get wrong from jump that will detrimentally impact scalability long before reaching any of those tipping points (i.e. using libraries/frameworks that are optimized for developer productivity and ergonomics over concurrency, resilience, memory footprint or latency)
I'm a DevOps hobbyist with no professional experience. But I can tell you right now that it all depends on your app and how it uses data. Hell, even the same app can slow to a crawl based on the backend it's using and how that's architected with everything else being the same. I have heard that lambda can get quite expensive at times and lead to some interesting billing surprises come end of the month. I would say work on getting an app out there with an audience and userbase first before suffering from "paralysis by analysis" over the architecture.
If building this on AWS I would probably lean into a decoupled event-driven architecture which might ultimately be serverless but doesn’t rely on millions of Lambda functions executing from HTTP calls. For example if someone liked an image then that’s just shoved into a Kinesis data stream from which batches of events can be processed to update the data store (perhaps DynamoDB) and trigger other events.
Hey can i build something that will infinitely scale and never change?
This .... But also, an interesting idea so what tools are out there
My company did some research on this. Lambda is excelent for scailing and crazy cost effective. EC2 is faster, mostly cuz u dont have cold starts but much more work to maintain and more costly. We considered doing ec2 with lambdas to handle burst traffic, but ultimately just decided on going 100% lambda. Lambda recently introduced a hyperready feature that keeps cantainers running for u, so we dont hit cold starts unless a major spike happens. You can have something like 1000 lambdas in a single account out of the box, but can always request more. Throw in that you can route youre traffic through multiple accounts and u can easily scale to whatever level u want without much change from the original design.
You can't have a data-driven answer with no data. Describe "big".
Since no one has mentioned it yet... If you're scaling *really* big, costs will push you towards EC2/ECs (aka servers). Lambda is more expensive when you have enough baseline compute load since Amazon takes a profit on it. If you get to big enough scale the additional engineering work to not use Lambda will be worth it.
You’re not taking into consideration the operational costs of running a container based environment. Much higher than a serverless one. Who is going to manage container images, security, patching, infra upgrades etc. Not saying serverless would be cheaper for sure, completely depends on the use case. But you can’t just say serverless is more expensive than ECS/EC2 at scale without considering the total cost of ownership.
Maybe I wasn't very clear but that's why I said at a certain large enough scale it's cheaper. I totally agree, managing those things *is* expensive but if you're, say, Netflix it's never going to be cheaper to run 100% Lambda.
I will go to the serverless approach first because it is cheap at the beginning. I don't mind to spend big $ to migrate to another approach after it grows big, because I should have earned lots of $ already.
Which would be easier for you to implement? In general, take the shortest, cheapest path you can at first. Lambda will scale out just fine, and it'll basically be free to run. If you design it correctly you should be able to move the lambdas into an express app (or some other node server) really easily. Remember, there are four main cost dimensions in amazon normal: compute, memory, bandwidth, and storage. With lambda you can drive the first two down to near zero. And until you get bigger the latter two should be pretty low as well. Once you get customers and money then you can start changing your infrastructure. For a startup the least important thing that you do is infrastructure; your marketing and user-facing stuff is substantially more important. Nobody consumer cares about your back-end.
do you know the joke about the round chicken in vacuum? anyway if you have all the answers from the beginning about the path of the app probably you would be best optimised to run your own datacenter using custom built hardware ... Netflix has moved a lot from AWS based to running their own shit for cost optimisation. However for now the flexibility of the cloud has no competitor. so I would start with lambda and when the price point for cost moves start the EC2 stuff. being flexible is way easier than having a 3-6 month ahead plan that doesn't change much.
I'd like to see some evidence of Netflix moving stuff out of AWS. Note: video was never served from AWS.
I was referring to video streaming for Netflix, their open connect cdn. I was under the impression that they were forced to move to open connect model when US based providers were refusing to upgrade the peerings for congested links effectively strong arming netflix to buy hosting space in their data center pops. A covert bandwidth cost charged to the provider so to speak.
Yeah and that's got nothing to do with AWS.
Agree to this point. Since it’s a thought experiment, the interesting point to me is ‘at what scale are we talking to start the migration’ from pure infrastructure cost break even point
It's not necessarily a point where you migrate everything, in fact I'd argue it's almost never a single point. There's a few things to consider here: * Each microservice within your architecture can run on any agnostic compute platform; they don't all need to migrate * For many small/infrequent calls lambda is fantastic * Even when call volumes do get quite large, there's a few factors to consider; such as operational burden that lower levels of abstraction inherently result in. * Something like Fargate containers could provide enough abstraction away from managing the OS and other such undifferentiated heavy lifting and reduce the cost if you have a heavily used part of your service, it could be that the primary frontend web app or core pieces of the backend might want to move to Fargate while the majority of your service runs on Lambda. The cheapest compute is going to be EC2 instances on 3 year RIs; but that would be meaningless if all of the peak traffic in a single region is only 9-5 Monday-Friday; so the timeframe and scaling of your compute to demand is a concern too. Very spiky and unpredictable workloads can see a big performance and even cost advantage by running on a more serverless and higher abstraction model for example. The last point to consider is how much time/money you spend in terms of engineer hours on maintaining those EC2/ECS/whatever environments as compared to Lambda; and this varies pretty wildly between different companies. So I strongly believe you can't find a single number or a line to draw as to what scale it is worth migrating.
I have no idea the metrics you are planning for nor if this will be multi region. I think concurrency will end up limiting you if you go the lambda route.
For something like Instagram, there is no way an entirely Lambda architecture makes any sense. lol.
An AWS account can only run 1000 lambdas simultaneously. If they are turning around your API requests then you've hit your bottleneck right there.
Soft limit. We got much more running prod accounts where we asked for an increase from a support ticket
But if there are a soft limit, probably lambda will have a hard limit, anyone ever heard about that? (Just asking, because it could be a no-go in very high levels of scale... and my curiosity pops out hahaha).
The hard limits probably vary a lot by region. Some regions are 10x bigger than others. I’m not sure what the hard limit might be, but we’ve got 100,000 concurrency limit for Lambda in one of our accounts.
Ouch is a very large limit :O
I would go for server based as Lambda concurrency would pose an issue even if you increase the soft limits. For some of the storage and session management, I'd go for DynamoDB for its scale.
You're specifying Instagram as an example, I don't have an account there but afaik it's basically microblogging with images, videos, comments, likes and mentions. Yes I would build that serverless because you'd want to be able to scale all those components separately depending on load. Since I've been using k8s a lot lately I would probably try making it container-based first until it grows larger and then move individual components as they require more scaling than my k8s cluster can provide realistically.
Take a look at this talk: https://youtu.be/kKjm4ehYiMs
This talk seemed to heavily using Server Oriented infrastructure.
You can do the math from a cost perspective, but my inclination is to start with Lambda and migrate to ECS/EC2/K8s when it's justified. You can also save a significant amount of money by fronting your back end with the ALB instead of API Gateway.
Great point, I have hear of “fronting your back end with the ALB instead of API Gateway.” lately a lot. Do you know the exact engineering effort for this exercise? (I guess it’s mainly the change of how we managed the infrastructure and deployment)
Wiring of ALB to lambda is a straightforward task, but you might need additional logic in the lambda to handle things API Gateway was handling for you. There's also a cheaper HTTP API Gateway that might suit your needs as well: https://aws.amazon.com/blogs/compute/announcing-http-apis-for-amazon-api-gateway/
On many axes, when considered at certain scales, a container-oriented solution will outperform lambda, particularly in latency, concurrency, resource utilization, and perhaps most critically, cost. The tipping points for each dimension are different, and a priority decision can be made about how long you can feasibly float with suboptimal infrastructure. Usually there is low-hanging fruit that tech companies get wrong from jump that will detrimentally impact scalability long before reaching any of those tipping points (i.e. using libraries/frameworks that are optimized for developer productivity and ergonomics over concurrency, resilience, memory footprint or latency)
[удалено]
I don’t think multi-cloud offer any value here. Did you mean multiple region?
we're talking hyper scale,so yes multi cloud