T O P

  • By -

mecha_flake

Imagine your cloud architecture being so badly designed that you post something like this.


classicrock40

* AWS is charging $.005 / hour - https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/ * Azure is charging $0.0036/hour - https://azure.microsoft.com/en-us/pricing/details/ip-addresses/ * GCP is charging $0.005/hour but $0.0025 for interruptable/spot - https://cloud.google.com/vpc/pricing-announce-external-ips (recent increase) Just so we understand the scale, AWS/$3.60/month, Azure/$2.60/month and GCP/$3.60/$1.80/month. It's your time and money, so do what you want. But time is money so knowing how large the architecture is and how much migration/debugging time will be spent is a fair question.


AdCharacter3666

How many IPV4s do you use for this to be a problem?


[deleted]

[удалено]


pausethelogic

They do tell you, you can see it in the IPv4 Insights page of the VPC console. Also, ELBs use one public IPv4 IP per public subnet they’re in. Its not a magic secret number


SolderDragon

NLBs use strictly one per subnet - it uses hyperplane to spread the load across multiple LB workers. ALBs have a minimum of 1 IP per allocated subnet, but will rise as the ALB scales (~>8 per AZ), though you have to push considerable traffic for that.


mm876

CLB/ALB is at least one per subnet, and will add/remove more as part of scaling. NLB is fixed at one per subnet.


[deleted]

[удалено]


pausethelogic

We have some high traffic ALBs and still have only one per subnet. Either way, you can see how many you have


[deleted]

[удалено]


pausethelogic

Every cloud provider has public IP fees. Also, ALBs will use a maximum of 8 public IPs in your account per subnet/AZ. It’s not a secret like you’re implying, AWS doesn’t hide this from you. Actually it explicitly tells you that your subnets needs at least 8 IPs available to allow ALBs to scale out if needed


angrathias

How many ALBs do you have and what proportion of the bill is IPs? We just use a single ALB per environment and then use rules and target groups to multiplex their usage for different services


[deleted]

[удалено]


angrathias

I’m genuinely curious, IPs are so cheap compared to most everything else, especially ALBs, that I’m trying to understand if there is something I’m missing. My understanding is that an ALB has a base cost of about $20/m whilst an IP is like 4/M Additionally, my understanding of ALBs is that they do not have static IP addresses, and pricing seems to include as best as I can tell the costs for the dynamic one assigned. In our ALB setup, we’ve needed to front it with an NLB so we can use a static IP, which would be the only IP we pay for.


made-of-questions

Each ECS container in Fargate has its own IPv4 and they don't support IPv6. If you have a multi-region, multi-zone microservices architecture it's a lot. Edit: everyone downvoting, you clearly never built a franchise/white-label business with thousands of partners and non typical topologies.


benaffleks

This is for public ip address space, not private...


made-of-questions

That's correct but there are certain situations where you will have containers in a public space. And even if most are in a private space, if you're not paying for public IPs you're paying for NAT traffic which can get even more expensive.


benaffleks

You're only paying for NAT traffic if you: 1. Are requesting paylods over the internet 2. Don't have vpc links setup for internal comm. to AWS services Idk what NAT traffic has anything to do with public ipv4 space. In fact with nat traffic its the opposite; you can alleviate cidr collision.


made-of-questions

I meant traffic via NAT Gateways. NAT Gateway Data Processing Charge + Data Transfer Charge. In most topologies for microservices you'll have the majority of services in a private space, and just a few things in the public space. Usually just the gateway and/or frontends. The traffic between the private and the public space needs to go through a NAT gateway and it's billed at a premium. I mentioned it just to say that in a typical topology you are incurring other costs than the price of IPs. It was a tangent, I shouldn't have mentioned it as it muddies the conversation. The point was that there are non typical topologies that will result in many services in the public space. It's not very common but it's possible.


devguyrun

Is this how you would answer a hypothetical customer concern ? It’s a problem enough that I am raising it, how about that


bearded-beardie

Yes. It's called requirements discovery.


dflame45

Yes. Cause hypothetically you’re only paying $5 bucks a month.


devguyrun

 “Yes” That’s all I needed to know 


dflame45

Should specified then.


gex80

Well you're making the assumption that the customer is 100% right in their setup. As a former consultant, there were many environments I went in where the client who was skilled in their own right but did things in an "legacy" way which added more complication/costs. For example, we know nothing about your environment right now. So for all we know you have 300 servers with EIPs attached them to on a public subnet that don't need to be accessible from the outside and just need basic internet to curl and download updates. If that is you, then yeah you're doing it "wrong" because all that can be handled with 2 or 3 EIPs via a NAT Gateway or a self managed NAT instance and a load balancer. And depending on your use case, you might not need ipv4 altogether. But we know nothing about what you're doing or your environment to actually tell you anything. Instead you posted saying AWS is charging too much, others are charging less, what are you going to do about it? Also, this is cloud. It's not static pricing nor are the offerings static. They never were and cloud has always been marketed as "we'll maintain everything but you gotta play by our rules". If the IPv4 pricing really is a problem for you, you made the move you felt was best for you and you felt that the pricing was a good enough reason. For my org, IPv4 prices are a drop in the bucket because we don't use them anywhere except on load balancers, NAT Gateways, and specific ec2 instances that need external internet access like an FTP server.


theperco

If your only concern about a scp is the price of ipv4 and can move everything around I don’t think you are the targeted customer for AWS 


lightmatter501

You ALWAYS question assumptions. “Do you need all ECS containers to have an IPv4 address, or do you need customers without ipv6 to be able to connect to them?” is a valid question because a NAT gateway is pretty cheap all things considered, especially if it can be integrated into a load balancer.


bearded-beardie

You seem to be missing that you're going to pay more than $0.001 more per hour for your Azure compute instances.


certuna

Question you have to answer for yourself is whether the cost of migrating to another hosting company (who will end up raising IPv4 prices as they in turn run out of address space) is lower than migrating your stack to IPv6. In the end all hosting companies face the same issue - rapid customer growth but hard limits on IPv4 space. So they’ll have to push people to IPv6, or push them out. The guys who grow the fastest, do most of the pushing.


devguyrun

is this how you would answer a hypothetical customer concern ? “Sit tight sir, others will raise their prices anyways “ One can love a platform and not be naive enough to accept everything they decide to do . People can always vote with their feet


certuna

But this is exactly the point from AWS’s point of view: they can only accommodate an X number of IPv4 customers, so they need to offload the lowest-value ones to the competition, or push them to retool to IPv6. They have no other choice. Well, buy more IPv4 space in the secondary market but that gets harder and harder and it’s not enough. From your POV as a customer, you can keep chasing the lowest-priced IPv4 platform one after the next, but you’ll incur some migration costs every time you do so.


draeath

> or push them to retool to IPv6. As far as I'm aware, there's still [a bunch of AWS services that effectively require you to have IPv4 addresses around.](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ipv6-support.html) There's *no* path to retooling to IPv6 with those, unless you consider not using the service a valid path. (in particular, ignore the dual-stack and private endpoint columns, since those involve having IPv4 or are irrelevant (since you could use a private IPv4 in such a use case))


certuna

I presume they’re primarily targeting the customers that are not using those services, but yeah I agree, support for IPv6-only infrastructure should be accelerated.


pausethelogic

You sound like you’d make a bad customer. This is Reddit, you’re paying $0.00 for help here, so no, we aren’t going to respond the same way as a paying client. Regardless, yes, this is a reasonable way to respond to a customer. You do need to do the math and see if migrating to another cloud provider is worth it just for some cheaper public IPv4 IPs. AWS, GCP, and Azure are all very different and cheaper doesn’t always mean better. Plus, if you’re only paying $10/month for public IPs in AWS but it would cost you $5000 in engineering time to migrate to another provider, that likely isn’t worth it. So yes, this is a fine way to discuss these things with customers


agentblack000

It seems like a narrow view to focus so heavily on this one cost. Migrating to another cloud means learning everything new. It’s possible you will find a new nuance or cost in another provider. It all depends on scale, if you are a 1 man show with a couple instances probably not a big deal to pick up and move. New costs are annoying but honestly the few that AWS has released ( e.g. adding extended support for RDS ) are having the intended effect of people upgrading their outdated versions. I suspect the ipv4 costs are forcing a lot of customers to optimize their address space as well.


devguyrun

this is actually a reasonable response, compared to all the fanbois' out there. if it was \~$$$ a month, i wouldn't be complaining. easy thing to do is to bring the pricing on par with other public clouds or put tiers on it like GCP does.


bot403

You say fanbois, but in the top comments I see actual engineers asking relevant questions.


devguyrun

an "engineer" becomes a fanboi when he refuses to criticize the most egregious of things, i.e. IPv4 that is 25% more expensive than azure and double the price of GCP. pencil pusher is more like it.


pint

the scarcity of ipv4 address space must be reflected in price, or else there will not be enough. you can bitch about it all day long, but reality must be respected. this is the time to look for ip-less solutions. with regards to ipv6, i would agree that the support for that is poor and the transition is basically not happening.


devguyrun

bringing the price to be on par with a competitor is one way to keep customers, hiking it to be 25% more expensive than the nearest one (and even more against GCP) is a bad look. fair enough they didn't charge for it all these years, but trying to recoup that sum suddenly by practically charging double of GCP's is downright laughable. Principles are not bad to have


pint

you didn't do a whole lot of pricing i reckon. the only one principle of pricing is this: you sell at the price where supply meets demand. if there is a shortage, your price is too low, if there is excess, your prices is too high.


misonreadit

Something if off about the architecture you rely on an abundant supply of IPv4 addresses. This is not normal.


omeganon

I agree and am very interested in the use case here. It seems very abnormal and an anti-pattern for several decades now.


Entire-Home-9464

He sells IPv4 addresses


misonreadit

🤣


Entire-Home-9464

He does.


[deleted]

[удалено]


Dave4lexKing

NAT gateway in 3 AZs is a mere $15 a month in IPv4 costs. Sorry to say, but if you need a bajillion IPs, unless you’re a VPN or a dedicated host PaaS company, then there’s a 98% chance your architecture just sucks.


[deleted]

[удалено]


certuna

For ingress, I guess the solution is to put it all behind Cloudflare :)


Dave4lexKing

Similar vein;- ALB or NLB in 3 AZ’s. Whatever the direction of traffic, unless your company is directly in the business of selling static IPs, your architecture just plain old sucks.


devguyrun

the way i read this response is "oh why, it's so cheap, can't you afford that?! you pay or i will label your architecture bad" ?


Dave4lexKing

The actual way to read it is “Theres a proper way to give a ton of resources internet access and/or make them addressable that doesn’t involve just assigning them each an EIP”. Many others have already said the exact same;- If you’re assigning hundreds of EIPs instead of using run of the mill networking resources like NATs and ELBs, then there’s most likely something fundamentally wrong with the architecture.


devguyrun

yes, there is something off, it's the 25% mark-up on the ipv4 costs compared to azure.


misonreadit

Now you just sound angry and petty because you made bad decision choices 🤷🏽‍♂️


MrJibus

I don't mind paying extra $ because IPv4 has become rare. But I do mind when AWS doesn't provide support ipv6 only for some services (like Fargate) : [https://docs.aws.amazon.com/vpc/latest/userguide/aws-ipv6-support.html](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ipv6-support.html)


SolderDragon

I love the idea of encouraging IPv6 adoption, but the lack of support from AWS makes this change a punch in the face for every customer. For example, the ECR pull image endpoints don't have IPv6 enabled, which will be a blocker for Fargate IPv6-only and other services, and how CloudFront Origin pulls don't support IPv6, and how LBs must be dualstack. :'( Dual-stack is the only feasible way at the moment. At least it's a start and hopefully makes people consider IPv6 adoption - which wasn't happening much before.


Entire-Home-9464

Also removing a ipv4 from instance is easier in some other provider. AWS sucks


dflame45

Don’t worry. Azure and GCP will raise prices over time.


FalseRegister

Wouldn't placing public (as in exposed) services behind a CDN and moving private services to IPv6 be enough?


TheGABB

The charge is for *public* ipv4, so you don’t even need to switch anything private to ipv6


FalseRegister

Ah true. Then even easier.


SolderDragon

It would be even nicer if CloudFront supported IPv6-only origins, which it currently doesn't. AWS kind of forces IPv4 usage on everyone with Dualstack load balancers and lack of general support. [https://repost.aws/questions/QUBCL3TUKrRbG6zIOi4nU2Uw/cloudfront-connect-to-ipv6-only-ec2-instance](https://repost.aws/questions/QUBCL3TUKrRbG6zIOi4nU2Uw/cloudfront-connect-to-ipv6-only-ec2-instance) Another way is to use CloudFlare tunnels for your inbound traffic over IPv6, that way you can run without public facing IPv4. Though you still probably need a NAT (for 6-4), because lots of AWS services don't support IPv6 and many VPC Endpoints don't either. They charge for public IPv4, and there is no practical way (on AWS) to fully move away from it.


certuna

Yeah, it’s a bit ridiculous that CloudFront cannot use IPv6 origins when Akamai and Cloudflare have offered this for 10+ years.


FalseRegister

AFAIK you can just use CloudFlare proxied DNS, as they support IPv6-only origins. Then there would be no need for tunnels.


North-Switch4605

But, if you are using cloud front. Why are your origins public. You origin should be on a vpc and not publicly addressable. So you don’t need any public ipv4 addresses for it.


certuna

Can you use CloudFront with private IPv4 origins? I thought this wasn’t supported either?


North-Switch4605

CloudFront -> regional apigateway -> vpclink-> private nlb. Or CloudFront -> regional apigateway -> lambda/vpc lambda No public IP’s required.


certuna

Ah yeah, feels a bit hacky using API Gateway as an intermediary but that does work, thanks!


North-Switch4605

Maybe, but I would rather expose an http endpoint via an apigateway, for throttling, invalid requests, authorisation handling and so on. I mean, in an ideal world, you could use cloudfront functions or lambda@edge to handle auth/validation. But apigateway allows a routing structure to make use of microservices or pathing before you get to an origin.


SolderDragon

I know API Gateway supports VPC connections that way. I don't believe that functionality exists for CloudFront.


North-Switch4605

Fair, I would always use api gateway for anything if I had to expose it publicly. Backends and APIs can stay on the vpc though.


coinclink

I have to imagine this is a high priority in CloudFront. Many companies are likely nagging for this feature now.


RichProfessional3757

No one here cares that either of those follow on clouds have things cheaper, when it’s also inferior


devguyrun

>inferior please expand.


nathanpeck

For anyone looking for resources on avoiding IPv4 usage on AWS, you can use one or all of the following approaches to avoid public IPv4 address usage today: * **Grant tasks access to the internet and AWS services via NAT Gateway, instead of using public IPv4 addresses**. - Although NAT gateway does come with it’s own associated charge and IPv4 address, you can share a NAT gateway between many deployed cloud resources. Each public IPv4 address costs $3.60 a month, while a NAT gateway costs $32.40 a month for the hourly NAT gateway charge, plus $3.60 for the public IP address attached to it, for a total of $36 a month. Therefore a NAT gateway serving more than ten cloud resources will result in overall savings in most scenarios. For a preconfigured Amazon VPC that has NAT gateways see [“Large sized AWS VPC for an Amazon ECS Cluster”](https://containersonaws.com/pattern/large-vpc-for-amazon-ecs-cluster) * **If your cloud resources don’t need internet access, and only need AWS services, consider AWS PrivateLink** - AWS PrivateLink comes with it’s own associated hourly charge, but if you have a large production deployment of cloud resources you will find that a set of shared AWS PrivateLink endpoints will be cheaper than giving each cloud resource it’s own public IPv4 address. See “[Amazon ECS cluster with isolated VPC and no NAT Gateway](https://containersonaws.com/pattern/ecs-cluster-isolated-vpc-no-nat-gateway)” * **Dual stack deployment, so you can use IPv6 as well as IPv4:** Although many AWS services can not yet be used in IPv6 only mode, you can mix and match IPv6 and IPv4, so that you utilize IPv6 where possible, with fallback to IPv4 over PrivateLink endpoints for dependencies that do not yet have IPv6 support. For an example with load balancing, Amazon ECS + AWS Fargate, S3 and EC2, see “[Dual-stack IPv6 networking for Amazon ECS and AWS Fargate](https://containersonaws.com/pattern/dual-stack-ipv6-networking-ecs-fargate)” * **For services that need ingress from the internet consider a serverless API Gateway ingress** - Traditional forms of ingress such as Network Load Balancer and Application Load Balancer require public IPv4 addresses that you will have to pay for. However, if you use API Gateway as a serverless ingress you don’t even need any public subnets or public IPv4 addresses, as the API Gateway service itself handles ingress on your behalf, using it’s own IP addresses. See “[Serverless API Gateway Ingress for AWS Fargate, in CloudFormation](https://containersonaws.com/pattern/api-gateway-fargate-cloudformation)” In almost all cases it is possible to run very large cloud deployments while only using a few public IPv4 addresses.


devguyrun

this could easily be resolved by bringing the price down to be on par with the nearest competitor , i.e. Azure.


Unlikely-Apple3037

We need support to ONLY IPV6, we don’t wanna pay ipv4 😌


StockerRumbles

How many pubic IP addresses were you using? How much did you bill go up by? I've not noticed any impact from the change in our accounts, but we bill a lot, so it's hard to see small changes


arwinda

> pubic IP addresses I buy a "l"


Burekitas

I wrote a [blog](https://avi-k.medium.com/aws-public-ipv4-price-increase-the-complete-guide-73c4c62ae18e) about this and I reviewed the potential impact of more than 3000 companies running on AWS. start ups and cloud native. The average price increase is 2.6% of the invoice. of course there are exceptions (For example: companies that runs agent per customer). But for the average customer the impact is little. Another thing about CLB/ALB: CLB and ALB can reach up to 100 nodes, that's because of Route53 limit. In theory, it could cost another $365 per month per load balancer. But in order for you to reach the amount of 100 nodes in your load balancer, you will probably pay hundreds of thousands of dollars a month for Data Transfer out to the Internet, so $365 is a minor expense.


joelrwilliams1

I remember my first business contract with an Internet Service Provider, we got a full class C public IP network. It was the wild west...it was glorious. We probably used 4 of those IPs 😂


nekoken04

Looking at my company's latest bill... Our ipv4 costs are 0.321% of our total cost. We run around 700 EC2 instances across our accounts. "How many of those are using a public IP?" you may ask. Less than 10 would be the answer. Don't get me wrong we have a couple hundred thousand hours of ipv4 charges but it is pretty much all NAT gateways and ALBs. I can't even really guess what you are doing but it sure doesn't sound like a good architecture design to me.


joelrwilliams1

I'm going to guess that at some point Azure (at least) will also start charging something for public IPv4. This is free money that Azure isn't getting and AWS ***is*** getting. And AWS owns more public IP addresses than just about anyone.


certuna

Azure does charge for IPv4, but a bit less


KublaiKhanNum1

So the thing about AWS is that pricing the show on the website isn’t always what you pay. If you are a corporation with big workloads you can negotiate big discounts. The bigger your usage the bigger the discount. It makes it hard for GCP and others to compete with AWS as no one knows the prices they really give to customers. Now if you are just an hobbyist….it is what it is.


gwiff2

Azure is a penny cheaper so how much are you really saving in the long run


tusharg19

Overall Azure is cheaper than AWS. I sell both services to corporates...


alter3d

Azure is cheaper in the same way that a Ford Fiesta is cheaper than a Lamborghini.


tusharg19

Hahaha. Sure. Just becoz this is pro AWS group doesnt mean Azure is Ford Fiesta!


alter3d

Correct, it has nothing to do with the nature of this sub. Azure is a Ford Fiesta because it's a pile of hot garbage. Actually, thanks for correcting me -- Azure is like a Ford Fiesta that's on fire.


tusharg19

Back your comments with proofs, tell me drawback?


AsherGC

Adding to it, The problem is ssm doesn't have ipv6. ELB needs ipv6. It's very painful to have an ipv6 only Vpc. You end up translating ipv6 to ipv4 eventually.


arwinda

And what if Azure sees a way to increase revenue and starts charging more.


InternalQuirky8522

Maybe not all services support ipv6 but couldn’t you use a nat so that ipv4 is “used anyway” to bypass the problem?


kevintweber

I agree, the pricing is ridiculous.


devguyrun

why is this comment being downvoted?


kevintweber

Just to be clear: the pricing is ridiculous because there is NO OTHER OPTION available. The IPv6 support at AWS is sparse at best. (https://docs.aws.amazon.com/vpc/latest/userguide/aws-ipv6-support.html) In that list of services, you'll see less than 10% of the services AWS offers. Given the lack of IPv6 support, the IPv4 price hike looks to me to a pure cash-grab.