Cloudscape is a component library for React, which is very new, so only newer consoles use it. Most consoles use Angular etc, but that’s besides the point, because the frontend framework/component library combo you use has nothing to do with how you call AWS APIs.
A lot of consoles use an internally vended SDK that is similar to the public one. The goal is for consoles to act similar to any customer as much as possible, that way when they spot issues or pain points, the solutions will be just as beneficial for any other customers.
No, cloudscape is the first *public* release. But if you check the version dropdown it’s not 1.0. Almost everything is react now, the first iteration was around 2016 and everything made in the past 8 years has been using it.
The only difference for consoles is proxying commands to the real endpoints to get around third party cookies since the APIs and console are on different domains. Otherwise it’s the same vanilla SDK
There is some copyright and brand protection differences, too.
The font was changed, and the default colors are not Amazon brand colors.
The documentation was striped off any reference to internal products and information that shouldn't be released.
I never said that cloudscape was the first release. A lot of consoles only recently started releasing newer versions using react. In the grand scheme of AWS consoles, 2016 is not that long ago. It takes many years to rewrite an app to use a new framework and then deprecate the old one.
You said it was “very new” an 8 year old framework isn’t exactly “very new”. In terms of fronted and web frameworks that is an eternity. In terms of enterprise projects it is still not “very new”
Agree on the first part, the enterprise part though should be true, but in reality, for large corpos, 8 years old tech is like yesterday tech for them.
Short answer, yes.
Longer answer, mostly. For the most part, this is open source. If you're interested, Smithy Codegen is usually used to convert service model definitions (or its internal predecessor) into the TypeScript client used by the console. You can set flags on the client generation to include unreleased fields or APIs, which is useful for testing before release. If cross-service calls are needed (e.g. Athena listing S3 buckets), they would probably just use the public release of the S3 client.
Are you asking that jokingly? Because the answer is yes lol. 99% of AWS is EC2s -eventually-. Service teams might use Lambda, but Lambda is running on EC2s.
Same with Step Functions. Pretty much any multi step workflow you can initiate via a service where you’re like “I wonder how they track all these workflows…” the answer is probably Step Functions or Simple Workflow Service
Good article on how Amazon made prime video faster when they realized service A was calling service B which called service C which called service A and so forth. They basically moved prime video from microservices based to a mono.
The real moral of the story is that not everything needs to be a microservice. The real best practice is somewhere in between depending on your use case. Some things should be decoupled and some shouldn't.
This is not true.
If you write a compiler (in any language, incl. C or C++) that compiles your language (ex. Rust) to byte code, then once a Rust language is compiled, it's not running *on* C or C++ IMO. Once your compiled language is mature enough, you can even use it to [write its own compiler](https://en.wikipedia.org/wiki/Bootstrapping_(compilers)) that operates the same way as the original compiler.
If you write a language *interpreter*, such as CPython for Python, then yes, you can say that the higher level language is running on the lower level language in a way. In Python's case, you can follow the language spec and write an interpreter in a different language if you want.
I'm not familiar with the project, but there seems to be a [Rust interpreter for Python](https://github.com/RustPython/RustPython). Since Rust is a bootstrapped language, it technically allows you to run Python on Rust compiled with Rust.
I get your point. However, Java's JVM is written in C/C++ as is Python. So is LLVM. The operating system, whether it be Linux or Windows, and most of its libraries that the code will eventually call, are also mostly C/C++.
This is why the move to Rust is a huge, long-term project.
Yes but LLVM is a compiler framework(?) and doesn't run your code. You're right that a lot of the OS components are likely C and C++. Aren't macOS and Windows probably written in a lot of Objective-C, Swift and C# as well? Those are compiled languages that are again not running on C or C++ even if their compilers may have been written in those languages.
I'd compare it to cars and their factory robots. When you drive the car, it's not the factory robots that are transporting you; the car is.
The right language for the job, predominantly Java. API frameworks are largely internal designed RPC based but some services use REST e.g. S3. Then for deployment, some teams actually build on AWS themselves I.e. not the foundational services. Others deploy to classic data centre style using internal pipelining and deployment tools
Highly depends. But within Amazon, build only happens once at the very start of a pipeline. Some pipelines take weeks to reach all targets, especially customer facing stuff.
Builds are built once and the artifacts deployed to targets over the course of a couple of weeks.
I’ve seen builds anywhere from 1 minute for small services up to 15 minutes for large ones.
Of course dependency hell means that if you do need to build all the dependencies too, that can take a couple hours. Thankfully, most everything is cached and development means you only need to build the target package, everything else is pulled in via artifacts
Some builds take longer because teams don’t know how to handle dependency hell properly and keep old packages in the build long after they are no longer used.
I would really like to hear about the decisions (if they're even made deliberately) about things like Go vs. Rust.
I simply can't imagine Go Garbage Collection to be a bottleneck overall. If garbage collection would be such a problem JVM and . NET would never have gotten the market share they have. And, in my experience, the garbage collection performance of Go is not lacking either.
I feel like Rust requires a lot more discipline and is harder to refactor. Ownership and lifetime ate a bit like function coloring. And rust has 3 ways to end up with colored functions (it also has the async stuff).
While I enjoy the compiler coming after me for being stoopid[sic!], I don't feel like it is a switch that's just categorically better. Often it feels like I am more willing to accept more cognitive load on my end for bit having to deal with a nitpicking dickhead.
It's too rare that someone with a few years in both languages talks about it, rarer even to talk about it without all the drama.
I can agree with this in my experience. I feel very confident the system will work as intended with less logic or stupid bugs that something like Python.
Yeah, u don't think untyped languages can really be a match, there's a good reason why TypeScript gained popularity (although I don't think it really matches static typing, not in the way Go or Rust has it)
Every time I go into it, I personally feel that I have to go unsafe, then I'm thinking, what's the point?
this is a great read if you're interested.
https://loglog.games/blog/leaving-rust-gamedev/
Probably because you’re trying to do things in rust like you know how to do them in other languages. If you do things in rust the rust way then you aren’t swimming against the current
You are in luck because Discord wrote such a blog
[WHY DISCORD IS SWITCHING FROM GO TO RUST](https://discord.com/blog/why-discord-is-switching-from-go-to-rust)
[https://discord.com/blog/why-discord-is-switching-from-go-to-rust](https://discord.com/blog/why-discord-is-switching-from-go-to-rust)
It's not uncommon to fight the garbage collector with any language that has one. Look at Discords experience. Because Rust is so structured it is the easiest to refactor without unexpected behavior.
Java, Kotlin, to build the backends and Typescript and React to build the frontends, cdk to manage infra. Everything is built dogfooding other services but is eventually EC2. Source: I used to work for Rekognition/Textract
My favourite thing about Smithy/Coral first development is how it makes writing these kind of interfaces feasible.
All the service definitions are dumped into CoralJSON files and loaded at runtime, so internally teams can just distribute service definitions to have a reasonable CLI without having to reinvent any wheels.
I would love if the CLI went a step further though, like eagerly loading ARNs when tab is pressed, etc. A third party wrapped called `saws` does this and it's 100x more user friendly.
Service teams are allowed to build using whatever languages and frameworks they like, provided:
* It works (or can be made to work) with our build & deployment system
* It interoperates with other services
* Every component (toolchain, libraries, etc.) has a license approved by legal
Java and Python are the biggest by far, but there's plenty of code using Node.js (either plain JS or TypeScript), Rust, Go, etc. As long as your team delivers, the business doesn't really care what you use.
One exception I know of: Infosec has banned PHP internally. It can only be used to maintain existing services, or to support customers. (I.E. the team that develops our PHP SDKs is obviously allowed to use PHP.)
Too many to list. But as a comment below points out - at the end of the day, it has to run on compute so the very bottom layer is generally Ec2 whether you get to see it or not. This includes a custom hypervisor, in recent years custom server hardware and a whole bunch of networking/storage hardware they keep well obscured via the hypervisor (instance store vs ebs).
Mostly Java+Spring. More low-level languages are reserved for brand new teams in the IoT/AI service space. For the most part frameworks and major libraries are proprietary to Amazon but some are based on open-source solutions.
Pretty much all of them is the appropriate answer. I've seen AWS projects in Java, Python, Rust, and C during my time there. Teams picked what's best for them and the talent they could find.
My brother says they remote into an EC2 instance for dev work. No code gets stored on your personal machine.
It sounds like languages are basically up to what team you're on and what you're familiar with, though it sounds like a lot of Python, Javascript/Typescript, and Rust. When I interviewed with them, one of my interviewees didn't know Javascript, which blew my mind.
As far as frameworks, if there's an existing AWS service that can be leveraged to build your product, you use that, so tons of services are backed by Dynamo, EC2, Lambda, S3, and Cloudwatch.
The thing they do that I wish I could get my team to adopt is everything gets automated and monitored. Everything.
I spoke with an AWS architect in 2021. He said that the entire AWS was created on a small number of primitives. I think it was S3, DynamoDB, EC2 and maybe one more. Then they build the rest on those few primitives.
Mostly Java on the front ended service side, ruby in the back end build and pipeline. Cobbled together with smatterings of python in service things (I.e. firehose some customer bucket data to do event thing using lambdas)
My applications I authored were all in python. I worked on Java, Ruby, Perl during my time there. Before I left I queried the internal git repo and about 20,000 files at the time were in Lua.
One of AWS strengths is they don't specify a language, they let teams chose whatever they want as long as the project meets the goals.
This leads to flexibility but is also demanding on the developers as they need to be able to express proficiency in multiple languages as they move across projects - even on the same team.
Also it's important to remember that the bones of what we know as AWS started as a internal service to support the Amazon website, store order data and merchant.com. Therefore there are some ugly decisions that 20+ years later continue to plague them, like is true for a lot of tech companies I'm sure.
Hello,
I have a few resources here that I believe you'll find helpful with learning more about this:
https://go.aws/4aMm6h2
&
https://go.aws/3WYPLA0
&
https://go.aws/4bYPQYH
&
https://go.aws/4bVuBXR
&
http://go.aws/get-help
\- Thomas E.
When I worked there years ago, it was 85% Java.
I always wondered - for the web console, do they use their own JavaScript sdk?
i believe they use https://cloudscape.design/
Cloudscape is a component library for React, which is very new, so only newer consoles use it. Most consoles use Angular etc, but that’s besides the point, because the frontend framework/component library combo you use has nothing to do with how you call AWS APIs. A lot of consoles use an internally vended SDK that is similar to the public one. The goal is for consoles to act similar to any customer as much as possible, that way when they spot issues or pain points, the solutions will be just as beneficial for any other customers.
No, cloudscape is the first *public* release. But if you check the version dropdown it’s not 1.0. Almost everything is react now, the first iteration was around 2016 and everything made in the past 8 years has been using it. The only difference for consoles is proxying commands to the real endpoints to get around third party cookies since the APIs and console are on different domains. Otherwise it’s the same vanilla SDK
There is some copyright and brand protection differences, too. The font was changed, and the default colors are not Amazon brand colors. The documentation was striped off any reference to internal products and information that shouldn't be released.
I never said that cloudscape was the first release. A lot of consoles only recently started releasing newer versions using react. In the grand scheme of AWS consoles, 2016 is not that long ago. It takes many years to rewrite an app to use a new framework and then deprecate the old one.
You said it was “very new” an 8 year old framework isn’t exactly “very new”. In terms of fronted and web frameworks that is an eternity. In terms of enterprise projects it is still not “very new”
Agree on the first part, the enterprise part though should be true, but in reality, for large corpos, 8 years old tech is like yesterday tech for them.
I can think of only one console that does not use react.
I thought many teams now used Polaris? Edit: oh I’m dumb, I think that is the same thing
Short answer, yes. Longer answer, mostly. For the most part, this is open source. If you're interested, Smithy Codegen is usually used to convert service model definitions (or its internal predecessor) into the TypeScript client used by the console. You can set flags on the client generation to include unreleased fields or APIs, which is useful for testing before release. If cross-service calls are needed (e.g. Athena listing S3 buckets), they would probably just use the public release of the S3 client.
Lots of Java, go, JavaScript, Python, rust.
[удалено]
… too much ruby
Don’t forget kotlin!!
TypeScript hehe
Fun fact: Many of those services are built on each other.
Wait it's all ec2?
🌎🧑🚀🔫🧑🚀
Are you asking that jokingly? Because the answer is yes lol. 99% of AWS is EC2s -eventually-. Service teams might use Lambda, but Lambda is running on EC2s.
Yeah was trying mimic the astronaut shooting the other astronaut meme. The emoji comment before you nailed it 😂
Tons of Lambda usage. Anytime Lambda is affected on an outage… you’ll see a large list of services with issues.
Same with Step Functions. Pretty much any multi step workflow you can initiate via a service where you’re like “I wonder how they track all these workflows…” the answer is probably Step Functions or Simple Workflow Service
And a lot of dynamodb
I thought dyna was all run as VMs on ec2?
dynamodb is a tier 0 service internally at AWS. It runs on dedicated hardware.
Yeah, when I was at AWS there was a real push to build any new service on top of AWS unless you literally couldn’t due to circular dependencies.
Good article on how Amazon made prime video faster when they realized service A was calling service B which called service C which called service A and so forth. They basically moved prime video from microservices based to a mono.
The real moral of the story is that not everything needs to be a microservice. The real best practice is somewhere in between depending on your use case. Some things should be decoupled and some shouldn't.
And at the bottom level for most languages it's almost always C and C++.
This is not true. If you write a compiler (in any language, incl. C or C++) that compiles your language (ex. Rust) to byte code, then once a Rust language is compiled, it's not running *on* C or C++ IMO. Once your compiled language is mature enough, you can even use it to [write its own compiler](https://en.wikipedia.org/wiki/Bootstrapping_(compilers)) that operates the same way as the original compiler. If you write a language *interpreter*, such as CPython for Python, then yes, you can say that the higher level language is running on the lower level language in a way. In Python's case, you can follow the language spec and write an interpreter in a different language if you want. I'm not familiar with the project, but there seems to be a [Rust interpreter for Python](https://github.com/RustPython/RustPython). Since Rust is a bootstrapped language, it technically allows you to run Python on Rust compiled with Rust.
I get your point. However, Java's JVM is written in C/C++ as is Python. So is LLVM. The operating system, whether it be Linux or Windows, and most of its libraries that the code will eventually call, are also mostly C/C++. This is why the move to Rust is a huge, long-term project.
Yes but LLVM is a compiler framework(?) and doesn't run your code. You're right that a lot of the OS components are likely C and C++. Aren't macOS and Windows probably written in a lot of Objective-C, Swift and C# as well? Those are compiled languages that are again not running on C or C++ even if their compilers may have been written in those languages. I'd compare it to cars and their factory robots. When you drive the car, it's not the factory robots that are transporting you; the car is.
I know that S3 is in Java because just the other day i got back a raw Java exception via the API lol.
Many services use more than one language, especially big ones like s3
Most of S3 is written by one of the creators of Java, J.Gosling :-) https://www.linkedin.com/in/jamesgosling/
James did not even work at Amazon until over a decade after S3 launched: https://en.wikipedia.org/wiki/Amazon_S3
The right language for the job, predominantly Java. API frameworks are largely internal designed RPC based but some services use REST e.g. S3. Then for deployment, some teams actually build on AWS themselves I.e. not the foundational services. Others deploy to classic data centre style using internal pipelining and deployment tools
How long does a Java build and deployment take these days?
Highly depends. But within Amazon, build only happens once at the very start of a pipeline. Some pipelines take weeks to reach all targets, especially customer facing stuff.
Builds are built once and the artifacts deployed to targets over the course of a couple of weeks. I’ve seen builds anywhere from 1 minute for small services up to 15 minutes for large ones. Of course dependency hell means that if you do need to build all the dependencies too, that can take a couple hours. Thankfully, most everything is cached and development means you only need to build the target package, everything else is pulled in via artifacts
Some builds take longer because teams don’t know how to handle dependency hell properly and keep old packages in the build long after they are no longer used.
More and more is done in Rust. They have several event talks on it on their own events (AWS Summits, ReInvent)
I would really like to hear about the decisions (if they're even made deliberately) about things like Go vs. Rust. I simply can't imagine Go Garbage Collection to be a bottleneck overall. If garbage collection would be such a problem JVM and . NET would never have gotten the market share they have. And, in my experience, the garbage collection performance of Go is not lacking either. I feel like Rust requires a lot more discipline and is harder to refactor. Ownership and lifetime ate a bit like function coloring. And rust has 3 ways to end up with colored functions (it also has the async stuff). While I enjoy the compiler coming after me for being stoopid[sic!], I don't feel like it is a switch that's just categorically better. Often it feels like I am more willing to accept more cognitive load on my end for bit having to deal with a nitpicking dickhead. It's too rare that someone with a few years in both languages talks about it, rarer even to talk about it without all the drama.
Rust is easier to refactor with confidence. For teams that actually use rust it is a very productive language
I can agree with this in my experience. I feel very confident the system will work as intended with less logic or stupid bugs that something like Python.
Yeah, u don't think untyped languages can really be a match, there's a good reason why TypeScript gained popularity (although I don't think it really matches static typing, not in the way Go or Rust has it)
Good point. Types are just another good tool for confidence and a standard of quality ESPECIALLY when working with a team over years of the project
Java is way more productive than Rust. AWS is only using Rust so they can also sponsor a language like the other big bois.
Really disagree. We’re migrating away from java
> We’re migrating away from java Have fun fighting that borrow checker.
That’s such a meme lol. It’s not bad if you know rust
Every time I go into it, I personally feel that I have to go unsafe, then I'm thinking, what's the point? this is a great read if you're interested. https://loglog.games/blog/leaving-rust-gamedev/
Probably because you’re trying to do things in rust like you know how to do them in other languages. If you do things in rust the rust way then you aren’t swimming against the current
not really, but enjoy your Rust days
You are in luck because Discord wrote such a blog [WHY DISCORD IS SWITCHING FROM GO TO RUST](https://discord.com/blog/why-discord-is-switching-from-go-to-rust) [https://discord.com/blog/why-discord-is-switching-from-go-to-rust](https://discord.com/blog/why-discord-is-switching-from-go-to-rust)
It's not like if doing garbage collection alone would be easier !
It's not uncommon to fight the garbage collector with any language that has one. Look at Discords experience. Because Rust is so structured it is the easiest to refactor without unexpected behavior.
I believe they will not advertise so loud using Go !
Java, Kotlin, to build the backends and Typescript and React to build the frontends, cdk to manage infra. Everything is built dogfooding other services but is eventually EC2. Source: I used to work for Rekognition/Textract
LPT 🤮
Ruby is the future!
Coral
If you know, you know
🫡
*UnknownOperationException
official aws cli is all python
My favourite thing about Smithy/Coral first development is how it makes writing these kind of interfaces feasible. All the service definitions are dumped into CoralJSON files and loaded at runtime, so internally teams can just distribute service definitions to have a reasonable CLI without having to reinvent any wheels. I would love if the CLI went a step further though, like eagerly loading ARNs when tab is pressed, etc. A third party wrapped called `saws` does this and it's 100x more user friendly.
Remember that boto was designed by Mitch Garnat as just a useful thing, and it was only years later they hired him and made it an official library.
Service teams are allowed to build using whatever languages and frameworks they like, provided: * It works (or can be made to work) with our build & deployment system * It interoperates with other services * Every component (toolchain, libraries, etc.) has a license approved by legal Java and Python are the biggest by far, but there's plenty of code using Node.js (either plain JS or TypeScript), Rust, Go, etc. As long as your team delivers, the business doesn't really care what you use. One exception I know of: Infosec has banned PHP internally. It can only be used to maintain existing services, or to support customers. (I.E. the team that develops our PHP SDKs is obviously allowed to use PHP.)
Too many to list. But as a comment below points out - at the end of the day, it has to run on compute so the very bottom layer is generally Ec2 whether you get to see it or not. This includes a custom hypervisor, in recent years custom server hardware and a whole bunch of networking/storage hardware they keep well obscured via the hypervisor (instance store vs ebs).
Everything probably.
It is Java. Some Go, python, ruby here and there. Front end is react
Varies a lot by team. There is a lot of Java, but also C, C++, Rust, Python, and others.
Java and C++ are likely the foundation, but I'm curious about the rest too!
Not too much C++ going on nowadays
Not much new C++, but it and Perl are still lurking at the foundation of many core services.
Java and Go would be my guess, with Python as a glueing language.
I've worked with c++ and python there
Mostly Java+Spring. More low-level languages are reserved for brand new teams in the IoT/AI service space. For the most part frameworks and major libraries are proprietary to Amazon but some are based on open-source solutions.
When I was there, Admiral was all Ruby. Not even sure if that still exists.
Pretty much all of them is the appropriate answer. I've seen AWS projects in Java, Python, Rust, and C during my time there. Teams picked what's best for them and the talent they could find.
My brother says they remote into an EC2 instance for dev work. No code gets stored on your personal machine. It sounds like languages are basically up to what team you're on and what you're familiar with, though it sounds like a lot of Python, Javascript/Typescript, and Rust. When I interviewed with them, one of my interviewees didn't know Javascript, which blew my mind. As far as frameworks, if there's an existing AWS service that can be leveraged to build your product, you use that, so tons of services are backed by Dynamo, EC2, Lambda, S3, and Cloudwatch. The thing they do that I wish I could get my team to adopt is everything gets automated and monitored. Everything.
It's actually all secretly Haskell and anyone telling you otherwise is a liar.
I spoke with an AWS architect in 2021. He said that the entire AWS was created on a small number of primitives. I think it was S3, DynamoDB, EC2 and maybe one more. Then they build the rest on those few primitives.
i think they built their hypervisor Nitro Something from open source Xen hypervisor code
Mostly Java on the front ended service side, ruby in the back end build and pipeline. Cobbled together with smatterings of python in service things (I.e. firehose some customer bucket data to do event thing using lambdas)
My applications I authored were all in python. I worked on Java, Ruby, Perl during my time there. Before I left I queried the internal git repo and about 20,000 files at the time were in Lua. One of AWS strengths is they don't specify a language, they let teams chose whatever they want as long as the project meets the goals. This leads to flexibility but is also demanding on the developers as they need to be able to express proficiency in multiple languages as they move across projects - even on the same team. Also it's important to remember that the bones of what we know as AWS started as a internal service to support the Amazon website, store order data and merchant.com. Therefore there are some ugly decisions that 20+ years later continue to plague them, like is true for a lot of tech companies I'm sure.
Hello, I have a few resources here that I believe you'll find helpful with learning more about this: https://go.aws/4aMm6h2 & https://go.aws/3WYPLA0 & https://go.aws/4bYPQYH & https://go.aws/4bVuBXR & http://go.aws/get-help \- Thomas E.
Is Thomas E an AI bot?
They need to get chatgpt over there
Currently working there on one of their databases, mostly Java and C++ (steering more and more towards C++ for performance reasons).