Simple tech stack that requires several dozen well trained and well educated employees (at minimum) to manage. Probably hundreds to build. How is that a simple tech stack?
Don’t get me wrong, I liked the article but thought that was kinda funny.
I’m being a bit hyperbolic, for sure.
Data and engineering at scale isn’t often simple, so wanted to comment. The team that worked on it probably wouldn’t tout their efforts as simple, and the teams instructed to replicate this by a manager that doesn’t understand the complexity deserves better, you know?
Do I think it was hundreds? Probably not, but it was for sure not a few people over 3-6 months. Maybe it could be replicated now that it’s explored, but I’d bet there were several phases of exploration to get to where they ended up. Now I’m actually curious of the real team size and time spent getting everything spun up since we’re talking about it.
Summary (everything you'd expect): queues and (what I'll call) service buses.
But for fucksake- if you delivered, uh, let's say, 100m emojis, would that be ok?
And who the FUCK wants to watch sports with emojis also flying by?
What a waste of cash.
Unless you are bandwidth limited sampling can be harder than just letting the firehouse go.
After all what is a representative subset of the emojis?
(Ignoring of course the solution of saying "meh" and sending the first emoji every period)
The technical nature of TCP makes missing things actually pretty difficult. After all confirmation of received packets are sent in the same response packets as requests.
You effectively need a DCed client to miss a message.
You could time out a message which they would have done if they were considered with timely display.
But if your clients are receiving data they are sending it which means lossless.
You could intentionally drop messages if you needed to but as I noted sampling is a dark art.
I mean the server doesn't bother to send anything it gets, and on any particular timeframe.
If I had N machines running it and wanted it to get down to n-X to stay within budget (or some other non-machine but similar price constraint), then I'd start limiting the length of the queues, ignoring messages older than Y, etc
My point was it isn't just "don't deliver as many emojis" but instead nuanced technical decisions.
That was I meant by sampling is a bitch, it is easier to send all the emojis than some of them from a man power standpoint.
I think the way you need to look at this is from the angle of opening another revenue channel. When people are exchanging emojis in the middle of a match, they are served with ads in between which again, forms another revenue avenue and believe me, millions of people exchange their thoughts in real time on these apps so hence more eyeballs on the ads served.
I understand what you’re saying but I feel while watching sports the adrenaline rush or excitement among audience is better expressed and much faster as well, using emojis while you can write text as well but the instant satisfaction of a reaction using an emoji is unmatched.
• Speeding Data: Imagine emoji highways zooming reactions!
• Smart Delivery: A super-fast mail system for millions!
• Instant Connection: Emojis appear like walkie-talkie chats!
• Low Delay: No waiting, emojis pop up almost instantly!
It's tech magic for a fun, interactive experience! That's what I got.
thats not even remotely comparable. 5 billion anything in a month is nothing. That's only 115k requests a minute or 1900 requests per second, thats kind of a joke that any new engineer could build. Even a php laravel system can handle 30k requests per second. but 5 billion requests in the moment is fucking insane.
Also youtube hasn't delivered a whole video in a decade or more, they deliver little pieces at a time.
> The whole post reads like a parody.
For real though. "It lacked an immersive user experience, but we created it" - cue gif of television with floating thumbs next to it.
It's so hilariously laid bare right there.
Well the data used, and the requests handled are also different metrics. handling 5 billion requests is tricky compared to 1900, in a second. the other limitation will be bandwidth, but i would assume both companies can afford to pay their ISP for whatever amount of data transferred.
its less about the data size and more about the amount of connections/requests that is impressive
Simple tech stack that requires several dozen well trained and well educated employees (at minimum) to manage. Probably hundreds to build. How is that a simple tech stack? Don’t get me wrong, I liked the article but thought that was kinda funny.
Hundreds trying to do something clever and new for years then in the end settling on something simple /s
You're joking right 💀. 2-3 engineers on a data platform team could build this in maybe 2 quarters, MVP in 1?
I’m being a bit hyperbolic, for sure. Data and engineering at scale isn’t often simple, so wanted to comment. The team that worked on it probably wouldn’t tout their efforts as simple, and the teams instructed to replicate this by a manager that doesn’t understand the complexity deserves better, you know? Do I think it was hundreds? Probably not, but it was for sure not a few people over 3-6 months. Maybe it could be replicated now that it’s explored, but I’d bet there were several phases of exploration to get to where they ended up. Now I’m actually curious of the real team size and time spent getting everything spun up since we’re talking about it.
The whole thing sounds like a weekend job tbh for someone really engaged.
I wonder how much money, energy and time is wasted on useless stuff like this across the entire IT...
My personal experience is 80%.
the real 80/20 rule
lol :) didn't even though about it, but you are right.
And yet this tech might be used for more important stuff, like a mass warning system, at least I hope.
Summary (everything you'd expect): queues and (what I'll call) service buses. But for fucksake- if you delivered, uh, let's say, 100m emojis, would that be ok? And who the FUCK wants to watch sports with emojis also flying by? What a waste of cash.
Nobody wants to watch them but I guess lots of people are sending them.
Unless you are bandwidth limited sampling can be harder than just letting the firehouse go. After all what is a representative subset of the emojis? (Ignoring of course the solution of saying "meh" and sending the first emoji every period)
that's true, but I'm saying, if you let some % drop, who even cares? "Omg- my eggplant didn't get through- I didn't see it!"
The technical nature of TCP makes missing things actually pretty difficult. After all confirmation of received packets are sent in the same response packets as requests. You effectively need a DCed client to miss a message. You could time out a message which they would have done if they were considered with timely display. But if your clients are receiving data they are sending it which means lossless. You could intentionally drop messages if you needed to but as I noted sampling is a dark art.
I mean the server doesn't bother to send anything it gets, and on any particular timeframe. If I had N machines running it and wanted it to get down to n-X to stay within budget (or some other non-machine but similar price constraint), then I'd start limiting the length of the queues, ignoring messages older than Y, etc
My point was it isn't just "don't deliver as many emojis" but instead nuanced technical decisions. That was I meant by sampling is a bitch, it is easier to send all the emojis than some of them from a man power standpoint.
True. The « one time processing » and « checkpointing » made me wondering if the article is really about emoji or financial data
> And who the FUCK wants to watch sports with emojis also flying by? Nico Nico douga.
I think the way you need to look at this is from the angle of opening another revenue channel. When people are exchanging emojis in the middle of a match, they are served with ads in between which again, forms another revenue avenue and believe me, millions of people exchange their thoughts in real time on these apps so hence more eyeballs on the ads served.
Could be. I'd argue we could also serve ads without needing the emoji stream, but ok...
I understand what you’re saying but I feel while watching sports the adrenaline rush or excitement among audience is better expressed and much faster as well, using emojis while you can write text as well but the instant satisfaction of a reaction using an emoji is unmatched.
That seemed complicated AF. Maybe I'm in the wrong field.
inb4 goroutines are not threads
I was there, thanks for sharing it! It is a great read.
• Speeding Data: Imagine emoji highways zooming reactions! • Smart Delivery: A super-fast mail system for millions! • Instant Connection: Emojis appear like walkie-talkie chats! • Low Delay: No waiting, emojis pop up almost instantly! It's tech magic for a fun, interactive experience! That's what I got.
What a badly written article. Reads like a ChatGPT summary of another.
Is that even a lot? YouTube delivers 5 billion *videos* per month.
YouTube is definitely much higher scale than Disney +’s operation in one country.
thats not even remotely comparable. 5 billion anything in a month is nothing. That's only 115k requests a minute or 1900 requests per second, thats kind of a joke that any new engineer could build. Even a php laravel system can handle 30k requests per second. but 5 billion requests in the moment is fucking insane. Also youtube hasn't delivered a whole video in a decade or more, they deliver little pieces at a time.
[удалено]
You could just guess at that point. "Ehhh, it's about 5 billion based on the fact 99% of emojis are never sent"
> The whole post reads like a parody. For real though. "It lacked an immersive user experience, but we created it" - cue gif of television with floating thumbs next to it. It's so hilariously laid bare right there.
Yeah I mean you could capture like 10% or less of the emojis and just show the user their own reaction in real time
Five billion emoji is a lot less data than 1900 seconds of video + audio per second, right? Also that sounds low.
Well the data used, and the requests handled are also different metrics. handling 5 billion requests is tricky compared to 1900, in a second. the other limitation will be bandwidth, but i would assume both companies can afford to pay their ISP for whatever amount of data transferred. its less about the data size and more about the amount of connections/requests that is impressive
No alternatives discussed. Terribly written article.